AES - Delivery - Digital Test Automation Engineer

You will own quality for data pipelines, microservices, APIs, GUIs, and LLM/ML integrations running on Databricks and Microsoft Azure. The role focuses on test strategy, test case design and execution, automation, ticket management, and agentic QA for LLM workflows. Minimum experience: 5 years Key Responsibilities Test Strategy and Planning Define and maintain QA strategy for data, API, GUI, microservices, and LLM/ML features; prioritize test coverage based on risk and business impact. Test Case Design and Execution Create clear, reusable test cases for functional, integration, regression, performance, and pricing scenarios; execute manual and automated tests and maintain test artifacts. Data and Database Testing Validate ETL/ELT pipelines on Databricks and Delta Lake; write and run T‑SQL validation queries; verify data lineage, completeness, and correctness. API Testing Design and run contract, integration, and negative tests for RESTful APIs; use Postman, OpenAPI/Swagger, and automated API test suites. GUI Testing Create and execute UI test plans for web applications; perform exploratory testing and maintain automated UI tests using Selenium, Playwright, or equivalent. Microservices Validation Test service contracts, message flows, resilience, and backward compatibility across containerized microservices; validate observability and error handling. LLM and ML QA Design evaluation metrics and test harnesses for ML models and LLM integrations; validate inference correctness, hallucination rates, latency, and cost tradeoffs. Prompt Engineering and Agentic QA Create minimalistic, cost‑aware prompts and prompt templates; implement agentic QA processes for autonomous agents including safety checks, guardrails, and evaluation pipelines. Pricing and Business Logic Testing Validate pricing engines, discount rules, billing flows, and edge cases; create scenario matrices and automated checks for pricing correctness. Automation and CI/CD Integrate tests into CI/CD pipelines; author automated test suites, test runners, and validation scripts; collaborate with SRE/DevOps on test environments. Ticketing and Incident Management Raise, triage, and manage tickets in Azure DevOps or Jira; document reproduction steps, root cause analysis, and remediation. Documentation and Collaboration Produce test plans, runbooks, release checklists, and postmortems; mentor junior QA engineers and participate in cross‑functional reviews. Required Skills and Experience Experience: Minimum 5 years in QA, software testing, or related roles. Databricks and Data Testing: Hands‑on experience validating Databricks pipelines, Delta Lake, and Spark outputs. Microsoft SQL Server: Medium proficiency in T‑SQL for validation queries and database testing. Azure: Practical experience with Azure Data Factory, ADLS/Blob Storage, Azure DevOps, and Azure Functions. API Testing: Strong experience with REST APIs, Postman, OpenAPI/Swagger, and automated API testing. GUI Testing: Experience with Selenium, Playwright, or equivalent UI automation tools and exploratory testing. Microservices: Familiarity testing containerized services, message queues, and service orchestration. LLM and ML Fundamentals: Understanding of ML lifecycle, model evaluation, prompt engineering basics, and LLM failure modes. Prompt Engineering: Ability to craft minimalistic prompts and maintain prompt libraries for reproducible tests. Agentic QA Awareness: Knowledge of agentic QA concepts, safety checks, and evaluation metrics for autonomous agents. Testing Tools and Automation: Experience with test frameworks, CI/CD integration, and test data management. Ticketing Tools: Experience using Azure DevOps, Jira, or similar. Soft Skills: Strong analytical thinking, clear written communication, and cross‑team collaboration. Nice to Have Certifications: ISTQB, Azure Data Engineer, or Databricks certifications. Programming: Python for test automation and data validation; familiarity with Scala or Java. Observability: Experience with Application Insights, Datadog, Prometheus, or Grafana. Performance Testing: Experience with JMeter, k6, or similar tools. LLM Ops: Experience with prompt versioning, cost monitoring, and model evaluation frameworks. Infrastructure as Code: Familiarity with Terraform or ARM templates for test environment provisioning. Interview and Evaluation Guide Technical screen: SQL validation problem and API testing scenarios. Practical exercise: Create test cases and automated checks for a sample Databricks ETL and an API that returns pricing. LLM prompt exercise: Provide minimal prompts for a given LLM task and design evaluation metrics for correctness and safety. System design: Design an agentic QA pipeline for autonomous agents including test harness, safety gates, and monitoring. Behavioral: Examples of ticket triage, incident postmortems, and cross‑team collaboration. Success Criteria at 6 Months Test Coverage: Comprehensive, automated test suites for data pipelines, APIs, and core GUI flows. LLM Readiness: Reproducible prompt templates, evaluation metrics, and agentic QA checks integrated into CI. Reliability: Reduced production incidents and faster mean time to resolution through clear runbooks and test automation. Pricing Confidence: End‑to‑end validated pricing flows with scenario tests and regression guards. Collaboration: Trusted QA partner to engineering and product teams with documented processes and mentoring outcomes.