AI QA Automation Lead

So, what’s the role all about?

We are looking for a strong AI QA Automation Lead with high technical skills to build methodology, provide vision and direction, and lead quality & automation throughout the organization. This role operates at the intersection of traditional software quality and AI-powered development: owning quality not just for code written by humans, but for systems designed, generated, and executed by AI agents.

How will you make an impact?  

• Serve as the quality leader for an entire domain, accountable for product quality across human-written, AI-generated, and agent-executed software
• Actively track the domain’s quality tasks, analyze the testing needs, and define QA activities and ownership
• Continuously analyze and challenge existing testing coverage
• Plan, develop, and execute test strategy across our Cloud Analytics platform , extending to AI-powered and agentic system components
• Review and approve tests cases and complex system scenarios
• Prepare and present data-driven quality insights, KPIs, risks, and trends to leadership through regular domain level reporting.
• Lead domain-level RCA reviews for customer-facing bugs and major incidents; detect trends and drive systemic improvements.
• Own post-deployment quality monitoring, drive sanity completion and rapid triage of production issues within the domain.
• Lead Quality assurance efforts that align with product objectives to ship products rapidly and with confidence
• Help the teams design the performance, concurrency and latency test scenarios and metrics
• Lead risk-based test strategy with product and engineering — define acceptance criteria and quality gates that support high delivery velocity without sacrificing customer-impacting quality
• Implement and evolve QA processes to get effective test signals and scale testing efforts across our services
• Define acceptance criteria for non-deterministic systems using golden-set evaluation, behavioral contracts, and statistical confidence thresholds
• Define adversarial and edge-case testing methodologies for AI features: prompt injection resistance, boundary condition handling, and graceful degradation
• Partner with engineering and DevOps teams to leverage automation for scalable solutions to prevent regressions and ensure reliability of product
• Track and improve CI/CD pipeline health across environments — expanding automation coverage, hardening test stability, and ensuring every environment produces trustworthy test signals.
• Plan and lead NFR testing
• Define guidelines and best practices for our automation and manual testing procedures
• Coordinate cross-organization/cross-region testing activities

Have you got what it takes?

Education:

• BSc in Computer Science/Software Engineering or equivalent
• 3 years QA Automation leadership experience: Team leader/Manager
• At least 4 years of automation experience
• Hands-on experience working within an agentic SDLC including AI-assisted code generation, agent-driven development workflows, and the quality, review, and governance practices that keep AI-accelerated delivery safe and reliable.
• Experience building AI-enabled automation workflows (e.g. Claude/OpenAI APIs, prompt patterns for test generation, repo-aware RAG, scripts/services that turn AI output into runnable tests)
• Experience with large scale production distributed systems and proven experience with designing and implementing tests for such systems
• Experience with systems that work with structured and unstructured data and with AI services and models
• Demonstrated technical leadership across teams — driving standards, roadmap execution, and stakeholder alignment
• Ability to define QA processes across development groups
• Hands on experience with CI/CD and Test-Driven Development
• Excellent communication skills (written and verbal) and interpersonal skills
• Experience with full software development lifecycle using Agile Development processes
• Experience designing evaluation harnesses for non-deterministic AI systems
• Cloud experience (AWS/Azure/GCP) and Infrastructure-as-Code

Agentic AI & LLM Proficiency

• Deep, hands-on understanding of agentic AI patterns: tool use, multi-agent orchestration, planning loops, memory architectures, and human-in-the-loop design
• Expert-level prompt engineering including chain-of-thought, few-shot, RAG, and structured output techniques applied to code and test generation
• Experience designing evaluation harnesses for non-deterministic AI systems — statistical confidence, behavioral consistency, and regression detection
• Familiarity with agent frameworks (LangChain, LlamaIndex, AutoGen, CrewAI, or equivalents) and their tradeoffs in production quality pipelines

Technical Depth

• Expert in Python and TypeScript; proficient in at least one compiled language (Java, Go, C#)

• Strong CI/CD architecture skills: Jenkins, GitHub Actions, pipeline-as-code, artifact management

What’s in it for you?

Join an ever-growing, market disrupting, global company where the teams – comprised of the best of the best – work in a fast-paced, collaborative, and creative environment! As the market leader, every day at NICE is a chance to learn and grow, and there are endless internal career opportunities across multiple roles, disciplines, domains, and locations. If you are passionate, innovative, and excited to constantly raise the bar, you may just be our next NiCEr!

Enjoy NiCE-FLEX!

At NiCE, we work according to the NiCE-FLEX hybrid model, which enables maximum flexibility: 2 days working from the office and 3 days of remote work, each week. Naturally, office days focus on face-to-face meetings, where teamwork and collaborative thinking generate innovation, new ideas, and a vibrant, interactive atmosphere.

Requisition ID: 11279
Reporting into: Vice President, Head of Engineering, Platform Research
Role Type: Individual Contributor

#LI-Hybrid