Director of AI Operations and Insurance Domain Quality

Accountabilities: As the operational leader of the AI evaluation function, you will oversee team development, quality governance, workflow performance, and continuous improvement. Your responsibilities will include: Program Stand-Up and Scaling Design and implement the operational framework for the AI evaluation function. Launch and manage an initial team of approximately 10 insurance domain evaluators. Scale team capacity and operational infrastructure in response to client demand. Monitor launch speed, queue capacity growth, and reviewer time-to-proficiency. Quality Assurance and Domain Governance Establish productivity and evaluation standards, quality audit rubrics, and calibration procedures. Conduct secondary reviews of complex insurance cases and provide targeted coaching to improve evaluation accuracy. Audit policy edge cases and identify recurring errors or inconsistencies in AI-generated outputs. Track quality metrics and implement corrective actions to reduce error rates across active evaluation queues. Operational Queue and SLA Management Build and manage daily workflow queues, ensuring efficient distribution of evaluation tasks. Monitor reviewer throughput, productivity, and operational capacity. Optimize resource allocation to meet strict client turnaround times and service-level agreements. Identify and eliminate workflow bottlenecks while maintaining consistent service quality. Cross-Functional Product and Engineering Collaboration Consolidate audit findings and identify systemic AI failure patterns. Lead recurring feedback sessions with product and machine learning engineering teams. Translate insurance domain insights into actionable recommendations for improving prompts, model behavior, and evaluation logic. Track defect reporting quality, model accuracy improvements, and reductions in recurring AI processing errors. Team Leadership and Development Recruit, mentor, manage, and evaluate junior and senior insurance domain evaluators. Partner with learning and development teams to create continuous training programs. Ensure evaluators remain up to date on insurance policy standards, domain practices, and AI audit tools. Monitor team retention, skill development, performance, and calibration consistency. Requirements The ideal candidate brings extensive commercial insurance experience, proven operational leadership, and the ability to apply specialized domain knowledge to emerging AI evaluation workflows. The following qualifications are expected: At least 8–10 years of progressive leadership experience in commercial Property & Casualty insurance operations, underwriting quality control, brokerage management, or insurtech program operations. Active Resident or Non-Resident Property & Casualty (P&C) Producer/Agent License. Professional designations such as CPCU, CIC, or ARM are strongly preferred. Deep technical knowledge of commercial P&C insurance lines, including General Liability, Commercial Property, Workers' Compensation, Umbrella/Excess, and Cyber. Strong understanding of standard insurance forms and documentation, including ACORD forms, ISO forms, and manuscript endorsements. Demonstrated experience building operational teams and establishing workflow processes and quality assurance frameworks from the ground up. Proven ability to manage team capacity, monitor operational SLAs, and scale teams from initial groups of approximately 10 employees to larger enterprise operations. Familiarity with AI/ML evaluation workflows, document extraction platforms, or Reinforcement Learning from Human Feedback (RLHF) methodologies. Experience with insurance or document management platforms such as Applied Epic, Vertafore, Hyperscience, or Glider is advantageous. Strong analytical and problem-solving skills, particularly in identifying quality issues and translating findings into operational improvements. Excellent executive communication and stakeholder management skills, with the ability to collaborate directly with client product leaders, machine learning engineers, and executive sponsors. Proven ability to lead, coach, and develop teams while maintaining high standards of quality and performance. Benefits Annual salary range of $135,000–$145,000 USD , depending on qualifications, relevant experience, technical expertise, and role requirements. Comprehensive healthcare coverage for employees and their families. Employee Assistance Program, including counseling and coaching support for employees and family members. 401(k) retirement plan with company matching contributions. Paid time off. Paid parental leave. Volunteer Day Off. Life insurance coverage. Business travel accident insurance. Remote work arrangement within the United States.