Director of Engineering, IAEF

Accountabilities:: The Director of Engineering will own the technical direction and execution of the platform while building a high-performing engineering organization, strengthening security and compliance practices, and representing the technical vision to external stakeholders. Own the Workbench’s Design → Test → Analyze → Publish (DTAP) workflow, Component Contract, Artifacts API, component registry, and canvas interface. Build a persistent, structured, and queryable evaluation history while integrating existing red-teaming, administrative dashboard, and human-in-the-loop data-capture capabilities. Evaluate build-versus-buy decisions for authentication, storage, orchestration, LLM-provider services, sandboxing, and other core platform capabilities. Recruit, develop, and retain the engineering organization in partnership with the HR function, including Staff/Senior Backend, Senior Frontend, DevOps/Site Reliability, Applied Machine Learning, and Senior Product Management roles. Establish effective planning, code review, on-call, incident response, and roadmap review practices. Track engineering progress, risks, and delivery outcomes for board and funder reporting. Partner with the CEO and Evaluation Science Lead to translate human-in-the-loop red teaming, automated benchmarks, LLM red teaming, LLM-as-judge, and drift detection into a reusable Component Contract. Ensure the platform supports the required Test artifact types and planned agentic components. Lead vendor risk reviews covering LLM providers, cloud infrastructure, and third-party evaluation tools. Coordinate with organizational leadership on required approvals, funder restrictions, and the appropriate separation of resources, data, and intellectual property across related entities and funded programs. Escalate unresolved legal, policy, or sponsor-related risks through the appropriate leadership channels. Represent the platform in technical discussions with customers, partners, funders, and press. Own technical content for investor fundraising materials, roadmaps, demonstrations, and due diligence reviews in collaboration with executive leadership. Shape the developer-relations narrative for SDK and ADK capabilities as the Component Contract becomes available to external developers. Partner with planned Developer Relations and Ecosystem leadership to support external developer adoption. Provide technical leadership across delivery, operational issues, organizational growth, and continuous platform improvement. Requirements: The ideal candidate brings deep software engineering expertise, senior technical leadership experience, and a strong understanding of distributed engineering, AI evaluation, platform architecture, security, and organizational development. 12+ years of software engineering experience, including at least 4 years in senior technical leadership within a startup, evaluation or audit firm, machine-learning infrastructure company, or research-grade platform team. Experience building and operating platforms with plugin, extension, or component models. Production experience with AI evaluation, red-teaming, machine-learning observability, model testing, or related systems. Proven ability to hire, develop, and retain senior engineers within a distributed organization. Experience building or leading teams that include applied machine-learning or agentic-systems expertise. Ability to design reusable platform components and work with human-in-the-loop transcripts, benchmark scores, and LLM red-teaming results. Strong understanding of platform architecture, engineering delivery, operational reliability, and technical decision-making. Ability to communicate complex technical decisions clearly to enterprise security leaders, foundation program officers, investors, customers, partners, and other non-technical stakeholders. Strong prioritization skills and the ability to balance competing technical, organizational, and delivery requirements. Ability to independently lead distributed engineering teams through delivery, operational challenges, and periods of growth. Understanding of the operational separation required between public-benefit organizations, nonprofit entities, and grant-funded programs. Strongly preferred experience implementing ISO 42001, the NIST AI Risk Management Framework, or SOC 2 requirements. Prior CTO, VP Engineering, or founding-engineer experience at an early-stage company with institutional funding is preferred. Experience enabling external developers to build on a platform is preferred. Experience developing evaluation methodologies with academic or civil-society partners is a plus. Experience working with enterprise security, compliance, vendor risk, or AI governance programs is valuable. A PhD or prior experience at a frontier AI research laboratory is not required. Strong communication, judgment, collaboration, and team-development skills. Benefits: $215,000–$315,000 annual compensation, commensurate with experience. Remote-first, full-time position. Health, dental, and vision insurance, with 100% of employee premiums and 85% of dependent premiums covered. $365 monthly employer contribution to a pre-tax Health Savings Account. Pre-tax Flexible Spending Account. Employer-paid AD&D insurance. Employer-paid short-term and long-term disability insurance. Employer-paid basic life insurance. Supplemental voluntary life insurance for employees, spouses, and dependent children. Pet insurance options, including Total Pet Plan and Supplemental Wishbone coverage. Employee Assistance Program supporting health and wellness. Wellness reimbursement program. 401(k) retirement plan with 6% employer matching. 15 days of paid time off annually, increasing to 20 days after the third anniversary and 25 days after the sixth anniversary. 16 paid holidays, including 14 common holidays and 2 flexible holidays. Paid bereavement leave. Paid parental leave for mothers and fathers, including two weeks after the first year. Approximately 15% travel for funder meetings, partner sites, conferences, and off-sites. Quarterly in-person meetings, with New York or the San Francisco Bay Area strongly preferred for proximity to key stakeholders. Opportunity to lead an impactful AI evaluation platform focused on trustworthiness, transparency, safety, and educational applications. High degree of technical ownership across architecture, engineering strategy, security, and platform growth. Opportunity to influence AI evaluation practices, external developer ecosystems, and responsible AI initiatives.