Senior Backend Engineer, AI Platform

Accountabilities: AI Platform Development: Design, build, and scale the backend infrastructure required to deliver production-grade AI features across a platform serving millions of users. LLM Integration and Serving: Develop and maintain services that integrate multiple LLM providers, supporting model routing, failover, token streaming, and cost and usage tracking. Retrieval-Augmented Generation: Build and optimize retrieval systems, including vector search, embedding pipelines, context assembly, and citation-supported responses. Backend Architecture and APIs: Design robust APIs and backend services, taking end-to-end ownership of service architecture, schemas, implementation, deployment, and operational support. Reliability and Observability: Anticipate and address production risks such as hallucinations, retrieval failures, provider outages, latency issues, and unexpected costs through appropriate monitoring and instrumentation. AI Evaluation and Quality: Develop practical, repeatable evaluation methods to assess response relevance, safety, source quality, and overall AI performance. Production Operations: Monitor live services, troubleshoot incidents, improve system resilience, and support ongoing performance and reliability improvements. Cross-Functional Collaboration: Work closely with platform architects and fellow developers to shape technical decisions, establish engineering best practices, and integrate AI capabilities with existing application services. Requirements Backend Engineering Experience: At least five years of backend engineering experience using a statically typed language such as Go, Java, Kotlin, or C#. Production LLM Expertise: Demonstrated experience shipping and maintaining LLM-powered features in production, including retrieval-augmented generation, streaming responses, and tool use. LLM Serving Infrastructure: Hands-on experience with model-serving systems, including multi-provider routing, failover strategies, token streaming, and cost and usage metering. Retrieval Systems: Experience developing vector search solutions, embedding pipelines, context assembly workflows, and systems that provide source-backed answers. Reliability and Observability: Strong understanding of AI failure modes and the ability to implement instrumentation, monitoring, and safeguards to detect and address production issues. AI Evaluation: Ability to assess AI output quality using straightforward, repeatable tests covering relevance, safety, and source quality rather than relying exclusively on academic benchmarks. API Design and Ownership: Strong API design skills and the ability to own backend services throughout their lifecycle, from data schema and implementation to deployment and on-call support. Location and Work Authorization: Must be based in the United States and authorized to work in the country. Preferred Programming Skills: Go experience is strongly preferred. Familiarity with Python is also beneficial. Preferred AI Infrastructure Experience: Experience with LLM gateways or serving frameworks such as LiteLLM, vLLM, TGI, or comparable technologies. Preferred Data and Model Experience: Familiarity with vector databases such as Qdrant or pgvector, embedding pipelines, and fine-tuning open-weight models using LoRA or full fine-tuning methods. Additional Advantages: Experience with content moderation or AI safety tooling, as well as familiarity with Ruby on Rails or React/TypeScript for application integration. Benefits Remote Work: Full-time remote work for candidates based in the United States, subject to the role's location requirements. Technical Ownership: The opportunity to build greenfield AI infrastructure and influence architectural decisions, development practices, and platform capabilities. High-Scale Engineering: Work on AI-powered features designed for a platform serving millions of users. Modern Technology Stack: Exposure to Go, Ruby on Rails, Python, React/TypeScript, PostgreSQL, Redis, and RabbitMQ, with services deployed across multiple data centers. Professional Growth: Opportunity to deepen expertise in LLM infrastructure, retrieval systems, AI evaluation, distributed services, and production engineering. Collaborative Environment: Direct collaboration with platform architects and developers in a fast-paced engineering organization. Salary and Additional Benefits: Compensation, healthcare coverage, retirement benefits, paid time off, and other employee perks are not specified in the available job description and should be confirmed during the hiring process.