Senior Software Engineer, Evaluation Engine
Vanta
Tailor my CV for this job, freeView job and applyYour CV rewritten for this role, from your real experience. Sign in with Google, nothing to install.
Got this interview? Our apps help you get the job.
Skills named in this job
Read from the description itself, not inferred.
This role on the market
151 open evaluation roles across 51 companies are on ApplySarthi right now, most of them in Bengaluru (5), Hyderabad (4), Delhi NCR (1).
- Attorney, Evaluation Operations (Contract)Jobgether
- Principal AI Engineer - Agentic Systems and Evaluation PlatformAmgen · hyderabad
- Attorney, Evaluation Operations (Contract)Harvey
- 2027 Summer Intern – Research and Development - Nondestructive EvaluationGeneralmotors
- Test & Evaluation Lab Tech (Electrical/Electronic Equipment) (Mid-Level)Boeing
What evaluation roles keep asking for: Python (52%), Machine learning (34%), LLMs (26%), C++ (22%), SQL (19%), PyTorch (18%), Observability (14%), Deep learning (13%) — counted across their open postings here.
Software Engineer jobs in the United States · Remote Software Engineer jobs · Data modelling jobs · GraphQL jobs · MongoDB jobs · Observability jobs
Vanta has 94 open roles listed here.
- Manager, Sales - EMEA
- Account Executive, LatAm (Brazil)
- Senior Revenue Enablement Manager, EMEA
- Solutions Engineer, Mid-Market
- Senior Director of Design
Counted across 14 company job boards, updated as roles open and close.
Preparing for this interview
Interviews for evaluation roles keep coming back to Python, Machine learning, LLMs, C++. Practise those questions before you sit with Vanta.
Questions you are likely to be asked
- Why do you want to join Vanta?
- What is your experience with Observability? Tell me one thing you learned the hard way.
- Walk me through a system you built. How was it designed, and what would you change now?
- Tell me about a hard bug you tracked down. How did you find the cause?
- How do you decide what to test, and what does good code review look like to you?
Prep Sarthi gives you a free mock interview: an AI interviewer asks you questions like these out loud, from your own CV and this job, and shows your score and your weakest answer.
Practise the Senior Software Engineer, Evaluation Engine at Vanta interview free →At Vanta, our mission is to help businesses earn and prove trust. We believe that security should be monitored and verified continuously, and we empower companies to practice better security and prove it with ease. Vanta has a kind and talented team, and while some have prior security experience, many have been successful at Vanta without it. Our Senior Software Engineers independently drive complex technical work, shape the systems and technical decisions within their teams, and enable other engineers to deliver high-quality, scalable solutions. Vanta's product monitors the security posture of thousands of companies, pulling tens of millions of API calls of data per day, pushing information from hundreds of thousands of laptop agents, and running tests against that data continuously to identify potential security threats. Our infrastructure and tooling need to stay ahead of exponential growth in our customer base. As a Senior Software Engineer at Vanta, you'll drive complex projects across our technical stack, contribute to the technical direction of your team, and mentor other engineers. Your past experience will be leveraged to enable and accelerate Vanta's growth. Visit our Vanta Engineering Blog to learn more about what our team is working on! Tests are at the heart of how Vanta continuously monitors security and compliance for our customers. The Evaluation Engine team builds the runtime platform that powers these checks. We own how Tests are scheduled and executed, how their results are persisted and exposed, and the systems that keep this runtime reliable as Vanta grows. In this role, you'll work on some of the core systems behind Vanta's Tests platform. You'll tackle problems around the reliability, correctness, and performance of Test execution, evolve the systems and abstractions that allow the platform to scale, and make it easier for other engineering teams to build on the Tests runtime. Many of these problems span multiple systems and teams and require a deep understanding of how the platform behaves in production. What you’ll do as a Senior Software Engineer on the Evaluation Engine team at Vanta: Own complex projects from problem definition and technical design through implementation, rollout, and operation Design and evolve the backend systems, APIs, and data models used to run Tests and process their results Improve the reliability, correctness, performance, and observability of the Tests runtime Investigate complex production issues, identify root causes, and build durable solutions Simplify existing systems and improve the abstractions and interfaces that other engineers build on Collaborate with engineers across Vanta on problems that span systems and team boundaries Give thoughtful feedback in design and code reviews, communicate technical tradeoffs, and help other engineers make good decisions Mentor other engineers and contribute to the engineering culture of the team How to be successful in this role: Have strong experience building and operating reliable, scalable backend systems in production Have independently delivered complex technical projects where the problem or solution was not always well defined Have strong systems thinking and fundamentals in backend system design, APIs, and data modeling Be a strong debugger who can reason about correctness, reliability, performance, and scalability across complex systems Make effective technical tradeoffs, take ownership of systems in production, and improve mature systems incrementally Communicate technical decisions clearly, give thoughtful feedback on designs, and collaborate effectively with engineers across teams Experience with TypeScript, GraphQL, MongoDB, and observability tools such as Datadog is a plus Open to using AI to amplify their skills and strengthen their work - demonstrating curiosity, a willingness to learn, and sound judgment in applying AI responsibly to improve efficiency and impact Physical location: you must reside in, and perform all work exclusively from within, the United States, accessing systems only from U.S. locations. Work Authorization Requirements: You must be authorized to work in the United States without the need for employer sponsorship What you can expect as a Vanta’n: Industry-competitive salary and equity Comprehensive medical, dental, and vision coverage, with 100% of employee-only benefit premiums covered for most medical plans 16 weeks fully-paid Parental Leave for all new parents Health & wellness stipend Remote workspace, internet, and cellphone stipend Commuter benefits for team members who report to the SF and NYC office Family planning benefits Matching 401(k) contribution with immediate vesting Flexible PTO policy, plus 80 hours of Sick Time 11 company-paid holidays Virtual team building activities, lunch and learns, and other company-wide events! Offices in SF, NYC, London, Dublin, Tel Aviv, and Sydney To provide greater transparency to candidates, we share base pay ranges for all US-based job postings regardless of state. We set standard base pay ranges for all roles based on function, level, and country location, benchmarked against similar-stage growth companies. Final offer amounts are determined by multiple factors and may vary based on candidate location, skills, depth of work experience, and relevant licenses/credentials. #LI-remote At Vanta, we are committed to hiring diverse talent of different backgrounds and as such, it is important to us to provide an inclusive work environment for all. We do not discriminate on the basis of race, gender identity, age, religion, sexual orientation, veteran or disability status, or any other protected class. As an equal opportunity employer, we encourage and welcome people of all backgrounds to apply. About Vanta We started in 2018, in the wake of several high-profile data breaches. Online security was only becoming more important, but we knew firsthand how hard it could be for fast-growing companies to invest the time and manpower it takes to build a solid security foundation. Vanta was inspired by a vision to restore trust in internet businesses by enabling companies to improve and prove their security. From our early days automating security monitoring for compliance standards like SOC 2, HIPAA and ISO 27001 to creating the world's leading Trust Management Platform, our vision remains unchanged. Now more than ever, making security continuous—not just a point-in-time check— is essential. Thousands of companies rely on Vanta to build, maintain and demonstrate their trust— all in a way that's real-time and transparent. Referral Instructions If you are being referred for the role, please contact that person to apply on your behalf.
Match this job to your CV
ApplySarthi scores your CV against this role, shows the skills you are missing, and writes a tailored version for the application.
Check my match →Similar open roles
- Director of Enterprise Sales, EMEAVanta
- Account Executive - [UK&I]Vanta
- Account Manager, Growth UK&IVanta
- Account Executive, Early Stage - DACHVanta
- Account Executive, Early Stage - UK&IVanta
- GRC Pre-Sales Consultant / Solutions Engineer – EMEAVanta
- Solutions Engineer (Upmarket, Pre-Sales) - EMEAVanta
- Account Executive, Growth - BeneluxVanta
Need answers during your interview? Try Live Sarthi.
Live Sarthi, an Interview Sarthi app, shows answer suggestions during the call.
- Hidden from supported screen sharingThe overlay stays out of supported Windows screen captures.
- Answers start in about 1.5 secondsResponse time varies with your connection and model.
- From your own CVYour projects and your experience, not a generic script.
- 30 minutes freeThen ₹149 for a 2-day pass with unlimited calls — you pay for the days you are interviewing, not a subscription.
A Windows app, from the same team as ApplySarthi.
Listed on ashby · posted 2026-08-21. ApplySarthi collects openings and links to application pages; the role is advertised by Vanta, not by us.