ApplySarthiSenior AI Engineer, Agent Evaluation and Governance
Niuro
Tailor my CV for this job, freeView job and applyYour CV rewritten for this role, from your real experience. Sign in with Google, nothing to install.
Got this interview? Our apps help you get the job.
Skills named in this job
Read from the description itself, not inferred.
This role on the market
119 open evaluation roles across 39 companies are on ApplySarthi right now, most of them in Hyderabad (9), Bengaluru (5), Delhi NCR (3).
- Senior Data Scientist AI EvaluationJobgether
- Senior Test and Evaluation Laboratory TechnicianBoeing
- Robotics Simulation and Evaluation PhD Intern - Summer 2027Nvidia
- Executive Director, Search and Evaluation OncologyAstrazeneca
- Senior Data Analyst - AI Evaluation (Life Sciences).Relx · chennai
What evaluation roles keep asking for: Python (45%), Machine learning (25%), LLMs (24%), C++ (23%), Observability (19%), SQL (15%), Java (13%), PyTorch (13%) — counted across their open postings here.
Remote AI Engineer jobs · LLMs jobs · Python jobs · RAG jobs · TypeScript jobs
Niuro has 4 open roles listed here.
Counted across 14 company job boards, updated as roles open and close.
Preparing for this interview
Interviews for evaluation roles keep coming back to Python, Machine learning, LLMs, C++. Practise those questions before you sit with Niuro.
Questions you are likely to be asked
- Why do you want to join Niuro?
- What is your experience with LLMs? Tell me one thing you learned the hard way.
- When would you not use machine learning for a problem?
- Walk me through a model you built, from the data to how it was used.
- How did you know your model was actually good, and not just good on your test set?
Prep Sarthi gives you a free mock interview: an AI interviewer asks you questions like these out loud, from your own CV and this job, and shows your score and your weakest answer.
Practise the Senior AI Engineer, Agent Evaluation and Governance at Niuro interview free →Niuro is hiring a Senior AI Engineer for an enterprise AI platform company where AI agents take real actions: sending messages, updating records and preparing decisions. You will own the evaluation and governance layer that decides when each agent capability is ready to go live and keeps it safe in production. ## You will - Design evaluation sets, metrics, quality thresholds and go-live gates per agent capability. - Detect regressions after model, prompt or instruction changes. - Build runtime safeguards in code: approvals, data restrictions, deterministic checks, kill switches and quality-based suspension. - Log and trace every model call for accountability and auditability. - Turn governance rules and acceptance scenarios into executable validation with platform, cloud and product teams. - Write clear design notes before implementation. ## You bring - 5+ years of software engineering experience. - Production experience with tool-using, multi-step LLM agents. - Evaluation sets, quality thresholds and regression detection in production. - Applied statistics: review sample sizing, confidence bounds, error-rate estimation. - Strong Python or TypeScript, with testing and code review. - C1 English. Nice to have: knowledge graphs and ontologies (RDF, OWL, SHACL, LinkML), RAG, human-in-the-loop systems, access control, multi-provider LLM integration, financial services. Contract: full-time, fully remote, monthly contract in USD. Apply through Niuro: https://app.niuro.io/source/CbmWjJkM?ref=ps_FtixrvAJ3nEyj0nPwo6ANNKEFFN0KXqr-c5os_3iSCU&utm_source=wellfound&utm_medium=job_board
Match this job to your CV
ApplySarthi scores your CV against this role, shows the skills you are missing, and writes a tailored version for the application.
Check my match →Similar open roles
Need answers during your interview? Try Live Sarthi.
Live Sarthi, an Interview Sarthi app, shows answer suggestions during the call.
- Hidden from supported screen sharingThe overlay stays out of supported Windows screen captures.
- Answers start in about 1.5 secondsResponse time varies with your connection and model.
- From your own CVYour projects and your experience, not a generic script.
- 30 minutes freeThen ₹99 for a 2-day pass with unlimited calls — you pay for the days you are interviewing, not a subscription.
A Windows app, from the same team as ApplySarthi.
Listed on wellfound · posted 2026-10-09. ApplySarthi collects openings and links to application pages; the role is advertised by Niuro, not by us.