ApplySarthi

AI Voice Evaluation Specialist

Jobgether

Tailor my CV for this job, freeView job and applyYour CV rewritten for this role, from your real experience. Sign in with Google, nothing to install.

Got this interview? Our apps help you get the job.

This role on the market

160 open evaluation roles across 50 companies are on ApplySarthi right now, most of them in Hyderabad (5), Bengaluru (5), Delhi NCR (1).

What evaluation roles keep asking for: Python (50%), Machine learning (33%), LLMs (25%), C++ (21%), SQL (18%), PyTorch (17%), Observability (13%), Deep learning (12%) — counted across their open postings here.

Jobgether has 4,317 open roles listed here.

Counted across 14 company job boards, updated as roles open and close.

Preparing for this interview

Interviews for evaluation roles keep coming back to Python, Machine learning, LLMs, C++. Practise those questions before you sit with Jobgether.

Questions you are likely to be asked

  1. Why do you want to join Jobgether?
  2. Tell me about a time the data was messy or wrong. What did you do?
  3. How would you explain your model's result to someone who is not technical?
  4. What would you check first if a model's accuracy dropped after going live?
  5. When would you not use machine learning for a problem?

Prep Sarthi gives you a free mock interview: an AI interviewer asks you questions like these out loud, from your own CV and this job, and shows your score and your weakest answer.

Practise the AI Voice Evaluation Specialist at Jobgether interview free →

Accountabilities:: Review assigned scenarios and conduct natural, voice-based roleplay conversations with two different AI models. Interact with Model A and Model B using the same scenarios, maintaining consistent conversational approaches and conditions to ensure fair comparisons. Keep approximately the same number of conversational turns with each model to support balanced and reliable evaluations. Record voice interactions and carefully assess the quality, accuracy, relevance, and overall behavior of each model's responses. Evaluate both AI models across five predefined quality dimensions, following established assessment criteria and evaluation guidelines. Identify, document, and categorize relevant error patterns and clusters occurring during audio interactions or conversations. Compare the performance of both models using structured evaluation frameworks and observations gathered throughout the interactions. Determine which model demonstrates stronger overall performance based on the defined quality standards and evaluation criteria. Prepare detailed written rationales supporting final evaluations, referencing specific conversational examples and observations from both models. Apply evaluation guidelines consistently across different scenarios and interactions to ensure objective, accurate, and dependable results. Maintain high standards of documentation, attention to detail, and evaluation consistency throughout all assigned tasks. Requirements: Bachelor's degree or equivalent relevant educational background. Excellent attention to detail, with the ability to identify subtle differences in conversational quality, response accuracy, and AI model behavior. Strong listening and comprehension skills, including the ability to understand spoken interactions and recognize inconsistencies or errors. Excellent written communication skills, with the ability to document findings clearly, objectively, and accurately. Ability to follow detailed instructions, structured evaluation frameworks, and predefined quality assessment guidelines. Confidence in speaking naturally and participating in roleplay conversations across different scenarios. Strong analytical and critical-thinking abilities, with the capacity to compare two interactions fairly and identify meaningful performance differences. Objectivity and impartiality when evaluating AI-generated responses, without allowing personal preferences to influence final assessments. High levels of reliability, consistency, and time management when completing repetitive and structured evaluation assignments. Genuine interest in artificial intelligence, conversational AI assistants, voice technologies, or emerging AI applications. Ability to work independently while maintaining accuracy and quality throughout the evaluation process. Benefits: Competitive hourly compensation ranging from CAD $20 to $25 per hour, depending on experience, skills, and qualifications. Full-time contract opportunity for an initial period of 3 months. 40-hour working week. Remote working arrangement within Canada. Opportunity to contribute directly to the evaluation and improvement of advanced AI voice technologies. Hands-on exposure to conversational AI systems and emerging artificial intelligence applications. Opportunity to develop practical experience in AI model evaluation, voice interaction analysis, and structured quality assessment. Collaborative, technology-focused working environment with clearly defined evaluation processes and quality standards.

Match this job to your CV

ApplySarthi scores your CV against this role, shows the skills you are missing, and writes a tailored version for the application.

Check my match →

Similar open roles

Need answers during your interview? Try Live Sarthi.

Live Sarthi, an Interview Sarthi app, shows answer suggestions during the call.

Try Live Sarthi free →

A Windows app, from the same team as ApplySarthi.

Listed on lever · posted 2026-11-17. ApplySarthi collects openings and links to application pages; the role is advertised by Jobgether, not by us.