Canonical page: https://taskbull.si/AIJobs/ai-evaluation-jobs/

[TaskBull](https://taskbull.si/) / [AI jobs](https://taskbull.si/AIJobs/) / AI evaluation & testing
SPECIALIST CAREERS · CANADA & THE US

# AI evaluation & testing jobs

Measure model behavior and test the systems around it. Find LLM evaluation, AI test automation and quality engineering opportunities.

Evaluation design Regression testing Python & test automation Experiment analysis

[Browse roles ↓](https://taskbull.si/AIJobs/ai-evaluation-jobs/#jobs)[Explore this career ↗](https://taskbull.si/AIJobs/ai-evaluation-jobs/#career-guide)

APPLY DIRECTLY WITH EMPLOYERS

## 12 current roles

[Search all AI jobs ↗](https://taskbull.si/AIJobs/#jobs)

I

Innodata Curated from employer

### [Gen AI Engineer](https://taskbull.si/AIJobs/jobs/innodata-gen-ai-engineer/)

Remote · Canada

Python LLMs Data pipelines

CAD $80,000–$90,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/innodata-gen-ai-engineer/) Source checked 2026-10-29

C

Cognizant Curated from employer

### [AI Test Automation Lead](https://taskbull.si/AIJobs/jobs/cognizant-ai-test-automation-lead/)

Toronto / Kitchener · Hybrid

AI testing RAG CI/CD

CAD $90,000–$117,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/cognizant-ai-test-automation-lead/) Source checked 2026-10-29

A

Anthropic Curated from employer

### [Cyber Evaluations Engineer](https://taskbull.si/AIJobs/jobs/anthropic-cyber-evaluations-engineer/)

United States · Remote-friendly · San Francisco / Washington, DC

Python Cybersecurity Model evaluation

USD $300,000–$405,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/anthropic-cyber-evaluations-engineer/) Source checked 2026-10-30

A

Anthropic Curated from employer

### [Research Engineer, Post-Training Model Evaluations](https://taskbull.si/AIJobs/jobs/anthropic-post-training-model-evaluations/)

San Francisco / Seattle · Hybrid

Python Model evaluation Statistics

USD $500,000–$850,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/anthropic-post-training-model-evaluations/) Source checked 2026-10-30

A

Anthropic Curated from employer

### [Performance Engineer, Inference Systems](https://taskbull.si/AIJobs/jobs/anthropic-performance-engineer-inference-systems/)

San Francisco / New York City / Seattle · Hybrid

Python Performance profiling SQL

USD $350,000–$850,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/anthropic-performance-engineer-inference-systems/) Source checked 2026-10-30

O

OpenAI Curated from employer

### [Software Engineer, Safety Engineering](https://taskbull.si/AIJobs/jobs/openai-software-engineer-safety-engineering/)

San Francisco · Hybrid · Full-time

JavaScript React Python

USD $207,000–$385,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/openai-software-engineer-safety-engineering/) Source checked 2026-10-30

D

Decagon Curated from employer

### [Research Engineer, Safety](https://taskbull.si/AIJobs/jobs/decagon-research-engineer-safety/)

San Francisco / New York City · In-office · Full-time

Python Red teaming Post-training

USD $200,000–$400,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/decagon-research-engineer-safety/) Source checked 2026-10-30

D

DRW Curated from employer

### [AI Inference Platform Engineer](https://taskbull.si/AIJobs/jobs/drw-ai-inference-platform-engineer/)

Chicago, Illinois · Confirm work model

NVIDIA GPUs vLLM Linux

USD $200,000–$250,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/drw-ai-inference-platform-engineer/) Source checked 2026-10-30

A

Apollo Research Curated from employer

### [Research Scientist/Engineer (Evaluations)](https://taskbull.si/AIJobs/jobs/apollo-research-scientist-engineer-evaluations/)

San Francisco · In-person · London option also listed · Full-time

Python Data analysis Red teaming

Pay: see employer [View role ↗](https://taskbull.si/AIJobs/jobs/apollo-research-scientist-engineer-evaluations/) Source checked 2026-10-30

B

Baseten Curated from employer

### [Forward Deployed Engineer (Training)](https://taskbull.si/AIJobs/jobs/baseten-forward-deployed-engineer-training/)

San Francisco / New York · Hybrid · Full-time

GPU workloads Model evaluation Distributed systems

USD $200,000–$400,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/baseten-forward-deployed-engineer-training/) Source checked 2026-10-30

L

LangChain Curated from employer

### [Deployed Engineer, Professional Services (San Francisco)](https://taskbull.si/AIJobs/jobs/langchain-deployed-engineer-professional-services-san-francisco/)

San Francisco, California · Hybrid · Up to 20% travel · Full-time

Python LangGraph Agent evaluation

USD $150,000–$300,000 base / year [View role ↗](https://taskbull.si/AIJobs/jobs/langchain-deployed-engineer-professional-services-san-francisco/) Source checked 2026-10-30

F

Firecrawl Curated from employer

### [Agent Experience Engineer](https://taskbull.si/AIJobs/jobs/firecrawl-agent-experience-engineer/)

Toronto / San Francisco · Hybrid · 3+ office days · Full-time

A/B testing Agent evaluation Product experiments

CAD $217,000–$233,000 / year [View role ↗](https://taskbull.si/AIJobs/jobs/firecrawl-agent-experience-engineer/) Source checked 2026-10-30

Each role shows its source-check date and known location restrictions. A listing can close before its review window ends; confirm current availability with the employer. [How we review listings](https://taskbull.si/AIJobs/about/).

UNDERSTAND THE WORK

## AI evaluation & testingcareer guide.

By TaskBull AIJobs · Updated October 30, 2026

### AI evaluation spans research and application quality

Evaluation work asks whether a model or AI application performs well enough for a particular use. Research teams may build measurements for model development. Product teams may test retrieval, agent behavior, APIs and connected workflows. These jobs can share techniques while needing different software, analysis and domain experience. Read what is being evaluated and how the results influence decisions.

### Engineering roles are different from annotation tasks

AI evaluator is also used for freelance rating, labeling or human-feedback work. This hub focuses on engineering and related technical evaluation roles in our Canada and US selection. A position building an evaluation platform is not interchangeable with a short-term data-rating task. Check employment type, responsibilities and qualifications before treating a search-title match as the right opportunity.

### Build an evaluation project around a decision

Choose a specific question, such as whether a retrieval change improves answer quality without introducing a regression. Explain your test cases, scoring approach, error analysis and limits. Keep a held-out set where appropriate and show how results can be reproduced. For quality engineering, demonstrate how checks fit into delivery and how a failing test leads to investigation rather than a misleading pass/fail label.

### Read the tooling, location and experience requirements

Evaluation vacancies can require Python, data analysis, automated testing or research methods. Senior quality roles may include CI/CD, stakeholder ownership and team leadership. Hybrid roles can specify several office days each week, and remote roles still have eligibility restrictions. Use the direct employer application to confirm the latest requirements, compensation and whether sponsorship is available.

### Sources and reading

Career guidance draws on the examples below and TaskBull’s editorial analysis. Employer sources describe their own roles; they do not endorse this board.

- [Anthropic — Post-Training Model Evaluations ↗](https://job-boards.greenhouse.io/anthropic/jobs/5198255008)
- [Cognizant — AI Test Automation Lead ↗](https://careers.cognizant.com/ca-en/jobs/00070676441/ai-test-automation-lead/)

BEFORE YOU APPLY

## Your careerquestions.

Are LLM evaluation jobs entry-level? Some related tasks are, but engineering and research positions can require substantial experience. We preserve the employer’s requirements rather than labeling every evaluation role entry-level.

How are AI testing jobs different from conventional QA? Many use familiar testing and delivery practices, with additional attention to variable model outputs, retrieval quality and evaluation design. The mix depends on the application and employer.

EXPLORE RELATED WORK

## Another way in.

[AI automation jobs ↗](https://taskbull.si/AIJobs/ai-automation-jobs/)[AI agent engineer jobs ↗](https://taskbull.si/AIJobs/ai-agent-jobs/)[AI safety engineer jobs ↗](https://taskbull.si/AIJobs/ai-safety-engineer-jobs/)

BUILDING AN AI TEAM?

## Find people who canmake it work.

Submit an active role for review. Choose a monthly employer plan from USD $99. Every submitted role is reviewed before publication.

[Submit your role ↗](https://taskbull.si/AIJobs/post-a-job/)