Senior AI Data Annotation Jobs in Seattle

Senior AI data annotation jobs in Seattle at Rex.zone focus on training-data quality for modern AI/ML systems—spanning data labeling, RLHF, LLM evaluation, prompt evaluation, and QA evaluation. You will apply annotation guidelines compliance, perform error analysis, and improve large language model evaluation signals used in LLM training pipelines. This remote full-time role supports enterprise and AI lab workflows across NLP, computer vision annotation, content safety labeling, and named entity recognition, helping drive model performance improvement with consistent, high-precision human feedback.

Job Image

LinkedIn Job Metadata — Senior AI Data Annotation Jobs in Seattle

Title: Senior AI Data Annotation Jobs in Seattle; Date: 25-02-2026; Company: Rex.zone; Country: US; Remote Type: Remote; Employment Type: FULL_TIME; Experience Level: Mid-Senior; Industry: Technology; Job Function: Engineering; Skills: AI data annotation, data labeling, RLHF, LLM evaluation, prompt evaluation, training data quality, QA evaluation, annotation guidelines compliance, named entity recognition, computer vision annotation, content safety labeling, error analysis; Salary Currency: USD; Salary Min: 63360; Salary Max: 126720; Pay Period: YEAR

About the Role

You will lead and execute high-impact AI data annotation and evaluation workflows that improve training data quality for LLMs and multimodal models. The work includes RLHF preference labeling, QA evaluation for instruction following, prompt evaluation, and targeted data labeling for NLP, named entity recognition, and computer vision annotation. You will contribute to model performance improvement by following annotation guidelines compliance, documenting edge cases, and partnering with cross-functional teams to refine taxonomies, rubrics, and acceptance criteria for large language model evaluation.

Key Responsibilities

Deliver senior-level AI data annotation across text, image, and mixed-modality tasks; Execute RLHF workflows including preference ranking, pairwise comparisons, and rationale capture where required; Perform LLM evaluation and QA evaluation against rubrics for helpfulness, honesty, harmlessness, and policy compliance; Run prompt evaluation to identify failure modes, jailbreak risk, and instruction-following gaps; Apply annotation guidelines compliance, report ambiguities, and propose guideline improvements; Conduct error analysis, label auditing, and consensus reviews to stabilize annotation quality; Support content safety labeling for toxicity, self-harm, violence, hate, and regulated content scenarios; Maintain throughput and precision targets while protecting data confidentiality and user privacy requirements; Collaborate with engineers and data operations on task design, schema updates, and quality monitoring signals; Produce clear documentation for edge cases and escalate issues impacting training data quality.

Required Qualifications

Mid-Senior experience delivering AI data annotation, data labeling, or LLM evaluation work in production settings; Strong understanding of RLHF concepts and how human feedback improves model alignment; Demonstrated ability to follow and improve annotation guidelines compliance with high attention to detail; Experience with QA evaluation methods such as spot checks, blind review, inter-annotator agreement, and calibration; Familiarity with NLP labeling (classification, sentiment, NER) and/or computer vision annotation (bounding boxes, segmentation, keypoints); Experience performing error analysis and translating findings into actionable guideline or rubric updates; Excellent written communication for documenting decisions, edge cases, and evaluation rationales; Ability to work remotely with consistent availability and secure handling of sensitive data.

Preferred Qualifications

Experience with content safety labeling and policy-driven evaluation workflows; Exposure to prompt evaluation for instruction following, tool-use outputs, and multi-turn conversations; Familiarity with model debugging signals such as confusion matrices, disagreement analysis, and rubric drift; Background supporting AI labs, tech startups, annotation vendors, or BPO-style data operations programs; Comfort working across multiple domains including NLP, computer vision annotation, and multimodal LLM training pipelines.

Tools and Workflows

Annotation platforms for data labeling and review queues; Rubric-based LLM evaluation and RLHF labeling interfaces; QA evaluation workflows including audits, sampling plans, and calibration sessions; Versioned guidelines, taxonomy management, and task change logs; Secure remote access practices and privacy-first handling of project data.

Compensation and Employment

This is a remote, full-time role with annual compensation in USD within the stated range. Final compensation may vary based on evaluated skills alignment, task complexity, and scope within AI data annotation, RLHF, and large language model evaluation programs.

How to Apply on Rex.zone

Apply through Rex.zone by submitting your profile with relevant AI data annotation, data labeling, RLHF, and QA evaluation experience. Include examples of guideline-driven work, calibration participation, or error analysis that led to measurable training data quality improvements.

Role Scope and Search Modifiers

This posting covers remote senior AI data annotation jobs in Seattle and may include adjacent work modes such as contract, freelance, entry-level, and senior roles across NLP, computer vision annotation, content safety labeling, and LLM training pipelines depending on project needs and availability.

Frequently Asked Questions

  • Q: Is this role remote even if the keyword includes Seattle?

    Yes. The role is explicitly marked Remote. “Seattle” reflects the target job search region and keyword intent, while work is performed remotely within the US.

  • Q: What does “Senior AI Data Annotation” mean in this posting?

    It refers to advanced data labeling and evaluation work that directly influences training data quality and model performance improvement, including RLHF, LLM evaluation, prompt evaluation, and QA evaluation with strong annotation guidelines compliance.

  • Q: What types of tasks will I work on?

    Typical tasks include RLHF preference ranking, rubric-based large language model evaluation, prompt evaluation for instruction following, named entity recognition labeling, computer vision annotation, and content safety labeling with structured QA review.

  • Q: What skills are most important to succeed?

    Strong AI data annotation fundamentals, data labeling accuracy, RLHF understanding, LLM evaluation and QA evaluation practice, prompt evaluation judgment, annotation guidelines compliance, and the ability to run error analysis and document edge cases clearly.

  • Q: Do you offer contract or freelance options?

    This posting is for FULL_TIME employment. Rex.zone may list other contract or freelance annotation roles separately depending on project demand.

  • Q: What industries or employer types does this work support?

    Projects commonly support AI labs, tech startups, enterprise AI teams, annotation vendors, and BPO-style data operations programs that require reliable human feedback for NLP, computer vision, content safety, and LLM training pipelines.

  • Q: How is quality measured in AI data annotation and evaluation?

    Quality is measured through QA evaluation methods such as audits, gold sets, calibration, inter-annotator agreement, rubric adherence, and error analysis focused on training data quality and downstream model performance improvement.

  • Q: Where do I apply?

    Apply via Rex.zone by submitting your profile and highlighting relevant AI data annotation, RLHF, LLM evaluation, and guideline-driven QA experience.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.