Senior Data Annotation Jobs in Los Angeles (Remote, Full Time)

Rex.zone is hiring for senior data annotation jobs in Los Angeles focused on building high-quality training data for AI systems. In this remote, full-time role at Rex.zone, you will lead data labeling and QA evaluation workflows that improve large language model evaluation, RLHF preference data, prompt evaluation, and computer vision annotation. You will translate annotation guidelines into consistent labels, audit training data quality, and partner with engineering to drive model performance improvement across NLP, content safety labeling, and LLM training pipelines. Apply on Rex.zone to join a modern AI data operations team supporting AI labs, tech startups, and annotation vendors.

Job Image

Job Overview

Title: Senior Data Annotation Specialist (Los Angeles) Date: 25-02-2026 Company: Rex.zone Country: US Remote Type: Remote Employment Type: FULL_TIME Experience Level: Mid-Senior Industry: Technology Job Function: Engineering Skills: senior data annotation, data labeling, RLHF, LLM evaluation, prompt evaluation, QA evaluation, annotation guidelines, training data quality, named entity recognition, computer vision annotation, content safety labeling Salary Currency: USD Salary Min: 63360 Salary Max: 126720 Pay Period: YEAR

About the Role

You will operate as a senior data annotation specialist supporting end-to-end training data workflows for AI/ML systems. Your scope includes data labeling for NLP and computer vision, RLHF preference ranking, prompt evaluation for LLM behavior, and content safety labeling. You will enforce annotation guidelines compliance, run QA evaluation sampling, analyze disagreement and ambiguity, and implement feedback loops that increase training data quality and model performance improvement.

Key Responsibilities

You will: - Lead senior data annotation queues for LLM training pipelines and multimodal datasets - Create and refine annotation guidelines, edge-case policies, and labeling taxonomies - Perform QA evaluation, inter-annotator agreement checks, and error analysis on labeled data - Execute RLHF workflows such as preference ranking, pairwise comparison, and rubric scoring - Conduct prompt evaluation to assess helpfulness, harmlessness, and instruction following - Deliver named entity recognition (NER) and text classification labels with high consistency - Support computer vision annotation tasks (bounding boxes, polygons, segmentation masks) when needed - Coordinate with engineering and ops to resolve data issues, tooling gaps, and throughput bottlenecks - Document decisions and ensure auditability for training data quality reviews

Required Qualifications

You have: - Mid-senior experience in data labeling, data annotation, or AI data operations - Demonstrated expertise with annotation guidelines compliance and QA evaluation processes - Hands-on familiarity with RLHF, LLM evaluation, or prompt evaluation methodologies - Strong written communication for rubric design, edge-case resolution, and reviewer feedback - Comfort working with structured schemas for NLP tasks (NER, sentiment, intent, toxicity) - Ability to work independently in a remote full-time environment while maintaining delivery cadence

Preferred Qualifications

Nice to have: - Experience with content safety labeling and policy-based evaluation frameworks - Background in computer vision annotation (detection, segmentation, keypoints) - Experience supporting AI labs, tech startups, BPOs, or annotation vendors - Familiarity with dataset versioning, sampling strategies, and disagreement analytics - Understanding of how training data quality impacts model performance improvement

What You Will Work On

Projects may include: - RLHF preference datasets for instruction-tuned LLMs - Prompt evaluation suites for regression testing and behavior monitoring - Named entity recognition corpora for domain-specific information extraction - Content safety labeling for policy categories and risk assessment - Multimodal datasets requiring computer vision annotation and text alignment

How to Apply on Rex.zone

Apply through Rex.zone with a concise summary of your senior data annotation experience, the task types you have shipped (RLHF, LLM evaluation, prompt evaluation, NER, computer vision annotation), and examples of how you improved training data quality via QA evaluation and guideline refinement.

Frequently Asked Questions

  • Q: Is this a remote role for candidates in Los Angeles?

    Yes. This posting targets senior data annotation jobs in Los Angeles while remaining Remote. You can work remotely while aligned with US-based operations.

  • Q: Is the job full-time or contract/freelance?

    This role is FULL_TIME. Rex.zone may also list contract or freelance data annotation roles separately, but this posting is full-time.

  • Q: What type of annotation work is included?

    Work commonly includes data labeling for NLP and computer vision, QA evaluation, prompt evaluation, RLHF preference ranking, named entity recognition, and content safety labeling tied to LLM training pipelines.

  • Q: What does “senior” mean in data annotation?

    Senior scope typically includes leading complex queues, refining annotation guidelines, mentoring reviewers, running training data quality audits, and partnering with engineering to drive model performance improvement.

  • Q: Do I need coding experience?

    Coding is not required for many senior data annotation workflows, but comfort with structured schemas, tooling, and analytical QA evaluation is expected. Any scripting or dataset analysis experience can be beneficial.

  • Q: What industries or employer types does Rex.zone support?

    Rex.zone connects talent to AI labs, tech startups, annotation vendors, and BPO-style data operations teams working on NLP, computer vision, and content safety programs.

  • Q: How is quality measured?

    Quality is typically measured through annotation guidelines compliance, QA evaluation sampling, inter-annotator agreement, rubric adherence for RLHF and prompt evaluation, and downstream model performance improvement signals.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.