Senior Data Annotator Jobs in San Francisco

Senior data annotator jobs in San Francisco (Remote, Full-Time) at Rex.zone support AI/ML training workflows by producing high-quality labeled data, RLHF preference judgments, prompt evaluation, and QA evaluation for large language models. You will apply annotation guidelines compliance, improve training data quality, and contribute to model performance improvement across NLP, computer vision, and content safety labeling. This role focuses on dataset curation, structured data labeling, and evaluation signals used in LLM training pipelines for AI labs, tech startups, and annotation vendors.

Job Image

Senior Data Annotator Jobs in San Francisco

Title: Senior Data Annotator Jobs in San Francisco Date: 25-02-2026 Company: Rex.zone Country: US Remote Type: Remote Employment Type: FULL_TIME Experience Level: Mid-Senior Industry: Technology Job Function: Engineering Skills: Data annotation, Data labeling, RLHF, LLM evaluation, Prompt evaluation, QA evaluation, Named entity recognition, Computer vision annotation, Content safety labeling, Annotation guidelines, Training data quality Salary Currency: USD Salary Min: 63360 Salary Max: 126720 Pay Period: YEAR

About the Role

As a Senior Data Annotator, you will label and evaluate multimodal and text-based data to create reliable supervision signals for machine learning models. Your work will include data labeling, RLHF ranking, prompt evaluation, and QA evaluation to support large language model evaluation and iterative model performance improvement. You will follow annotation guidelines compliance requirements, document edge cases, and collaborate with data operations, ML engineers, and QA to maintain consistent training data quality in production-grade LLM training pipelines.

Key Responsibilities

You will: - Produce high-accuracy labels for NLP and computer vision annotation tasks (classification, extraction, segmentation, bounding boxes as needed) - Perform RLHF preference ranking and rubric-based QA evaluation for LLM outputs - Conduct prompt evaluation and response quality scoring (helpfulness, correctness, safety, policy compliance) - Apply named entity recognition (NER) and structured information extraction with clear justifications - Flag ambiguous items, propose guideline updates, and maintain annotation guidelines compliance - Run self-check and peer-check workflows to improve training data quality and reduce label noise - Document recurring error patterns to support model performance improvement and evaluator calibration - Support content safety labeling (toxicity, self-harm, harassment, sexual content, hate, violence) with consistent taxonomy usage

Required Qualifications

- Mid-Senior experience in data annotation, data labeling, or LLM evaluation - Demonstrated ability to follow detailed annotation guidelines and apply consistent decision rules - Experience with RLHF-style ranking, side-by-side evaluation, or rubric scoring for model outputs - Familiarity with NLP tasks such as named entity recognition, text classification, and prompt evaluation - Strong written communication for documenting rationales, edge cases, and QA findings - Ability to meet productivity and quality targets in remote, asynchronous workflows

Preferred Qualifications

- Experience with computer vision annotation (bounding boxes, polygons, keypoints) and visual QA - Prior work on content safety labeling, trust & safety, or policy-based evaluation - Exposure to LLM training pipelines, dataset curation, and evaluation set construction - Experience collaborating with AI labs, tech startups, BPOs, or annotation vendors - Comfort using annotation platforms, QA dashboards, and review queues

Work Model, Location, and Scheduling

- Location keyword alignment: San Francisco, US - Remote Type: Remote (role remains remote regardless of location modifier) - Employment Type: FULL_TIME - Schedule: Remote, outcome-driven production with quality-first review cycles

How You Will Be Evaluated

Success metrics typically include: - Training data quality: accuracy, consistency, and low rework rates - Annotation guidelines compliance: adherence to rubrics and taxonomy - QA evaluation performance: agreement rates, rationale clarity, and calibration stability - Throughput: consistent delivery while maintaining quality - Model performance improvement contributions: actionable feedback and error pattern reporting

Why This Role at Rex.zone

Rex.zone connects experienced annotators and evaluators with AI/ML data operations work across NLP, computer vision, and content safety. You will help create dependable supervision data—data labeling, RLHF signals, and QA evaluation—that directly impacts large language model evaluation and training outcomes. Explore and apply through Rex.zone to work on real-world LLM training pipelines used by leading technology teams.

Frequently Asked Questions

  • Q: Are these senior data annotator jobs in San Francisco remote?

    Yes. Remote Type is Remote, and the role remains remote while targeting the San Francisco job search modifier for discovery and matching.

  • Q: Is this a full-time role?

    Yes. Employment Type is FULL_TIME with yearly compensation (Pay Period: YEAR).

  • Q: What types of tasks are included in senior data annotation?

    Typical work includes data labeling, RLHF preference ranking, prompt evaluation, QA evaluation, named entity recognition, computer vision annotation, and content safety labeling aligned to annotation guidelines.

  • Q: What skills are most important for this role?

    Strong data annotation fundamentals, annotation guidelines compliance, training data quality focus, RLHF/LLM evaluation experience, prompt evaluation, QA evaluation, and domain familiarity across NLP and computer vision.

  • Q: What kinds of employers use Rex.zone for data annotation work?

    Rex.zone supports hiring needs across AI labs, technology teams at startups, annotation vendors, and BPO-style data operations groups that contribute to LLM training pipelines.

  • Q: Do you hire contract or freelance annotators too?

    This page is for FULL_TIME roles, but search modifiers like contract and freelance may appear across other Rex.zone job listings depending on project needs.

  • Q: How does RLHF relate to data annotation?

    RLHF uses human preference judgments (rankings or ratings) as supervision signals. Senior annotators help produce consistent, rubric-based preference data that improves model behavior and supports large language model evaluation.

  • Q: What does quality assurance (QA) mean in this context?

    QA evaluation includes reviewing labels or model outputs for correctness and policy compliance, calibrating with other evaluators, documenting edge cases, and improving training data quality through feedback loops.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.