Senior Data Annotation Jobs in San Francisco

Senior data annotation jobs in San Francisco at Rex.zone focus on producing high-quality training data for AI/ML systems through data labeling, RLHF (Reinforcement Learning from Human Feedback), LLM evaluation, prompt evaluation, and QA review. You will apply rigorous annotation guidelines compliance to improve model performance, reduce hallucinations, and strengthen content safety in real-world LLM training pipelines across NLP and computer vision. These remote, full-time roles support AI labs, tech startups, and annotation vendors by delivering training data quality that scales, with measurable impact on model evaluation, named entity recognition, and policy-based labeling outcomes. Explore and apply on Rex.zone.

Senior Data Annotation Jobs in San Francisco

About the Role

You will lead and execute complex data annotation workflows for LLM and multimodal model training, including RLHF preference ranking, prompt-response evaluation, content safety labeling, and structured data labeling for NLP tasks such as named entity recognition and intent classification. You will produce gold-standard labels, conduct QA evaluation, calibrate with other annotators, and help maintain annotation guidelines compliance so training data quality consistently supports model performance improvement.

Key Responsibilities

You will deliver high-accuracy labels across NLP, computer vision annotation, and content moderation datasets; perform RLHF ranking and critique to improve helpfulness, harmlessness, and honesty; run prompt evaluation to detect policy violations, unsafe outputs, and low-quality generations; execute QA review cycles (spot checks, inter-annotator agreement checks, adjudication); document edge cases and propose guideline updates that reduce ambiguity; collaborate with data operations and engineering to refine labeling tools, taxonomies, and sampling strategies; track metrics tied to training data quality, annotation throughput, and defect rates; support model evaluation sets and offline benchmarks used for large language model evaluation.

Required Qualifications

Demonstrated experience with data annotation or data labeling at scale; strong understanding of annotation guidelines compliance and QA evaluation practices; familiarity with RLHF, preference data, or human feedback workflows; ability to apply consistent judgment on ambiguous content for content safety labeling; strong written communication for documenting decisions, edge cases, and rationale; comfort working in remote, production-driven environments with measurable quality targets.

Preferred Qualifications

Hands-on experience with LLM training pipelines, prompt evaluation, and large language model evaluation; exposure to named entity recognition, sentiment, toxicity, or instruction-following datasets; experience with computer vision annotation (bounding boxes, polygons, keypoints) and multimodal evaluation; experience with annotation tooling, audit sampling, and inter-annotator agreement methodologies; background supporting AI labs, tech startups, BPOs, or annotation vendors with strict turnaround and quality requirements.

What Success Looks Like

Consistent delivery of high-quality labeled data with low defect rates; measurable improvements in training data quality and model performance improvement signals; clear documentation that reduces guideline ambiguity and improves annotation consistency; reliable QA evaluation practices that catch policy issues and mislabeled samples early; strong calibration outcomes across projects involving RLHF, prompt evaluation, and content safety labeling.

How to Apply

Apply via Rex.zone and be ready to complete a short labeling or evaluation exercise covering annotation guidelines compliance, QA review, and prompt evaluation. Your application should highlight relevant data labeling, RLHF, LLM evaluation, and training data quality experience aligned to senior data annotation jobs in San Francisco.

Frequently Asked Questions

Q: Are these senior data annotation jobs in San Francisco remote?
Yes. The roles are explicitly Remote while targeting candidates aligned to the San Francisco job market and AI ecosystem.
Q: What kind of work is included in senior data annotation?
Typical work includes data labeling, QA evaluation, RLHF preference ranking, prompt evaluation, content safety labeling, and building or validating gold-standard datasets for LLM training pipelines.
Q: Do I need experience with RLHF and LLM evaluation?
It is strongly preferred. Senior-level performance often requires comfort with RLHF, large language model evaluation, and structured critique workflows that improve model behavior.
Q: What domains are supported: NLP, computer vision, or content safety?
These roles can span NLP (including named entity recognition), computer vision annotation, and content safety labeling depending on project needs.
Q: Is this a full-time role and what is the pay range?
Yes, it is FULL_TIME. The listed annual range is USD 63360 to USD 126720 with Pay Period: YEAR.
Q: What employers does this work typically support?
Senior data annotation work commonly supports AI labs, tech startups, annotation vendors, and BPO-style data operations teams that deliver training data quality at scale.
Q: What skills should I emphasize to match the role?
Emphasize senior data annotation, data labeling, RLHF, LLM evaluation, prompt evaluation, QA evaluation, annotation guidelines, and named entity recognition, along with examples of training data quality and model performance improvement outcomes.

230+Domains Covered

120K+PhD, Specialist, Experts Onboarded

50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.