Senior Data Labeling Jobs in North America

Senior data labeling jobs in North America at Rex.zone focus on building high-quality training datasets for AI/ML systems through data labeling, RLHF evaluation, and QA review. You will apply annotation guidelines compliance to improve training data quality for LLM training pipelines, NLP and computer vision models, and content safety labeling workflows. This remote, full-time role supports model performance improvement via prompt evaluation, named entity recognition, and systematic error analysis in production annotation operations. Explore and apply through Rex.zone to join teams powering real-world AI products across AI labs, tech startups, annotation vendors, and BPO delivery partners.

Job Image

Senior Data Labeling Specialist (North America, Remote)

Title: Senior Data Labeling Specialist (North America, Remote) Date: 25-02-2026 Company: Rex.zone Country: US Remote Type: Remote Employment Type: FULL_TIME Experience Level: Mid-Senior Industry: Technology Job Function: Engineering Skills: data labeling, data annotation, RLHF, LLM evaluation, prompt evaluation, QA evaluation, training data quality, annotation guidelines compliance, named entity recognition, computer vision annotation, content safety labeling, dataset curation Salary Currency: USD Salary Min: 63360 Salary Max: 126720 Pay Period: YEAR You will lead day-to-day senior labeling execution and quality assurance across NLP, LLM evaluation, and computer vision annotation projects. This role requires strong judgment, consistent rubric use, and the ability to translate ambiguous model behaviors into clear labels that increase model performance. Key Responsibilities: - Own training data quality across multiple queues by enforcing annotation guidelines compliance and resolving edge cases - Perform RLHF-style ranking, preference labeling, and rubric-based QA evaluation for large language model evaluation - Conduct prompt evaluation and response assessment for helpfulness, factuality, toxicity, and policy compliance (content safety labeling) - Label and review tasks including named entity recognition, text classification, summarization grading, and conversation evaluation - Support computer vision annotation programs (bounding boxes, polygons, keypoints) with strict quality checks when required - Run error analysis, identify systematic label noise, and propose guideline updates to reduce ambiguity - Mentor labelers, calibrate decision-making, and maintain inter-annotator agreement targets - Create and maintain lightweight documentation for task definitions, examples, and escalation paths - Partner with operations and engineering stakeholders to ensure smooth throughput, QA gates, and audit readiness Qualifications: - Experience in production data labeling or data annotation with demonstrated QA review ownership - Comfort with large language model evaluation, rubric design, and ranking / preference labeling (RLHF workflows) - Strong written reasoning skills and attention to detail in long-form prompt evaluation - Familiarity with annotation tools, dataset curation, and quality metrics (precision/recall proxies, agreement rates) - Ability to work independently in a remote environment with consistent delivery and documentation Preferred: - Exposure to NLP tasks such as named entity recognition, intent classification, and information extraction - Exposure to computer vision annotation or multimodal evaluation - Experience supporting content safety labeling programs and policy-aligned evaluation What You’ll Deliver: - Higher training data quality and reduced rework through clear decisions and reliable QA evaluation - Better model performance improvement signals through consistent RLHF preference labeling and prompt evaluation - Durable annotation guidelines compliance via calibration, documentation, and feedback loops How to Apply: - Apply through Rex.zone with a resume highlighting data labeling, QA evaluation, RLHF, and LLM evaluation experience

What You’ll Work On

This senior data labeling role spans LLM training pipelines and evaluation workflows used by AI labs, tech startups, annotation vendors, and BPO delivery teams. Typical work includes RLHF preference judgments, prompt evaluation, content safety labeling, named entity recognition, and computer vision annotation with a focus on training data quality and annotation guidelines compliance.

Quality Standards and Evaluation

You will be measured on training data quality, consistency of rubric application, and the ability to reduce ambiguity in guidelines. Success includes strong QA evaluation outcomes, stable inter-annotator agreement, low defect rates, and clear documentation that improves model performance improvement signals for large language model evaluation.

Remote Work Expectations (North America)

This is a Remote, FULL_TIME position aligned to North America collaboration needs. You will operate with structured queues, clear QA gates, and asynchronous communication, while participating in calibration sessions to keep annotation guidelines compliance and training data quality consistent across projects.

Frequently Asked Questions

  • Q: What are senior data labeling jobs in North America?

    Senior data labeling jobs in North America focus on producing and reviewing high-quality labeled data used to train and evaluate AI models. The work commonly includes data labeling, QA evaluation, RLHF preference labeling, prompt evaluation, named entity recognition, content safety labeling, and sometimes computer vision annotation, all aimed at improving training data quality and model performance.

  • Q: Is this role remote and full-time?

    Yes. The role is explicitly Remote and FULL_TIME, aligned to North America collaboration requirements.

  • Q: What is RLHF and why is it part of this job?

    RLHF (Reinforcement Learning from Human Feedback) is a workflow where humans provide preference signals—such as ranking two model responses or scoring outputs against a rubric. These labels become training and evaluation signals that help improve large language model behavior, making RLHF evaluation and preference labeling core tasks in senior data labeling roles.

  • Q: What tools or tasks might I use day to day?

    You may use web-based annotation tools and QA dashboards to label and review text classification, named entity recognition, prompt evaluation, and conversation evaluation. Depending on project needs, you may also support computer vision annotation (boxes, polygons, keypoints) or content safety labeling with policy-based rubrics.

  • Q: What does success look like in this role?

    Success includes consistent annotation guidelines compliance, strong QA evaluation results, improved training data quality, reduced rework, and clear documentation and calibration practices that generate reliable signals for model performance improvement in LLM training pipelines.

  • Q: Who hires for these roles through Rex.zone?

    Openings may support AI labs, technology companies, startups, annotation vendors, and BPO delivery partners running production AI/ML data operations, including NLP, computer vision, and content safety labeling programs.

  • Q: Do you offer contract or freelance options?

    This posting is for FULL_TIME remote work. Rex.zone may also list contract or freelance data labeling roles separately depending on project needs.

  • Q: How do I apply?

    Apply via Rex.zone and include details on data labeling, data annotation, QA evaluation, RLHF, LLM evaluation, and any experience with named entity recognition, computer vision annotation, or content safety labeling.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.