Senior Data Annotation Jobs in Dublin

Senior data annotation roles at Rex.zone focus on building high-quality training datasets for AI/ML systems through data labeling, RLHF evaluation, QA review, and model-aligned prompt evaluation. You will apply annotation guidelines compliance to improve training data quality for large language model evaluation, NLP tasks like named entity recognition, computer vision annotation, and content safety labeling. These remote full-time jobs support LLM training pipelines used by AI labs, tech startups, and annotation vendors, with clear quality metrics tied to model performance improvement. Explore and apply via Rex.zone to join a distributed team delivering reliable human feedback for next-generation AI systems.

Job Image

Job Opening: Senior Data Annotation Specialist

Date: 25-02-2026 Company: Rex.zone Country: US Remote Type: Remote Employment Type: FULL_TIME Experience Level: Mid-Senior Industry: Technology Job Function: Engineering Skills: Senior data annotation, data labeling, RLHF, LLM evaluation, QA evaluation, prompt evaluation, annotation guidelines compliance, training data quality, named entity recognition, computer vision annotation, content safety labeling Salary Currency: USD Salary Min: 63360 Salary Max: 126720 Pay Period: YEAR You will lead complex annotation workflows that turn raw text, image, and multi-modal inputs into model-ready training signals. This includes RLHF preference labeling, rubric-based QA evaluation, and prompt evaluation for large language model evaluation. You will partner with operations and engineering to refine annotation guidelines compliance, reduce ambiguity, and raise training data quality across programs. Key outcomes include measurable model performance improvement through consistent labels, calibrated judgments, and accurate edge-case handling. You will contribute to NLP tasks (named entity recognition, classification, summarization, conversation quality), computer vision annotation (bounding boxes, polygons, attributes), and content safety labeling (policy-driven categorization). You will also help root-cause quality issues, audit sampling plans, and mentor annotators to maintain reliability at scale.

What You Will Do

You will: - Execute senior-level data labeling and review for text, image, and multi-modal datasets - Perform RLHF ranking, preference labeling, and rationale writing aligned to evaluation rubrics - Run QA evaluation cycles: spot checks, double-pass review, adjudication, and error taxonomy - Perform prompt evaluation for helpfulness, correctness, safety, and instruction-following - Maintain annotation guidelines compliance; propose clarifications and decision rules - Track training data quality metrics (accuracy, agreement, coverage) and drive improvements - Support dataset governance: versioning notes, issue logs, and escalation of ambiguous cases - Collaborate with cross-functional partners supporting LLM training pipelines (ops, QA, engineering)

Required Qualifications

You have: - Professional experience in data annotation, data labeling, or QA evaluation for AI/ML - Demonstrated capability with rubric-based judgment tasks and edge-case reasoning - Familiarity with large language model evaluation and RLHF-style workflows - Strong written communication for rationales, guideline updates, and audit notes - High attention to detail and consistent application of policy and annotation guidelines - Comfort working with structured tooling, queues, and quality sampling processes

Preferred Qualifications

Nice to have: - Experience with named entity recognition, taxonomy design, or ontology-driven labeling - Exposure to computer vision annotation (boxes, polygons, keypoints) and quality review - Experience with content safety labeling and policy interpretation for sensitive content - Experience measuring inter-annotator agreement and improving reviewer calibration - Familiarity with prompt evaluation patterns and failure modes in instruction-following models

Tools and Workflows

You may work with: - Annotation platforms and review tooling for text, vision, and multi-modal tasks - QA evaluation workflows: gold sets, consensus review, adjudication, and audits - Dataset documentation: labeling specs, edge-case catalogs, and decision logs - Performance dashboards tied to training data quality and model performance improvement

Compensation and Benefits

This role is FULL_TIME and Remote. Compensation details: - Salary Currency: USD - Salary Min: 63360 - Salary Max: 126720 - Pay Period: YEAR Benefits may include health coverage options, paid time off, and remote-work support depending on engagement terms and location.

How to Apply on Rex.zone

Apply through Rex.zone with: - A resume highlighting senior data annotation, RLHF, QA evaluation, and guideline compliance work - Examples of projects involving training data quality improvement, LLM evaluation, or content safety labeling - Availability for Remote full-time work and any relevant domain strengths (NLP, computer vision, safety) Rex.zone supports candidates seeking remote, contract, freelance, full-time, entry-level, and senior roles across AI labs, tech startups, BPOs, and annotation vendors. This posting targets senior data annotation jobs in Dublin aligned to global remote teams.

Frequently Asked Questions

  • Q: What does a senior data annotation specialist do?

    A senior data annotation specialist leads complex data labeling and QA evaluation work that produces reliable training data for AI/ML. The role often includes RLHF preference ranking, prompt evaluation, guideline refinement, audits, and mentoring to improve training data quality and downstream model performance.

  • Q: Is this role remote and full-time?

    Yes. The Remote Type is Remote and the Employment Type is FULL_TIME. Work is performed remotely with structured queues, review cycles, and quality metrics.

  • Q: What AI domains are covered in these senior data annotation jobs?

    Common domains include large language model evaluation, NLP (named entity recognition, classification, summarization), computer vision annotation (boxes, polygons, attributes), and content safety labeling aligned to policy and safety rubrics.

  • Q: How is quality measured in senior data annotation work?

    Quality is typically measured via training data quality metrics such as accuracy against gold sets, inter-annotator agreement, reviewer calibration, audit pass rates, and consistency with annotation guidelines compliance. These metrics connect to model performance improvement in evaluation benchmarks and production behavior.

  • Q: What is RLHF and why is it important here?

    RLHF (Reinforcement Learning from Human Feedback) uses human preference judgments and rankings to train or align model behavior. Senior annotators help ensure RLHF labels are consistent, well-rationalized, and aligned with rubrics so they can be used safely in LLM training pipelines.

  • Q: Do I need engineering skills for a data annotation role listed under Job Function: Engineering?

    You do not need to be a software engineer, but you should be comfortable with structured workflows, tooling, and careful reasoning. The Job Function reflects close collaboration with engineering and evaluation teams and the impact of labeling decisions on model training and evaluation systems.

  • Q: Where do I apply and what should I include?

    Apply via Rex.zone. Include a resume emphasizing senior data annotation, data labeling, QA evaluation, RLHF or prompt evaluation experience, and examples of guideline compliance, audits, and training data quality improvements.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.