Senior Data Annotation Jobs in Rome

Senior data annotation is a Mid-Senior, full-time remote role focused on training data quality for AI/ML systems—labeling, RLHF, prompt evaluation, and QA evaluation that directly improve model performance. At Rex.zone, you will help build reliable LLM training pipelines by applying annotation guidelines compliance, auditing datasets, and refining instructions for human feedback. This job connects real production workflows across NLP, computer vision annotation, named entity recognition, and content safety labeling, supporting AI labs, tech startups, and annotation vendors. Explore and apply through Rex.zone to work remotely while delivering measurable improvements in large language model evaluation and data labeling operations.

Job Heading: Senior Data Annotation Jobs in Rome

About the Role

You will lead high-impact data labeling and evaluation work for AI/ML systems, ensuring training data quality and consistent annotation guidelines compliance. Your day-to-day includes RLHF preference labeling, prompt evaluation, QA evaluation, dataset audits, and feedback loops that drive model performance improvement. You will collaborate with project leads and tooling teams to resolve edge cases, calibrate annotators, and improve rubrics for large language model evaluation across NLP and computer vision annotation tasks.

What You Will Do

Own and execute complex annotation tasks across NLP and CV, including named entity recognition, classification, ranking, and span labeling. Perform RLHF workflows (preference comparisons, rationale checks, policy-aligned scoring) and prompt evaluation for LLM training pipelines. Run QA evaluation using sampling plans, inter-annotator agreement checks, and targeted error analysis. Maintain annotation guidelines compliance by documenting decisions, escalating ambiguous cases, and updating rubrics. Partner with stakeholders across AI labs, tech startups, BPO teams, and annotation vendors to deliver production-ready datasets on schedule.

Key Workflows and Task Types

Large language model evaluation: instruction-following checks, factuality/error tagging, safety policy labeling, and response ranking. RLHF: pairwise preference labeling, rubric-based scoring, and calibration sessions to reduce variance. Data labeling: taxonomy design support, label normalization, and structured feedback for guideline improvements. Computer vision annotation: bounding boxes, polygons, keypoints, and attribute tagging where required. Content safety labeling: policy interpretation, severity grading, and contextual risk tagging for harmful content categories.

Required Qualifications

Demonstrated experience in data annotation, data labeling, or QA evaluation in production AI/ML environments. Strong understanding of training data quality, guideline-driven labeling, and error taxonomy development. Hands-on familiarity with RLHF or large language model evaluation concepts, including prompt evaluation and preference ranking. High attention to detail, consistent decision-making, and comfort working with ambiguous language/data edge cases. Ability to communicate clearly in written form and maintain precise documentation for annotation guidelines compliance.

Preferred Qualifications

Experience with named entity recognition, information extraction, or other NLP labeling tasks. Background in computer vision annotation workflows and quality auditing. Exposure to content safety labeling, policy-based moderation, or safety evaluation for LLMs. Experience supporting distributed teams (BPOs or vendors) with calibration and QA evaluation routines. Familiarity with dataset versioning, sampling strategies, and feedback loops for model performance improvement.

Tools and Working Style

You will work in annotation platforms and QA dashboards, applying structured rubrics and maintaining consistent labeling decisions. Expect iterative guideline revisions, calibration rounds, and evaluation cycles aligned with LLM training pipelines. Remote work requires strong time management, clear written updates, and steady throughput without compromising training data quality.

What Success Looks Like

High annotation accuracy with strong inter-annotator agreement and consistent annotation guidelines compliance. Clear QA evaluation reporting, actionable error analysis, and documented edge-case resolutions. Measurable improvements in dataset reliability that contribute to model performance improvement in large language model evaluation and RLHF training. On-time delivery of labeled data and evaluations across NLP, computer vision annotation, and content safety labeling projects.

How to Apply on Rex.zone

Apply through Rex.zone with a resume highlighting senior data annotation experience, RLHF or prompt evaluation exposure, and examples of QA evaluation or training data quality initiatives. Include task domains you have worked in (NLP, computer vision annotation, content safety labeling) and the annotation guidelines compliance practices you follow in production workflows.

Frequently Asked Questions

Q: Is this role remote and full-time?
Yes. The position is Remote and FULL_TIME, aligned with the metadata shown under the job heading.
Q: What does “Senior data annotation” mean in an AI/ML context?
It typically means owning complex data labeling and evaluation tasks, driving training data quality, leading QA evaluation routines, and contributing to guideline design and calibration for LLM training pipelines, including RLHF and prompt evaluation.
Q: Which domains are covered in these senior data annotation jobs?
Common domains include NLP (e.g., named entity recognition), large language model evaluation (e.g., response ranking and rubric scoring), computer vision annotation (e.g., boxes/polygons), and content safety labeling (policy-based tagging).
Q: Do I need RLHF experience to apply?
RLHF experience is strongly relevant for senior roles, but candidates with strong data labeling, QA evaluation, and guideline-driven annotation experience can still be considered if they can ramp quickly on preference labeling and prompt evaluation.
Q: What skills are most important for ranking performance in this role?
Training data quality focus, annotation guidelines compliance, strong QA evaluation habits, comfort with ambiguity, and the ability to perform large language model evaluation tasks such as prompt evaluation and RLHF preference ranking.
Q: What kinds of employers use Rex.zone postings for data annotation work?
Work may support AI labs, tech startups, BPOs, and annotation vendors—often contributing labeled datasets and evaluation results used in production AI/ML systems.

230+Domains Covered

120K+PhD, Specialist, Experts Onboarded

50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.