Senior Data Labeling Jobs Madrid

Senior data labeling jobs Madrid: Lead end-to-end data labeling and LLM evaluation workflows that improve training data quality, annotation guidelines compliance, and model performance improvement for AI/ML systems on Rex.zone. You will oversee RLHF evaluation, prompt evaluation, named entity recognition, computer vision annotation, and content safety labeling across large language model evaluation pipelines, partnering with engineering and QA to drive measurable dataset accuracy, inter-annotator agreement, and production-ready training data for AI labs, tech startups, and annotation vendors.

LinkedIn Job Metadata — Senior Data Labeling Jobs Madrid

About the Role

As a Senior Data Labeling professional, you will lead labeling operations for AI/ML training workflows, ensuring training data quality across NLP and computer vision datasets. You will translate product and research goals into annotation guidelines, run calibration sessions, and manage QA evaluation to reduce label noise and improve downstream model metrics. You will support RLHF evaluation and prompt evaluation for large language model evaluation, coordinate edge-case handling, and enforce content safety labeling policies to ensure safe and reliable model behavior.

What You Will Do

Own annotation guidelines compliance, including taxonomy design, labeling instructions, and rubric creation for NLP and CV tasks. Lead RLHF evaluation and prompt evaluation queues, including preference ranking, response grading, and safety checks for LLM training pipelines. Run QA evaluation programs: gold set creation, blind review, inter-annotator agreement tracking, and root-cause analysis of disagreements. Partner with engineering to operationalize labeling tooling, audit trails, and dataset versioning for repeatable production workflows. Triage edge cases (ambiguity, multilingual content, policy conflicts) and update guidelines to improve consistency and throughput. Report weekly on training data quality, defect rates, and model performance improvement signals linked to labeling changes.

Required Qualifications

Experience leading data labeling programs for AI/ML, with demonstrated ownership of training data quality and QA evaluation. Strong familiarity with RLHF evaluation, prompt evaluation, and large language model evaluation concepts. Hands-on experience with named entity recognition and/or computer vision annotation, including guideline development and reviewer calibration. Ability to design quality metrics (accuracy, consistency, coverage) and implement annotation guidelines compliance processes. Clear written communication for policy-driven labeling, including content safety labeling and escalation protocols.

Preferred Qualifications

Experience supporting multilingual datasets and locale-specific annotation standards, including Spanish-language evaluation. Familiarity with dataset governance practices: versioning, sampling strategies, and error taxonomy design. Prior work with AI labs, tech startups, BPOs, or annotation vendors delivering production datasets. Exposure to automated QA heuristics, model-assisted labeling, and active learning workflows.

Work Setup

Remote role marked Remote, full-time, supporting distributed labeling and evaluation operations. You will collaborate with cross-functional stakeholders across engineering, research, and quality teams, using structured queues and documented rubrics to manage throughput and accuracy. This job is aligned with senior data labeling jobs Madrid search intent while remaining remote and available through Rex.zone.

How to Apply on Rex.zone

Apply through Rex.zone by submitting your resume and a short overview of the labeling programs you have led, including QA evaluation methods, guideline examples, and any RLHF evaluation or prompt evaluation work. If applicable, include brief summaries of named entity recognition, computer vision annotation, and content safety labeling projects, along with metrics that demonstrate training data quality and model performance improvement.

Frequently Asked Questions

Q: Are these senior data labeling jobs Madrid remote or on-site?
These roles are explicitly marked Remote and remain Remote. The page targets the senior data labeling jobs Madrid keyword while offering remote work through Rex.zone.
Q: What kinds of tasks are included in senior data labeling work?
Typical tasks include data labeling, QA evaluation, annotation guideline creation, calibration, named entity recognition, computer vision annotation, content safety labeling, and LLM evaluation such as RLHF evaluation and prompt evaluation.
Q: What does RLHF evaluation involve in this role?
RLHF evaluation commonly includes ranking model outputs, scoring responses against rubrics, identifying failure modes, and providing structured preference data that improves large language model evaluation and training outcomes.
Q: How is quality measured in training data quality programs?
Quality is measured through annotation guidelines compliance, gold set performance, inter-annotator agreement, blind review pass rates, error taxonomy trends, and the impact of labeling changes on model performance improvement.
Q: Is this job full-time and what is the salary range?
Yes, Employment Type is FULL_TIME. The salary range is USD 63360 to USD 126720 per year, as listed in the job metadata.

230+Domains Covered

120K+PhD, Specialist, Experts Onboarded

50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.