Remote Data Labeling Jobs in Madrid

Remote data labeling jobs in Madrid at Rex.zone focus on training-data creation and quality control for AI/ML systems. In this full-time remote role, you will label and evaluate text, images, audio, and conversations used in large language model evaluation and computer vision annotation workflows. Your work improves training data quality, annotation guidelines compliance, and model performance improvement through RLHF-style preference judgments, prompt evaluation, content safety labeling, and QA evaluation. You will follow clear rubrics, document edge cases, and partner with data operations to ship reliable datasets used by AI labs, tech startups, and annotation vendors.

Remote Data Labeling Jobs in Madrid

About the Role

You will produce and review labeled datasets that power AI training pipelines. Tasks include text classification, intent labeling, named entity recognition (NER), summarization and prompt-response scoring, RLHF preference ranking, and quality assurance checks. You will also support computer vision annotation (bounding boxes, polygons, keypoints) and content safety labeling to ensure policy-aligned outputs.

What You’ll Work On

You will execute labeling workflows using annotation guidelines, handle ambiguous cases with consistent rationale, and perform QA evaluation to reduce disagreement and rework. You will run prompt evaluation for LLM outputs, compare responses for helpfulness/harmlessness/honesty, and create clear feedback that improves rubric reliability and large language model evaluation quality.

Key Responsibilities

You will: (1) label and review training data across NLP and computer vision annotation tasks, (2) apply RLHF-style ranking and pairwise preference judgments, (3) follow annotation guidelines compliance standards and document edge cases, (4) conduct QA evaluation using checklists and sampling plans, (5) track error patterns and propose rubric improvements that drive model performance improvement, and (6) collaborate asynchronously with data ops, QA, and project leads in a remote environment.

Required Qualifications

Mid-Senior experience in data labeling or data annotation operations, strong attention to detail, and comfort working with structured rubrics. Familiarity with NLP concepts (classification, NER, sentiment), LLM training pipelines, and prompt evaluation. Ability to explain decisions clearly, resolve ambiguity, and maintain consistent quality under production timelines.

Preferred Qualifications

Experience with RLHF, large language model evaluation, content safety labeling, and audit-ready QA workflows. Exposure to computer vision annotation formats (bbox, polygon, keypoints) and inter-annotator agreement methods. Prior work with AI labs, tech startups, BPOs, or annotation vendors is helpful.

Quality Standards and Workflow

You will work within measurable accuracy targets, adhere to training data quality requirements, and participate in calibration sessions. Expect iterative guideline updates, spot-checking, gold tasks, and structured feedback loops aimed at reducing variance, improving annotation consistency, and ensuring reliable datasets for downstream training and evaluation.

Employment Details

Remote Type: Remote. Employment Type: FULL_TIME. This role is designed for sustained delivery across ongoing projects, with opportunities to support multiple domains such as NLP, computer vision, and content safety. Work is fully remote while remaining aligned to the Madrid job-search intent for remote opportunities.

How to Apply

Apply through Rex.zone to be considered for remote data labeling jobs in Madrid. Include brief notes on your annotation experience (NLP, RLHF, QA evaluation, computer vision annotation, or content safety labeling), examples of guideline-driven work, and your availability for full-time remote projects.

Frequently Asked Questions

Q: Are these remote data labeling jobs in Madrid fully remote?
Yes. Remote Type is Remote, and the workflow is designed for asynchronous collaboration, online annotation tools, and remote QA evaluation processes.
Q: What kinds of data will I label?
You may label text for NLP (classification, NER, summarization), evaluate LLM responses through prompt evaluation and RLHF-style ranking, and support computer vision annotation and content safety labeling depending on project needs.
Q: Is this a full-time role or contract/freelance?
This posting is FULL_TIME. Rex.zone may also host contract or freelance roles on other pages, but this role is explicitly full-time and remote.
Q: What experience level is required?
The Experience Level is Mid-Senior. You should be comfortable interpreting rubrics, maintaining annotation consistency, and contributing to training data quality improvements.
Q: What skills are most important for success?
Core skills include data labeling and data annotation, annotation guidelines compliance, QA evaluation, prompt evaluation, RLHF, named entity recognition, computer vision annotation, content safety labeling, and familiarity with LLM training pipelines.
Q: How does this work connect to real AI/ML training workflows?
Your labels and evaluations become training and validation datasets used to fine-tune models, run large language model evaluation, measure model performance improvement, and ensure safe, policy-compliant outputs.

230+Domains Covered

120K+PhD, Specialist, Experts Onboarded

50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.