Senior Data Annotation Jobs in Austin

Senior data annotation jobs in Austin at Rex.zone focus on building and evaluating training data for AI/ML systems, including LLM training pipelines, RLHF, data labeling, and QA evaluation. In this full-time remote role, you will apply annotation guidelines compliance, prompt evaluation, and content safety labeling to improve model performance improvement across NLP and computer vision workflows. You will partner with engineers and AI teams to deliver training data quality, accurate named entity recognition, and consistent rubric-based scoring that supports reliable large language model evaluation and safer deployments.

Job Image

Job Heading: Senior Data Annotation (Austin)

Title: Senior Data Annotation (Austin) Date: 25-02-2026 Company: Rex.zone Country: US Remote Type: Remote Employment Type: FULL_TIME Experience Level: Mid-Senior Industry: Technology Job Function: Engineering Skills: Senior data annotation, data labeling, RLHF, QA evaluation, prompt evaluation, named entity recognition, computer vision annotation, content safety labeling, LLM training pipelines, annotation guidelines compliance, training data quality, large language model evaluation Salary Currency: USD Salary Min: 63360 Salary Max: 126720 Pay Period: YEAR

About the Role

You will lead end-to-end data annotation and evaluation workflows used in AI/ML training, including RLHF preference labeling, prompt-response evaluation, and rubric-based scoring for large language model evaluation. Your work will directly impact training data quality, model performance improvement, and safety outcomes by producing consistent labels, QA reviews, and clear feedback on annotation guidelines compliance.

What You Will Do

Core responsibilities include: - Execute and review data labeling tasks for NLP and computer vision datasets - Perform RLHF evaluations (preference ranking, pairwise comparisons, and structured scoring) - Conduct QA evaluation audits for accuracy, consistency, and policy alignment - Perform prompt evaluation on model outputs for helpfulness, correctness, and safety - Apply named entity recognition tagging and taxonomy mapping where required - Support content safety labeling (policy-based classification, sensitive content handling) - Write and refine annotation guidelines, edge-case notes, and decision logs - Track dataset health metrics (agreement rate, defect density, rework rate) and propose fixes - Collaborate with engineering on tooling feedback for labeling UI, workflow automation, and sampling strategy

Required Qualifications

We are looking for candidates with: - Experience in data annotation, data labeling, or QA evaluation for AI/ML systems - Familiarity with RLHF concepts and large language model evaluation methodologies - Ability to follow and improve annotation guidelines compliance under ambiguity - Strong attention to detail and comfort making consistent judgment calls using rubrics - Experience with NLP tasks such as classification, summarization evaluation, and named entity recognition - Professional communication skills for documenting edge cases and explaining labeling decisions

Preferred Qualifications

Nice to have: - Experience with computer vision annotation (bounding boxes, polygons, segmentation masks) - Prior work on content safety labeling, trust & safety, or policy-driven evaluation - Exposure to dataset sampling, inter-annotator agreement, and evaluation design - Experience with annotation platforms, internal tooling, or vendor/queue management - Comfort working with product and engineering partners in an AI training pipeline context

Work Model and Location

This is a full-time Remote role for candidates in the US, aligned to Austin hiring intent. You will collaborate asynchronously with distributed teams and participate in scheduled calibration sessions to maintain training data quality and rubric consistency.

Compensation

Salary Range: 63360 to 126720 USD per year, based on experience level, evaluation scope, and QA ownership. This role remains explicitly Remote.

How to Apply

Apply through Rex.zone and include: - A short summary of your data annotation, RLHF, and QA evaluation experience - Examples of annotation guidelines you have followed or improved (describe, no confidential data) - The task domains you have worked in (NLP, computer vision annotation, content safety labeling) - Your availability for full-time Remote work in the US

Frequently Asked Questions

  • Q: Are these senior data annotation jobs in Austin remote or onsite?

    These roles are explicitly marked Remote. The Austin keyword aligns the job with the Austin market, but day-to-day work is performed remotely within the US.

  • Q: What does “senior” mean for data annotation in this role?

    Senior scope includes higher ownership of training data quality, QA evaluation, calibration leadership, edge-case documentation, and consistent rubric application across RLHF and prompt evaluation workflows.

  • Q: What kinds of tasks are included?

    Tasks include data labeling, named entity recognition, prompt evaluation, RLHF preference judgments, QA evaluation audits, and (depending on project) computer vision annotation and content safety labeling.

  • Q: What is RLHF and why is it part of data annotation?

    RLHF (Reinforcement Learning from Human Feedback) uses human preference labels and structured evaluations to guide model behavior. In practice, it involves ranking responses, scoring against rubrics, and providing feedback signals used in LLM training pipelines.

  • Q: Do I need engineering experience to apply if Job Function is Engineering?

    You do not need to be a software engineer, but you should be comfortable with structured processes, quality measurement, and collaborating with engineering teams on labeling tools, workflow improvements, and dataset requirements.

  • Q: Which domains does the job cover: NLP, computer vision, or content safety?

    The role can include NLP and large language model evaluation as the core, with optional coverage of computer vision annotation and content safety labeling depending on the dataset and client needs.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.