Remote Data Annotation Jobs in New York

Remote data annotation jobs in New York are full-time roles focused on creating high-quality labeled datasets for AI/ML systems. At Rex.zone (Rex.zone), you will label and evaluate text, images, and conversational outputs used in LLM training pipelines, RLHF, prompt evaluation, and training data quality programs. Your work supports annotation guidelines compliance, QA evaluation, and model performance improvement across NLP, computer vision annotation, named entity recognition, and content safety labeling. Explore Rex.zone to apply to remote, contract, freelance, entry-level, and senior opportunities aligned to real production workflows used by AI labs, tech startups, BPOs, and annotation vendors.

Job Image

Remote Data Annotation Jobs in New York

Keyword: Remote Data Annotation Jobs in New York | Job Title: Remote Data Annotation Specialist in New York | Date Posted: 25-02-2026 | Company: Rex.zone | Country: US | Workplace Type: Remote | Employment Type: FULL_TIME | Experience Level: Mid-Senior | Industry: Technology | Job Function: Engineering | Skills: data annotation, data labeling, RLHF, QA evaluation, prompt evaluation, named entity recognition, computer vision annotation, content safety labeling, LLM evaluation, annotation guidelines compliance | Salary Currency: USD | Salary Min: 63360 | Salary Max: 126720 | Pay Period: YEAR

About the Role

You will perform remote data annotation and evaluation tasks that directly improve AI training pipelines. Work includes labeling text and image data, reviewing model outputs, ranking responses for RLHF, and completing QA evaluation checks to ensure training data quality and annotation guidelines compliance. Projects may span NLP datasets (classification, named entity recognition, retrieval relevance), conversational AI (prompt evaluation and response grading), computer vision annotation (bounding boxes, polygons, segmentation), and content safety labeling for policy adherence.

What You Will Do

Core responsibilities include: (1) Apply labeling taxonomies and annotation guidelines to text, image, and multimodal data, (2) Perform RLHF-style preference ranking and prompt evaluation for large language model evaluation, (3) Conduct QA evaluation, audits, and adjudication to resolve edge cases and improve inter-annotator agreement, (4) Document rationales and error patterns that drive model performance improvement, (5) Collaborate with project leads to refine rubrics, update policies, and calibrate labeling decisions for consistent outcomes.

Projects and Domains You May Support

You may work on: NLP classification, named entity recognition, sentiment and intent labeling, retrieval relevance judgments, prompt evaluation for assistant behavior, RLHF ranking and critique tasks, content safety labeling (harassment, hate, self-harm, sexual content, violence), computer vision annotation (object detection, keypoints, segmentation), and multilingual evaluation. Employer environments may include AI labs, tech startups, BPOs, and annotation vendors delivering production-grade datasets.

Requirements

Requirements include: (1) Professional experience with data annotation, data labeling, evaluation, or QA workflows, (2) Strong written communication and ability to justify decisions using rubrics, (3) Comfort working with annotation tools, spreadsheets, and web-based labeling platforms, (4) Ability to maintain high accuracy under detailed policies and changing guidelines, (5) Reliable remote setup and ability to meet full-time productivity and quality targets.

Preferred Qualifications

Preferred: experience with RLHF or large language model evaluation, familiarity with prompt evaluation and rubric design, exposure to NLP tasks (NER, text classification) or computer vision annotation, understanding of content safety labeling policies, and prior QA evaluation work (audits, sampling, adjudication, inter-annotator agreement).

Quality Standards and How Success Is Measured

Success is measured by training data quality, annotation guidelines compliance, throughput aligned to project targets, and consistency across edge cases. Metrics may include accuracy against gold data, QA pass rates, inter-annotator agreement, rework percentage, and calibration performance on evaluation sets. You will be expected to follow secure handling practices and maintain clear rationales that support dataset reliability.

Compensation and Benefits

Compensation for this full-time remote role is budgeted between 63360 and 126720 USD per year, depending on skills alignment and project complexity. Benefits and scheduling details vary by project assignment through Rex.zone, and may include stable full-time allocation, clear performance feedback loops, and opportunities to move into QA, adjudication, or guideline development tracks.

How to Apply on Rex.zone

Apply through Rex.zone by completing your profile, confirming your availability for full-time remote work, and selecting data annotation projects aligned to your strengths (NLP, computer vision annotation, RLHF, prompt evaluation, content safety labeling). Keep examples of prior labeling or evaluation work ready, and be prepared for calibration tasks that validate annotation guidelines compliance and QA evaluation readiness.

Frequently Asked Questions

  • Q: Are these remote data annotation jobs limited to New York City?

    No. The keyword targets New York, but the roles are Remote and can typically be performed from anywhere in New York State (and sometimes broader US eligibility depending on project requirements). Always follow the Rex.zone posting details for location eligibility.

  • Q: What kinds of data will I annotate?

    Typical tasks include text data labeling (classification, named entity recognition), LLM evaluation and prompt evaluation, RLHF preference ranking, content safety labeling, and computer vision annotation such as bounding boxes or segmentation.

  • Q: What is RLHF and why is it part of data annotation?

    RLHF (Reinforcement Learning from Human Feedback) uses human judgments—such as ranking model responses—to train preference models and improve assistant behavior. It is a common extension of data labeling and QA evaluation for LLM training pipelines.

  • Q: Is this full-time or can it be contract/freelance?

    This specific role is listed as FULL_TIME and Remote. Rex.zone may also host contract, freelance, entry-level, and senior variations, depending on project needs and availability.

  • Q: How is quality checked?

    Quality is typically verified through gold-standard test items, QA evaluation sampling, audits, adjudication of disagreements, and calibration exercises designed to ensure annotation guidelines compliance and training data quality.

  • Q: Do I need a technical background?

    A strong technical background is not always required, but comfort with detailed rubrics, consistent decision-making, and basic tooling is essential. Experience with evaluation workflows, content policies, or structured labeling improves fit for Mid-Senior projects.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.