Remote Data Annotator Jobs in New York

Remote Data Annotator jobs in New York on Rex.zone focus on data labeling and human-in-the-loop evaluation for AI/ML systems. You will annotate text, image, and multimodal datasets to improve training data quality, support RLHF feedback loops, and run QA evaluation that drives model performance improvement for large language models. Typical workflows include prompt evaluation, named entity recognition, computer vision annotation, and content safety labeling under strict annotation guidelines compliance. Explore full-time, contract, freelance, entry-level, and senior remote roles supporting AI labs, tech startups, BPOs, and annotation vendors—then apply through Rex.zone.

Remote Data Annotator Jobs in New York — LinkedIn Job Metadata

About the Role

As a Remote Data Annotator supporting New York-based programs, you will label and evaluate datasets used to train and validate machine learning models. Your work directly impacts training data quality, annotation guidelines compliance, and model performance improvement across NLP, computer vision, and LLM workflows.

What You Will Do

You will produce high-accuracy labels for text, images, and multimodal inputs; perform RLHF-style preference ranking and prompt evaluation; complete QA evaluation checks, audits, and consistency reviews; apply named entity recognition and taxonomy tagging; conduct content safety labeling for policy and harm categories; document edge cases and guideline gaps to improve labeling instructions; and collaborate asynchronously with reviewers, QA, and data operations teams to meet throughput and quality targets.

Core Workflows You May Support

Common assignments include large language model evaluation, prompt-response grading, training data curation, classification and ranking tasks, entity extraction, sentiment and intent labeling, computer vision annotation (bounding boxes, polygons, keypoints), OCR correction, conversation quality scoring, and safety policy enforcement for sensitive content.

Required Qualifications

You have professional experience in data annotation or data labeling, strong written English comprehension, and a track record of following detailed guidelines with high precision. You are comfortable with repetitive, detail-oriented work, can manage ambiguity by escalating edge cases, and can maintain consistent decisions across long labeling sessions.

Preferred Qualifications

Experience with RLHF, LLM training pipelines, prompt evaluation, or QA evaluation is preferred. Familiarity with named entity recognition, content safety labeling, and computer vision annotation tools is a plus. Prior work with AI labs, tech startups, BPOs, or annotation vendors is helpful.

Tools and Quality Expectations

You will work in web-based annotation platforms and internal QA dashboards. Success is measured through training data quality metrics such as accuracy, inter-annotator agreement, calibration against gold sets, throughput, and adherence to annotation guidelines compliance.

Remote Work Notes (New York)

These roles are Remote and can be performed from New York, with collaboration across distributed teams. Some projects may require availability windows aligned with US business hours or structured QA review cycles.

Why Rex.zone

Rex.zone aggregates remote data annotator jobs across NLP, computer vision, content safety, and LLM evaluation programs. You can compare full-time, contract, and freelance opportunities and apply to roles aligned with your domain interests and experience level.

How to Apply

Apply via Rex.zone using a resume that highlights annotation accuracy, guideline-driven decision making, QA evaluation experience, and any RLHF or prompt evaluation work. Include examples of domain expertise (NLP, CV, or safety) and your approach to handling edge cases.

Frequently Asked Questions

Q: What does a Remote Data Annotator do in AI/ML workflows?
A Remote Data Annotator creates labeled datasets used for model training and evaluation. Work commonly includes data labeling for classification, named entity recognition, prompt evaluation for LLMs, RLHF preference ranking, QA evaluation, and content safety labeling to improve model reliability and reduce harmful outputs.
Q: Are these Remote Data Annotator jobs only for candidates in New York?
This page targets Remote Data Annotator jobs in New York, meaning the work is Remote and aligned to New York-based hiring needs or candidate pools. Exact location eligibility can vary by client, compliance needs, or time zone coverage.
Q: What skills should match a Remote Data Annotator job in New York?
Skills that commonly match include data annotation, data labeling, RLHF, QA evaluation, prompt evaluation, named entity recognition, computer vision annotation, content safety labeling, and familiarity with LLM training pipelines and annotation guidelines compliance.
Q: Do remote roles include full-time and contract options?
Yes. Remote data annotation hiring commonly spans full-time, contract, and freelance roles, and may include entry-level through senior positions depending on project complexity, QA expectations, and domain requirements such as NLP or computer vision.
Q: What is RLHF and how does it relate to data annotation?
RLHF (Reinforcement Learning from Human Feedback) uses human judgments—such as preference rankings or quality ratings—to train reward models and improve LLM behavior. Data annotators contribute by grading responses, ranking outputs, identifying safety issues, and ensuring consistent evaluations under clear guidelines.
Q: What types of annotation tasks are common for computer vision and NLP?
For computer vision annotation, common tasks include bounding boxes, polygons, keypoints, segmentation masks, and image-level classification. For NLP, tasks include named entity recognition, intent labeling, sentiment classification, summarization evaluation, and prompt-response scoring for large language model evaluation.
Q: How is quality measured in data labeling work?
Quality is often measured by accuracy against gold data, inter-annotator agreement, calibration scores, error rates by category, adherence to annotation guidelines compliance, and consistency over time—plus QA evaluation audits and reviewer feedback.
Q: Where can I find and apply for Remote Data Annotator jobs in New York?
You can browse and apply through Rex.zone, which lists Remote Data Annotator roles across LLM evaluation, NLP labeling, computer vision annotation, and content safety labeling for teams such as AI labs, tech startups, BPOs, and annotation vendors.

230+Domains Covered

120K+PhD, Specialist, Experts Onboarded

50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.