[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"job-remote-ai-data-annotation-jobs-new-york":3},{"Ques":4,"Slug":31,"Header":32,"job_category":62},{"title":5,"content":6},"Frequently Asked Questions",[7,10,13,16,19,22,25,28],{"A":8,"Q":9},"Yes. The Remote Type is Remote, and the work is completed online using labeling and evaluation tools.","Are these roles fully remote?",{"A":11,"Q":12},"Common tasks include data labeling for NLP and computer vision, named entity recognition, content safety labeling, prompt evaluation, RLHF preference ranking, and QA evaluation for training data quality.","What kinds of annotation tasks are included?",{"A":14,"Q":15},"This posting targets Mid-Senior experience, emphasizing consistent quality, calibration, and ownership of guideline adherence.","Is this job entry-level or senior?",{"A":17,"Q":18},"RLHF work typically involves ranking or comparing model outputs, scoring responses with rubrics, writing rationales, and helping align model behavior for large language model evaluation.","What does RLHF mean in day-to-day work?",{"A":20,"Q":21},"Not always, but experience with computer vision annotation (bounding boxes, polygons, segmentation) is a plus because projects may span multimodal datasets.","Do I need a computer vision background?",{"A":23,"Q":24},"Some projects involve content safety labeling. If assigned, you will follow defined policies and escalation paths to label content consistently and safely.","Will I review sensitive content?",{"A":26,"Q":27},"Success comes from strong attention to detail, annotation guidelines compliance, stable rubric calibration, clear rationale writing, and a focus on training data quality that supports model performance improvement.","What makes candidates successful in AI data annotation?",{"A":29,"Q":30},"Rex.zone is the platform where you explore the role, apply, and complete any assessments or calibration tasks required for production AI\u002FML labeling workflows.","How does Rex.zone fit into the application process?","remote-ai-data-annotation-jobs-new-york",{"desc":33,"title":34,"content":35},"Remote AI Data Annotation Jobs in New York at Rex.zone focus on training data quality for modern AI systems, including data labeling, RLHF evaluation, prompt evaluation, and QA review across NLP and computer vision workflows. You will follow annotation guidelines compliance, apply rubric-based judgments, and help drive model performance improvement for large language model evaluation and content safety labeling. These roles support LLM training pipelines used by AI labs, tech startups, and annotation vendors, with tasks like named entity recognition, image and video annotation, and policy-aligned safety classification. Explore full-time remote opportunities designed for consistent throughput, high accuracy, and measurable quality metrics.","Remote AI Data Annotation Jobs in New York",[36,38,41,44,47,50,53,56,59],{"h2":34,"desc":37},"Date: 25-02-2026 | Company: Rex.zone | Country: US | Remote Type: Remote | Employment Type: FULL_TIME | Experience Level: Mid-Senior | Industry: Technology | Job Function: Engineering | Skills: AI data annotation, data labeling, RLHF, prompt evaluation, QA evaluation, annotation guidelines compliance, named entity recognition, computer vision annotation, content safety labeling, LLM training pipelines | Salary Currency: USD | Salary Min: 63360 | Salary Max: 126720 | Pay Period: YEAR",{"h2":39,"desc":40},"About the Role","You will perform remote AI data annotation supporting production AI\u002FML training workflows. Your work will include data labeling for NLP and computer vision, RLHF-style preference ranking, prompt evaluation for instruction-following behavior, and QA evaluation using calibrated scoring rubrics. You will interpret and apply detailed annotation guidelines, document edge cases, and resolve ambiguity through consistent rationale. You will also contribute to training data quality initiatives by analyzing disagreement patterns, reducing label noise, and proposing improvements that support model performance improvement and reliable offline evaluation.",{"h2":42,"desc":43},"Key Responsibilities","Produce high-accuracy labels for text, image, and multimodal datasets aligned to project schemas. Execute RLHF tasks such as pairwise preference judgments, ranking, and rationale writing when required. Complete prompt evaluation and response grading for helpfulness, correctness, and policy alignment. Perform QA evaluation, including peer review, spot checks, and adjudication with clear decision logs. Apply named entity recognition and taxonomy tagging for NLP datasets, including span selection and attribute assignment. Support computer vision annotation such as bounding boxes, polygons, keypoints, and segmentation masks per spec. Conduct content safety labeling across categories (harassment, self-harm, sexual content, hate, violence) using policy definitions and threshold rules. Track throughput and quality metrics, follow escalation paths, and maintain annotation guidelines compliance across batches.",{"h2":45,"desc":46},"What Success Looks Like","Consistently meets accuracy targets on gold sets and maintains stable inter-annotator agreement. Demonstrates strong calibration on ambiguous cases and applies rubrics with minimal variance. Produces clear rationales that improve reviewer alignment and reduce rework. Identifies recurring dataset issues (schema gaps, unclear instructions, class overlap) and proposes actionable fixes. Delivers reliable throughput while maintaining training data quality and auditability for downstream large language model evaluation.",{"h2":48,"desc":49},"Required Qualifications","Experience in AI data annotation, data labeling, QA evaluation, or related data operations. Demonstrated ability to follow annotation guidelines compliance standards and work within rubric-based evaluation. Familiarity with NLP concepts such as named entity recognition, intent classification, and text categorization. Ability to write concise rationales, handle sensitive content, and maintain confidentiality on proprietary datasets. Strong attention to detail, comfort with repetitive tasks, and ability to meet productivity expectations in a remote environment.",{"h2":51,"desc":52},"Preferred Qualifications","Hands-on experience with RLHF, preference ranking, prompt evaluation, or large language model evaluation. Exposure to computer vision annotation tasks such as bounding boxes, polygons, and segmentation. Experience supporting content safety labeling and policy-driven moderation datasets. Prior work with AI labs, tech startups, BPOs, or annotation vendors in production labeling pipelines. Familiarity with quality programs such as gold data creation, calibration sessions, and disagreement analysis.",{"h2":54,"desc":55},"Tools and Workflow","Work is completed in web-based labeling and evaluation tools with task queues, guideline wikis, and QA workflows. You will use structured rubrics, gold checks, and reviewer feedback loops to improve training data quality. Projects may span NLP classification, named entity recognition, computer vision annotation, RLHF preference ranking, and content safety labeling. Documentation and communication happen asynchronously with clear handoffs, versioned guidelines, and audit-ready decision notes.",{"h2":57,"desc":58},"Work Conditions and Location","Remote role based in the US, aligned with New York job search intent. You must be able to perform focused labeling and evaluation work, handle occasional sensitive content for content safety labeling, and maintain consistent availability during agreed working hours. This is a full-time remote position supporting ongoing AI\u002FML training pipelines.",{"h2":60,"desc":61},"How to Apply on Rex.zone","Apply via Rex.zone by submitting your profile, selecting Remote AI Data Annotation Jobs in New York, and completing any required skills checks. Keep your work samples focused on data labeling, QA evaluation, RLHF-style judgments, or prompt evaluation when applicable. Qualified candidates may receive calibration tasks to confirm annotation guidelines compliance and training data quality standards.","AI Data Operations"]