Entry Level AI Data Annotation Specialist (United States, Remote)
Title: Entry Level AI Data Annotation Specialist (United States, Remote) Date: 25-02-2026 Company: Rexzone Country: US Remote Type: Remote Employment Type: FULL_TIME Experience Level: Mid-Senior Industry: Technology Job Function: Engineering Skills: Data Labeling, RLHF, Prompt Evaluation, QA Evaluation, Training Data Quality, Annotation Guidelines, Named Entity Recognition, Content Safety Labeling, LLM Evaluation, Computer Vision Annotation Salary Currency: USD Salary Min: 63360 Salary Max: 126720 Pay Period: YEAR You will label and review text, image, and conversation datasets used in LLM training pipelines and computer vision training. You will apply annotation guidelines compliance checks, resolve edge cases, and document labeling decisions to improve training data quality. Responsibilities: • Perform data labeling across NLP and CV tasks (classification, ranking, segmentation, and NER) • Execute QA evaluation and inter-annotator agreement checks to reduce label noise • Support RLHF workflows by ranking responses and applying preference rubrics • Run prompt evaluation to assess instruction-following, factuality, and safety behaviors • Flag content safety labeling issues (policy, toxicity, sensitive content) with clear rationale • Track model performance improvement signals by auditing recurring model failure patterns Qualifications: • Familiarity with large language model evaluation concepts (helpfulness, harmlessness, honesty) • Comfort following detailed rubrics and making consistent judgments at scale • Strong written communication for edge-case notes and escalation summaries • Basic understanding of NLP and computer vision annotation task types How Success Is Measured: • Training data quality improvements (accuracy, consistency, and reduced rework) • Annotation throughput with high guideline adherence • Clear documentation that enables downstream engineering and model iteration



