Key Responsibilities
Produce high-quality annotations for NLP (NER, sentiment, classification), LLM evaluation (pairwise ranking, preference labeling, prompt evaluation), computer vision (bounding boxes, polygons, keypoints), ASR/audio tagging, and content safety labeling. Follow annotation guidelines, taxonomy definitions, and edge-case policies to ensure training data quality. Perform QA evaluation with consensus checks, spot audits, and adjudication. Provide structured feedback to improve guidelines and enable model performance improvement. Support RLHF and alignment tasks by scoring outputs and documenting reasoning traces.



