Key Responsibilities
Execute text, image, audio, and video annotation aligned with detailed guidelines. Perform RLHF tasks including pairwise preference ranking, rubric-based scoring, and red-teaming for safety. Conduct prompt evaluation and LLM output grading for factuality, helpfulness, and harmlessness. Complete NER tagging, intent classification, and sentiment labeling for NLP datasets. Carry out computer vision annotation such as bounding boxes, segmentation, and keypoints. Label content safety categories (toxicity, hate, self-harm) with high precision. Follow QA workflows, resolve edge cases, and provide feedback to improve guidelines. Use inter-annotator agreement metrics to maintain consistency and escalate ambiguities.



