Remote Data Annotator Jobs in Portland

Remote data annotator jobs in Portland at Rex.zone focus on data labeling, RLHF evaluation, and training data quality for modern AI/ML systems. You will follow annotation guidelines compliance to create high-signal datasets that support LLM training pipelines, prompt evaluation, content safety labeling, named entity recognition, and computer vision annotation. This role connects day-to-day annotation workflows to measurable model performance improvement through QA evaluation, gold set validation, and error analysis. Explore and apply for full-time remote opportunities supporting AI labs, tech startups, annotation vendors, and enterprise teams—without leaving Portland.

Job Image

Job Overview

This Portland-based remote role supports AI training workflows by producing labeled datasets used in NLP, computer vision, and content safety systems. You will annotate text, images, and multimodal items; complete RLHF and preference ranking tasks; and perform prompt evaluation and response quality scoring. Your work directly improves training data quality, reduces model hallucinations through better supervision, and increases model reliability via consistent guideline application.

LinkedIn Job Metadata (Keyword + Job Title)

Title: Remote Data Annotator Jobs in Portland; Date: 25-02-2026; Company: Rex.zone; Country: US; Remote Type: Remote; Employment Type: FULL_TIME; Experience Level: Mid-Senior; Industry: Technology; Job Function: Engineering; Skills: Data Annotation, Data Labeling, RLHF, LLM Evaluation, Prompt Evaluation, QA Evaluation, Named Entity Recognition, Computer Vision Annotation, Content Safety Labeling, Annotation Guidelines; Salary Currency: USD; Salary Min: 63360; Salary Max: 126720; Pay Period: YEAR

Key Responsibilities

You will label and review training data across domains (NLP, CV, and safety), execute RLHF comparison and ranking tasks, and complete rubric-based LLM evaluation. You will apply taxonomy rules for named entity recognition, sentiment, intent, and policy categories; annotate bounding boxes, polygons, and segmentation masks for computer vision annotation; and document edge cases to improve annotation guidelines. You will run QA evaluation checks, participate in calibration sessions, and help maintain training data quality through audits and error analysis.

Required Qualifications

You have experience with structured labeling workflows, careful reading, and consistent decision-making under detailed guidelines. You can interpret rubrics for prompt evaluation and large language model evaluation, understand label taxonomy, and maintain high agreement scores. You are comfortable working in web-based annotation tools, tracking tasks, and communicating blockers clearly in a remote environment.

Preferred Qualifications

Experience with RLHF, preference data creation, red-teaming style content safety labeling, or policy enforcement labeling is preferred. Familiarity with NLP tasks such as named entity recognition, summarization evaluation, instruction-following evaluation, or retrieval relevance judgment is a plus. Exposure to computer vision annotation (bounding boxes, keypoints, segmentation) and dataset QA concepts (gold sets, inter-annotator agreement, sampling plans) is beneficial.

Tools and Workflow

You will work in annotation platforms and internal labeling tools to complete tasks, review feedback, and resolve disagreements. The workflow includes reading task instructions, labeling items, performing self-checks, escalating ambiguous cases, and participating in periodic calibration. Quality is maintained through guideline updates, spot checks, gold tasks, and targeted rework based on QA evaluation outcomes.

Quality Standards

Success is measured through training data quality metrics such as accuracy against gold labels, consistency across similar cases, throughput with low rework rates, and clear documentation of edge cases. You will be expected to demonstrate annotation guidelines compliance, contribute to model performance improvement by reducing label noise, and support reliable evaluation signals for LLM training pipelines.

Who You Will Support

Projects may support AI labs, tech startups, annotation vendors, and enterprise product teams building search, chat, moderation, and multimodal assistants. Domains can include NLP intent classification, prompt evaluation for instruction-following, content safety labeling for moderation, and computer vision annotation for detection and segmentation tasks.

How to Apply

Apply through Rex.zone to be considered for remote data annotator jobs in Portland. Ensure your profile highlights experience with data labeling, RLHF or preference ranking, QA evaluation, and any domain exposure to NLP, computer vision annotation, or content safety labeling. Qualified candidates may be invited to complete a short skills calibration or sample annotation task aligned with project guidelines.

Frequently Asked Questions

  • Q: Are these remote data annotator jobs in Portland fully remote?

    Yes. The role is explicitly Remote and can be performed from Portland, Oregon, with online tools and remote team communication.

  • Q: What type of data will I annotate?

    You may annotate text for NLP tasks (classification, named entity recognition, summarization evaluation), run prompt evaluation and LLM response scoring, perform RLHF preference ranking, and label images for computer vision annotation (bounding boxes, polygons, segmentation).

  • Q: What is RLHF and why does it matter for this job?

    RLHF (Reinforcement Learning from Human Feedback) uses human preference data and rubric-based judgments to improve model behavior. As an annotator, your rankings and evaluations become training signals that improve helpfulness, safety, and instruction-following.

  • Q: How is quality measured?

    Quality is evaluated through annotation guidelines compliance, accuracy on gold tasks, inter-annotator agreement, audit results, and rework rates. Clear edge-case notes and consistent labeling are critical for training data quality.

  • Q: Is this role full-time or contract/freelance?

    This posting is for FULL_TIME remote work. Some projects in the broader market may be contract or freelance, but this role’s employment type remains FULL_TIME.

  • Q: Do I need prior experience with annotation tools?

    Prior tool experience is helpful but not always required. What matters most is careful guideline reading, consistent decisions, and the ability to learn labeling workflows quickly.

  • Q: What domains are commonly involved?

    Common domains include NLP, computer vision, content safety labeling, and large language model evaluation for LLM training pipelines, including prompt evaluation and QA evaluation.

  • Q: How do I increase my chances of being selected on Rex.zone?

    Highlight data labeling experience, examples of QA-focused work, familiarity with RLHF or evaluation rubrics, and any specialization in named entity recognition, content safety labeling, or computer vision annotation. Emphasize consistency, documentation habits, and remote collaboration skills.

230+Domains Covered
120K+PhD, Specialist, Experts Onboarded
50+Countries Represented

Industry-Leading Compensation

We believe exceptional intelligence deserves exceptional pay. Our platform consistently offers rates above the industry average, rewarding experts for their true value and real impact on frontier AI. Here, your expertise isn't just appreciated - it's properly compensated.

Work Remotely, Work Freely

No office. No commute. No constraints. Our fully remote workflow gives experts complete flexibility to work at their own pace, from any country, any time zone. You focus on meaningful tasks - we handle the rest.

Respect at the Core of Everything

AI trainers are the heart of our company. We treat every expert with trust, humanity, and genuine appreciation. From personalized support to transparent communication, we build long-term relationships rooted in respect and care.

Ready to Shape the Future of AI Data Operations?

Apply Now.