Remote Data Annotation Specialist

About the role

As a Remote Data Annotation Specialist at Rex.zone, you will be responsible for labeling and assessing multi-modal AI training data to enhance the evaluation of large language models and their performance. This position is full-time and entirely remote, primarily focusing on search intent and local data relevant to Barcelona.

Key facts

What you'll do

• Carry out data labeling for natural language processing tasks, including intent tagging, sentiment/topic classification, and named entity recognition.
• Conduct preference ranking and pairwise comparisons for reinforcement learning from human feedback, providing clear written justifications.
• Perform quality assurance evaluations on labeled datasets, address discrepancies, and document exceptional cases.
• Annotate visual data as needed, including tasks such as bounding boxes, polygons, keypoints, and image classification.
• Implement content safety labeling, ensuring compliance with policies regarding toxicity, self-harm, hate speech, and sexual content.
• Monitor production metrics such as throughput, accuracy, and inter-annotator agreement to assist in calibration efforts.
• Collaborate asynchronously using Rex.zone tools while adhering to secure data handling protocols.

Requirements

• A minimum of 3-5 years of experience in data annotation, data labeling, quality assurance evaluation, or search relevance assessment.
• Strong ability to follow guidelines and make consistent decisions, demonstrating high compliance with annotation standards.
• Proficient written English skills for providing rationale-based evaluations and prompt responses.
• Familiarity with natural language processing concepts and workflows for evaluating large language models.
• Comfortable using web-based annotation tools and managing tasks in a spreadsheet-like environment.

Nice to have

• Experience with computer vision data annotation.

Skills & tools

• Proficient in web-based annotation platforms and spreadsheet applications.
• Knowledge of NLP and large language model evaluation processes.

Practical notes

• This role is open to candidates in the United States.
• Compensation is hourly and varies based on the complexity of projects and accuracy of calibration.