remote
Entry - Rex.zone
AI Evaluator
Entry-level role supporting AI/ML engineering workflows with data labeling, RLHF evaluation, and QA for large language models.
About the role
Key Responsibilities
- Perform data labeling for text and/or vision datasets to meet training data quality targets
- Complete RLHF-style preference ranking and response quality scoring to support model performance improvement
- Run prompt evaluation and QA evaluation to identify errors, hallucinations, policy violations, and formatting issues
- Apply named entity recognition and taxonomy tagging using clear annotation guidelines compliance
- Conduct content safety labeling for sensitive, policy, and restricted content categories
- Audit datasets for consistency, duplicates, label noise, and distribution gaps; propose fixes
Requirements
- STEM fundamentals and strong analytical reasoning
- Comfort with Python and SQL for basic data checks and simple scripts
- Understanding of NLP and/or computer vision annotation concepts
- Experience applying annotation guidelines compliance and documenting decisions
- Ability to perform QA evaluation, spot inconsistencies, and propose improvements
Skills
pythonsqlnlpcomputer visionannotation guidelinesqa evaluation