remote
Software Engineer - Data Science - Mercor
Data Scientist
Contract software engineer specializing in data science to design prompts, test AI reasoning on technical documents, and evaluate model performance for leading AI research labs.
About the role
Key Responsibilities
- Design and author prompt tasks that assess AI reasoning over complex technical documentation.
- Develop evaluation pipelines to benchmark AI models on accuracy, relevance, and logical consistency.
- Analyze model outputs, identify failure modes, and recommend improvements.
- Collaborate with research and engineering teams to integrate evaluation results into model development cycles.
- Maintain reproducible codebases and documentation for all testing frameworks.
Requirements
- Strong proficiency in Python and experience with ML/NLP libraries such as PyTorch, TensorFlow, or Hugging Face.
- Demonstrated ability to design and execute AI model evaluation experiments.
- Solid understanding of natural language processing, prompt engineering, and data analysis techniques.
- Experience working with large language models and interpreting their reasoning processes.
- Excellent problem‑solving skills and ability to work independently on a part‑time remote schedule.
Skills
pythonmachine learningnatural language processingdata analysis