onsite
AI/LLM Safety Engineer - propio
Research Engineer
Design and implement safety evaluations, guardrails, and red‑team testing for LLM‑driven agents, ensuring responsible behavior and secure tool usage in production environments.
About the role
Key Responsibilities
- Design, implement, and maintain automated safety evaluation pipelines for large language models and autonomous agents.
- Develop and enforce guardrails that prevent unsafe tool usage, unintended actions, and policy violations.
- Lead red‑team exercises to discover adversarial prompts, jailbreaks, and other failure modes before release.
- Collaborate with product, engineering, and research teams to integrate safety checks into CI/CD and production monitoring.
- Analyze incident logs, produce root‑cause reports, and iterate on mitigation strategies.
Requirements
- Strong programming skills in Python and experience with ML frameworks (e.g., PyTorch, TensorFlow).
- Deep understanding of LLM architectures, prompt engineering, and evaluation methodologies.
- Hands‑on experience in AI safety, red‑team testing, or security research for generative models.
- Ability to design robust test suites, metrics, and automated guardrails for production systems.
- Excellent problem‑solving skills and a proactive mindset for identifying and closing safety gaps.
Skills
pythonmachine learning