onsite
Cloud Data & AI Engineer - ruhrdot GmbH
AI Engineer
Lead the design and implementation of scalable data and AI platforms on Azure or AWS using Databricks, Python, Spark, and MLflow, driving end‑to‑end solutions that are production‑ready and maintainable.
About the role
Key Responsibilities
- Architect, develop, and maintain end‑to‑end data pipelines and AI workflows on Databricks within Azure or AWS environments.
- Collaborate with cross‑functional teams to translate business requirements into scalable, production‑ready solutions.
- Implement and optimize Spark jobs, data models, and ML pipelines using Python, SQL, and MLflow.
- Ensure data quality, governance, and security across the platform, applying best practices for monitoring and observability.
- Mentor junior engineers and share knowledge on emerging cloud data technologies.
Requirements
- Proven experience with Databricks, Azure Data Lake, or AWS Glue/EMR.
- Strong programming skills in Python and familiarity with Spark SQL.
- Hands‑on experience with MLflow, Delta Lake, and data versioning.
- Solid understanding of cloud architecture, security, and cost optimization.
- Excellent problem‑solving skills and a collaborative mindset.
Skills
databricksazureawspythonapache sparkmlflow