onsite
Senior Data Engineer - AI & Agentic Pipelines - SIXT
Data Engineer
Senior Data Engineer responsible for designing, building, and operating AI‑driven data pipelines and agentic workflows, leveraging Python, Spark, Airflow, and cloud services to enable real‑time analytics and machine‑learning model deployment.
About the role
Key Responsibilities
- Design and implement scalable, fault‑tolerant data pipelines that feed AI and agentic systems.
- Develop, schedule, and monitor ETL workflows using Apache Airflow and Spark.
- Integrate streaming data sources (e.g., Kafka) and batch processes into unified data platforms.
- Collaborate with data scientists and ML engineers to operationalize models in production.
- Ensure data quality, governance, and security across cloud (AWS) and on‑prem environments.
- Optimize performance and cost of data infrastructure through automation and best practices.
Requirements
- 5+ years of professional experience in data engineering, with a strong focus on AI/ML pipelines.
- Proficiency in Python and SQL; hands‑on experience with Apache Spark and Airflow.
- Solid understanding of streaming technologies such as Kafka and cloud platforms, preferably AWS.
- Experience building and deploying production‑grade machine‑learning workflows.
- Strong problem‑solving skills, ability to work autonomously, and excellent communication with cross‑functional teams.
Skills
pythonsqlapache sparkkafkaaws