remoteonsite
Forward Deployed Data Engineering Expert - SAP
Software Engineer
Senior data engineering specialist driving end‑to‑end data pipelines, cloud architecture, and real‑time streaming solutions across enterprise domains, leveraging Python, Spark, AWS, Kafka, and Airflow to deliver scalable, high‑performance analytics platforms.
About the role
Key Responsibilities
- Design, build, and maintain large‑scale data pipelines and ETL workflows using Python, Spark, and SQL across on‑prem and cloud environments.
- Implement real‑time streaming solutions with Apache Kafka and orchestrate workflows with Airflow, ensuring data reliability and low latency.
- Collaborate with data scientists and product teams to translate business requirements into robust data models and analytics services.
- Optimize performance and cost of data infrastructure on AWS, including S3, Redshift, EMR, and Glue.
- Mentor junior engineers, conduct code reviews, and promote best practices in data engineering and DevOps.
Requirements
- 5+ years of professional experience in data engineering or related field.
- Strong proficiency in Python, SQL, and Spark for batch and streaming data processing.
- Hands‑on experience with AWS services (S3, Redshift, EMR, Glue) and Kafka ecosystem.
- Knowledge of workflow orchestration tools such as Airflow and CI/CD pipelines.
- Excellent problem‑solving skills, ability to work in a fast‑paced, cross‑functional environment.
Skills
pythonsqlapache sparkawsairflow