onsite
Data Engineer - LBBW
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and infrastructure using Python, SQL, Spark, and AWS services. Focus on data quality, performance, and collaboration with analytics teams.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines and ETL processes using Python, SQL, and Apache Spark.
- Implement and manage data storage solutions on AWS (S3, Redshift, RDS) ensuring high availability and security.
- Collaborate with data scientists and analysts to provide clean, well‑documented datasets for modeling and reporting.
- Monitor pipeline performance, troubleshoot issues, and optimize for scalability and cost efficiency.
- Automate deployment and monitoring using Docker, Kubernetes, and CI/CD pipelines.
Requirements
- 3+ years of experience in data engineering or related field.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, Athena).
- Knowledge of containerization (Docker) and orchestration (Kubernetes) is a plus.
- Strong problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlapache sparkawsdockerkafka