onsite
Big Data Engineer - FERCHAU GmbH Niederlassung Leipzig
Data Engineer
Lead the design, implementation, and optimization of large-scale data pipelines using Hadoop, Spark, and Python, ensuring high performance and reliability on AWS infrastructure.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes using Hadoop, Spark, and Python.
- Implement data ingestion from diverse sources, ensuring data quality and consistency across the platform.
- Optimize query performance and resource utilization on AWS services such as EMR, S3, and Redshift.
- Collaborate with data scientists and analysts to support advanced analytics and machine learning initiatives.
- Monitor, troubleshoot, and improve system reliability, scalability, and security.
Requirements
- Proven experience with Big Data technologies (Hadoop, Spark) and Python programming.
- Strong SQL skills and familiarity with data warehousing concepts.
- Hands‑on experience deploying and managing workloads on AWS.
- Knowledge of streaming platforms such as Kafka or Flink is a plus.
- Excellent problem‑solving skills and a collaborative mindset.
Skills
hadooppythonsqlawskafka