onsite
Technical Specialist - Data Engineer
Data Engineer
Seeking a Data Engineer with expertise in Apache Airflow, Beam, Hadoop, and Spark. This role focuses on building and optimizing data pipelines and infrastructure.
About the role
Key Responsibilities
- Design, build, and maintain scalable data pipelines using technologies like Apache Airflow, Apache Beam, Apache Hadoop, and Apache Spark.
- Develop and optimize data processing workflows to ensure efficiency and reliability.
- Collaborate with cross-functional teams to understand data requirements and deliver robust solutions.
- Implement data quality checks and monitoring to ensure data integrity.
- Troubleshoot and resolve issues related to data pipelines and infrastructure.
Requirements
- Proven experience as a Data Engineer or in a similar role.
- Strong proficiency with big data technologies such as Apache Hadoop and Apache Spark.
- Hands-on experience with workflow orchestration tools like Apache Airflow.
- Familiarity with data processing frameworks like Apache Beam.
- Excellent problem-solving and analytical skills.
Skills
apache beamapache spark