onsite
Data Engineer - Halian Technology Limited
Data Engineer
Lead the design and rebuild of a modern, cloud‑first data platform, driving end‑to‑end data pipelines, lake architecture, and scalable analytics solutions using Python, SQL, AWS, and Spark.
About the role
Key Responsibilities
- Architect and implement a scalable data lake and lakehouse on AWS, ensuring high availability and performance.
- Design, develop, and maintain robust ETL pipelines using Python, SQL, and Apache Spark.
- Collaborate with data scientists and business stakeholders to translate analytical requirements into production‑ready data solutions.
- Implement data governance, security, and compliance controls across the data platform.
- Continuously optimize data workflows, monitor performance, and troubleshoot issues.
Requirements
- 5+ years of experience building enterprise data platforms in cloud environments.
- Strong proficiency in Python, SQL, and Spark for data processing.
- Hands‑on experience with AWS services such as S3, Redshift, Glue, and Lake Formation.
- Solid understanding of data modeling, lakehouse architecture, and data governance best practices.
- Excellent problem‑solving skills and a proactive, collaborative mindset.
Skills
pythonsqlawsapache spark