onsite
Data Engineer - Activate Interactive Pte Ltd
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and infrastructure using Python, SQL, Spark, and AWS services.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines to ingest, transform, and load data from diverse sources into data warehouses.
- Implement and optimize ETL processes using Python, SQL, and Apache Spark for high-volume, real‑time data streams.
- Collaborate with data scientists and analysts to ensure data quality, consistency, and accessibility across the organization.
- Deploy and manage data infrastructure on AWS (S3, Redshift, EMR, Glue) and containerize services with Docker.
- Automate workflow orchestration with Airflow, monitor job performance, and troubleshoot production issues.
Requirements
- 3+ years of experience as a Data Engineer or similar role.
- Strong proficiency in Python, SQL, and Spark for data processing.
- Hands‑on experience with AWS data services and Docker containerization.
- Knowledge of data modeling, schema design, and performance tuning.
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonsqlapache sparkawsdockerairflow