remoteonsite
Senior Data Engineer - Ciklum
Data Engineer
Senior Data Engineer crafting scalable data pipelines, transforming complex relational databases into agent‑ready layers using Python, SQL, ETL, and AWS services, while leveraging Spark for big‑data processing.
About the role
Key Responsibilities
- Design, build, and maintain robust data pipelines that ingest, transform, and load data from diverse relational sources into a unified, agent‑ready data layer.
- Implement ETL processes using Python, SQL, and Spark to ensure data quality, consistency, and performance at scale.
- Collaborate with cross‑functional teams to define data models, schemas, and data governance standards.
- Optimize data storage and retrieval on AWS services (Redshift, S3, Glue) to support analytics and reporting.
- Monitor pipeline health, troubleshoot issues, and continuously improve reliability and efficiency.
Requirements
- 5+ years of experience as a Data Engineer or similar role.
- Experience designing data warehouses and implementing data governance practices.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlawsapache spark