remoteonsite
Mid-level Data Engineer Databricks - CGI
Data Engineer
Mid-level Data Engineer focused on building and maintaining scalable data pipelines on Databricks, leveraging Spark, Python, and SQL to transform and model data for analytics and reporting.
About the role
Key Responsibilities
- Design, develop, and maintain data pipelines on Databricks using Apache Spark and Python.
- Implement ETL processes to ingest, clean, and transform large datasets into structured data warehouses.
- Optimize Spark jobs for performance and cost efficiency, including tuning partitioning and caching strategies.
- Collaborate with data scientists and business analysts to deliver high-quality data models and dashboards.
- Ensure data quality, lineage, and compliance with security and governance policies.
Requirements
- 3+ years of experience in data engineering with a focus on Spark and Databricks.
Skills
databricksapache sparkpythonsql