onsite
Lead Data Engineer - Python & GCP - Egen Solutions Inc
Data Engineer
Lead Data Engineer focused on designing, building, and maintaining scalable ETL/ELT pipelines on GCP, leveraging Python and core GCP services to deliver high‑quality, customer‑centric data solutions.
About the role
Key Responsibilities
- Lead end‑to‑end design and architecture of data pipelines using GCP services such as BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
- Develop and maintain robust ETL/ELT workflows in Python, ensuring data quality, consistency, and performance.
- Collaborate directly with customers to gather requirements, conduct discovery sessions, and translate business needs into technical specifications.
- Implement data ingestion, transformation, and validation logic, applying best practices for data governance and security.
- Provide post‑delivery support, monitoring, and optimization of data pipelines to meet SLAs and business objectives.
Requirements
- Proven experience as a Lead or Senior Data Engineer with a strong focus on Python and GCP.
- Deep knowledge of GCP data services (BigQuery, Dataflow, Pub/Sub, Cloud Storage) and experience building production‑grade pipelines.
- Strong understanding of ETL/ELT concepts, data modeling, and data quality practices.
- Excellent communication skills and ability to work closely with customers and cross‑functional teams.
- Experience with CI/CD, monitoring, and performance tuning of data pipelines is a plus.