onsite
Data Engineer - National Grid
Data Engineer
Data Engineer responsible for building scalable data pipelines, optimizing data warehouses, and enabling analytics for the Electricity Transmission business unit using Python, SQL, AWS, and Spark.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest, transform, and load large volumes of data from diverse sources into cloud data warehouses.
- Implement and optimize ETL processes using Python, SQL, and Apache Spark to ensure high performance and reliability.
- Collaborate with data scientists and analysts to provide clean, well‑documented datasets that support advanced analytics and reporting.
- Monitor pipeline health, troubleshoot issues, and implement automated alerts to maintain data quality and availability.
- Leverage AWS services (S3, Redshift, Glue, Lambda) to build scalable, cost‑effective data solutions.
Requirements
- Proven experience as a Data Engineer or similar role, with a strong background in Python, SQL, and ETL development.
- Hands‑on experience with AWS data services and cloud architecture.
- Familiarity with data warehousing concepts and experience building data models.
- Knowledge of Spark or other big‑data processing frameworks.
- Strong problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlawsapache spark