remote
Senior/Staff Data Engineer - Gridsight
Data Engineer
Lead the design and delivery of scalable data pipelines and analytics infrastructure for a cutting‑edge electricity grid platform, leveraging Python, Spark, Airflow, and cloud services.
About the role
Key Responsibilities
- Architect, build, and maintain high‑performance data pipelines that ingest, transform, and store large volumes of grid telemetry and market data.
- Design and implement data models and schemas to support real‑time analytics, forecasting, and reporting for utility customers.
- Develop and operationalize ETL workflows using Apache Spark, Airflow, and Kafka, ensuring reliability, scalability, and low latency.
- Collaborate with data scientists, product managers, and software engineers to translate business requirements into robust data solutions.
- Implement best practices for data quality, monitoring, and security on AWS (S3, Redshift, Glue, etc.).
Requirements
- 5+ years of professional experience building large‑scale data pipelines in cloud environments.
- Strong proficiency in Python and SQL, with hands‑on experience in Apache Spark or similar distributed processing frameworks.
- Experience with workflow orchestration tools (e.g., Airflow) and streaming platforms such as Kafka.
- Deep understanding of data modeling, warehousing, and ELT/ETL design patterns.
- Proven ability to work autonomously, mentor junior engineers, and drive technical decisions in a fast‑moving, mission‑critical environment.
Skills
pythonsqlapache sparkairflowawskafka