remoteonsite
Software Engineer - ETL Developer - Empower
Software Engineer
Lead the design, development, and maintenance of scalable ETL pipelines using Python, SQL, and Spark on AWS, ensuring data quality and performance for enterprise analytics.
About the role
Key Responsibilities
- Design, develop, and optimize ETL workflows to ingest, transform, and load large datasets from diverse sources into data warehouses.
- Implement data quality checks, monitoring, and alerting using Airflow and custom scripts.
- Collaborate with data scientists and business analysts to understand data requirements and deliver actionable insights.
- Maintain and improve existing pipelines, ensuring high availability, scalability, and cost efficiency on AWS.
- Document architecture, code, and best practices for future maintenance and knowledge transfer.
Requirements
- 3+ years of experience building ETL pipelines in a production environment.
- Strong proficiency in Python, SQL, and Apache Spark.
- Hands‑on experience with AWS services (S3, Redshift, Glue, EMR).
- Familiarity with workflow orchestration tools such as Airflow.
- Excellent problem‑solving skills and a collaborative mindset.
Skills
pythonsqlawsapache sparkairflow