remoteonsite
Principal - Data Engineer - Algoworks Solutions
Data Engineer
Lead data engineering initiatives, architecting scalable pipelines with Python, Spark, and AWS to transform complex datasets into actionable insights for Fortune 500 clients.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines using Python, Apache Spark, and AWS services (S3, Redshift, Glue).
- Collaborate with data scientists and product teams to define data models, schema, and governance standards.
- Optimize query performance and resource utilization across large‑scale data warehouses.
- Implement robust monitoring, alerting, and automated testing for data workflows.
- Mentor junior engineers and drive best practices in data engineering and DevOps.
Requirements
- 8+ years of experience in data engineering with a strong focus on big data technologies.
- Proficiency in Python, SQL, and Spark (PySpark).
- Hands‑on experience with AWS data services (S3, Redshift, Glue, EMR).
- Deep understanding of data modeling, ETL design, and performance tuning.
- Excellent communication skills and a proven ability to lead cross‑functional teams.
Skills
pythonapache sparksqlaws