remote
Lead Data Engineer - Create Music Group
Data Engineer
Lead Data Engineer driving scalable data pipelines and architecture for a high‑volume music streaming platform, leveraging Python, Spark, and AWS to deliver insights and support business growth.
About the role
Key Responsibilities
- Design, build, and maintain large‑scale data pipelines that ingest, transform, and store terabytes of streaming and metadata from multiple sources.
- Lead the migration of legacy ETL processes to a modern, cloud‑native architecture using AWS services (Glue, Redshift, S3, Lambda).
- Collaborate with data scientists and product teams to define data models, schemas, and performance metrics that enable real‑time analytics and reporting.
- Implement data quality, governance, and security best practices, ensuring compliance with industry standards.
- Mentor and grow a team of data engineers, fostering a culture of continuous improvement and innovation.
Requirements
- 10+ years of experience in data engineering, with 5+ in a leadership role.
- Proficiency in Python, SQL, and Apache Spark for large‑scale data processing.
- Hands‑on experience with AWS data services (Glue, Redshift, S3, Lambda, Athena).
- Strong background in data modeling, ETL design, and data warehousing concepts.
- Excellent communication skills and a proven ability to translate business needs into technical solutions.
Skills
pythonsqlapache sparkaws