remote
People Data Engineering Lead - Millennium Management
Software Engineer
Lead the design, development, and scaling of people‑data pipelines and platforms, leveraging Python, SQL, Spark, and AWS to deliver reliable, high‑performance data solutions for talent analytics and HR insights.
About the role
Key Responsibilities
- Architect and build end‑to‑end data pipelines that ingest, transform, and store employee‑related data from multiple sources.
- Lead a team of data engineers, providing technical guidance, mentorship, and performance management.
- Collaborate with HR analytics, product, and business stakeholders to define data requirements and deliver actionable insights.
- Design and maintain scalable data models and warehouses on AWS (Redshift, S3, Glue) ensuring data quality, security, and compliance.
- Implement best‑in‑class ETL processes using Python, SQL, and Apache Spark, optimizing for performance and cost.
- Establish monitoring, alerting, and governance frameworks for data pipelines and infrastructure.
Requirements
- 5+ years of hands‑on data engineering experience, with at least 2 years in a lead or managerial role.
- Strong proficiency in Python, SQL, and Spark for large‑scale data processing.
- Deep experience with AWS data services (Redshift, S3, Glue, Lambda) and modern data warehousing concepts.
- Proven ability to design robust ETL architectures and data models for HR or people‑analytics domains.
- Excellent communication and stakeholder management skills, with a track record of delivering data solutions that drive business impact.
Skills
pythonsqlapache sparkaws