onsite
Sr. Lead Data Engineer
Data Engineer
Lead data engineering initiatives focused on automating anomaly detection pipelines on AWS, leveraging BigQuery for analytics, and managing change across data environments.
About the role
Key Responsibilities
- Design, build, and maintain scalable data pipelines for anomaly detection using AWS services and BigQuery.
- Automate data ingestion, processing, and alerting workflows to ensure real‑time insights.
- Implement robust change management practices, including version control, testing, and deployment automation.
- Collaborate with data scientists and product teams to refine detection models and improve data quality.
- Monitor system performance, troubleshoot issues, and optimize resource utilization.
Requirements
- 10+ years of experience in data engineering with a strong focus on cloud platforms.
- Proficiency in AWS (Lambda, Glue, Redshift, S3) and BigQuery.
- Hands‑on experience with anomaly detection techniques and machine learning pipelines.
- Expertise in automation tools (Airflow, Terraform, CI/CD) and change management processes.
- Excellent problem‑solving skills and ability to lead cross‑functional teams.