remoteonsite
Senior Engineer Data Foundry - Compliance Ops - Empower
Systems Engineer
Senior Engineer in Data Foundry – Compliance Ops, building robust data pipelines and ensuring regulatory compliance using Python, SQL, Spark on AWS. Lead data quality, governance, and automation for financial services.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines for compliance reporting using Python and Spark on AWS.
- Implement data quality checks, lineage, and metadata management to meet regulatory standards.
- Collaborate with data scientists and product teams to translate compliance requirements into technical solutions.
- Automate data ingestion, transformation, and validation workflows, ensuring high availability and performance.
- Monitor and troubleshoot production data pipelines, providing root cause analysis and continuous improvement.
Requirements
- 5+ years of experience in data engineering with a focus on compliance or regulated industries.
- Strong proficiency in Python, SQL, and Spark for large‑scale data processing.
- Hands‑on experience with AWS services (Glue, Redshift, S3, Lambda).
- Deep understanding of data governance, lineage, and data quality frameworks.
- Excellent problem‑solving skills and ability to work cross‑functionally in a fast‑paced environment.