remote
Senior Data Engineer - Experian
Data Engineer
Senior Data Engineer building scalable data pipelines and lakehouse solutions using Python, Spark, and AWS services to deliver high‑quality analytics for global clients.
About the role
Key Responsibilities
- Design, develop, and maintain large‑scale data pipelines that ingest, transform, and store structured and unstructured data across cloud platforms.
- Implement data modeling and schema design for data lakehouse architectures, ensuring optimal query performance and data integrity.
- Collaborate with data scientists and business analysts to translate analytical requirements into robust, production‑ready data solutions.
- Leverage AWS services (S3, Glue, Redshift, Athena) and Spark to build scalable, fault‑tolerant ETL workflows.
- Monitor pipeline health, troubleshoot issues, and continuously improve data quality and processing efficiency.
Requirements
- 5+ years of experience in data engineering with a strong focus on cloud data platforms.
- Proficiency in Python, SQL, and Apache Spark for data processing and transformation.
- Hands‑on experience with AWS data services (S3, Glue, Redshift, Athena, EMR).
- Solid understanding of data modeling, schema design, and ETL best practices.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlapache sparkaws