remote
Data Platform Engineer - Northern Trust Corp.
Data Engineer
Lead the design, build, and maintenance of scalable data pipelines and lakehouse architecture using Python, Spark, and AWS services, ensuring high data quality and performance for enterprise analytics.
About the role
Key Responsibilities
- Design, develop, and optimize large‑scale data pipelines using Python and Apache Spark on AWS.
- Implement data ingestion, transformation, and loading processes into a lakehouse architecture.
- Collaborate with data scientists and analysts to define data models and ensure data quality.
- Monitor pipeline performance, troubleshoot issues, and implement automated alerts.
- Document architecture, processes, and best practices for future maintenance.
Requirements
- 3+ years of experience building data pipelines in a cloud environment.
- Strong proficiency in Python, SQL, and Spark.
- Hands‑on experience with AWS services such as S3, Glue, Redshift, and EMR.
- Solid understanding of data modeling, ETL best practices, and data governance.
- Excellent problem‑solving skills and ability to work cross‑functionally.
Skills
pythonsqlapache sparkaws