remote
Senior Data Engineer - UST
Data Engineer
Senior Data Engineer driving enterprise data pipelines and architecture using Python, Spark, and AWS, ensuring scalable, high‑performance data solutions for analytics and AI initiatives.
About the role
Key Responsibilities
- Design, build, and maintain large‑scale data pipelines and lakehouse architectures on AWS using Python, Spark, and SQL.
- Implement robust ETL processes, data quality checks, and monitoring to support analytics, reporting, and machine learning workloads.
- Collaborate with data scientists, product owners, and business stakeholders to translate requirements into scalable data solutions.
- Optimize performance and cost of data services, leveraging AWS services such as S3, Redshift, Glue, and Athena.
- Document data models, pipeline logic, and best practices for reproducibility and knowledge transfer.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python, SQL, and Spark.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, Athena, EMR).
- Proven ability to design and implement scalable, fault‑tolerant data pipelines.
- Strong analytical skills and a passion for data quality and governance.
- Excellent communication skills and ability to work cross‑functionally in a fast‑paced environment.
Skills
pythonsqlapache sparkaws