onsite
Senior Data Engineer - Six Feet Up Inc.
Data Engineer
Senior Data Engineer leading the design, build, and maintenance of secure, scalable data pipelines using Python, Spark, and AWS, while ensuring data quality and supporting machine learning initiatives across structured, unstructured, and time‑series datasets.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines that ingest, transform, and store structured, unstructured, sensor‑based, and high‑volume data.
- Implement robust ETL workflows using Python and Apache Spark, optimizing for performance and scalability on AWS infrastructure.
- Ensure data quality and integrity through automated validation, monitoring, and alerting mechanisms.
- Collaborate with data scientists and product teams to support machine learning projects, providing clean, feature‑ready datasets.
- Lead technical decision‑making, mentor junior engineers, and drive best practices in data architecture and governance.
Requirements
- 5+ years of experience as a data engineer or similar role, with a strong background in Python, SQL, and Spark.
- Proven expertise building scalable data pipelines on AWS (EMR, Redshift, S3, Glue).
- Deep understanding of data modeling, ETL design, and data quality principles.
- Excellent communication skills and ability to translate complex data challenges into actionable solutions.
- Experience with time‑series and sensor data is a plus.
Skills
pythonsqlapache sparkaws