remote
Data Engineer - Milestone Technologies
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and lakehouse architectures using Python, SQL, Spark, and AWS services to enable data-driven insights across the organization.
About the role
Key Responsibilities
- Design, develop, and optimize large-scale data pipelines using Python, SQL, and Apache Spark.
- Implement and maintain data lakehouse solutions on AWS (S3, Glue, Redshift, Athena).
- Collaborate with data scientists and business analysts to translate requirements into robust data models and schemas.
- Ensure data quality, lineage, and security through automated testing, monitoring, and documentation.
- Participate in performance tuning, cost optimization, and continuous improvement of data infrastructure.
Requirements
- 3+ years of experience in data engineering or related field.
Skills
pythonsqlapache sparkaws