remote
Principal Data Engineer - Data Architect II - UST
Data Engineer
Lead the design and delivery of a cloud‑scale data platform, driving distributed processing, streaming, and data product development using technologies such as Spark, Kafka, AWS, and Snowflake.
About the role
Key Responsibilities
- Architect, build, and evolve a scalable, cloud‑native data platform supporting batch and real‑time workloads.
- Design and implement distributed data pipelines using Apache Spark, Kafka, and SQL across AWS services.
- Define and enforce engineering standards, best practices, and governance for data ingestion, storage, and processing.
- Mentor and guide engineering teams, fostering a culture of code quality, testing, and continuous delivery.
- Collaborate with product and analytics stakeholders to translate business requirements into robust data models and services.
Requirements
- 10+ years of experience in data engineering or data architecture, with a strong focus on cloud platforms (AWS) and modern data stack tools.
- Proficiency in Python and Java for building high‑performance data pipelines.
- Deep expertise in Apache Spark, Apache Kafka, and SQL‑based data warehouses such as Snowflake.
- Hands‑on experience designing data models, ETL/ELT processes, and streaming architectures.
- Demonstrated ability to lead technical teams, set standards, and drive architectural decisions.
Skills
pythonjavaapache sparkawssnowflakesql