onsite
Sr. Data Engineer, iCloud - Apple
Data Engineer
Senior Data Engineer responsible for designing and operating large‑scale, reliable data pipelines that power iCloud services, leveraging Python, Spark, SQL, and cloud infrastructure to deliver timely, accurate data across Apple’s ecosystem.
About the role
Key Responsibilities
- Design, build, and maintain high‑throughput data pipelines that ingest, transform, and store petabytes of iCloud user data.
- Develop scalable data models and schemas to support cross‑product analytics for Photos, Mail, Messages, FaceTime, Calendar, and more.
- Implement real‑time streaming solutions using Kafka and Spark Structured Streaming to provide up‑to‑date information to downstream services.
- Collaborate with product, security, and reliability teams to ensure data quality, privacy compliance, and operational excellence.
- Optimize performance and cost of data workloads on AWS, leveraging services such as S3, Redshift, EMR, and Glue.
Requirements
- 5+ years of experience building large‑scale data pipelines in a cloud environment.
- Strong proficiency in Python and SQL, with hands‑on experience in Apache Spark or similar distributed processing frameworks.
- Deep understanding of data modeling, schema design, and ETL best practices.
- Experience with streaming platforms (e.g., Kafka) and AWS data services.
- Demonstrated ability to solve complex, high‑impact problems at scale while maintaining data integrity and security.
Skills
pythonsqlapache sparkawskafka