remote
Senior Data Engineer - Consumer Subscriptions - Xai
Data Engineer
Lead end‑to‑end data pipeline development for consumer subscription analytics, leveraging Python, Spark, and AWS to deliver scalable, high‑quality data solutions that drive business insights.
About the role
Key Responsibilities
- Design, build, and maintain robust data pipelines that ingest, transform, and store large volumes of subscription data across AWS services.
- Collaborate with data scientists and product teams to define data models, schemas, and metrics that support analytics and reporting.
- Optimize ETL workflows using Apache Spark and SQL, ensuring performance, reliability, and cost efficiency.
- Implement data quality checks, monitoring, and alerting to guarantee data integrity and availability.
- Document architecture, processes, and best practices for internal knowledge sharing.
Requirements
- 5+ years of experience as a data engineer in a fast‑paced environment.
- Proficiency in Python, SQL, and Spark for large‑scale data processing.
- Hands‑on experience with AWS services (S3, Redshift, Glue, EMR, Athena).
- Strong understanding of data modeling, schema design, and ETL best practices.
- Excellent communication skills and a collaborative mindset.
Skills
pythonsqlapache sparkaws