remote
Senior Data Engineer - Warner Bros. Discovery
Data Engineer
Senior Data Engineer responsible for designing, building, and maintaining scalable data pipelines and warehouses using Python, SQL, Spark, Airflow, and AWS to enable analytics and machine‑learning initiatives.
About the role
Key Responsibilities
- Design, develop, and optimize end‑to‑end data pipelines that ingest, transform, and store large‑scale datasets.
- Implement robust ETL processes using Apache Spark and Apache Airflow to support real‑time and batch workloads.
- Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver reliable data models.
- Maintain and enhance data warehouse solutions on AWS (Redshift, S3, Glue) ensuring high performance and cost efficiency.
- Monitor pipeline health, troubleshoot issues, and implement automated testing and alerting.
Requirements
- 5+ years of professional experience in data engineering or related fields.
- Strong proficiency in Python and SQL for data manipulation and scripting.
- Hands‑on experience with Apache Spark and workflow orchestration tools such as Airflow.
- Deep understanding of AWS data services (Redshift, S3, Glue, Lambda) and cloud‑based architecture.
- Solid knowledge of data modeling, ETL best practices, and performance tuning.
Skills
pythonsqlapache sparkaws