remote
Data Engineer 3 - Reston, VA - Freewheel - Comcast
Data Engineer
Senior Data Engineer responsible for designing, building, and maintaining scalable data pipelines and lakehouse architecture using Python, Spark, and AWS services, ensuring data quality, governance, and compliance across global advertising platforms.
About the role
Key Responsibilities
- Design, develop, and optimize large‑scale data pipelines using Python, Apache Spark, and AWS Glue to ingest, transform, and load data into a unified data lake.
- Implement robust data modeling and schema evolution strategies to support analytics, reporting, and machine‑learning workloads.
- Ensure data quality, lineage, and governance through automated testing, monitoring, and metadata management.
- Collaborate with cross‑functional teams to translate business requirements into technical solutions, maintaining compliance with privacy and regulatory standards.
- Continuously evaluate and adopt emerging data technologies and best practices to improve performance, cost, and reliability.
Requirements
- 5+ years of experience in data engineering, with strong proficiency in Python, SQL, and Spark.
- Hands‑on experience with AWS services (S3, Redshift, Athena, Glue, Lambda).
- Deep understanding of data lake architecture, ETL processes, and data governance.
- Excellent problem‑solving skills and ability to work independently in a fast‑paced environment.
- Strong communication skills and a collaborative mindset.
Skills
pythonsqlapache sparkaws