remote
Data Engineer 2 - Reston, VA - Freewheel - Comcast
Data Engineer
Data Engineer 2 focused on building and optimizing data pipelines, managing data storage, and supporting analytics and machine learning initiatives using Python, SQL, Spark, and AWS services.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Python, SQL, and Apache Spark to ingest, transform, and load data across multiple sources.
- Implement and optimize data storage solutions on AWS (S3, Redshift, Athena) ensuring high availability and performance.
- Collaborate with data scientists and analysts to provide clean, reliable datasets for advanced analytics and machine learning projects.
- Monitor pipeline health, troubleshoot issues, and implement automated alerts and logging for data quality and performance.
- Document data models, pipeline architecture, and best practices for future maintenance and onboarding.
Requirements
- 3+ years of experience in data engineering with strong proficiency in Python and SQL.
- Hands‑on experience with Apache Spark and distributed data processing.
- Solid understanding of AWS data services (S3, Redshift, Athena, Glue).
- Experience designing data models and building ETL workflows.
- Strong problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlapache sparkaws