onsite
Jr Data Engineer - Custom Solutions - Citylitics
Data Engineer
Junior Data Engineer building scalable data pipelines in Python and SQL on AWS, orchestrating workflows with Airflow, and modeling data for real‑time infrastructure insights.
About the role
Key Responsibilities
- Design, develop, and maintain data pipelines that ingest, transform, and load large volumes of infrastructure data using Python and SQL.
- Collaborate with data scientists and product teams to understand data requirements and deliver clean, well‑documented datasets.
- Implement and monitor Airflow DAGs to automate ETL processes and ensure data quality and reliability.
- Utilize AWS services (S3, Redshift, Glue) to build scalable, cost‑effective data storage and processing solutions.
- Perform data profiling, validation, and performance tuning to optimize query execution and pipeline throughput.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or related field.
- Proficiency in Python, SQL, and experience with data pipeline frameworks.
- Hands‑on experience with AWS data services and Airflow orchestration.
- Strong analytical skills and ability to troubleshoot complex data issues.
- Excellent communication skills and a collaborative mindset.
Skills
pythonsqlairflowaws