onsite
Senior Data Engineer Team Lead - Neighborly Software
Data Engineer
Lead a data engineering team to design, build, and maintain scalable data pipelines on AWS, leveraging Python, Spark, and SQL to support analytics for government funding programs.
About the role
Key Responsibilities
- Architect and develop robust, scalable data pipelines using Python, Spark, and SQL on AWS services (Redshift, S3, Glue).
- Lead a small team of data engineers, providing mentorship, code reviews, and technical guidance.
- Design and maintain data models, ensuring data quality, lineage, and compliance with governance standards.
- Implement and manage Airflow workflows for automated data ingestion, transformation, and reporting.
- Collaborate with data scientists and product stakeholders to translate business requirements into technical solutions.
Requirements
- 5+ years of experience in data engineering with a strong focus on cloud-based architectures.
- Proficiency in Python, SQL, and Spark for large-scale data processing.
- Hands‑on experience with AWS services (Redshift, S3, Glue, Lambda, EMR).
- Solid understanding of data modeling, ETL best practices, and data governance.
- Excellent communication skills and a proven ability to lead and mentor junior engineers.
Skills
pythonsqlawsairflow