onsite
Data Engineer III - TechniPros, LLC
Data Engineer
Senior Data Engineer responsible for designing, building, and optimizing scalable cloud data pipelines and models using Python, SQL, AWS, Spark, and Airflow to deliver real‑time analytics and reporting.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ingestion frameworks on AWS for enterprise analytics.
- Implement and optimize Spark jobs and Airflow DAGs to support near real‑time data delivery.
- Model and document data structures, ensuring high quality and performance for downstream reporting and BI.
- Collaborate with data scientists, analysts, and product teams to translate business requirements into robust data solutions.
- Monitor pipeline health, troubleshoot issues, and continuously improve reliability and efficiency.
Requirements
- 5+ years of experience as a Data Engineer in a cloud environment.
- Proficiency in Python, SQL, and AWS services (S3, Redshift, Glue, EMR).
- Hands‑on experience with Apache Spark and Airflow for ETL orchestration.
- Strong data modeling skills and familiarity with dimensional modeling concepts.
- Excellent problem‑solving, communication, and collaboration abilities.
Skills
pythonsqlawsapache spark