onsite
Data Engineer III - TechniPros
Data Engineer
Experienced Data Engineer III to design, build, and optimize cloud‑native data pipelines and models using Python, Spark, Airflow, and AWS services for real‑time analytics and reporting.
About the role
Key Responsibilities
- Design and implement scalable, fault‑tolerant data pipelines for batch and streaming workloads on AWS.
- Develop ETL/ELT processes using Python, SQL, and Apache Spark to ingest, transform, and load data into Snowflake and other analytical stores.
- Orchestrate workflows with Apache Airflow, ensuring reliable scheduling, monitoring, and alerting.
- Collaborate with data analysts, scientists, and product teams to define data models that support reporting, dashboards, and near‑real‑time insights.
- Optimize performance and cost of data solutions through partitioning, indexing, and resource tuning.
Requirements
- 5+ years of professional experience building data pipelines in cloud environments, preferably AWS.
- Strong proficiency in Python, SQL, and Spark programming.
- Hands‑on experience with workflow orchestration tools such as Apache Airflow.
- Deep understanding of data warehousing concepts and experience with Snowflake or similar platforms.
- Solid grasp of data modeling, schema design, and best practices for data quality and governance.
Skills
pythonsqlapache sparkairflowawssnowflake