onsite
Senior Data Engineer - Iris Software
Data Engineer
Lead end‑to‑end data pipeline development using Python, Spark, and AWS services, designing scalable ETL workflows and data warehouses to support analytics and AI initiatives.
About the role
Key Responsibilities
- Design, build, and maintain large‑scale data pipelines and ETL processes using Python and Apache Spark.
- Implement data ingestion from diverse sources, ensuring data quality, consistency, and performance.
- Develop and optimize data models in AWS Redshift or Snowflake, supporting downstream analytics and reporting.
- Collaborate with data scientists and business stakeholders to translate requirements into robust data solutions.
- Monitor pipeline health, troubleshoot issues, and continuously improve reliability and scalability.
Requirements
- 5+ years of experience in data engineering or related roles.
- Strong proficiency in Python, SQL, and Spark.
- Hands‑on experience with AWS data services (S3, Glue, Redshift, Athena).
- Solid understanding of data warehousing concepts and ETL best practices.
- Excellent problem‑solving skills and ability to work in a fast‑paced environment.
Skills
pythonsqlapache sparkaws