onsite
Senior IT Data Engineer - Xai
Data Engineer
Senior Data Engineer responsible for designing, building, and maintaining scalable data pipelines and warehouses using Python, SQL, Airflow, Spark, and AWS services.
About the role
Key Responsibilities
- Design, develop, and maintain robust ETL pipelines to ingest and transform large‑scale datasets.
- Implement workflow orchestration using Apache Airflow for reliable, repeatable data processing.
- Build and optimize data models and warehouses on AWS (Redshift, S3, Glue) to support analytics and machine‑learning teams.
- Develop high‑performance data processing jobs with Apache Spark and Python.
- Collaborate with cross‑functional teams to define data requirements, ensure data quality, and drive continuous improvement.
Requirements
- 5+ years of hands‑on experience in data engineering, including pipeline development and data warehousing.
- Strong proficiency in Python and SQL for data manipulation and analysis.
- Experience with Apache Airflow and Apache Spark in production environments.
- Deep knowledge of AWS data services (Redshift, S3, Glue, Lambda) and best practices for cloud‑based data solutions.
- Excellent problem‑solving, communication, and prioritization skills in a fast‑paced, collaborative setting.
Skills
pythonsqlapache sparkaws