onsite
Data Engineer - PA Consulting
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and warehouses using Python, SQL, Spark, and AWS services to support analytics and AI initiatives across diverse industry domains.
About the role
Key Responsibilities
- Design, develop, and optimize large-scale data pipelines using Python, SQL, and Apache Spark to ingest, transform, and load data from heterogeneous sources.
- Implement and maintain data warehouses on AWS (Redshift, S3, Glue) ensuring high availability, performance, and security.
- Collaborate with data scientists, analysts, and business stakeholders to define data models, quality metrics, and reporting requirements.
- Automate data workflows with Airflow or similar orchestration tools, monitor job health, and troubleshoot failures.
- Document data architecture, pipeline logic, and best practices for reproducibility and compliance.
Requirements
- 3+ years of experience in data engineering or related field.
Skills
pythonsqlapache sparkaws