onsite
Data Engineer - Whitefield Careers Private Limited
Data Engineer
Data Engineer with 3–5 years of experience building scalable analytics pipelines on AWS, mastering Redshift, Glue, EMR, and Airflow, and proficient in Python and Spark for data transformation and quality assurance.
About the role
Key Responsibilities
- Design, develop, and maintain high‑performance data pipelines using Amazon Redshift, AWS Glue, and Apache Airflow.
- Implement data modeling and schema design for analytical workloads, ensuring optimal query performance.
- Leverage Apache Spark for large‑scale data processing and transformation tasks.
- Write and optimize complex SQL queries across Redshift and other data stores.
- Collaborate with data scientists and analysts to enforce data quality, validation, and best practices.
- Monitor, troubleshoot, and improve pipeline reliability and scalability.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or related field.
- 3–5 years of hands‑on data engineering experience.
- Advanced SQL skills and deep experience with Amazon Redshift.
- Proficiency in AWS data services (Glue, S3, EMR) and orchestration with Apache Airflow.
- Hands‑on experience with Apache Spark and at least one programming language (Python, Java, or Scala).
Skills
sqlapache sparkpython