onsite
Data Engineer II - Travelers Insurance Company
Data Engineer
Data Engineer II responsible for designing, building, and maintaining scalable data pipelines on AWS, leveraging Python, SQL, and Spark to transform and model data for analytics and machine learning teams.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines using Python, SQL, and Apache Spark on AWS services (Glue, EMR, Redshift).
- Implement data ingestion, transformation, and loading processes to support analytics and ML workloads.
- Collaborate with data scientists and analysts to understand data requirements and deliver high‑quality, well‑documented datasets.
- Optimize pipeline performance, monitor job health, and troubleshoot issues in production environments.
- Ensure data quality, lineage, and security compliance across all data assets.
Requirements
- 3+ years of experience in data engineering or related field.
- Experience with data modeling, ETL design, and performance tuning.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlawsapache spark