onsite
Senior Data Engineer - Arrive Logistics
Data Engineer
Senior Data Engineer responsible for designing, building, and optimizing large‑scale data pipelines on cloud platforms, leveraging Python, SQL, Airflow, Snowflake, and Spark to enable real‑time analytics and support business growth.
About the role
Key Responsibilities
- Design, develop, and maintain robust ETL/ELT pipelines using Python, Apache Airflow, and SQL to ingest and transform high‑volume transportation data.
- Architect and manage data warehouses on Snowflake, ensuring optimal performance, scalability, and data integrity.
- Implement data processing solutions with Apache Spark for batch and streaming workloads, supporting real‑time analytics.
- Collaborate with data scientists, product managers, and engineering teams to define data requirements and deliver reliable data products.
- Monitor, troubleshoot, and optimize data workflows in AWS, applying best practices for security, cost efficiency, and reliability.
Requirements
- 5+ years of professional experience building data pipelines and data warehouses in a cloud environment.
- Strong proficiency in Python and SQL, with hands‑on experience in Apache Airflow and Apache Spark.
- Deep knowledge of Snowflake (or similar cloud data warehouse) architecture and performance tuning.
- Experience with AWS services such as S3, Redshift, Lambda, and IAM for data storage and processing.
- Excellent problem‑solving skills, ability to work independently, and strong communication for cross‑functional collaboration.
Skills
pythonsqlsnowflakeawsapache spark