remote
Senior Data Engineer - a5labs
Data Engineer
Lead end‑to‑end data pipeline development, architecting scalable Big Data solutions with Python, SQL, and Spark to deliver high‑quality data for analytics and machine learning.
About the role
Key Responsibilities
- Design, build, and maintain robust data pipelines from diverse sources into enterprise data warehouses.
- Implement ETL processes using Python, SQL, and Spark, ensuring data quality, performance, and reliability.
- Collaborate with data scientists and analysts to understand data requirements and deliver actionable insights.
- Optimize storage and query performance in Big Data environments (Hadoop, Hive, or similar).
- Document architecture, data models, and best practices for future maintenance and scalability.
Requirements
- 5+ years of experience in data engineering or related roles.
- Proficiency in Python, SQL, and Apache Spark.
- Strong understanding of data warehousing concepts and ETL design patterns.
- Experience with Big Data ecosystems (Hadoop, Hive, or similar).
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlapache spark