remote
Senior Data Engineer - Managed Services - 3Cloud
Data Engineer
Lead end‑to‑end data pipelines for managed services, leveraging Python, Spark, and AWS to deliver scalable, high‑quality data solutions. Drive automation, performance tuning, and cross‑functional collaboration in a fast‑paced environment.
About the role
Key Responsibilities
- Design, build, and maintain large‑scale data pipelines using Python, Spark, and AWS services (S3, Redshift, Glue).
- Implement and manage workflow orchestration with Airflow, ensuring reliable, scheduled data processing.
- Optimize data models and queries for performance, scalability, and cost efficiency.
- Collaborate with data scientists, analysts, and product teams to translate business requirements into technical solutions.
- Containerize services with Docker and manage deployment pipelines.
- Monitor pipeline health, troubleshoot issues, and implement proactive alerts.
Requirements
- 5+ years of data engineering experience in a managed services or SaaS environment.
- Strong proficiency in Python, SQL, and Spark.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, Athena).
- Solid knowledge of workflow orchestration (Airflow) and containerization (Docker).
- Excellent problem‑solving skills and a collaborative mindset.
Skills
pythonsqlapache sparkawsairflowdocker