onsite
Senior Data Engineer - techblocks
Data Engineer
Senior Data Engineer building scalable, cloud‑native data pipelines using Python, Airflow, Spark, Beam, and SQL, with deep expertise in data warehousing and modern big‑data architectures.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines that ingest, transform, and load large volumes of data into cloud data warehouses.
- Implement and orchestrate workflows with Apache Airflow, ensuring reliability, scalability, and observability.
- Leverage Apache Spark and Beam for batch and streaming data processing, optimizing performance and resource usage.
- Write efficient, maintainable SQL queries and data models to support analytics and reporting.
- Collaborate with data scientists, analysts, and product teams to translate business requirements into robust data solutions.
- Monitor pipeline health, troubleshoot issues, and continuously improve data quality and processing efficiency.
Requirements
- 5+ years of experience as a Data Engineer or similar role.
- Experience with version control, CI/CD, and containerization (Docker, Kubernetes).
- Excellent problem‑solving skills and a passion for building scalable, production‑grade data systems.
Skills
pythonapache sparkapache beamsql