onsite
AVP Data Engineer - Barclays
Data Engineer
Senior Data Engineer leading the design and operation of scalable, real‑time data platforms using Databricks, Spark, and cloud streaming technologies to power digital banking services.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines on Databricks and Apache Spark.
- Implement real‑time streaming solutions using technologies such as Kafka to support low‑latency banking applications.
- Build, schedule, and monitor data jobs in cloud environments (AWS or Azure) ensuring reliability and performance.
- Collaborate with data scientists, analysts, and product teams to deliver data products that drive digital banking initiatives.
- Establish best practices for data security, governance, and cost‑effective cloud resource utilization.
Requirements
- 5+ years of hands‑on experience building data pipelines with Databricks and Apache Spark.
- Proficiency in Python for data transformation and orchestration.
- Strong knowledge of cloud platforms (AWS or Azure) and experience deploying scalable data solutions.
- Experience with streaming technologies such as Kafka or equivalent.
- Solid understanding of data modeling, ETL processes, and performance optimization.
Skills
databricksapache sparkpython