remote
Data Engineering Lead - LSEG (London Stock Exchange Group)
Software Engineer
Lead a high‑performing data engineering team delivering real‑time, cloud‑native data pipelines for a global CCP, leveraging Python, Spark, Kafka and AWS to support risk and analytics platforms.
About the role
Key Responsibilities
- Design, build, and maintain scalable, low‑latency data pipelines that ingest, process, and distribute market and risk data across the organization.
- Lead a team of data engineers, providing technical mentorship, setting coding standards, and driving best practices in CI/CD and automated testing.
- Collaborate with product owners, traders, and risk analysts to translate business requirements into robust data solutions and data models.
- Own the end‑to‑end data platform on AWS, including data lake, streaming services (Kafka), and compute clusters (EMR, Spark).
- Ensure data quality, governance, and security compliance across all data assets.
Requirements
- 5+ years of hands‑on experience in data engineering, with at least 2 years in a lead or managerial role.
- Strong proficiency in Python, SQL, and big‑data frameworks such as Apache Spark.
- Deep experience with streaming technologies (Kafka) and cloud platforms, preferably AWS (S3, Redshift, EMR, Lambda).
- Proven ability to design data models and ETL processes for high‑volume, low‑latency financial data.
- Excellent communication skills and a track record of delivering complex data solutions in regulated, fast‑paced environments.
Skills
pythonsqlapache sparkkafkaaws