onsite
Market Data Engineer - BIPB
Data Engineer
Design and maintain high‑performance market data pipelines, integrating real‑time feeds with cloud platforms using Python, SQL, Kafka and AWS to deliver reliable, low‑latency data for analytics and trading applications.
About the role
Key Responsibilities
- Develop, test, and deploy scalable data ingestion pipelines for real‑time market feeds using Python and Kafka.
- Design and optimise data models and SQL queries for high‑throughput storage and retrieval in cloud data warehouses.
- Implement and maintain cloud infrastructure on AWS (S3, Kinesis, Lambda, Redshift) to ensure low‑latency, fault‑tolerant data delivery.
- Monitor pipeline performance, troubleshoot latency or data quality issues, and implement automated alerting.
- Collaborate with quantitative analysts and trading teams to translate business requirements into robust data solutions.
Requirements
- 3+ years of experience building market data or financial data pipelines.
- Proficiency in Python programming and SQL for data transformation and analysis.
- Hands‑on experience with Kafka (or similar streaming platforms) and AWS services.
- Strong understanding of data modelling, time‑series data, and low‑latency system design.
- Ability to work in an agile environment, communicate technical concepts clearly, and deliver production‑ready code.