onsite
Software Developer - Datametrics Software Systems
Software Engineer
Software Developer focused on cloud data architecture, migrating on‑prem DB2 pipelines to Amazon Redshift, building PySpark ingestion, and managing near real‑time data sync with Qlik Replicate on AWS.
About the role
Key Responsibilities
- Design and implement end‑to‑end data architecture on AWS, handling 350+ tables from on‑prem systems.
- Lead migration of enterprise data pipelines from DB2 to Amazon Redshift, enhancing scalability and performance.
- Build and maintain a PySpark‑based ingestion framework to extract data from DB2 and store it in optimized Parquet format on Amazon S3.
- Implement batch and near real‑time ingestion using Qlik Replicate with CDC for continuous data synchronization.
- Design multi‑layer architecture including Qlik Amazon Aurora for staging and batch control.
Requirements
- Proven experience with AWS services (Redshift, S3, Aurora).
- Strong proficiency in PySpark and data processing pipelines.
- Hands‑on experience migrating legacy DB2 databases to cloud platforms.
- Knowledge of Qlik Replicate and CDC concepts.
- Ability to design scalable, maintainable data architectures.