onsite
Data Pipeline & Integration Engineer
Implementation Engineer
Design, build, and maintain scalable data pipelines on AWS, leveraging DBT for transformation while ensuring robust data governance, lineage tracking, and seamless API integrations.
About the role
Key Responsibilities
- Architect and implement end‑to‑end data pipelines on AWS services (e.g., S3, Redshift, Glue) to ingest, process, and store large‑scale datasets.
- Develop and maintain DBT models for reliable, version‑controlled data transformations and testing.
- Design, develop, and manage API integrations to bring external data sources into the data platform, ensuring data quality and security.
- Implement data governance frameworks, including metadata management, data cataloging, and lineage documentation.
- Monitor pipeline performance, troubleshoot failures, and continuously optimize for cost and latency.
Requirements
- 3+ years of experience building data pipelines on AWS with strong knowledge of services such as S3, Lambda, Glue, and Redshift.
- Proficiency in DBT for data modeling, testing, and documentation.
- Hands‑on experience with RESTful and SOAP API integration, authentication mechanisms, and data format handling (JSON, XML, CSV).
- Solid understanding of data governance principles, metadata management, and data lineage tracking tools.
- Strong SQL skills and familiarity with Python or similar scripting languages for data manipulation.