onsite
Consultant Data Engineer
Data Engineer
Lead end‑to‑end data engineering projects on AWS and Azure, architecting scalable Big Data pipelines, automating deployments with CI/CD, and managing secure certificate lifecycles to ensure compliance and high availability.
About the role
Key Responsibilities
- Design, build, and maintain scalable data pipelines on AWS and Azure platforms, ensuring high throughput and low latency.
- Implement CI/CD workflows for data services, automating testing, deployment, and rollback processes.
- Manage certificate lifecycle across cloud services, ensuring secure communication and compliance with industry standards.
- Collaborate with data scientists and business analysts to translate requirements into robust data solutions.
- Monitor and optimize pipeline performance, troubleshoot issues, and implement best practices for data quality and governance.
Requirements
- Proven experience with AWS and Azure data services (e.g., Redshift, Synapse, Data Lake, Databricks).
- Strong background in Big Data technologies such as Spark, Hadoop, or similar frameworks.
- Hands‑on expertise in CI/CD tools (e.g., GitHub Actions, Azure DevOps, Jenkins) for data workflows.
- Experience managing TLS/SSL certificates and integrating them into cloud environments.
- Excellent problem‑solving skills and ability to work independently in a fast‑paced environment.