onsite
Senior Cloud Data Engineer AWS - Hiflylabs
Data Engineer
Senior Cloud Data Engineer focused on designing and deploying scalable data solutions on AWS, leveraging Python, SQL, Docker, and CI/CD pipelines to automate and optimize data pipelines.
About the role
Responsibilities
Data Solution Design and Development:
- Design, develop, and implement complex and scalable data solutions on the AWS cloud platform.
- Design, build, and optimize data pipelines to ensure efficiency and reliability.
Programming & Scripting:
- Write clean, efficient, and maintainable SQL and Python code.
- Develop and maintain shell scripts to support automation.
DevOps & Automation:
- Utilize Docker containers for containerizing applications and data pipelines.
- Establish and maintain CI/CD pipelines (e.g., GitHub Actions, GitLab CI/CD) for rapid and reliable deployments.
- Automate data pipeline processes.
Cloud Data Services:
- Deliver solutions on AWS data services (e.g., S3, Redshift, Glue, EMR, Lambda).
- Design and implement data modeling and data warehousing solutions.
Project and Team Management:
- Own and manage the technical aspects of deliveries (at least at the component level).
- Participate in project planning, task estimation, testing, and liaising with business stakeholders.
- Provide technical leadership to smaller teams.
- Create technical documentation.
Requirements
Programming & Scripting
- Must have Proficiency in both SQL and at least one imperative programming language, particularly in Python, emphasizing clean, efficient, and maintainable code.
- Experience with shell scripting.
Devops & Automation:
- Familiarity with Docker and at least one CI/CD tool (e.g., Github Actions, GitLab CI/CD) to streamline deployment and manage data pipelines.
Cloud Data Services
- A minimum of 5 years of hands-on experience in data engineering, with a strong focus on cloud-based solutions.
- Has delivered at least one complex and scalable data solution on at least one cloud platform’s data services (Primary AWS data engineering skills required, GCP/Azure experience beneficial.)
- Experience in working with dbt or SQLMesh is a plus.
Soft and other skills
- Strong team player with a proactive mindset, eager to take on challenges and drive solutions.
- Great problem-solving skills, analytical thinking, and a structured approach to decision-making.
- Excellent written and verbal communication skills in English
Why us?
- Diverse projects : In each assignment there is always something new either on the technical or on the business side that helps you grow.
- Cutting edge technology : We work with many of the most up-to-date technologies and tools.
- Strong and motivating team : We stress the importance of working together in tight-k