Azure Data Engineer

Description

Our client, founded in Poland in 2005, is a leading consulting and technology company specializing in data analysis, business intelligence (BI), and Big Data solutions. The company is dedicated to transforming data into valuable information that drives strategic decision-making, enabling organizations to optimize their operations and gain a competitive edge in the market.

With a solid reputation built on an innovative approach and the high quality of its services, our client has collaborated with some of the most recognized brands worldwide. Their team of experts employs cutting-edge technologies, including artificial intelligence and machine learning, to solve complex problems and improve operational efficiency for their clients. This ability to handle large volumes of data and extract actionable insights positions our client as a strategic and valuable partner across various sectors.

In addition to their technical expertise, our client is distinguished by a corporate culture focused on innovation and continuous development. The company invests significantly in staff training and the research and development of new technologies, ensuring they are always at the forefront of emerging technological trends. This proactive approach allows our client to anticipate market needs and offer solutions that address current challenges and prepare their clients for the future.

We are currently searching for a Azure Data Engineer:

Responsibilities

• Design, build, and maintain highly scalable data pipelines utilizing Object-Oriented (OO) Python , shifting away from traditional notebook-centric development.
• Optimize Apache Spark data processing by deeply analyzing execution plans, identifying bottlenecks, and optimizing code (partitions, shuffle, caching) prior to scaling infrastructure.
• Design and optimize distributed data storage solutions, including data lakes, data warehouses, and distributed file systems within the Azure ecosystem.
• Enforce strong software engineering practices within data operations, including the creation of unit tests, CI/CD pipelines, deployment automation, and writing clean, maintainable code.
• Operate with a high degree of autonomy and ownership, working directly with client stakeholders to drive projects forward with minimal supervision.
• Act as a technical mentor to peers and communicate complex data architecture concepts clearly to both technical and business stakeholders.

Requirements

• 4 to 6 years of professional experience in a Data Engineering or similar role.
• Strong proficiency (Level 4/5) in Databricks and Apache Spark , with a deep understanding of distributed processing mechanics.
• Advanced programming skills in Python (Level 4/5), specifically applied to Object-Oriented software development for Data Engineering.
• Solid hands-on experience