remote
Senior Data Engineer Microsoft Fabric
Data Engineer
Senior Data Engineer responsible for designing, building, and optimizing data pipelines on Microsoft Fabric, leveraging Apache Airflow, Spark, and Azure AI services to deliver scalable analytics solutions.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines on Microsoft Fabric, ensuring high performance and reliability.
- Orchestrate workflow automation using Apache Airflow, integrating data ingestion, transformation, and loading processes.
- Implement large‑scale data processing jobs with Apache Spark, optimizing Spark code for speed and cost efficiency.
- Utilize Azure AI services and AI Studio to embed machine‑learning models and advanced analytics into data workflows.
- Collaborate with data scientists, analysts, and product teams to translate business requirements into robust data solutions.
- Monitor, troubleshoot, and continuously improve data pipeline health, implementing best practices for data governance and security.
Requirements
- 5+ years of professional experience in data engineering, with a strong focus on cloud‑based platforms.
- Proficiency in Microsoft Fabric, Apache Airflow, and Apache Spark.
- Hands‑on experience with Azure AI services, AI Studio, and Python for data processing and model integration.
- Solid understanding of relational and NoSQL databases, data modeling, and ETL/ELT patterns.
- Excellent problem‑solving skills, ability to work independently, and strong communication abilities.