remoteonsite
Z2 Azure Data Engineer with DevOps - NTT DATA
Data Engineer
We're looking for a Data Engineer focused on designing and building scalable technical solutions. This mid level role requires 9+ years of relevant experience.
About the role
- Lead the end-to-end architecture and execution of BI FSL 2.0 transformation from SAP BW/BW4HANA to neXtollo/Azure Databricks.
- Translate existing SAP BW data models, transformations, process chains, extractors, and reporting logic into scalable Databricks Lakehouse patterns.
- Define the target-state data architecture covering ingestion, bronze/silver/gold layers, semantic consumption, Power BI integration, and reusable data products.
- Design SAP-to-Azure extraction and integration patterns using relevant approaches such as ODP, SLT, BW Bridge, APIs, CDC, or custom ETL frameworks.
- Drive fit-gap analysis between existing SAP BW capabilities and future-state neXtollo architecture.
- Establish architecture standards, design principles, reusable engineering patterns, and governance guidelines for the transformation program.
- Collaborate with business stakeholders, SAP functional teams, platform teams, data engineers, Power BI teams, and program leadership to ensure aligned execution.
- Design, build, and optimize enterprise-scale data pipelines using Azure Databricks, PySpark, Delta Lake, Azure Data Factory, and ADLS Gen2.
- Implement CI/CD pipelines for notebooks, workflows, data pipelines, infrastructure, configuration, and deployment across development, test, and production environments.
- Automate infrastructure provisioning and environment setup using Infrastructure as Code, preferably Terraform or equivalent Azure-native tooling.
- Define release management, branching strategy, code review process, deployment gates, rollback approach, and environment promotion standards.
- Implement observability, monitoring, alerting, logging, data quality checks, reconciliation, and operational dashboards for production-grade data platforms.
- Embed DevSecOps practices including secure secret handling, service principals, managed identities, RBAC, Key Vault integration, policy compliance, and audit readiness.
- Troubleshoot migration defects, pipeline failures, data mismatches, performance bottlenecks, and production issues with structured root cause analysis.
- Provide technical leadership to data engineers, developers, analysts, and external partners throughout the migration lifecycle.
- Own architecture decisions and ensure alignment with enterprise cloud, security, data governance, and platform standards.
- Support roadmap planning, sprint execution, dependency management, technical risk mitigation, and delivery governance.
- Create solution documents, architecture decision records, migration playbooks, runbooks, and design review material for leadership and engineering teams.
- Act as a bridge between business outcomes and technical implementation, ensuring the platform enables reliable, faster, and future-ready analytics.
- Azure Databricks and Databricks Workflows