onsite
Systems Integration Specialist - NTT DATA
Software Engineer
Lead integration of cloud data services, architecting pipelines with Azure Data Bricks, Data Factory, and Data Lake Storage, while leveraging PySpark and Airflow for scalable ETL workflows.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines using Azure Data Bricks, Azure Data Factory, and Azure Data Lake Storage.
- Implement and optimize PySpark jobs for large‑scale data processing.
- Orchestrate workflows with Airflow, ensuring reliability and monitoring.
- Collaborate with data scientists and business analysts to translate requirements into technical solutions.
- Document architecture, code, and best practices for future maintenance.
Requirements
- 5+ years of experience in data engineering and system integration.
- Proficiency with Azure cloud services (ADB, ADF, ADLS).
- Strong programming skills in Python and PySpark.
- Hands‑on experience with Airflow for workflow orchestration.
- Excellent problem‑solving and communication skills.