remote
Data Engineering, Management & Governance Specialist - Accenture
Software Engineer
Lead end‑to‑end data pipelines using Apache NiFi and Starburst, ensuring robust ETL/ELT workflows, data governance, and real‑time processing for critical operational systems.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end ETL/ELT pipelines for batch and real‑time data processing across diverse source and target systems.
- Build and optimize complex data flows in Apache NiFi, leveraging processors, process groups, templates, controller services, and parameter contexts.
- Implement data transformation, enrichment, validation, routing, and error‑handling mechanisms to support high‑availability operational solutions.
- Collaborate with cross‑functional teams to integrate Starburst for analytical workloads and ensure seamless data access.
- Provide production support, troubleshoot performance issues, and continuously improve pipeline reliability and scalability.
Requirements
- Proven experience with Apache NiFi and Starburst in a production environment.
- Strong background in ETL/ELT design, data modeling, and data governance principles.
- Hands‑on expertise in real‑time data processing, data transformation, and error‑handling strategies.
- Excellent problem‑solving skills and ability to work independently in a fast‑paced setting.
- Effective communication skills for collaborating with technical and non‑technical stakeholders.