Senior Data Pipeline Engineer - Netbuilder

Key Responsibilities

• Architect, build and maintain large‑scale telemetry pipelines that ingest, transform and route observability data across cloud and on‑prem environments.
• Implement data ingestion solutions with Apache Kafka, stream processing with Apache Spark, and orchestration via Apache Airflow.
• Deploy and manage pipeline components on AWS using services such as S3, EMR, Lambda, and ECS, ensuring high availability and cost efficiency.
• Containerize services with Docker and orchestrate with Kubernetes or ECS, applying best practices for CI/CD and automated testing.
• Collaborate with cross‑functional teams to define data quality, security, and compliance requirements, and enforce them through monitoring and alerting.

Requirements

• 5+ years of experience designing and operating data pipelines in production environments.
• Proficiency in Python, Spark, Airflow, Kafka, and AWS services.
• Hands‑on experience with Docker, Kubernetes, and Terraform for infrastructure as code.
• Strong understanding of observability concepts, metrics, logs, and tracing.
• Excellent communication skills and ability to work independently in client‑facing roles.