onsite
Senior Data Engineer - Advertising Engineering
Data Engineer
Senior Data Engineer specializing in advertising platforms, designing and scaling data pipelines on AWS and Azure using Apache Airflow, Spark, and modern CI/CD practices.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines for advertising data ingestion, transformation, and delivery.
- Leverage Apache Airflow for orchestrating complex workflows and ensure reliable scheduling across cloud environments.
- Implement high‑performance data processing using Apache Spark on both AWS and Azure platforms.
- Build and maintain CI/CD pipelines to automate testing, deployment, and monitoring of data engineering code.
- Collaborate with analytics, product, and engineering teams to translate business requirements into robust data solutions.
- Monitor, troubleshoot, and optimize data workflows for cost efficiency and low latency.
Requirements
- 5+ years of professional experience in data engineering, preferably in advertising or digital media.
- Strong expertise with AWS services (e.g., S3, Redshift, EMR) and Azure data offerings.
- Hands‑on experience building pipelines with Apache Airflow and processing large datasets using Apache Spark.
- Proficiency in implementing CI/CD pipelines using tools such as GitHub Actions, Jenkins, or Azure DevOps.
- Solid programming skills in Python or Scala and a deep understanding of data modeling, ETL best practices, and cloud security.
Skills
awsapache sparkazurecicd