onsite
Data Engineer - LexisNexis Risk Solutions
Data Engineer
Design and maintain scalable data pipelines and models on AWS, using Python, SQL, Spark, and Airflow to enable reliable customer data foundations for analytics and marketing decisions.
About the role
Key Responsibilities
- Develop, test, and deploy robust ETL pipelines that ingest, transform, and load large‑scale customer data from multiple sources.
- Design data models and schemas that support analytics, reporting, and machine‑learning use cases.
- Implement workflow orchestration using Apache Airflow to ensure reliable, repeatable data processing.
- Leverage AWS services (e.g., S3, Redshift, Glue, Lambda) to build cost‑effective, scalable data solutions.
- Collaborate with data scientists, analysts, and product teams to understand requirements and translate them into technical specifications.
- Monitor pipeline performance, troubleshoot issues, and continuously optimize for speed and reliability.
Requirements
- 3+ years of hands‑on experience building data pipelines in Python and SQL.
- Proficiency with big‑data processing frameworks such as Apache Spark.
- Experience using workflow orchestration tools, preferably Apache Airflow.
- Strong knowledge of AWS data services (S3, Redshift, Glue, Lambda) and cloud‑based architecture.
- Solid understanding of data modeling, ETL best practices, and data‑quality assurance.
Skills
pythonsqlapache sparkairflowaws