remote
Data Engineer, Quality Intelligence - Anduril
Data Engineer
Data Engineer focused on building scalable data pipelines for real‑time quality intelligence, leveraging Python, SQL, Airflow, AWS, and Spark to support AI and computer vision workloads.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest, transform, and deliver high‑quality data streams for AI and computer vision applications.
- Implement and optimize ETL workflows using Airflow, ensuring reliability, scalability, and low latency.
- Collaborate with data scientists and software engineers to integrate machine learning models into production pipelines.
- Leverage AWS services (S3, Redshift, Glue, Lambda) to build cloud‑native data solutions.
- Monitor pipeline performance, troubleshoot issues, and continuously improve data quality and processing efficiency.
Requirements
- 3+ years of experience in data engineering with a strong foundation in Python and SQL.
- Proficiency in building and orchestrating data pipelines using Airflow and Spark.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, Lambda).
- Solid understanding of data modeling, ETL best practices, and data quality principles.
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonsqlairflowawsmachine learning