onsite
Engineer II, Data - CarMax
Software Engineer
Data Engineer II responsible for designing, building, and optimizing scalable data pipelines using Python, SQL, Airflow, and Spark on AWS, ensuring data quality, governance, and security for analytics and business users.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines that ingest, transform, and load data from diverse sources into data warehouses and lakehouse environments.
- Leverage Airflow for orchestration, ensuring reliable scheduling, monitoring, and alerting of ETL workflows.
- Implement data quality checks, lineage tracking, and governance controls to meet compliance and security standards.
- Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver curated datasets that enable faster insights.
- Optimize pipeline performance using Spark, SQL, and AWS services (Glue, Redshift, S3) to reduce latency and cost.
Requirements
- 3+ years of experience building production data pipelines in a cloud environment.
Skills
pythonsqlairflowapache sparkaws