onsite
Systems Integration & Data Pipeline Engineer - CACI International
Software Engineer
Engineer specialized in building secure, automated data pipelines and integration solutions, leveraging Python, SQL, AWS, Docker, and Airflow to support DoD data modernization and AI/ML initiatives.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines that ingest, transform, and load large, heterogeneous datasets.
- Implement secure, automated CI/CD workflows using Docker, Git, and orchestration tools to ensure repeatable deployments.
- Integrate disparate systems and services across cloud (AWS) and on‑prem environments, applying best‑in‑class security practices.
- Collaborate with data scientists and analysts to enable AI/ML model training and inference pipelines.
- Monitor pipeline performance, troubleshoot failures, and continuously optimize for scalability and reliability.
Requirements
- Active TS/SCI clearance.
- 5+ years of experience in data engineering, pipeline automation, and systems integration.
- Proficiency in Python, SQL, and Linux scripting.
- Hands‑on experience with AWS services (e.g., S3, Lambda, Glue) and containerization (Docker, Kubernetes).
- Familiarity with workflow orchestration tools such as Apache Airflow and CI/CD platforms (Jenkins, GitLab CI).
Skills
pythonsqlawsdockercicdlinux