remote
Senior Data Engineer - General Dynamics Information Technology
Data Engineer
Lead the design, development, and maintenance of enterprise‑wide data pipelines and lakehouse solutions using Python, Spark, and AWS, enabling advanced analytics and AI for defense missions.
About the role
Key Responsibilities
- Architect and implement scalable data pipelines in Python and Apache Spark to ingest, transform, and store mission‑critical data across AWS services.
- Design and maintain data lake and lakehouse structures, ensuring high availability, security, and compliance with DoD standards.
- Collaborate with data scientists, analysts, and cyber operations teams to deliver actionable insights and support AI/ML initiatives.
- Optimize query performance and resource utilization for large‑scale analytics workloads.
- Document data models, pipeline logic, and best practices for cross‑team knowledge sharing.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python, SQL, and Spark.
- Proven expertise in AWS data services (S3, Glue, Redshift, Athena, Lake Formation).
- Solid understanding of data lake architecture, ETL processes, and data governance.
- Experience with security and compliance frameworks relevant to defense and federal data.
- Excellent problem‑solving skills and ability to work in a fast‑paced, mission‑critical environment.
Skills
pythonsqlapache sparkaws