onsite
Data Engineer - NSD
Data Engineer
Senior Data Engineer responsible for designing, building, and maintaining robust data pipelines and ETL workflows using Apache Hadoop, SQL, NoSQL, and Python to support mission‑critical national security analytics.
About the role
Key Responsibilities
- Design, develop, and optimize scalable data pipelines for ingestion, transformation, and delivery of large datasets.
- Implement ETL processes using Apache Hadoop, Spark, and related tools to support analytics and AI initiatives.
- Integrate data from diverse sources, ensuring quality, consistency, and compliance with security standards.
- Collaborate with data scientists and analysts to provide reliable data feeds for machine learning models.
- Monitor pipeline performance, troubleshoot issues, and implement improvements for reliability and efficiency.
Requirements
- Proven experience in data engineering, pipeline development, and ETL design.
- Strong knowledge of Apache Hadoop ecosystem, SQL, and NoSQL databases.
- Proficiency in Python for data manipulation and automation.
- Experience with data integration tools and best practices for data quality.
- Active Enhanced DV Clearance (West) required.