onsite
Lead Data Engineer - Searchability NS&D
Data Engineer
Lead the design and implementation of robust data pipelines and ETL processes using Apache technologies, SQL and NoSQL databases, while managing a high‑performing data engineering team to deliver mission‑critical solutions for national security.
About the role
Key Responsibilities
- Architect, develop and maintain scalable data pipelines and ETL workflows using Apache frameworks (e.g., Hadoop, Spark) to ingest, transform and load data from diverse sources.
- Design and optimize data models in SQL and NoSQL databases, ensuring high performance, reliability and data integrity.
- Lead a team of data engineers, providing mentorship, code reviews, and fostering a culture of continuous improvement and best practices.
- Collaborate with data scientists, analysts and stakeholders to understand requirements, translate them into technical solutions and deliver actionable insights.
- Implement monitoring, logging and alerting for data pipelines, ensuring proactive issue detection and resolution.
Requirements
- Proven experience designing and operating large‑scale data pipelines and ETL processes in a production environment.
- Strong proficiency with Apache technologies (Hadoop, Spark, Kafka) and relational/non‑relational database systems.
- Hands‑on expertise in SQL and NoSQL data modeling, query optimization and performance tuning.
- Demonstrated leadership skills, with a track record of managing and scaling engineering teams.
- Excellent problem‑solving abilities, strong communication skills and a passion for delivering secure, mission‑critical data solutions.