Data Engineer - Apache Kafka / NiFi - Atlas Technologies, Inc.
Data Engineer
Data Engineer focused on designing, building, and maintaining Kafka and NiFi pipelines to ingest, transform, and store data for DoD and NIWC applications.
About the role
Key Responsibilities
Design, develop, and maintain scalable ETL pipelines using Apache Kafka and NiFi to ingest data from diverse sources.
Collaborate with data scientists and analysts to define data models, schemas, and transformation logic.
Implement data quality checks, monitoring, and alerting to ensure pipeline reliability and performance.
Optimize data flow and storage solutions for high-volume, low-latency requirements in a defense environment.
Document architecture, processes, and best practices for future maintenance and onboarding.
Requirements
3+ years of experience in data engineering with hands‑on Kafka and NiFi.
Strong proficiency in Python and SQL for data transformation and querying.
Experience designing and deploying production‑grade ETL pipelines and data pipelines.
Knowledge of data governance, security, and compliance in regulated environments.
Excellent problem‑solving skills and ability to work independently in a hybrid setting.