remote
Lead Data Platform Engineer - Digital Strategy LLC
Data Engineer
Lead a team to migrate legacy SQL Server/SSIS workloads to an AWS‑hosted Databricks platform, building scalable cloud‑native data pipelines and AI‑enabled analytics for mission‑critical federal programs.
About the role
Key Responsibilities
- Architect, design, and implement a cloud‑native data platform on AWS using Databricks and Apache Spark.
- Lead a cross‑functional engineering team through the migration of legacy SQL Server databases and SSIS packages to modern, scalable pipelines.
- Develop and operationalize data ingestion, transformation, and enrichment workflows with Python/Scala in Databricks notebooks.
- Collaborate with architects, analysts, and federal stakeholders to define data models, governance, and security standards.
- Implement AI‑enabled analytics solutions, ensuring performance, reliability, and cost‑effectiveness in a production environment.
Requirements
- 5+ years of hands‑on experience with AWS services (e.g., S3, Redshift, IAM) and Databricks.
- Deep expertise in SQL Server, SSIS, and modern data pipeline frameworks (Spark, Delta Lake).
- Proficiency in Python (or Scala) for data engineering and analytics.
- Demonstrated technical leadership, mentoring engineers and driving large‑scale migration projects.
- Strong understanding of data security, compliance, and federal IT standards.
Skills
awsdatabricksapache sparkpython