onsite
Senior Data Platform Engineer - Karthik Consulting, LLC
Data Engineer
Lead the design, build, and secure operation of enterprise data platforms on AWS, leveraging Python, SQL, and Spark to deliver scalable, compliant data pipelines for government clients.
About the role
Key Responsibilities
- Architect and maintain large‑scale data lake and warehouse solutions on AWS, ensuring high availability and performance.
- Develop and optimize ETL pipelines using Python, SQL, and Apache Spark to ingest, transform, and load data from diverse sources.
- Implement robust security controls, including encryption, access management, and compliance with government security standards.
- Collaborate with data scientists, analysts, and stakeholders to define data models, governance policies, and data quality metrics.
- Monitor, troubleshoot, and tune data platform components, providing proactive incident response and root cause analysis.
Requirements
- 5+ years of experience building enterprise data platforms in cloud environments, preferably AWS.
- Strong proficiency in Python, SQL, and Spark for data engineering tasks.
- Hands‑on experience with data lake architectures (Lake Formation, S3, Glue) and data warehouse services (Redshift, Snowflake).
- Deep understanding of security best practices, encryption, IAM, and compliance frameworks (FedRAMP, NIST).
- Excellent problem‑solving skills, strong communication, and ability to work in a fast‑paced, mission‑critical environment.
Skills
pythonsqlawsapache spark