remote
Staff Software Engineer, Data - DC Tech Lead - Afresh
Software Engineer
Lead data platform engineering for a high‑scale AI grocery platform, driving architecture, performance, and reliability across distributed cloud services using Python, Java, Spark, AWS, and Kubernetes.
About the role
Key Responsibilities
- Architect, design, and implement large‑scale data pipelines and services that process billions of events daily.
- Lead a cross‑functional data engineering team, setting technical direction, coding standards, and best practices.
- Collaborate with product, ML, and operations teams to translate business requirements into robust, low‑latency data solutions.
- Optimize performance, cost, and reliability of cloud‑native infrastructure on AWS, leveraging Kubernetes, serverless, and container orchestration.
- Mentor senior engineers, conduct code reviews, and drive continuous improvement through automation and testing.
Requirements
- 10+ years of software engineering experience, with at least 5 years leading data‑intensive platforms.
- Strong proficiency in Python and Java, and deep experience with Apache Spark or similar distributed processing frameworks.
- Extensive hands‑on experience building, deploying, and scaling services on AWS (e.g., EMR, S3, Lambda, Redshift) and Kubernetes.
- Proven track record delivering production‑grade machine‑learning pipelines and data products.
- Excellent problem‑solving skills, ability to influence technical strategy, and a passion for mentoring engineers.
Skills
pythonjavaapache sparkawskubernetesmachine learning