remote
Senior Software Engineer - Agentic Applications - Databricks
Software Engineer
Lead the design and implementation of agentic data applications, leveraging Python, Scala, and Spark on the Databricks platform to deliver scalable, AI‑driven solutions across cloud environments.
About the role
Key Responsibilities
- Architect and develop high‑performance data pipelines and agentic services using Python, Scala, and Apache Spark on Databricks.
- Integrate Delta Lake and MLflow to ensure data reliability, reproducibility, and model lifecycle management.
- Collaborate with cross‑functional teams to translate business requirements into scalable, cloud‑native solutions.
- Optimize performance and cost across AWS and Kubernetes clusters, implementing best practices for resource utilization.
- Mentor junior engineers, conduct code reviews, and champion engineering excellence.
Requirements
- 5+ years of software engineering experience in data‑centric environments.
- Proficiency in Python, Scala, and Spark, with hands‑on experience in Delta Lake and MLflow.
- Strong understanding of cloud infrastructure (AWS) and container orchestration (Kubernetes).
- Excellent problem‑solving skills and a passion for building scalable, maintainable systems.
- Effective communication and collaboration skills in a fast‑paced, distributed team.
Skills
pythonscalaapache sparkmlflowkubernetesaws