remote
Python ,Databricks, AWs- Lead II - Software Engineering - UST
Software Engineer
Lead Software Engineer with 7‑9 years of experience building scalable, cloud‑native backend services in Python on AWS, designing REST APIs and managing Databricks pipelines for multi‑tenant data integration with HBase.
About the role
Key Responsibilities
- Design, develop, and maintain high‑performance backend services in Python on AWS, ensuring scalability and reliability.
- Build and expose RESTful APIs for data integration and platform services, following best practices for security and versioning.
- Develop, schedule, and monitor data pipelines and workloads using Databricks, optimizing performance and cost.
- Implement and support the HONI connector for Databricks‑to‑HBase integration, ensuring data consistency and fault tolerance.
- Architect and manage multi‑tenant configurations, observability, and monitoring across the Enrolment Team’s services.
Requirements
- 7‑9 years of professional experience in Python development and AWS cloud services.
- Proven expertise in building REST APIs and managing Databricks pipelines.
- Strong knowledge of HBase and data integration patterns.
- Experience with observability tools (e.g., CloudWatch, Prometheus) and multi‑tenant architecture design.
- Excellent problem‑solving skills and a collaborative mindset.
Skills
pythonawsdatabricksrest api