onsite
Databricks in AWS SME/Architect - HoonarTek
Software Engineer
Lead the design and implementation of scalable data pipelines on Databricks within AWS, driving architecture, performance, and best practices for large‑scale analytics and machine learning workloads.
About the role
Key Responsibilities
- Architect and deliver end‑to‑end data solutions on Databricks, leveraging AWS services such as S3, Glue, Redshift, and EMR.
- Design and optimize Spark jobs in Python, Scala, or Java to process terabytes of data with high performance.
- Collaborate with data scientists, engineers, and product teams to translate business requirements into robust data pipelines and dashboards.
- Implement security, governance, and monitoring best practices, including IAM, Lake Formation, and Databricks Unity Catalog.
- Mentor junior team members and conduct knowledge‑sharing sessions on Databricks and AWS best practices.
Requirements
- 5+ years of experience in data engineering or analytics architecture, with deep expertise in Databricks and AWS.
- Proficient in Spark programming (Python/Scala/Java) and SQL.
- Strong understanding of cloud data architecture, ETL/ELT patterns, and data lakehouse concepts.
- Experience with CI/CD for data pipelines and automated testing.
- Excellent communication skills and a collaborative mindset.
Skills
databricksawsapache sparkpython