onsite
Senior Data Scientist - DTEL Engineering & Consultants Inc
Data Scientist
Senior Data Scientist skilled in designing scalable ETL/ELT pipelines, leveraging Databricks, Spark, SQL and AWS to build lakehouse architectures and enable AI/ML, GenAI and Retrieval‑Augmented Generation workloads.
About the role
Key Responsibilities
- Design, develop, and maintain high‑performance ETL/ELT pipelines for both structured and unstructured data using Databricks, Apache Spark, and SQL.
- Implement and optimize lakehouse (medallion) architecture layers—bronze, silver, and gold—on AWS cloud services.
- Establish data quality, validation, lineage, and monitoring frameworks to ensure reliable data delivery.
- Collaborate with AI/ML teams to provide clean, feature‑rich datasets for model training, GenAI, and Retrieval‑Augmented Generation (RAG) use cases.
- Continuously evaluate and adopt emerging data processing technologies to improve scalability and cost efficiency.
Requirements
- 5+ years of hands‑on experience in data engineering and data science, with a strong focus on ETL/ELT pipeline development.
- Proficiency in Databricks, Apache Spark, SQL, and AWS services (e.g., S3, Glue, Redshift, Lambda).
- Demonstrated ability to design lakehouse/medallion architectures and manage data quality and monitoring processes.
- Experience supporting AI/ML, GenAI, or RAG workloads, including feature engineering and data preparation.
- Strong problem‑solving skills and ability to work cross‑functionally in fast‑paced environments.
Skills
databricksapache sparksqlawsmachine learning