remote
Data Engineer AI/ML & MLOps - Bright Sol
Data Engineer
Senior Data Engineer focused on AI/ML and MLOps, building scalable data pipelines on Databricks, leveraging Python, Spark, and AWS to deliver production‑ready machine learning workflows.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines using Databricks and Apache Spark to support AI/ML workloads.
- Implement MLOps practices, including model versioning, CI/CD, and automated deployment to production environments.
- Collaborate with data scientists and ML engineers to translate model requirements into scalable data solutions.
- Optimize data storage and processing using SQL, Delta Lake, and cloud services (AWS).
- Monitor pipeline performance, troubleshoot issues, and continuously improve data quality and reliability.
Requirements
- 5+ years of experience in data engineering with a strong focus on AI/ML and MLOps.
- Hands‑on experience with MLOps tools (MLflow, Kubeflow, or similar) and CI/CD pipelines.
- Solid understanding of cloud platforms, preferably AWS, and experience with data lake architecture.
- Excellent problem‑solving skills and ability to work independently in a remote setting.
Skills
pythondatabricksmlopsapache sparksqlaws