remote
Senior Data Infrastructure Engineer - 10Beauty
Devops Engineer
Lead the design, implementation, and scaling of data pipelines and infrastructure for a robotics startup, leveraging Python, SQL, and cloud services to support real‑time analytics and machine learning workloads.
About the role
Key Responsibilities
- Architect and maintain scalable data pipelines that ingest, transform, and store large volumes of sensor and operational data from autonomous manicure robots.
- Design and manage cloud-based data infrastructure on AWS, including S3, Redshift, Glue, and Lambda, ensuring high availability and cost efficiency.
- Implement containerized services with Docker and orchestrate them using Kubernetes, automating deployments with Helm and CI/CD pipelines.
- Develop and schedule ETL workflows with Apache Airflow, monitoring job health and optimizing performance.
- Collaborate with data scientists and ML engineers to provide clean, reliable datasets for model training and inference.
- Enforce data governance, security best practices, and compliance with industry standards.
Requirements
- 5+ years of experience building production data pipelines and infrastructure.
- Proficiency in Python, SQL, and experience with Spark or similar big‑data frameworks.
- Hands‑on expertise with AWS services (S3, Redshift, Glue, Lambda, EMR).
- Strong knowledge of containerization (Docker) and orchestration (Kubernetes).
- Experience with Airflow or similar workflow orchestration tools.
Skills
pythonsqlawsdockerkubernetesairflow