onsite
Staff Data Engineer - LVT (LiveView Technologies)
Data Engineer
Lead the design, implementation, and scaling of data pipelines and analytics platforms using Python, SQL, AWS, and Spark to empower AI-driven insights across physical sites.
About the role
Key Responsibilities
- Architect and maintain end‑to‑end data pipelines that ingest, transform, and store large volumes of sensor and operational data.
- Collaborate with data scientists and product teams to define data models, schemas, and quality metrics for AI and analytics workloads.
- Optimize performance and cost of cloud‑based data services (AWS S3, Redshift, Athena, Glue) and on‑prem Hadoop clusters.
- Implement CI/CD for data workflows using Airflow, Git, and automated testing frameworks.
- Mentor junior engineers, conduct code reviews, and promote best practices in data engineering.
Requirements
- 10+ years of experience in data engineering, with a proven track record of building scalable, production‑grade pipelines.
- Expertise in Python, SQL, and distributed processing frameworks such as Apache Spark.
- Deep knowledge of AWS data services (S3, Redshift, Athena, Glue, EMR) and experience with on‑prem Hadoop ecosystems.
- Strong understanding of data modeling, ETL design, and data quality assurance.
- Excellent communication skills and ability to work cross‑functionally in a fast‑paced environment.
Skills
pythonsqlawsapache sparkairflow