remote
Senior Data Engineer - Sand Tech Holdings Limited
Data Engineer
Lead end‑to‑end data engineering for large‑scale AI projects, building robust pipelines on AWS, optimizing Spark workloads, and designing data models that power critical infrastructure solutions.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Python, SQL, and Apache Spark on AWS.
- Implement data ingestion, transformation, and quality checks for high‑volume, real‑time datasets.
- Collaborate with data scientists and product teams to deliver reliable data assets for AI models.
- Optimize performance and cost of data workflows through architecture tuning and resource management.
- Document data models, pipeline logic, and best practices for cross‑functional teams.
Requirements
- 5+ years of experience in data engineering, with strong Python and SQL skills.
- Proficiency in Spark (PySpark) and experience building large‑scale ETL pipelines.
- Hands‑on experience with AWS services (S3, Redshift, EMR, Glue, Athena).
- Solid understanding of data modeling, schema design, and data warehousing concepts.
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonsqlapache sparkaws