onsite
Senior Data Engineer - www.velansys.com
Data Engineer
Senior Data Engineer leading large‑scale data migration projects using Python, Kafka, Spark, Snowflake, and Iceberg, with strong expertise in CI/CD, Kubernetes, and data modeling to deliver robust, scalable data pipelines.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines for large‑scale datastore migration projects.
- Implement real‑time streaming solutions with Kafka and batch processing with Apache Spark.
- Build and optimize Snowflake and Apache Iceberg data models for performance and cost efficiency.
- Collaborate with DevOps to automate CI/CD workflows and deploy services on Kubernetes.
- Ensure data quality, governance, and compliance across all data assets.
Requirements
- 6–12 years of experience in data engineering, with strong proficiency in Python or Java.
- Hands‑on experience with Kafka, Apache Spark, Snowflake, and Apache Iceberg.
- Solid knowledge of ANSI SQL, Hadoop ecosystem (HDFS/Hive), and data modeling concepts.
- Experience with CI/CD pipelines and Kubernetes orchestration.
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonkafkaapache sparksnowflakekubernetes