remote
Data engineer - Induansh Private Limited
Data Engineer
Senior Data Engineer specializing in Databricks and PySpark, building scalable cloud‑based data pipelines and optimizing big data solutions for enterprise analytics.
About the role
Key Responsibilities
- Design, develop, and maintain high‑performance data pipelines using Databricks and PySpark.
- Implement ETL/ELT workflows to ingest, transform, and load large datasets into data warehouses.
- Collaborate with data scientists and analysts to ensure data quality, lineage, and accessibility.
- Optimize Spark jobs for cost, speed, and resource utilization on cloud platforms.
- Document architecture, processes, and best practices for future scalability.
Requirements
- 6–8 years of experience in data engineering, data warehousing, and big data solutions.
- Proficiency with Databricks, PySpark, and Apache Spark.
- Strong background in ETL/ELT design and implementation.
- Experience with cloud services (AWS, Azure, or GCP) and related data services.
- Excellent problem‑solving skills and ability to work independently in a remote environment.
Skills
databricksapache spark