remoteonsite
DBT/SPARK - Persistent Systems
Software Engineer
Lead data engineering initiatives using DBT and Spark to build scalable data pipelines, transform raw data into actionable insights, and optimize performance across enterprise platforms.
About the role
Key Responsibilities
- Design, develop, and maintain data pipelines using DBT and Spark to ingest, transform, and load large datasets.
- Collaborate with data scientists and business stakeholders to translate analytical requirements into efficient ETL workflows.
- Optimize Spark jobs for performance, cost, and scalability on cloud platforms.
- Implement data quality checks, monitoring, and alerting to ensure pipeline reliability.
- Document data models, pipeline logic, and best practices for future maintenance.
Requirements
- 3+ years of experience in data engineering with a focus on Spark and DBT.
- Experience with version control, CI/CD for data pipelines.
- Excellent problem‑solving skills and ability to work in a fast‑paced environment.