remote
Data Solutions Engineer - PowerSecure Inc.
Software Engineer
Designs, builds, and operates scalable Azure data pipelines using Azure Data Factory, Databricks (PySpark/Spark SQL) and Delta Lake to deliver reliable, self‑service analytics and BI solutions.
About the role
Key Responsibilities
- Develop and maintain batch and streaming data pipelines on Azure using Data Factory, Databricks, and Delta Lake.
- Implement ELT processes with PySpark and Spark SQL to transform raw data into analytics‑ready datasets.
- Design and manage Delta Live Tables for automated data quality checks and incremental processing.
- Collaborate with BI developers, data architects, and business stakeholders to define data requirements and ensure timely delivery.
- Monitor pipeline performance, troubleshoot issues, and provide operational support for production workloads.
Requirements
- 2+ years of hands‑on data engineering experience, preferably on Azure.
- Proficiency with Azure Data Factory, Databricks, PySpark, Spark SQL, and Delta Lake technologies.
- Experience building ELT workflows and implementing data quality frameworks.
- Strong problem‑solving skills and ability to work cross‑functionally with analytics teams.
- Solid understanding of cloud‑based data storage, orchestration, and security best practices.