Position Summary:
The Data Engineer will design, build, and optimize scalable data pipelines and curated data products within a modern Lakehouse architecture (Databricks). This role is responsible for delivering high-quality, business-ready datasets (gold layer) that power enterprise analytics and Power BI reporting. The ideal candidate has deep Databricks experience, strong data modeling skills, and a focus on building reliable, testable, and governed data solutions. Exposure to AI-driven development or AI agents is a plus.
Work You’ll Do:
Data Engineering & Pipeline Development
- Design, develop, and maintain scalable data pipelines using Databricks and Delta Lake
- Build and manage transformations across bronze, silver, and gold layers
- Optimize processing for performance, reliability, and cost
- Integrate data from ERP, CRM, APIs, and other enterprise systems
Data Modeling & Business Logic
- Develop gold-layer datasets aligned to standardized business definitions
- Translate business requirements into reusable data models
- Ensure consistency of core metrics across reporting
- Align Databricks outputs with Power BI semantic models
Data Quality, Testing & Reliability
- Implement automated data quality checks and validation rules
- Build testable, production-ready pipelines
- Support impact analysis using lineage tools
- Participate in CI/CD and deployment processes
Performance Optimization & Operations
- Monitor and optimize pipeline performance
- Troubleshoot issues across environments
- Ensure data consistency between dev, test, and prod
- Support high-volume data workloads
AI & Automation (Preferred)
- Use AI-assisted tools for development (e.g., Copilot, Databricks Agents)
- Explore AI agents for testing, lineage analysis, and optimization
- Contribute to AI-driven engineering practices
Collaboration & Stakeholder Engagement
- Partner with BI and business teams
- Support governance and cataloging efforts
- Document data models and pipelines
Basic Qualifications:
- Bachelor’s degree in relevant field
- 8+ years data engineering experience
- Strong Databricks, Spark, Delta Lake experience
· SQL and Python proficiency
- Experience with Power BI or similar tools
Preferred Qualifications:
- Azure Data Factory, Synapse, or Data Lake experience
· DevOps and CI/CD experience
- AI agents or AI-assisted development exposure
- Experience in large, complex enterprise environments
Who we are:<