onsite
Senior Data Engineer - Globaldev Group
Data Engineer
Senior Data Engineer responsible for designing and scaling streaming and batch pipelines, implementing data quality controls, and performing feature engineering for an industrial anomaly detection solution using Python, Spark, Kafka, Airflow, and AWS.
About the role
We are looking for a Senior Data Engineer to join an innovative project for a prominent European renewable energy company. In this role, you will help design and scale a cross-business anomaly detection solution (PoC) for complex industrial data. We need a senior-level, independent contributor with a strong engineering mindset who thrives in cross-functional environments and enjoys translating complex data into smart, reusable models.
Responsibilities:
- Design and implement streaming and batch data pipelines for data integration, ingestion, data quality management (quality gating), transformations, and auditing.
- Perform feature engineering and data exploration.
- Collaborate with business and operations teams to align on requirements, and to request and standardize waveform/sensor data.
- Build and maintain the Gold data layer, key KPIs, and PoC evaluation frameworks.
- Set up and maintain solid engineering practices, including CI/CD pipelines, Infrastructure as Code (IaC), version control (Git), unit and integration testing, and proper documentation.
Requirements
- Strong, hands-on experience with Azure and the Databricks platform.
- Solid experience with MLflow for lifecycle management.
- Proven track record in setting up production-ready engineering practices (CI/CD, IaC, Git, unit/integration testing).
- Senior-level engineering mindset with the ability to work independently in a complex data environment.
- Excellent communication skills and fluent English.
Will be a plus:
- Experience with Snowflake .
- Additional Data Science / ML capabilities, especially with anomaly detection (real-time or streaming scenarios).
- Experience with Databricks-based ML workflows, model serving endpoints, model versioning, testing, and tuning.
- Knowledge of statistical normalization techniques (e.g., z-score) and signal processing (e.g., FFT transformation).
What We Offer
- Direct cooperation with the already successful and growing project.
- Truly competitive salary.
- 20 days of paid vacation per year.
- Paid Polish public holidays.
- Private medical insurance package.
- Opportunities for professional growth in an international environment and cutting-edge energy sector domain.
- Help and support from our caring HR team.
Originally posted on Himalayas