WE ARE HIRING: Engineer – Data Engineering
Location: Sri Lanka Employment Type: Contract Company Industry: Software
Role Overview
We are seeking a motivated and technically skilled Engineer – Data Engineering to design, develop, and maintain scalable enterprise data solutions and cloud-based data-processing applications.
The selected candidate will be responsible for building reliable data pipelines, integrating multiple source systems, processing large volumes of data, and supporting solutions throughout the complete software-development lifecycle. The role requires strong programming skills, hands-on cloud experience, and the ability to work effectively within an Agile and DevOps environment.
Key Responsibilities
- Design and develop scalable enterprise-grade data-processing solutions.
- Build distributed and highly available data applications that support large-scale business requirements.
- Design, develop, test, and maintain data pipelines connecting multiple source systems and target platforms.
- Develop ETL and ELT workflows for batch and near-real-time data processing.
- Process, transform, and validate large volumes of structured and unstructured data.
- Develop cloud-native data solutions using Google Cloud Platform services.
- Build data-processing and orchestration workflows using tools such as Airflow and Cloud Composer.
- Work with services such as BigQuery, Dataflow, Dataproc, Datastream, Pub/Sub, Cloud Functions, and Cloud Run.
- Support data-processing solutions using AWS services such as Glue, Lambda, EMR, and Data Pipeline when required.
- Develop data-processing applications using Python, Shell scripting, and SQL.
- Design and optimize solutions using relational databases, NoSQL databases, and distributed storage engines.
- Support streaming-data applications using technologies such as Kafka, Pub/Sub, Spark, or similar tools.
- Contribute to the development of data warehouses, data marts, data lakes, and data-mesh solutions.
- Apply software-engineering best practices to ensure clean, reusable, maintainable, and high-quality code.
- Participate in code reviews, technical design discussions, and solution-architecture activities.
- Implement and maintain Continuous Integration and Continuous Delivery pipelines.
- Use code-management and automation tools such as GitHub, GitLab, Jenkins, or equivalent platforms.
- Follow DevOps principles throughout development, testing, deployment, and production support.
- Monitor application health, data-pipeline performance, and system reliability.
- Investigate and resolve data-processing failures, performance issues, and production incidents.
- Use monitoring platforms such as Datadog or equivalent tools when required.
- Collaborate with software engineers, data engineers, qua