onsite
Senior Data Engineer - OPUS Technologies
Data Engineer
Senior Data Engineer responsible for designing, building, and optimizing large‑scale data pipelines and warehouses using SQL, Python, Spark, and cloud services.
About the role
Key Responsibilities
- Design, develop, and maintain high‑performance data pipelines and ETL processes for real‑time and batch workloads.
- Architect and optimize relational and columnar databases, ensuring data quality, scalability, and security.
- Implement data models and schemas that support analytics, reporting, and machine‑learning use cases.
- Collaborate with cross‑functional teams to integrate streaming platforms (e.g., Kafka) and cloud services (AWS) into the data ecosystem.
- Monitor, troubleshoot, and tune data workflows to meet SLAs and cost‑efficiency targets.
Requirements
- 5+ years of experience in data engineering, with deep expertise in advanced SQL and relational/NoSQL databases.
- Proficiency in Python and big‑data frameworks such as Apache Spark.
- Hands‑on experience with streaming technologies (Kafka) and cloud platforms (AWS, including S3, Redshift, Glue).
- Strong understanding of data modeling, warehousing concepts, and ETL best practices.
- Ability to work autonomously in an agile environment and communicate technical solutions to stakeholders.
Skills
sqlpythonapache sparkkafkaaws