onsite
Senior Data Engineer - DiversiTech Corporation
Data Engineer
Design, build, and maintain a scalable data platform that powers analytics, data science, and AI initiatives, ensuring secure, reliable data flow across enterprise systems.
About the role
Key Responsibilities
- Architect, develop, and operate a modern data platform using cloud services (AWS) and big‑data technologies (Spark, Redshift, S3).
- Design and implement robust ETL/ELT pipelines with Apache Airflow, ensuring data quality, lineage, and governance.
- Collaborate with analytics, data science, and application teams to translate business requirements into scalable data solutions.
- Optimize data storage, query performance, and cost efficiency across relational and distributed data stores.
- Implement security best practices, access controls, and monitoring to protect data assets.
Requirements
- 5+ years of hands‑on experience building data pipelines and platforms in a cloud environment.
- Strong programming skills in Python and advanced SQL for data transformation and modeling.
- Proficiency with Apache Spark (or similar distributed processing frameworks) and workflow orchestration tools such as Airflow.
- Deep understanding of AWS services (S3, Redshift, Glue, Lambda) and data‑warehouse design patterns.
- Experience with data governance, security, and performance tuning in large‑scale environments.
Skills
pythonsqlapache sparkawsairflow