Senior Data Engineer Databricks 26DE02ES

As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems using Apache Spark and other big data technologies. You will work closely with cross-functional teams to develop and implement data governance policies and ensure data quality and integrity. You will also be responsible for implementing CI/CD pipelines to automate data processing and deployment. Experience with cloud-based data platforms, such as AWS, is highly desirable. The ideal candidate will have a strong background in software engineering and a passion for working with big data technologies.

Key Responsibilities:

• Design and develop large-scale data processing systems using Apache Spark and other big data technologies.
• Collaborate with cross-functional teams to develop and implement data governance policies.
• Implement CI/CD pipelines to automate data processing and deployment.
• Ensure data quality and integrity by implementing data validation and quality control processes.
• Work closely with data scientists and analysts to understand business requirements and develop data solutions.

Requirements:

• 5+ years of experience in software engineering with a focus on big data technologies.
• Strong background in Apache Spark and other big data technologies.
• Experience with cloud-based data platforms, such as AWS.
• Strong understanding of data governance policies and data quality control processes.
• Excellent communication and collaboration skills.