onsite
Senior Data Engineer - Senior Manager - State Street
Data Engineer
Lead the design, development, and optimization of cloud‑based data pipelines, driving performance, reliability, and data quality across AWS, Snowflake, Databricks, and Redshift environments.
About the role
Key Responsibilities
- Architect, build, and maintain end‑to‑end data pipelines on AWS, Snowflake, Databricks, and Redshift, ensuring scalability and high availability.
- Develop robust ingestion, transformation, and validation frameworks using Python, SQL, and Shell scripting.
- Optimize pipeline performance through profiling, indexing, and query tuning, while enforcing data quality and governance standards.
- Collaborate with data scientists, analysts, and business stakeholders to translate requirements into efficient data solutions.
- Lead testing, deployment, and monitoring of pipelines, implementing CI/CD practices and automated alerting.
Requirements
- 8+ years of experience in data engineering, ETL, and data warehousing.
- Proven expertise with AWS services, Snowflake, Databricks, and Redshift.
- Strong programming skills in Python, SQL, and Shell scripting.
- Experience designing data models, optimizing queries, and ensuring data quality.
- Excellent communication and leadership abilities, with a track record of mentoring junior engineers.
Skills
pythonsqlawssnowflakedatabricks