remote
Lead Data Engineer - Wells Fargo
Data Engineer
Lead Data Engineer responsible for designing, building, and optimizing large‑scale data pipelines and lake architectures using Python, Spark, and AWS services, while mentoring a team and ensuring high performance and reliability.
About the role
Key Responsibilities
- Architect and develop scalable data pipelines and lake solutions using Python, Spark, and AWS services (Glue, Redshift, S3).
- Lead a small team of data engineers, providing mentorship, code reviews, and best‑practice guidance.
- Design and enforce data modeling standards, ensuring data quality, lineage, and governance across the enterprise.
- Collaborate with data scientists, analysts, and product owners to translate business requirements into robust ETL workflows.
- Monitor, troubleshoot, and optimize pipeline performance, implementing cost‑effective solutions.
Requirements
- 5+ years of experience in data engineering with a strong focus on large‑scale batch and streaming pipelines.
- Solid understanding of data modeling, lakehouse architecture, and data governance principles.
- Excellent communication skills and proven ability to lead and mentor junior engineers.
- Experience with CI/CD for data pipelines and monitoring tools (Datadog, CloudWatch) is a plus.
Skills
pythonsqlapache sparkaws