onsite
Staff Data Engineer - MissionWired
Data Engineer
Lead the design and implementation of scalable data pipelines and analytics platforms, leveraging Python, Spark, Airflow, and cloud services to enable data‑driven fundraising insights.
About the role
Key Responsibilities
- Architect, build, and maintain high‑performance data pipelines that ingest, transform, and store large volumes of fundraising and engagement data.
- Develop and optimize SQL and Spark jobs for real‑time and batch processing, ensuring data quality and reliability.
- Design and implement workflow orchestration using Airflow, managing dependencies, scheduling, and monitoring across multiple environments.
- Collaborate with data scientists, analysts, and product teams to translate business requirements into robust data models and analytical solutions.
- Implement best practices for cloud infrastructure on AWS and BigQuery, including security, cost optimization, and scalability.
Requirements
- 5+ years of professional experience building data pipelines and analytics platforms in a fast‑moving environment.
- Strong proficiency in Python, SQL, and Apache Spark for large‑scale data processing.
- Hands‑on experience with workflow orchestration tools such as Airflow.
- Deep knowledge of cloud services, particularly AWS (S3, Redshift, Lambda) and Google BigQuery.
- Demonstrated ability to design data models, ensure data integrity, and mentor junior engineers.
Skills
pythonsqlapache sparkairflowaws