remote
Senior Data Engineer - SRS Acquiom
Data Engineer
Lead end‑to‑end data engineering for complex M&A and loan transaction platforms, building scalable pipelines on AWS, optimizing SQL and Spark workloads, and designing robust data models to support real‑time analytics.
About the role
Key Responsibilities
- Design, develop, and maintain large‑scale data pipelines using Python, SQL, and Apache Spark on AWS.
- Implement data ingestion, transformation, and loading (ETL) processes for M&A and loan transaction datasets.
- Collaborate with data scientists and product teams to define data models and schema that support analytics and reporting.
- Optimize query performance and ensure data quality through automated testing and monitoring.
- Manage data storage solutions (Redshift, S3, Glue) and implement security best practices.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python and SQL.
- Proficiency with AWS services (Redshift, S3, Glue, EMR) and experience building Spark jobs.
- Solid understanding of data modeling, ETL design, and performance tuning.
- Experience with version control (Git), CI/CD pipelines, and containerization (Docker).
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonsqlawsapache spark