onsite
Data Engineer II, Identity Security & Abuse Prevention - Amazon
Data Engineer
Build and maintain scalable data pipelines for identity security, turning complex behavioral data into actionable intelligence to detect and prevent abuse and unauthorized access.
About the role
Key Responsibilities
- Design, develop, and operate robust data pipelines that ingest, transform, and store large volumes of identity and authentication logs.
- Implement ETL processes using Python, SQL, and Apache Spark to enable real‑time security analytics and investigations.
- Collaborate with security analysts and engineers to translate detection requirements into data models and dashboards.
- Maintain and optimize data infrastructure on AWS, ensuring high availability, scalability, and cost efficiency.
- Monitor pipeline health, troubleshoot data quality issues, and continuously improve performance.
Requirements
- 2+ years of experience building data pipelines and ETL solutions in a cloud environment.
- Proficiency in Python, SQL, and Spark for large‑scale data processing.
- Hands‑on experience with AWS services such as S3, Redshift, Glue, and Lambda.
- Strong understanding of security data, identity logs, and abuse detection concepts.
- Ability to work cross‑functionally, communicate technical ideas clearly, and prioritize tasks in a fast‑paced environment.
Skills
pythonsqlawsapache spark