remote
Senior Security Data Engineer - Obsidian Security
Data Engineer
Senior Security Data Engineer building scalable data pipelines and analytics on AWS to detect and mitigate SaaS security threats, leveraging Python, SQL, and advanced data engineering practices.
About the role
Key Responsibilities
- Design, develop, and maintain large-scale data pipelines ingesting security telemetry from Microsoft 365, Salesforce, and other SaaS platforms.
- Implement data models and transformations in Python and SQL to enable real‑time threat detection and incident response.
- Deploy and manage data infrastructure on AWS (S3, Redshift, Glue, Lambda) ensuring high availability and cost efficiency.
- Collaborate with security analysts to translate threat intelligence into actionable data products and dashboards.
- Optimize query performance and storage costs through partitioning, indexing, and materialized views.
Requirements
- 5+ years of experience in data engineering with a focus on security or compliance data.
- Proficiency in Python, SQL, and AWS data services (Glue, Redshift, Athena, Lambda).
- Strong understanding of SaaS security concepts and threat detection workflows.
- Experience with CI/CD pipelines, version control, and automated testing for data solutions.
- Excellent problem‑solving skills and ability to communicate complex technical concepts to cross‑functional teams.