onsite
Software Engineer, Data Platform - Profound
Software Engineer
Build and scale data pipelines and analytics platforms using Python, SQL, Spark, and AWS services to enable real‑time AI‑driven insights for enterprise customers.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines that ingest, transform, and store large volumes of structured and unstructured data.
- Implement robust ETL/ELT workflows using Apache Spark and Airflow to support real‑time and batch processing needs.
- Collaborate with data scientists, product managers, and engineers to deliver reliable data services that power AI‑driven analytics and reporting.
- Optimize data storage and query performance on AWS services such as S3, Redshift, and Athena.
- Ensure data quality, reliability, and security through testing, monitoring, and best‑practice governance.
Requirements
- 3+ years of professional experience building data platforms or large‑scale data pipelines.
- Proficiency in Python and SQL, with hands‑on experience in Apache Spark.
- Strong knowledge of AWS data services (S3, Redshift, Athena, Lambda) and infrastructure‑as‑code tools.
- Experience with workflow orchestration tools such as Airflow or similar.
- Ability to work cross‑functionally, communicate technical concepts clearly, and troubleshoot complex data issues.
Skills
pythonsqlapache sparkawsairflow