remote
Data Platform Engineer II - Advisor360 Llc
Data Engineer
Senior Data Platform Engineer responsible for building scalable data pipelines, managing cloud infrastructure, and optimizing data workflows using Python, SQL, AWS, Spark, Airflow, and Docker.
About the role
Key Responsibilities
- Design, develop, and maintain large-scale data pipelines and ETL processes using Python and SQL.
- Implement and manage data workflows on AWS services (S3, Redshift, Glue, EMR) and orchestrate with Airflow.
- Optimize Spark jobs for performance and cost-efficiency, ensuring data quality and reliability.
- Collaborate with data scientists and analysts to provide clean, accessible datasets for modeling and reporting.
- Automate deployment and monitoring of data infrastructure using Docker, CI/CD pipelines, and CloudWatch.
Requirements
- 3+ years of experience in data engineering or related roles.
- Experience with workflow orchestration (Airflow) and containerization (Docker).
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonsqlawsapache sparkairflowdocker