remote
Data engineer - bp
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and architectures using Python, SQL, Spark, Hadoop, and AWS to enable large‑scale data ingestion, processing, and analytics.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines and ETL processes using Python, SQL, and Spark.
- Build and optimize data storage solutions on Hadoop and AWS services (S3, Redshift, EMR).
- Collaborate with data scientists and analysts to ensure data quality, consistency, and accessibility.
- Implement data modeling best practices and maintain metadata catalogs.
- Monitor performance, troubleshoot issues, and continuously improve pipeline efficiency.
Requirements
- 3+ years of experience in data engineering or related field.
Skills
pythonsqlapache sparkhadoopaws