remote
Data Engineer - General Motors (GM)
Data Engineer
Design, develop, and maintain Databricks‑based pipelines and medallion‑layer data products, integrating enterprise applications and supporting analytics, reporting, and AI use cases.
About the role
Key Responsibilities
- Architect and build scalable Databricks pipelines using Python and Apache Spark to ingest, transform, and deliver data across the organization.
- Develop and maintain medallion‑layer data products, ensuring data quality, lineage, and governance.
- Integrate enterprise applications with Databricks, leveraging DataStage and other data movement patterns.
- Collaborate with data scientists, analysts, and business stakeholders to support analytics, reporting, and AI initiatives.
- Implement robust monitoring, alerting, and troubleshooting processes for data pipelines.
Requirements
- Proven experience with Databricks, Python, and Apache Spark in a production environment.
- Strong SQL skills and experience with data modeling and ETL design.
- Hands‑on experience with DataStage or similar enterprise data integration tools.
- Solid understanding of data governance, security, and compliance best practices.
- Excellent communication skills and ability to work collaboratively in a hybrid setting.
Skills
databrickspythonapache sparksql