onsite
Data Engineer Intermediate or Senior - Teacher Retirement System of Texas
Data Engineer
Intermediate to senior Data Engineer building scalable data pipelines, designing data models, and optimizing analytics workloads using Python, SQL, Spark, Airflow, and AWS services.
About the role
Key Responsibilities
- Design, develop, and maintain robust ETL pipelines to ingest, transform, and load data from diverse sources into enterprise data warehouses.
- Collaborate with data scientists, analysts, and business stakeholders to define data requirements and deliver high‑quality, reproducible datasets.
- Implement data modeling best practices, ensuring optimal schema design for performance and scalability.
- Leverage AWS services (S3, Redshift, Glue, EMR) and Spark to process large volumes of data efficiently.
- Automate workflow orchestration with Apache Airflow, monitor job health, and troubleshoot failures.
- Document data lineage, metadata, and data quality metrics to support governance and compliance.
Requirements
- 3+ years of experience as a Data Engineer or similar role.
Skills
pythonsqlawsapache spark