Nelnet Business Services (NBS), a division of Nelnet, Inc., provides payment technology and education services to more than 1,200 higher education institutions, nearly 12,000 K-12 schools, and millions of individual students, families, and supporters across the globe. Our culture of service enables us to form long-lasting and trusted partnerships, while our focus on creativity and innovative solutions empowers our customer communities to thrive.
Source-System Ingestion
- Build and maintain CDC and batch pipelines from source systems into the Snowflake Bronze layer using Redpanda, Python, and Dagster.
- Manage source-specific adapters across relational (SQL Server, Oracle), document (MongoDB), API, and file-based sources, and stand up new sources as the platform grows.
- Handle the complexity of consolidating single-tenant source databases into a multi-tenant Snowflake model.
- Implement idempotent load patterns with deduplication so pipelines can re-run safely without producing duplicate or missing records.
Bronze Layer Ownership
- Design and maintain raw snapshot tables that preserve source-system state with minimal transformation — the immutable audit trail back to source.
- Implement historical tracking so no data is silently overwritten.
- Handle hard deletes from source systems by tracking deletions in Snowflake rather than losing them.
- Maintain source-schema documentation and change detection — alert when upstream schemas drift unexpectedly.
Silver Build — Contribute and Grow (with the Analytics Engineer)
- Build conformed Silver models from the business rules, grain, and conforming logic the Analytics Engineer defines — starting with close guidance and taking on more independence over time.
- Own Bronze-to-Silver reconciliation: row counts and totals tie out, no records lost or double-counted across the transformation.
- Help build and automate the mechanical contract gates — reconciliation tie-out, freshness, and schema validation — as CI/CD checks.
- Apply classification tags at Bronze so columns can promote to Silver, partnering with the data owner and Data Governance on classification decisions.
- Handle incremental logic and performance tuning as models grow in volume, with support as needed.
Pipeline Reliability and Observability
- Run and monitor Dagster orchestration — scheduling, retries, failure alerting — and grow into fuller ownership of it, including SLA monitoring, over time.
- Implement data-quality checks at the ingestion boundary (row counts, schema validation, freshness checks).
- Monitor Snowflake concurrent-write limits and tune ingestion patterns to avoid bottlenecks.
- Participate in severity-1 incident response — failed file loads, latency cascades, duplicate transactions, and similar.
Platform Collaboration