Tags / data engineering

Posts tagged "data engineering"

3 posts found

Nov 26, 2025
Migrating a Dagster Pipeline from Pandas to Polars and Delta Lake
Why and how I'm migrating a production ETL pipeline from Pandas/SQLite to Polars/Delta Lake with a medallion architecture.
- healthcare-dag
- Dagster
- Polars
- Delta Lake
- Pandas
- data engineering
- medallion architect…
Nov 26, 2025
How I Went from Zero Tests to Comprehensive Dagster Test Coverage
The journey from no tests to data-driven testing with pytest fixtures, mock resources, and proper asset validation.
- healthcare-dag
- Dagster
- testing
- pytest
- Python
- test-driven develop…
- data engineering
Nov 26, 2025
Testing Dagster Assets Without Real Databases: The Mock Resource Pattern That Actually Works
How I learned to test Dagster assets properly using mock resources and pytest fixtures, moving from brittle tests to reliable ones.
- healthcare-dag
- Dagster
- testing
- Python
- pytest
- Delta Lake
- data engineering