Back to Work
Making Everything Fast and Cheap
A pipeline redesign that delivered major cost and latency wins by fixing architecture, scaling, and query strategy.
The problem
The pipeline was slow, expensive, and operationally awkward. The underlying issue was architecture: always-on compute, inefficient loading, and unnecessary work.
The goal was not micro-optimizations — it was structural improvement.
What I built / changed
- Redesigned the pipeline stack end-to-end.
- Parallelized data loaders.
- Moved from App Service Plans (always-on, expensive) to Container Apps (scales to zero, pay only when running).
- Optimized queries with lazy evaluation to fetch less data.
- Tuned memory allocation settings.
- Upgraded from a resource-constrained B3 dev plan to appropriate specs.
Result
86% cost reduction and 88% latency improvement.
Stack / concepts
PolarsContainer AppsLazy evaluationInfrastructure
Interested in discussing engineering challenges?
Get in touch