Backtesting the full 5 TB history, for two dollars a run
A quant desk runs the entire tick history nightly on a right-sized L4 for ~$2, instead of a 10% sample that hid tail risk.
The challenge
5 TB of tick history doesn't fit a laptop's disk, so the desk backtested on a 10% sample, quietly hiding tail risk. The instinct was to throw a big GPU at it.
Workload understanding
One saw a 5 TB working set (exceeds local disk → burst) that is IO-bound, not compute-bound, so raw GPU horsepower wouldn't help.
Best hardware for the job
Matched 1× NVIDIA L4, cheaper than the naive 2× T4 pick AND the H100 pick, because the bottleneck is data, not FLOPs.
Benchmark
| Pick | Hardware | Time | Cost | Verdict |
|---|---|---|---|---|
| undersized | 2× T4 | ~362 min | $4.22 | slower and pricier |
| One's match | 1× L4 | ~182 min | $2.12 | right tool for an IO-bound job |
| oversized | 1× H100 | ~18 min | $3.00 | a frontier GPU mostly idle |
Completion
Full-dataset backtest ran in the desk's own cloud, results returned, instance torn down.
Outcome
“The H100 felt right and was almost pure waste. One put it on an L4 and it cost two dollars.”
Representative composite story. Placement, hardware matching, and completion are real system behavior; figures are transparent, editable model inputs.