Benchmarks we ran, mistakes we watch customers make, and the arithmetic behind hardware choices.
Weights are the number everyone quotes, but the KV cache is what actually pushes a long-context job off an 80 GB card. Here is the arithmetic.
Full post coming soonHopper and Ada advertise double throughput at FP8. We measured what survives contact with a real training loop.
Full post coming soonLoading 70B of weights from NVMe takes about a minute. At what traffic level does keeping a warm instance become the cheaper option?
Full post coming soonThe gap between SXM and PCIe cards is invisible on a single GPU and decisive across eight. Numbers from our own fleet.
Full post coming soonHourly rounding quietly adds 15-30% to a bursty workload. We did the modelling and decided not to charge for it.
Full post coming soonStart with the VRAM your model needs, not the card you have heard of. A short flowchart that gets most people to the right answer.
Full post coming soon