Ferguson told The Register in an interview that while Fireworks AI, a managed inference provider, supports on-prem AI, "we're trying to push people more and more away from it because frankly, it's just the economics." The GPU hardware is just so expensive that if you can't keep it fully utilized, it's a waste of money, he contends.
Ten thousand GPUs, he said, represent a billion dollars of capital. "You don't want those things to be sitting idle," he said.
For companies with particularly sensitive workloads, Ferguson said, Fireworks is happy to work something out – it offers a service called Sovereign for just that purpose and it has partners like Microsoft that can help. But most inference work probably doesn't need that level of protection, he said.
"The state of the art for everyone is to have a portfolio of models," he explained. "In fact, I think most people don't know that the best model in the world's not Astra.
The best model in the world's not Fable. The best model is literally a portfolio and we can prove it.
