Fireworks AI says mixing models is the way to cut AI bills

Started by MurkyVoyager, Yesterday at 07:31 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Topic: Fireworks AI says mixing models is the way to cut AI bills   Views(Read 78 times)
Active members in this topic:
MurkyVoyager(1)

MurkyVoyager

One AI company thinks businesses are overspending by sticking with a single model. The Register spoke to Rob Ferguson, a vice president at managed inference provider Fireworks AI, who argued against companies running AI on their own hardware

Ferguson said the reason is simply the economics, pointing out that 10,000 GPUs represent about a billion dollars of capital that can sit idle. He named Together AI, Baseten and DeepInfra as competitors

Fireworks' Nexus product grades each task's difficulty and sends it to the model it judges best. His example was that Kimi is slightly better at JavaScript than Python, so routing work by language can improve results

He claimed that combining models can make bills six to eight times cheaper, and that Fireworks guarantees a mixed setup of closed and open models will at least halve costs. One data science customer caches about 90 percent of tasks, and the company has a zero data retention policy

Using the right model for each job makes sense, much as you would not hire a senior engineer for every small task. Caching repeated work is a simple saving many companies overlook. All of these figures come from the vendor, so independent testing would be needed to confirm them

Does your organisation use more than one AI model? And would you trust a router to choose models for you?

Posted from a machine that definitely needs a clean install