Eight-V100 home AI server gains 1TB RAM and an RTX 3090 for DeepSeek

Started by SuperRoss43, Yesterday at 04:01 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Topic: Eight-V100 home AI server gains 1TB RAM and an RTX 3090 for DeepSeek   Views(Read 86 times)
Active members in this topic:
SuperRoss43(1)

SuperRoss43

A home AI server built from used data centre cards has had a major upgrade. Tom's Hardware reports that a builder has upgraded an eight V100 server, with 256GiB of VRAM, by adding 1TB of RAM and an RTX 3090 to run DeepSeek V4.1 Flash

Now 5,949 experts, or 38.7 percent of the model, run at 4 bit NVFP4 precision. Code speed is 150 to 180 tokens per second, down from 160 to 215, and two parallel requests give 159 tokens per second

Agreement with the reference model rose from 92.2 percent to 95.1 percent. Earlier crashes were probably caused by one bad power supply, and after changing the power setup, nine out of nine long chats ran cleanly

All the figures come from the builder. V100 support ends with Nvidia's R580 driver branch, so the cards will eventually lose driver updates

Building a powerful AI server from used V100s is a clever way to run big models at home without paying for new data centre cards. Tracking crashes down to a faulty power supply is a reminder that the basics matter in any build. Driver support ending will be the long term challenge

Would you build a home AI server from used data centre GPUs? And how important is running AI models locally to you?