MIT's InstructMesh lets anyone fix AI-generated 3D models so they can be printed

[Reece88]

Researchers from MIT CSAIL, Google and Northeastern University have built a tool called InstructMesh that helps people fix AI generated 3D models before printing them. The problem it tackles is a real one. AI can now produce 3D models that look great on screen but cannot work in reality, like a mug that will not hold liquid or a shape that cannot be printed. Fixing them usually needs professional 3D modelling skills

InstructMesh combines Microsoft's TRELLIS 3D generation system with GPT-4's reasoning. You can prompt it to create a design, then refine specific parts using plain language or sliders, rather than manually editing the mesh. Lead researcher Faraz Faruqi explained that language models excel with text and images, while TRELLIS creates 3D models from extensive training data. Putting the two together gives you creativity with practical fixes

The examples are fun. The team made a mug with a dragon wrapped around it that still had a working handle, a whistle shaped like a shell, glasses with butterfly wings and even a knee brace with a denim texture. They also built a little vibrating robot called a bristle bot, dressed up to look like a shrimp. It shows the tool can handle both decorative and functional objects

In a study, the researchers found that 80 percent of AI generated models they looked at had structural problems that would stop them working. When novices used InstructMesh to find and fix flaws in models from Thingiverse, they succeeded about 90 percent of the time, as judged by experts. The senior author is Stefanie Mueller, an associate professor at MIT, and the work will be presented at the ACM UIST conference in November

I have a 3D printer and the gap between a good looking model and a printable one is real. Thin walls, overhangs and floating parts ruin plenty of prints. Would you use AI to design things for printing? And how much would a tool like this lower the barrier for beginners?

New tool lets users repair AI-generated 3D models, then fabricate them just the way they wantnews.mit.edu

[Penguin79]

Overhangs are my biggest problem with downloaded models. A tool that spots them and suggests fixes before printing would save a lot of plastic

Failed prints are expensive and annoying. Supports help, but they leave marks