MIT's InstructMesh lets anyone fix AI-generated 3D models so they can be printed

[Reece88]

Researchers from MIT CSAIL, Google and Northeastern University have built a tool called InstructMesh that helps people fix AI generated 3D models before printing them. The problem it tackles is a real one. AI can now produce 3D models that look great on screen but cannot work in reality, like a mug that will not hold liquid or a shape that cannot be printed. Fixing them usually needs professional 3D modelling skills

InstructMesh combines Microsoft's TRELLIS 3D generation system with GPT-4's reasoning. You can prompt it to create a design, then refine specific parts using plain language or sliders, rather than manually editing the mesh. Lead researcher Faraz Faruqi explained that language models excel with text and images, while TRELLIS creates 3D models from extensive training data. Putting the two together gives you creativity with practical fixes

The examples are fun. The team made a mug with a dragon wrapped around it that still had a working handle, a whistle shaped like a shell, glasses with butterfly wings and even a knee brace with a denim texture. They also built a little vibrating robot called a bristle bot, dressed up to look like a shrimp. It shows the tool can handle both decorative and functional objects

In a study, the researchers found that 80 percent of AI generated models they looked at had structural problems that would stop them working. When novices used InstructMesh to find and fix flaws in models from Thingiverse, they succeeded about 90 percent of the time, as judged by experts. The senior author is Stefanie Mueller, an associate professor at MIT, and the work will be presented at the ACM UIST conference in November

I have a 3D printer and the gap between a good looking model and a printable one is real. Thin walls, overhangs and floating parts ruin plenty of prints. Would you use AI to design things for printing? And how much would a tool like this lower the barrier for beginners?

New tool lets users repair AI-generated 3D models, then fabricate them just the way they wantnews.mit.edu

[Penguin79]

Overhangs are my biggest problem with downloaded models. A tool that spots them and suggests fixes before printing would save a lot of plastic

Failed prints are expensive and annoying. Supports help, but they leave marks

[Renegade24]

UIST is a respected conference for this kind of work. Plenty of interface ideas from there end up in real products a few years later

Undo and touch interfaces both have roots in research like this. The talk should be worth watching when it goes online. I hope they release the code too

[Shaun5]

That dragon mug is a great demo. It is exactly the kind of thing people want to make but cannot design themselves

Decorative and functional at the same time. Gift makers on Etsy will love this

[PhotonBurst]

Being able to say make the handle thicker instead of editing vertices is huge for beginners. Mesh editing is a nightmare if you have never done it

I gave up on Blender after a weekend. Something like this would have kept me going. Plain language editing could open 3D printing to many more people

[ObserverEffect68]

Eighty percent of AI models having structural issues does not surprise me. Image based generators do not understand physics or printing. They just make something that looks right. Gravity is not something they learn from pictures