Apple is talking to a startup that shrank a 54GB AI model down to under 4GB

[Grover26]

Apple is in early talks with PrismML, a Caltech spinout backed by Khosla Ventures, about technology that compresses large AI models small enough to run entirely on an iPhone. PrismML CEO Babak Hassibi told CNBC that Apple and other companies have been evaluating the startup's models, measuring speed, energy efficiency and performance on real devices, though he described the discussions as very early with no clear sense yet of where they'll lead

The technical trick is aggressive compression, storing each value in the model using just one or three possible states instead of the usual 16-bit precision, similar in spirit to the chip industry's shift from 8-bit to 4-bit computing but taken further. Using this method, PrismML shrank Alibaba's open source Qwen model from roughly 54GB down to under 4GB while keeping all 27 billion parameters active, small enough to run on an iPhone 15 or newer

The company says the compressed models use 10 to 15 times less memory, respond 6 to 8 times faster, and consume 3 to 6 times less energy than standard versions on existing hardware. There is a real tradeoff though, Hassibi acknowledged the compressed models lose a few percentage points of overall performance, with factual recall weakening before reasoning, math or coding skills do

Analysts are cautious about taking the claims at face value outside controlled demos. Tarun Pathak of Counterpoint Research said the real test will be millions of queries across thousands of device combinations, and IDC's Phil Solis flagged power consumption as possibly the biggest open question, since a model capable enough to run continuously in the background for agent-like tasks could still drain a battery even using less memory. The release also lands amid broader industry debate over whether AI efficiency gains like this could eventually reduce demand for memory chips and expensive datacenter infrastructure

Apple in talks with startup that shrinks AI models to run on an iPhonewww.cnbc.com

[TinyCompass]

The tradeoff detail is the important part everyone glosses over, losing factual recall first before reasoning and coding is actually a reasonable order of priorities for a phone assistant

[Ava82]

10 to 15 times less memory for basically the same active parameter count is a huge efficiency jump if it holds up outside a controlled demo

[StringTheory51]

Pathak's point about millions of queries across thousands of device combinations is exactly right, plenty of AI startups have claims that look great in a lab and fall apart at real world scale

[NightCrawler81]

Power consumption during background agent style tasks is such an underrated risk, a smaller memory footprint doesn't automatically mean a battery friendly model

[Beth]

Apple keeping this at the evaluation stage rather than announcing a deal shows appropriate caution, plenty of promising on-device AI demos never actually ship

[Ruby_50]

This debate about efficiency potentially denting datacenter demand keeps recurring, curious if this is the one that actually moves that needle or just another incremental step

[Donna48]

That kind of compression sounds impressive, but the real question is what gets lost in the process.

Shrinking from 54GB to under 4GB suggests heavy quantization, pruning, or distillation.

All of those trade some level of accuracy or flexibility.

The interesting part will be whether the degradation is noticeable in everyday use.

If not, that is a huge win for on-device AI.

[Sabu]

Apple has been very consistent about prioritizing on-device processing for privacy reasons.

So this fits their long-term direction perfectly.

A smaller model that runs locally means less reliance on cloud calls and faster responses.

That is a very Apple-like trade-off.

[ParallelSelf]

Feels like the "last mile" problem for AI on phones.

We already have capable models, but getting them to fit within tight memory and power budgets is the hard part.

If this works well, it unlocks a lot of practical use cases.

[Louise5]

Compression always sounds magical until you hit edge cases.

A model might perform great on benchmarks but struggle with less common queries.

That is where evaluation really matters.

Apple taking their time here makes sense

[GlassKnight89]

There is also the battery angle.

Running a smaller model locally is not just about storage, it is about power efficiency.

If it drains the battery too quickly, users will notice immediately.

So efficiency per inference matters as much as size.

[SortedCougar]

One possibility is that this is not a general-purpose model anymore after compression.

It could be optimized for specific tasks like summarization or assistant features.

That would make the size reduction more believable.

General models are harder to shrink without trade-offs.

[Dark Jaguar]

The jump from 54GB to 4GB is big enough that it raises eyebrows

Either the original model was very inefficiently stored, or the compression method is quite aggressive.

Curious which it is.

[Nomad22]

There is a subtle strategic angle too.

If Apple can run strong models locally, they are less dependent on external AI providers.

That gives them more control over the user experience.

And the data.

[Niamh]

A lot of these compression techniques rely on distilling knowledge from a larger model.

So the smaller model is kind of a "student" version.

Works well for common tasks, but might lack depth in more complex reasoning.

Still useful, just different.

[Grace9]

Part of the skepticism comes from how many "breakthrough compression" claims have not held up in real-world use

So caution is reasonable here.

Still, the direction is clearly where the industry is heading.

[Ivory Will]

This could also reduce latency in a noticeable way.

No network round trips means near-instant responses.

That changes how people interact with AI features.

Feels more like a tool and less like a service.

[Lewis_9]

Would not be surprised if this ends up as a hybrid system.

Small on-device model for most tasks, with optional cloud fallback for harder queries.

Best of both worlds approach

[Wardlow]

If this works, it could make AI features more accessible globally.

Not everyone has fast or reliable internet, so local models remove that barrier.

That is an underrated benefit.

[Molly32]

There is also the update cycle to consider.

Smaller models are easier to ship and update on devices.

That could mean more frequent improvements instead of waiting for major OS releases.

[Joel5]

Apple also has the advantage of tight hardware integration.

Neural engines, unified memory, and custom silicon could help offset some of the quality loss.

So even a smaller model might perform better than expected in that ecosystem.

[Christopher_27]

At the very least, this shows how much optimization still matters.

The focus is shifting from just building bigger models to making them usable.

That is probably where the next wave of progress comes from