Zuck: Everyone Can Run AI at Home, All They Need Is $30,000+ Computers
Zuck has clearly lost touch with the average consumer. Meta announced that everyone can now run Meta’s Spark open model on their own Mac hardware…all you need is, wait for it, 2 maxxed out linked Mac Studios ($15K+ each).
For rich hobbyists, this might be a cool announcement and a viable option. For most people, on-device inference isn’t a viable option…yet. But it’s close.
Fanboys, don’t start about quantization and running Qwen or Gemma on your old Dell laptop. Most consumers won’t do that either.
Make no mistake, on-device inference is coming for consumer grade hardware. It’s just 6-12 months away. There are a few important unlocks:
- Most LLMs are dense MoEs. I know a lab releasing a frontier sparse MoE soonish that requires far less memory and can run much faster. More will follow this architecture path.
- Hardware upgrade cycles need to catch up so the baseline of consumer hardware is capable.
- Harnesses need to bake the LLM directly into their harness on-device to maximize system efficiency and continual learning.
Meta’s announcement is cool (and a great showcase of how powerful Apple silicon is), but it’s not really viable today. It is a sign of things to come. On-device inference is the future.