M4 Ultra Mac Pro, starting at $7,999. With 192GB unified memory, the total approaches $10,000.
What can it run? Llama 3.1 70B, Qwen 2.5 72B, Mixtral 8x22B — quantized versions of these models. Inference speed depends on quantization precision: ~15-20 tok/s for 4-bit, ~8-12 tok/s for 8-bit.
Is it enough? For daily chat and simple code generation, yes. For scenarios needing high precision — legal contract analysis, medical Q&A, financial data processing — no. Quantization doesn't lose a few percentage points of accuracy; it loses the model's reliability in long-tail scenarios.
If You Only Count Money, API Wins
A post on HN ran the numbers: $10,000 Mac, if all spent on OpenRouter, how many model calls is that?
At current OpenRouter pricing, Llama 3.1 70B costs ~$0.3/million input tokens and ~$0.5/million output tokens. A medium-complexity conversation consumes about 5,000 tokens (input + output), costing ~$0.004. $10,000 buys about 2.5 million calls.
Assuming a developer runs 100 inferences daily (heavy usage), that's 36,500 per year. $10,000 covers nearly 70 years.
Purely mathematically, the economics of running models locally don't make sense.
But You Can't Just Count Money
Three factors that pure math can't capture.
Data privacy. If your work involves customer data, internal code, trade secrets — can you send this data to the cloud? Many companies' compliance departments will flatly say no. In these cases, local inference isn't an economic choice; it's the only choice.
Latency and offline availability. APIs have network latency, typically 1-3 seconds. Local inference can do sub-second. And local doesn't depend on the network — on a plane, in a weak signal area, when disconnected, your AI tools still work.
Mental accounting. This is a behavioral economics concept: when marginal cost is zero (the model is already running on your machine), your usage frequency increases significantly. Every API call has a visible price tag, and this "I'm spending money every time" psychological signal inhibits exploratory usage.
My own workflow is an example. Since getting an M2 Max, my local inference usage is 5x what it was with APIs. Not because local is faster or better, but because the "it's free anyway" mentality makes me much more willing to experiment with prompts, models, and scenarios.
So How to Choose
If you care about data privacy, need offline use, or do heavy exploratory usage — run locally.
If you want the strongest model capabilities, don't want to manage infrastructure, and have moderate usage — use APIs.
If you want both the strongest model and privacy — that's indeed a tough problem. The current optimal solution is probably hybrid: daily exploration on local models, critical tasks on cloud's strongest models.
But is this $10,000 Mac actually worth it? If you're a developer who works with AI daily, it's not just a tool — it's a workbench. You don't calculate workbench investment by usage count.
Primary sources:
- Hacker News Discussion — William Angel's original analysis
- OpenRouter Pricing
- Apple M4 Ultra Technical Specifications