DeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAI
DeepSeek, a Chinese AI startup, has recently launched a groundbreaking large language model known as DeepSeek-V3-0324. This 641-gigabyte model has made waves in the AI industry due to its capabilities and unique deployment strategy. What sets this model apart is its MIT license, allowing for commercial use, and reports suggesting it can run on consumer-grade hardware like Apple’s Mac Studio with the M3 Ultra chip.
The release of DeepSeek-V3-0324 was characterized by a stealthy launch strategy, with no whitepaper or marketing push accompanying its debut on Hugging Face. Early testers have reported significant improvements over previous versions, with AI researcher Xeophon praising its performance as surpassing other models like Claude Sonnet 3.5. The fact that DeepSeek-V3-0324’s weights are freely available for download further sets it apart from its competitors.
The model’s architecture, which includes a mixture-of-experts (MoE) approach, Multi-Head Latent Attention (MLA), and Multi-Token Prediction (MTP), enhances efficiency and output speed. This breakthrough technology allows the model to achieve high performance while reducing computational demands significantly. Additionally, a 4-bit quantized version of the model reduces the storage footprint, enabling it to run on consumer hardware like the Mac Studio.
DeepSeek’s open-source approach challenges the closed-garden model typically seen in Western AI companies. By making their models freely available, Chinese AI companies like DeepSeek are transforming the AI ecosystem in China and accelerating innovation in the field. This shift towards open-source AI models has also been observed in other Chinese tech giants like Baidu, Alibaba, and Tencent.
The release of DeepSeek-V3-0324 hints at the upcoming DeepSeek-R2 model, which is expected to focus on reasoning capabilities. This open-source reasoning model could democratize access to advanced AI systems and challenge models like GPT-5 from OpenAI. DeepSeek’s open-source strategy is redefining the global AI landscape, narrowing the gap between China and the US in AI capabilities and promoting broader access to advanced technology.
In conclusion, DeepSeek’s quiet release of DeepSeek-V3-0324 signals a new era in AI development, where the sharing of technology plays a crucial role in shaping the future of artificial intelligence. As this model becomes more widely available, the competition in the AI space will no longer be solely about creating the most powerful AI but about empowering more people to innovate with AI. DeepSeek’s approach may ultimately determine the influence AI has on reshaping our world.
