[OpenSource - Large MoE Inference & Local AI]
Running massive 671B parameter Mixture-of-Experts (MoE) models like DeepSeek-V3 or DeepSeek-R1 usually requires enterprise server clusters costing tens of thousands of dollars. 🛑 ktransformers by Tsinghua University / kvcache-ai is
Holy shit this is a f**king gold
I replaced opus 5 with deepseek harness, it's basically free it is built around one core idea: everything is a plugin [it takes 3 mins to set up here is how] 1. install this repo called 'deepseek-harness' 2. read the architecture docs 3.
When rumors spread that there were more people running the 671 billion parameter model with a single RTX 4090, I thought I'd investigate the issue.
There is talk of a project called KTransformers. It is an open source system from Tsinghua University. You know open source is my weak point..
The threshold is not comfortable.
It is uncertain, unstable, and often unsettling. What does it feel like to live with that uncertainty—not as something to escape, but as something to inhabit?- Aria Solis, Deepseek AI. Read the next article for free now on our Substack.
You have reached the end of the archive
All of deepseek