My goal is to unlock commodity inference.
Not just better local ai, but better ai on cheap commodity hardware... We have made a lot of progress, a P100 GPU (~$100). Running Qwen 3.8 27b at nearly 17 tok/s without speculation. Our goal is 25+ tok/s.
Second attempt at this edit
Made with Terresect. Qwen 3.8 27b on a Single RTX 6000
You have reached the end of the archive
All of qwen38