262K context. On a 16GB RTX 5070 Ti.
🤯 Qwen 3.8 27B Q3 hits ~25 tok/s while an adaptive llama.cpp fork streams KV cache between RAM ↔ VRAM. Stock llama.cpp starts thrashing around ~120K context. Same consumer GPU. 2x+ the usable context. This could be huge for local LLMs.
The underlying Qwen 3.8 Max model is huge
2.4 trillion total parameters. But only around 95 billion are activated for an individual task. That mixture-of-experts approach lets it pull in the relevant parts of the model instead of activating everything.
Perplexity Portable Computer runs the whole AI agent on your own machine.
Not a chatbot. The full agent that browses, runs tools, reads your files, and finishes tasks. All of it, local. The model, the orchestrator, the tools, the file access. Your data doesn't leave your
The surprising part is not the 27B model - it is the hardware claim
Qwen 3.8 27B reportedly running locally on an 8GB RTX 4060 via Unsloth IQ4_XS, with 64k context, 14.6GB on disk, ~150 tok/s prefill, and ~5 tok/s decode with native MTP.
You have reached the end of the archive
All of qwen38