资讯
Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s
📌 概要
I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. E
⚡ 关键要点
- ▸I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on
- ▸Comments URL: https://news.ycombinator.com/item?id=49524447
- ▸Points: 11
I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift.
It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next
Comments URL: https://news.ycombinator.com/item?id=49524447
Points: 11
# Comments: 0