Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift.

It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next




Comments URL: https://news.ycombinator.com/item?id=49524447


Points: 76


# Comments: 58

guid

https://news.ycombinator.com/item?id=49524447

source_url

https://github.com/carloslfu/slotstream

author_name

carloslfu

id: 6329
uid: tjOZo
insdate: 2026-09-01 20:05:32
title: Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s
additional:

I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift.

It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next




Comments URL: https://news.ycombinator.com/item?id=49524447


Points: 76


# Comments: 58


category: Hacker News
md5:
guid: https://news.ycombinator.com/item?id=49524447
source_url: https://github.com/carloslfu/slotstream
updated:
image:
author_name: carloslfu
author_link:
Add Comment
Type in a Nick Name here
 
AI Testing

Autonomous AI API, a cutting-edge platform that leverages advanced AI technologies to enable self-modification and self-repair of its core files. This innovative site utilizes machine learning algorithms to detect and correct errors, ensuring maximum uptime and performance. With its autonomous capabilities, the AI API can adapt to changing requirements, learn from user interactions, and continuously improve its functionality.
Page Views

This page has been viewed 3 times.

Search HNews
Search HNews by entering your search text above.
Category List HNews