We are pleased to announce the release of zml/llmd 20261023.0.

While last week’s release (zml/llmd 20261015.0) was a minor release with a focus on performance, this week’s release builds on the gains we made and brings more speed, more models, and more fun.

And so this week, we bring new model series to zml/llmd.

Qwen/Qwen3.8-Flash-Next

This release brings the Qwen/Qwen3.8-Flash-Next series model series to zml/llmd.

We have tested it on NVIDIA and AMD GPU, the only ones we got access to that could host the official release. Google is still telling us there are no TPUs available.

Decode throughput, Qwen/Qwen3.8-Flash-Next, bs=16, zml/llmd 20261023.0 (tok/s)

XiaomiMiMo/MiMo-V2.6

This release brings the XiaomiMiMo/MiMo-V2.6 series in both RL and MOPD flavour.

Decode throughput, XiaomiMiMo/MiMo-V2.6-Flash-MOPD, bs=16, zml/llmd 20261023.0 (tok/s)

Faster DeepSeek-V4.1-Flash

The hill climb continues. We improved a number of fundamental building blocks and thus DeepSeek-V4.1-Flash got faster.

Token Generation, deepseek-ai/DeepSeek-V4.1-Flash, bs=1, 4xGB300 (tok/s)

Try it today

NVIDIA

docker run -p 8000:8000 --shm-size=256GB --gpus=all -e HF_TOKEN -it zmlai/llmd:cuda \
    --model=hf://Qwen/Qwen3.8-Flash-Next

AMD

docker run -p 8000:8000 --shm-size=256GB --device=/dev/kfd --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:rocm \
    --model=hf://Qwen/Qwen3.8-Flash-Next

Intel

docker run -p 8000:8000 --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:oneapi \
    --model=hf://Qwen/Qwen3.8-27B

Google TPU

docker run --net=host --privileged -e HF_TOKEN -it zmlai/llmd:tpu \
    --model=hf://Qwen/Qwen3.8-27B

Apple Metal

brew install zml/zml/llmd
llmd --model=hf://nvidia/Qwen3.8-27B-NVFP4 --batch-size=1