We are pleased to announce the release of zml/llmd 2026.11.3. This week brings an embeddings API, support for decision models, and MXFP4/NVFP4 on Intel oneAPI. You’ll also find general bug fixes and improvements throughout.

A new versioning scheme

The version number looks a little different this week. Releases now use YYYY.MM.<revision>, with a revision number for each release within the month.

Embeddings API

zml/llmd now implements the embeddings endpoints through POST /v1/embeddings and POST /embeddings with initial support for the Qwen3-Embedding series. You can pass a single string or a list of strings and request a smaller output vector with dimensions when the model supports it.

curl http://localhost:8000/v1/embeddings \
    -H 'Content-Type: application/json' \
    -d '{"model":"Qwen/Qwen3-Embedding-0.6B","input":["First document", "Second document"], "dimensions": 32}'

Decision Models

This week, zml/llmd adds decision models via the /v1/systemone endpoint. Decision models are designed to answer multiple-choice questions and do it a lot faster than general-purpose LLMs. They are trained to reason about the choices and provide a single answer.

We have also implement a StartLux extension in which questions in a request can depend on one another, even when a dependency appears later in the list. The server schedules questions when their dependencies are complete and rejects cycles.

Here is a sample request/response:

curl -sS http://127.0.0.1:8000/v1/systemone \
    -H 'Content-Type: application/json' \
    -d '{
        "model": "startlux-models/StartLux-Decision-v1.1-0.8B-Instant",
        "state": "The animal sits on the mat, purrs and chases a red laser dot.",
        "questions": {
            "animal": {"type": "choice", "instructions": "Which animal is it?", "criteria": {"cat": null, "dog": "barks"}}
        }
    }'
{
  "answers": {
    "animal": {
      "type": "choice",
      "choice": "cat",
      "confidence": 0.9999995231628418,
      "probabilities": {
        "cat": 0.9999997615814209,
        "dog": 2.0826452384881122E-7
      }
    }
  },
  "usage": {
    "input_tokens": 54,
    "output_tokens": 0
  },
  "model": "startlux-models/StartLux-Decision-v1.1-0.8B-Instant"
}

Here’s a little demo of Google’s Diffusion Gemma playing Pac-Man via zml/llmd:

StartLux Decision Models

As a first release, we added support for the StartLux Decision models:

StartLux Decision models come in two variants. Instant answers in one pass. Autothink can spend more time reasoning and try again when the first decision is uncertain.

Here are the performance measurements vs vLLM and SGLang:

StartLux Decision 0.8B performance evaluation StartLux Decision 9B performance evaluation StartLux Decision 27B performance evaluation

Regular LLMs in decision mode

You can send decision requests to a general-purpose LLM you’re already serving through zml/llmd. The same checkpoint can handle chat requests and answer multiple-choice questions through /v1/systemone. Decision mode requests are batched with regular completion requests inside the same forward pass, so you can use the same model for both workloads if you choose to do so.

This release supports decision mode for Llama, Qwen3.5/3.6/3.8, Qwen 3.8 Flash Next, Gemma 4, LFM 3, and GLM 5.2. DeepSeek-V4.1 support is deferred pending a chat template fix.

DeepSeek-V4.1 Data Parallelism

DeepSeek-V4.1 gains support for --data-parallel. As with Llama, this lets you distribute requests and their KV caches across groups of devices.

Intel oneAPI MXFP4/NVFP4 support

The Intel oneAPI backend can now run compatible MXFP4 and NVFP4 checkpoints. This lets you run models that were quantized for NVIDIA’s NVFP4 format transparently, with a nice performance boost and more memory savings.

Qwen3.8-27B, pp1024/tg128, bs=1, 4xArc B70, more is better (tg tok/s)
Qwen3.8-27B, pp1024/tg128, bs=1, 4xArc B70, more is better Qwen/Qwen3.8-27B: 22 tg tok/s. Qwen/Qwen3.8-27B-FP8: 52 tg tok/s. unsloth/Qwen3.8-27B-NVFP4: 64 tg tok/s. Bars start at zero. Qwen/Qwen3.8-27B 22 tg tok/s Qwen/Qwen3.8-27B-FP8 52 tg tok/s unsloth/Qwen3.8-27B-NVFP4 64 tg tok/s

Try it today

NVIDIA

docker run -p 8000:8000 --shm-size=256GB --gpus=all -e HF_TOKEN -it zmlai/llmd:cuda \
    --model=hf://Qwen/Qwen3-Embedding-0.6B

AMD

docker run -p 8000:8000 --shm-size=256GB --device=/dev/kfd --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:rocm \
    --model=hf://Qwen/Qwen3.8-Flash-Next

Intel

docker run -p 8000:8000 --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:oneapi \
    --model=hf://Qwen/Qwen3.8-27B

Google TPU

docker run --net=host --privileged -e HF_TOKEN -it zmlai/llmd:tpu \
    --model=hf://Qwen/Qwen3.8-27B

Apple Metal

brew install zml/zml/llmd
llmd --model=hf://Qwen/Qwen3-Embedding-0.6B