https://zml.ai/tags/huggingface/Recent content in Huggingface on ZML - Model to MetalHugoen-usTue, 07 Apr 2026 09:00:00 +0100https://zml.ai/posts/iree-tokenizer/Tue, 07 Apr 2026 09:00:00 +0100https://zml.ai/posts/iree-tokenizer/<p>Today we merged a pretty nice performance improvement to ZML.</p> <p><a href="https://github.com/zml/zml/pull/416">PR #416</a> switches <code>zml.tokenizer.Tokenizer</code> from using the Hugging Face <a href="https://crates.io/crates/tokenizers"><code>tokenizers</code></a> crate to using the <a href="https://github.com/iree-org/iree/blob/main/runtime/src/iree/tokenizer/README.md">IREE project tokenizer</a> when using Hugging Face <code>tokenizer.json</code> files.</p> <p><img src="https://zml.ai/img/posts/iree-tokenizer/1-benchmark.jpg" alt=""></p> <h1 id="the-upcoming-problem">The (upcoming) problem</h1> <p>Tokenizers performance is often overlooked, but it can have a significant impact on the overall latency of LLM inference. In some cases, it can even become the bottleneck of the entire system. This is particularly important as context are becoming bigger and bigger. For code generation, a 1M context window is regular. But a 1M context means 4MB of data (<code>u32</code>).</p>