Run MiMo 2.6 Distill 9B on a Mac: Setup and a Real Code Check

MiMo-V2.6-Distill-Qwen-9B can run locally on the Mac configuration tested here: an Apple M3 Pro with 18 GiB of unified memory, using a community Q3_K_M GGUF and llama.cpp. Loading and generating text succeeded. The small coding task we tried did not pass its full acceptance test.

That distinction is the useful result. A model that fits in memory is not automatically reliable for your task, and the distilled 9B checkpoint is not the hosted MiMo 2.6 Pro model.

What we actually ran

The original Xiaomi checkpoint and community GGUF distribution are different artifacts. The latter is a quantized conversion. Preserve both the publisher and revision rather than calling it an official Xiaomi Q3 release.

Download a pinned file and verify it

Install llama.cpp from a trusted package source; on the test Mac we used Homebrew. Then download the exact revision used here into a directory with enough free space:

brew install llama.cpp
mkdir -p mimo26-test
cd mimo26-test
curl -fL --retry 2 \
  'https://huggingface.co/bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF/resolve/4371da10c84fb26da3592d4cf312d24aa82b7b65/MiMo-V2.6-Distill-Qwen-9B-Q3_K_M.gguf' \
  -o model-Q3_K_M.gguf
shasum -a 256 model-Q3_K_M.gguf

The verified SHA-256 is:

d98e54ec9650b6554cfdeea531e2420009207d50471170d0c3640483df8e87eb

Keep additional disk space for the runtime, logs and normal system operation. The GGUF size is a download figure, not a complete memory requirement. A newly installed llama.cpp may differ from the tested build; record llama-cli --version before comparing results.

Run a short, bounded request

/usr/bin/time -l llama-cli \
  -m model-Q3_K_M.gguf \
  -c 2048 -n 256 \
  --single-turn --temp 0 --seed 42 --no-warmup \
  -p 'Write a Python function unique_in_order(values) that removes duplicates while preserving first occurrence order. Inputs are hashable. Return only the function, with no explanation.'

This is the command structure we executed. /usr/bin/time -l is the macOS measurement option used here; do not assume the same flag exists on Linux. The context is deliberately small. This run does not validate a much larger context, multimodal input or concurrent requests.

Generated code: the important failure

The first run returned:

def unique_in_order(values):
    if not values:
        return []
    result = [values[0]]
    for value in values[1:]:
        if value != result[-1]:
            result.append(value)
    return result

It removes adjacent duplicates, but does not remove values that reappear later. We checked the generated function against four cases:

A second run added the non-adjacent-duplicate example to the prompt and still failed those two cases. A third run repeated the original prompt and returned the original function. These are three smoke-test runs on one small task, not a representative accuracy benchmark or proof that other quantizations behave the same way.

Timing and memory: what these numbers mean

Wall time includes startup and loading. The first two runs overlapped a local blog build, while the third ran after that build finished. Treat the measurements as observations from this machine, not a controlled speed comparison. We did not measure end-to-end time to a correct solution because the task never passed all cases.

macOS reported maximum process resident sizes of approximately 1.91, 3.86 and 3.84 GiB. These are OS process counters, not dedicated GPU VRAM measurements or the model’s complete unified-memory requirement. Memory mapping, shared buffers and other applications affect how such counters should be interpreted. This test supports “ran on this 18 GiB Mac,” not “requires only 4 GiB” or “fits any 8 GiB GPU.”

The downloadable test record includes the generated functions, expected and actual outputs, and measurement settings.

What to try next

Use a task with explicit acceptance cases and inspect the generated code before executing it. If local output fails, changing the prompt, quantization or model is an experiment that needs its own result; none of those changes is guaranteed to fix the failure above.

For hosted inference, our MiMo API guide explains the separate connection path, and the pricing guide covers direct-service rates. Hosted Pro and this local 9B conversion are different models, so do not interpret a result from one as a test of the other.