Provider: Core (local mistral.rs)

vibecli-mistralrs runs models inside the VibeCLI daemon via mistral.rs — no Ollama, no sidecar, no second process. It appears in VibeCoder’s toolbar dropdown as Core. It is the other local option alongside Ollama: Ollama is a separate server you install and manage, this one is the daemon you are already running.

Weights are pulled from Hugging Face on first use and cached; inference is in-process.

Availability

The backend is compiled in, not installed at runtime, so whether it exists depends on how your vibecli was built:

No bundled chat model — downloaded at daemon startup instead

As of 2026-09-18, no desktop installer (macOS, Windows, Linux) ships a chat model inside the package. scripts/fetch-model-assets.{ps1,sh} still stages the embedding model used for semantic index (sentence-transformers/all-MiniLM-L6-v2) into each shell’s tauri.*.conf.json resources, but the chat model was dropped to keep the .pkg/.dmg/.msi size down.

Instead, the daemon calls Backend::warm_default_model() (vibecli-cli/src/inference/backend.rs, implemented in mistralrs.rs) once at startup (serve.rs), which downloads recommended_default_model() via the ordinary hf-hub path in the background — the same download get_or_load would otherwise trigger lazily on the first chat request. Net effect: a blank machine with no Ollama and no API keys still needs one network connection the first time the app runs, but pays that latency during startup rather than on the user’s first message, and every later launch finds the model already cached.

Cargo features cannot be conditioned on target OS, so vibecli/vibecli-cli/build.rs computes the union and emits a single cfg(mistralrs_enabled). Accelerator variants (mistralrs-cuda, mistralrs-metal, mistralrs-flash-attn) are selected in vibe-infer.

Check what your binary has:

curl -s localhost:7878/health | jq '.mistralrs_recommended_default'

A value means the backend is present. No daemon, no answer — see Daemon startup.

Configure VibeCody

There is no API key. Authentication is the daemon’s own bearer token, read from ~/.vibecli/daemon.token on every request — so a daemon restart, which rotates that token, needs no reconfiguration.

vibecli --provider vibecli-mistralrs

In VibeCoder it appears in the toolbar model dropdown with no key configured. Both vibecli-mistralrs and vibecli_mistralrs are accepted as the provider id.

Config file (~/.vibecli/config.toml) — only needed to point at a non-default daemon:

[vibecli_mistralrs]
enabled = true
model = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
api_url = "http://localhost:7878"   # optional, default shown

The provider speaks the Ollama HTTP wire format against the daemon but pins the backend by sending X-VibeCLI-Backend: mistralrs, which outranks the daemon default and any per-model pin. Backend selection precedence is header → body field → per-model pin → daemon default.

Gated weights and HF_TOKEN

The Llama models are gated on Hugging Face — loading one without an accepted licence returns 401. VibeCody handles this rather than letting you discover it at first token:

• /health reports mistralrs_recommended_default, which is meta-llama/Llama-3.1-8B-Instruct when HF_TOKEN is set and Qwen/Qwen2.5-Coder-1.5B-Instruct (Apache-2.0, ungated) when it is not.
• VibeCoder reads that field on launch and swaps its picker default in place, so it never pre-selects a model that would 401.

Set the token the encrypted way — it is a credential, so it does not belong in config.toml:

vibecli set-key huggingface "hf_..."

The daemon hydrates HF_TOKEN from the ProfileStore at startup, so downstream libraries see it without a shell export.

Model Selection

Sub-1.5B models (SmolLM2-360M, Llama-3.2-1B, the Qwen2.5 0.5B pair) were dropped from the picker 2026-09-18: hallucination rate on open-ended questions is too high below ~1.5B params.

General-purpose entries refreshed 2026-09-23 from Qwen2.5 to Qwen3 — each id was checked against the Hugging Face API (exists, ungated) and against the loader registry of the pinned mistral.rs revision, which supports Qwen3ForCausalLM. Coder slots stay on Qwen2.5-Coder: Qwen3-Coder ships MoE-only (30B-A3B and up), too large for a zero-config local pick.

Override from the CLI:

vibecli --provider vibecli-mistralrs --model Qwen/Qwen2.5-Coder-1.5B-Instruct

Any model mistral.rs can load works — the list above is what the picker offers, not an allow-list.

Best For

• Fully offline work — no network once weights are cached
• One less moving part — no Ollama install or sidecar to keep running
• Apple Silicon — Metal acceleration is on by default in macOS builds

Verify Connection

vibecli --provider vibecli-mistralrs -c "Write a Python async web scraper"

The daemon starts downloading the default model’s weights on its own at startup, so by the time you run this the download may already be complete; if the daemon just started, the first response will still be slow while that finishes.

Troubleshooting

Provider missing from the picker

The binary was built without the backend. Check /health as above; rebuild with --features vibe-mistralrs on Linux/Windows.

401 from Hugging Face

A gated model without a token. Either accept the licence and vibecli set-key huggingface, or switch to Qwen/Qwen2.5-Coder-1.5B-Instruct. The daemon substitutes the ungated fallback and says so when it can.

401 from the daemon

~/.vibecli/daemon.token is missing or stale. The token rotates on every daemon start; the provider re-reads the file per request, so this usually means the daemon is not running rather than that the token is wrong.

Out of memory on load

Drop to a smaller model — Qwen/Qwen2.5-Coder-1.5B-Instruct (the zero-config default) or Qwen/Qwen3-1.7B. Model size is the dominant term; a 7B at fp16 wants roughly 14 GB before cache.