How to run DeepSeek V4 Flash locally in Home Assistant

Verified 2026-07-31. Start with the hardware gate: DeepSeek V4 Flash is a 284B-parameter mixture-of-experts model with about 13B active parameters per token, a 1M-token context window, an MIT license, and a preview release dated April 24, 2026.[1][2] The 13B number is not a RAM requirement. For local inference, the model weights still have to be resident. In the current GGUF path, even the smallest practical quantization is an 80GB-plus-class download, Unsloth’s recommended UD-IQ3_XXS is about 103GB, and the lossless UD-Q8_K_XL is 162GB with roughly 169GB of RAM/VRAM needed.[3]

Also rule out the shortcut before you install anything: the Ollama library entry for DeepSeek V4 Flash is a :cloud hosted tag, not a local model tag you can pull onto your own machine.[4] If your plan was “install Ollama, paste the model name into Home Assistant, done,” that is not the local setup described here.

The working route, narrowed down

As of July 31, 2026, the local path worth documenting is: serve a DeepSeek V4 Flash GGUF from a llama.cpp build that includes the July 7 V4 support merge, expose llama-server’s OpenAI-compatible /v1 endpoint, then point Home Assistant at that endpoint through the HACS Local OpenAI LLM integration.[3][6]

Home Assistant’s native Ollama integration is still useful for models Ollama can actually run locally, but it does not turn the hosted DeepSeek V4 Flash cloud tag into a local model.[7] LM Studio-style convenience wrappers are in the same caution bucket for this recipe unless they are clearly shipping the same current llama.cpp V4 support. For V4 Flash, the runtime date matters.

Before you build: choose a GGUF that fits the machine, not the wish list

The practical local choice is not between “13B” and “284B.” It is between quantizations of a 284B model. For this Home Assistant recipe, the useful starting point is Unsloth’s UD-IQ3_XXS at about 103GB. The smaller IQ2_XS-XL class exists around 81GB, but it is a harsher compromise. The UD-Q8_K_XL lossless option is a 162GB file and belongs on machines that can keep roughly 169GB available for the model path.[3]

Do not set the context window to 1M just because the model card says the model supports it. Home Assistant tool use benefits from a larger-than-chat context, but every extra token of context has a runtime cost. For an Assist agent, start at 16k if the machine can hold it; the Local OpenAI LLM project recommends at least 10k context for Assist-style tooling, and the entity list should stay small enough that the prompt does not become the workload.[6]

Step 1 — Build or update llama.cpp with V4 support

Status: Confirmed as of 2026-07-31. DeepSeek V4 support landed in llama.cpp on July 7, 2026, and a related multi-turn KV-cache quantization fix also landed that day.[3] If your llama.cpp checkout is older than that, update before testing Home Assistant. A first-turn success followed by broken second-turn behavior is exactly the kind of failure that wastes time if you start from an old build.

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
git pull

# NVIDIA CUDA build
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

# The server binary is usually here after build:
./build/bin/llama-server --help

On Apple Silicon, use the Metal build path instead of the CUDA line. On a CPU-only box, this model is the wrong target for Home Assistant interactivity even if the file technically loads. The question is not whether a process can start; it is whether Assist can answer before the person in the room gives up.

Step 2 — Put the GGUF somewhere boring and permanent

Status: Confirmed. Download the DeepSeek V4 Flash GGUF from the current Unsloth DeepSeek V4 page, then store it outside Home Assistant’s config directory. This is not an add-on asset; it is a workstation-scale model file. The path below is only an example layout.

sudo mkdir -p /models/deepseek-v4-flash
sudo chown "$USER":"$USER" /models/deepseek-v4-flash

# Place the selected GGUF here, for example:
# /models/deepseek-v4-flash/DeepSeek-V4-Flash-UD-IQ3_XXS.gguf

ls -lh /models/deepseek-v4-flash/

If the model lives on removable storage, a network share, or a drive that spins down aggressively, fix that before debugging Home Assistant. The first clean test should remove storage weirdness, permission errors, and disappearing paths from the list.

Step 3 — Start llama-server with the DeepSeek-friendly flags

Status: Confirmed route, with local tuning required. Unsloth’s V4 notes call out the llama.cpp path and DeepSeek-recommended settings including flash attention, fit behavior, Jinja templating, temperature 1.0, and top-p 1.0.[3] Start conservatively, confirm multi-turn chat, then raise context or change quantization only after the endpoint is stable.

./build/bin/llama-server \
  --model /models/deepseek-v4-flash/DeepSeek-V4-Flash-UD-IQ3_XXS.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  --ctx-size 16384 \
  --flash-attn \
  --fit \
  --jinja \
  --temp 1.0 \
  --top-p 1.0

If your llama-server build uses a slightly different binary path, adjust the path, not the route. What matters for Home Assistant is that the server exposes an OpenAI-compatible API under /v1 and keeps the model loaded.

curl http://127.0.0.1:8080/v1/models

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "local",
    "messages": [
      {"role": "system", "content": "You are a concise home automation assistant."},
      {"role": "user", "content": "Say ready in one sentence."}
    ],
    "temperature": 1.0,
    "top_p": 1.0
  }'

Run at least two turns against the endpoint before Home Assistant enters the picture. A single curl response only proves that the model loaded and produced a token stream once. It does not prove that the KV cache, chat template, and context handling are usable for an Assist session.

Step 4 — Add the Local OpenAI LLM bridge in Home Assistant

Status: Confirmed bridge path. The HACS Local OpenAI LLM integration is designed to connect Home Assistant to local or self-hosted OpenAI-compatible LLM endpoints and expose them as conversation agents.[6] That is the important fit here: llama-server speaks enough of the OpenAI-style API, and Home Assistant gets an agent it can assign to Assist.

1. Install the Local OpenAI LLM custom integration through HACS, following the project’s current repository instructions.
2. Restart Home Assistant if HACS asks for it.
3. Add the Local OpenAI LLM integration from Settings → Devices & services.
4. Use your llama-server endpoint as the API base URL, for example http://WORKSTATION_IP:8080/v1.
5. If the integration requires an API key field for OpenAI compatibility, use a placeholder value unless your reverse proxy enforces a real key.
6. Set the model name to the value your endpoint accepts. With llama.cpp, this is often not strict, but keep it stable once configured.

Do not point Home Assistant at localhost unless llama-server is running inside the same Home Assistant network namespace. For the common setup — Home Assistant on one box, model server on a workstation — use the workstation’s LAN address, then verify from the Home Assistant host that the /v1 endpoint is reachable.

# From the Home Assistant host or a shell add-on with network access:
curl http://WORKSTATION_IP:8080/v1/models

Step 5 — Give Assist enough context, but not your entire house

Status: Confirmed recommendation from the bridge documentation, with local tuning. Set the LLM context at 10k or higher for Home Assistant tool use, and expose fewer than 25 entities at first.[6] That second number is easy to ignore until the model spends its context budget reading every light, sensor, scene, and switch you forgot was exposed.

• Start with one room or one function: lights, climate, or media — not the whole installation.
• Prefer obvious entity names over clever aliases. The model has enough work to do without guessing that “north glow” means the office lamp.
• Add entities in batches only after successful tool calls, not after successful chat replies.
• Keep automations with real consequences behind normal Home Assistant safeguards. A local LLM should not be the only thing preventing a bad service call.

The right first test is not “tell me a joke.” It is a small, reversible command against one exposed entity, followed by a second-turn reference to the same entity. For example: turn on the office lamp, then ask it to turn it back off without repeating the full entity name. That tests the bridge, the tool call, and the model’s short conversation state.

Step 6 — Assign it to Assist and, optionally, a voice pipeline

Status: Confirmed Home Assistant configuration path, with model latency dependent on hardware. Once the Local OpenAI LLM agent is present, assign it under Home Assistant’s Assist or voice assistant settings as the conversation agent. If you already run local Whisper for speech-to-text and Piper for text-to-speech, keep them; the LLM swap is the middle of the pipeline, not a replacement for wake word, STT, or TTS.

Voice makes latency more obvious than chat. A reply that feels acceptable in a web UI can feel broken when someone is standing in a kitchen waiting for the lights to change. Get chat tool calls stable first, then move the same agent into a voice pipeline.

Troubleshooting the two failures that matter most

Second turn breaks, stalls, or produces nonsense

Status: Confirmed fix. Update llama.cpp. The July 7, 2026 V4 work included a fix for the multi-turn KV-cache quantization bug.[3] If your first prompt works and the second turn fails, do not start by rewriting Home Assistant prompts or reinstalling HACS. Confirm the llama.cpp commit date, rebuild, and retest directly against /v1/chat/completions before bringing Assist back into the loop.

cd llama.cpp
git pull
cmake --build build --config Release -j

# Restart llama-server after rebuilding.

HTTP 400: reasoning_content must be passed back

Status: Workaround for Home Assistant-style tool agents; documented outside HA. The specific HTTP 400 failure where “reasoning_content must be passed back” appears when a DeepSeek reasoning model is combined with tool use is documented in an n8n agent case, not as a reproduced Home Assistant bug.[8] The relevance to Home Assistant is cautious but practical: Assist agents also combine conversation, tool calls, and follow-up turns.

If you see this class of error through the bridge or through an adjacent automation stack, avoid thinking mode for tool-calling use. Use the non-thinking chat path for Home Assistant control unless your entire client chain preserves and returns the reasoning content exactly as the upstream API expects. A smart-home agent that cannot complete a tool call is worse than a less flashy model that reliably turns the device on.

The model loads, but Home Assistant feels slow

Status: Local tuning. Reduce the exposed entity count first, then reduce context, then reconsider quantization. Do not chase speed claims from unrelated hardware posts as if they were benchmarks. The available hardware reports are useful for deciding whether to attempt the setup, but they are still early community ballparks rather than controlled Home Assistant latency tests.[5]

• If chat is slow before Home Assistant is involved, the bottleneck is the model server or hardware.
• If raw chat is acceptable but Assist is slow, inspect entity exposure, prompt size, and tool-call retries.
• If first-token delay dominates voice, test chat and TTS separately instead of blaming the full pipeline.

When to use the hosted API instead

Below the memory line, the honest alternative is a hosted API or a smaller local model. That is not as satisfying as a fully local V4 Flash agent, but it is cleaner than pretending a 284B MoE becomes laptop-friendly because only 13B parameters are active per token. If privacy, availability, or cost rules out hosted inference, pick a smaller model that your machine can keep resident and make reliable inside Assist.

If you do use a hosted provider, separate that decision from this local recipe in your Home Assistant notes. Label the integration as hosted, record the endpoint, and keep any API key handling out of automations and dashboards that do not need it.

Status table — verified 2026-07-31

References

1. DeepSeek-V4-Flash, Hugging Face.
2. DeepSeek-V4-Flash, DeepSeek API Docs, April 24, 2026.
3. DeepSeek V4, Unsloth Docs.
4. deepseek-v4-flash, Ollama.
5. Run DeepSeek V4 Flash Locally: Hardware Reality Check, Modem Guides.
6. skye-harris/hass_local_openai_llm, GitHub.
7. Ollama, Home Assistant.
8. DeepSeek AI Agent fails with 400 “reasoning_content must be passed back” when using tools, n8n Community.