Run the model on your own GPUs

Local mode with your own model server. No H API key needed.

your code ──► Client.local() ──► local runtime ──► your model server
                                      └──► your browser or desktop

Serve Holo4

GPU server (vLLM)
Mac (llama.cpp)

vLLM

vllm serve Hcompany/Holo4-35B-A3B-FP8 \
  --served-model-name holo4-35b-a3b \
  --max-model-len 262144 \
  --enable-prefix-caching \
  --chat-template-content-format openai \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --limit-mm-per-prompt '{"image": 5, "video": 0}'

Base URL: http://localhost:8000/v1.

llama.cpp

brew install llama.cpp
llama-server -hf Hcompany/Holo4-35B-A3B-GGUF

Base URL: http://localhost:8080/v1. Any model ID works.

Other sizes and precisions: Local inference.

Run a session

Install

pip install "hai-agents[browser]"

Python

from hai_agents import Client
from hai_agents_local.runtime import Inference

agent = {
    "name": "local-web",
    "description": "Drives Chrome on this machine.",
    "model": "holo4-35b-a3b",
    "environments": [{"id": "browser", "kind": "web", "host": "user_device"}],
}
inference = Inference.self_hosted("http://localhost:8000/v1", model="holo4-35b-a3b")

with Client.local(inference=inference) as client:
    result = client.run_session(agent=agent, messages="Open example.com and tell me the page heading.")
    print(result.status, result.answer)

Two model names

Field	Value
Agent `model`	The Holo version you serve: `holo4-35b-a3b`, `holo4-27b`, or `holo3-1-35b-a3b`. Sets the prompt format.
`Inference.self_hosted(model=...)`	Your server’s model name.

GPUs on another machine

No API key is sent to your server, so keep it private. Use an SSH tunnel and keep http://localhost:8000/v1:

SSH tunnel

ssh -N -L 8000:localhost:8000 you@gpu-host

Troubleshooting

Error	Fix
`No health endpoint at http://.../health`	The base URL must end in `/v1`, and the server must answer `GET /health`.
`inference selection cannot reconfigure an existing runtime`	Close the other client, or pass `local_options={"port": 18796}`.
`Unknown model '...'`	Use an agent `model` from Two model names.

Get started

Concepts

Browser and desktop

Integrations

Guides

Operate

Run the model on your own GPUs

Serve Holo4

Run a session

Two model names

GPUs on another machine

Troubleshooting

Serve Holo4

Run a session

​Two model names

​GPUs on another machine

​Troubleshooting

Two model names

GPUs on another machine

Troubleshooting