# vLLM

> Serve an open model with vLLM for ShopAssist, with tool calling enabled.

Source: https://shopassist.coec.dk/docs/ai/vllm · Updated 2026-10-01

## Run vLLM [#run]

Start vLLM's OpenAI-compatible server with automatic tool choice and the tool-call parser for your model family:

```bash
vllm serve Qwen/Qwen3-14B --max-model-len 32768 \
  --enable-auto-tool-choice --tool-call-parser hermes \
  --api-key "$VLLM_API_KEY"
```

Expose it on a public HTTPS URL behind a reverse proxy (ShopAssist never calls private addresses).

## Settings [#settings]

* Preset: **vLLM**, base URL `https://llm.example.com/v1`.
* API key: the `--api-key` value. Add up to 3 extra headers if an access proxy needs them.
* Model: the served model name.

## Data location [#data-location]

> **Data location:** Data goes wherever this endpoint runs. Self-hosted endpoints must be reachable over HTTPS with authentication.

## Troubleshooting [#troubleshooting]

* **`tools_unsupported`:** `--enable-auto-tool-choice` or the right `--tool-call-parser` is missing.
* **`context_too_small`:** raise `--max-model-len`.