Skip to content
ShopAssist Docs
AI

vLLM

Serve an open model with vLLM for ShopAssist, with tool calling enabled.

View as Markdown

Run vLLM

Start vLLM's OpenAI-compatible server with automatic tool choice and the tool-call parser for your model family:

vllm serve Qwen/Qwen3-14B --max-model-len 32768 \
  --enable-auto-tool-choice --tool-call-parser hermes \
  --api-key "$VLLM_API_KEY"

Expose it on a public HTTPS URL behind a reverse proxy (ShopAssist never calls private addresses).

Settings

  • Preset: vLLM, base URL https://llm.example.com/v1.
  • API key: the --api-key value. Add up to 3 extra headers if an access proxy needs them.
  • Model: the served model name.

Data location

Data location

Data goes wherever this endpoint runs. Self-hosted endpoints must be reachable over HTTPS with authentication.

Troubleshooting

  • tools_unsupported: --enable-auto-tool-choice or the right --tool-call-parser is missing.
  • context_too_small: raise --max-model-len.

Last updated on

Ollama

Run ShopAssist on Ollama's cloud or your own Ollama server, with the context window set high enough.

LM Studio

Use a model served by LM Studio with ShopAssist; only models with a tool chat template work.

On this page

Run vLLMSettingsData locationTroubleshooting