Skip to content

Knowledge assistant PRO and Helpdesk with self-hosted AI

Use this guide for Easy8 AI Knowledge assistant PRO or Easy8 AI Helpdesk when ChromaDB, the LLM and the embeddings all run on your infrastructure. Both products have the same infrastructure requirements.

Before you start, read Prerequisites.

Supported models

  • LLM: gemma-4-26b served through vLLM. The endpoint in LLM_URL must expose this model. If your endpoint exposes a deployment-specific model id, map it with VLLM_MODEL_ALIASES as described in Custom vLLM served model aliases.
  • Embeddings: BAAI/bge-m3 served through TEI. The endpoint in EMBEDDINGS_URL must expose this model.

For GPU requirements, see LLM hardware recommendations and Embeddings hardware recommendations. The LLM and the embeddings can share one GPU. If you run both on one GPU, lower --gpu-memory-utilization in the vLLM deployment example so that TEI keeps enough memory.

Environment configuration

Set the variables from Common configuration, then add:

LLM_URL=https://llm.example.com
EMBEDDINGS_URL=https://tei.example.com
CHROMADB_URL=http://chroma:8000
VLLM_API_KEY=your-api-key

If an endpoint uses HTTPS with a certificate from your own CA, configure the CA bundle as described in Custom CA certificates.

If your vLLM endpoint, or a reverse proxy in front of it, requires authentication, set VLLM_API_KEY. Chain sends it as an Authorization: Bearer <VLLM_API_KEY> header. If the TEI endpoint requires a key, set TEI_API_KEY the same way.

Set the API key in the reverse proxy (for example nginx) rather than in vLLM. vLLM protects only some URLs with authentication, while a reverse proxy enforces it for all requests.

Installation steps

  1. If you are preparing a new GPU host, follow Self-hosted AI server setup.
  2. Deploy the LLM with the vLLM deployment example.
  3. Deploy the embeddings with the TEI deployment example.
  4. Check that both endpoints are reachable from the chain container.
  5. Add the environment variables to the Easy8 .env file.
  6. Check that the package includes docker-compose_easy_ai.yml.

  7. Install and start Easy8:

    docker compose run --rm app install
docker compose up -d

  8. Check that app, chain, chroma, celery and redis are running.

  9. Add the reverse proxy rule from Nginx configuration.

  10. Open both public Chain URLs:

    • https://your.easy.host/ai/chain/sys/monitoring.json
    • https://your.easy.host/ai/chain/api/info

    The info endpoint must report the vLLM provider and the embeddings model. See Verification and troubleshooting.