Skip to content

Easy8 AI Knowledge assistant PRO / Easy8 AI Helpdesk - Self-Hosted AI

Use this guide when:

  • the customer uses Easy8 AI Knowledge assistant PRO or Easy8 AI Helpdesk
  • ChromaDB runs on the customer infrastructure
  • the LLM and embeddings services also run on customer infrastructure

Easy8 AI Knowledge assistant PRO and Easy8 AI Helpdesk share the same infrastructure requirements.

Required services

This scenario requires:

  • chain
  • celery
  • chroma
  • local LLM endpoint
  • local embeddings endpoint

Environment configuration

Set the common variables from Common configuration, then add:

LLM_URL=http://vllm-host:8081
EMBEDDINGS_URL=http://embeddings-host:8085
CHROMADB_URL=http://chroma:8000
VLLM_API_KEY=your-api-key

If your VLLM endpoint (or reverse proxy in front of it, such as nginx) requires authentication, set VLLM_API_KEY. The value will be sent as Authorization: Bearer <VLLM_API_KEY> header.

Recommendation: We recommend setting the API key in the reverse proxy (e.g., nginx) rather than in VLLM itself. VLLM only protects certain URLs with authentication, but not all endpoints. A reverse proxy can enforce authentication consistently across all requests.

Supported self-hosted LLM

Easy8 AI Knowledge assistant PRO and Easy8 AI Helpdesk currently support:

  • gemma-4-26b
  • served through vLLM

The endpoint configured in LLM_URL must expose gemma-4-26b from a reachable vLLM deployment.

Supported self-hosted embeddings

For self-hosted retrieval, the supported embeddings setup is:

  • BAAI/bge-m3
  • served through a TEI endpoint configured by EMBEDDINGS_URL

The endpoint configured in EMBEDDINGS_URL must expose BAAI/bge-m3 from a reachable TEI deployment.

Embeddings hardware requirements

For BAAI/bge-m3 served through TEI:

  • only NVIDIA GPU deployment is supported
  • minimum supported GPU memory is 4 GB VRAM
  • CPU-only deployment is not supported
  • 8 GB VRAM or more is recommended for faster indexing and higher throughput

For the detailed hardware reference, see Embeddings hardware recommendations.

Note that you can run embeddings alongside LLM on a single GPU. See our docker compose example that limits LLM to 90% of GPU memory.

Installation steps

  1. If the customer is preparing a new GPU host for self-hosted AI, follow Self-hosted AI server setup first.
  2. Deploy the local LLM endpoint by following the vLLM deployment example.
  3. Deploy the local embeddings endpoint by following the TEI deployment example.
  4. Make sure the local LLM and embeddings endpoints are running and reachable from the chain container.
  5. Add the required environment variables to the Easy8 deployment .env file.
  6. Verify the package includes docker-compose_easy_ai.yml.
  7. Run:
docker compose run --rm app install
docker compose up -d
  1. Confirm app, chain, chroma, celery, and redis are running.
  2. Configure the reverse proxy for browser access to Easy8 AI before testing activation.

Use the reverse proxy rule from Common configuration.

  1. Verify both public chain URLs work through the reverse proxy:

  2. https://your.easy.host/ai/chain/sys/monitoring.json

  3. https://your.easy.host/ai/chain/api/info

The runtime info endpoint should report the expected local vLLM provider and the embeddings model. See Verification and troubleshooting.

Notes

  • LLM_URL should point to a reachable vLLM endpoint serving gemma-4-26b
  • EMBEDDINGS_URL should point to a reachable TEI endpoint serving BAAI/bge-m3
  • the TEI deployment for BAAI/bge-m3 should run on NVIDIA GPU infrastructure with at least 8 GB VRAM