Skip to content

Easy8 AI Basic - Self-Hosted AI

Use this guide when:

  • the customer uses Easy8 AI Basic
  • the LLM runs on customer infrastructure
  • no knowledge-base features are required

Easy8 AI Basic does not require ChromaDB or an embeddings service.

Environment configuration

Set the common variables from Common configuration, then configure the local LLM endpoint.

Example:

LLM_URL=http://vllm-host:8081

Supported self-hosted LLM

Easy8 AI Basic currently supports:

  • gemma-4-26b
  • served through vLLM

The endpoint configured in LLM_URL must expose gemma-4-26b from a reachable vLLM deployment.

Installation steps

  1. If the customer is preparing a new GPU host for self-hosted AI, follow Self-hosted AI server setup first.
  2. Deploy the local LLM endpoint by following the vLLM deployment example.
  3. Make sure the local LLM endpoint is running and reachable from the chain container.
  4. Add LLM_URL and the common Easy8 AI variables to the Easy8 deployment .env file.
  5. Verify the package includes docker-compose_easy_ai.yml.
  6. Run:
docker compose run --rm app install
docker compose up -d
  1. Confirm app, chain, celery, and redis are running.
  2. Configure the reverse proxy for browser access to Easy8 AI before testing activation.

Use the reverse proxy rule from Common configuration.

  1. Verify both public chain URLs work through the reverse proxy:

  2. https://your.easy.host/ai/chain/sys/monitoring.json

  3. https://your.easy.host/ai/chain/api/info

The runtime info endpoint should report the expected local vLLM provider and model. See Verification and troubleshooting.

Notes

  • LLM_URL can contain one or more comma-separated endpoints.
  • only gemma-4-26b served through vLLM is supported for this self-hosted scenario
  • embeddings are not needed for this scenario