Skip to content

TEI (Text Embeddings Inference)

Overview

TEI (Text Embeddings Inference) is a HuggingFace library for deploying optimized text embeddings models. This setup uses the BAAI/bge-m3 model, which is the supported embeddings model for self-hosted Easy AI Chain deployments.

Docker Compose Setup

This example sets up TEI with an nginx reverse proxy for API key authentication.

Directory Structure

tei/
├── tei.env
├── tei-nginx.conf
└── docker-compose.yaml

Environment Configuration (tei.env)

HUGGING_FACE_HUB_TOKEN=your_huggingface_token
API_KEY=your_api_key
LOCAL_MODEL_CACHE_DIR=/data/llm/tei-bge-m3

Docker Compose (docker-compose.yaml)

services:
  tei:
    image: ghcr.io/huggingface/text-embeddings-inference:89-latest
    pull_policy: always
    container_name: tei-bge-m3
    expose:
      - 80
    volumes:
      - ${TEI_MODEL_CACHE_DIR:-./data/tei}:/data
    environment:
      - HUGGING_FACE_HUB_TOKEN=${HUGGING_FACE_HUB_TOKEN}
    restart: always
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    command:
      - '--model-id'
      - 'BAAI/bge-m3'
      - '--hf-api-token'
      - '${HUGGING_FACE_HUB_TOKEN}'

  nginx:
    image: nginx:stable
    container_name: nginx
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./tei-nginx.conf:/etc/nginx/conf.d/default.conf:ro
      - /etc/certs:/etc/certs:ro
    restart: always

Nginx Configuration (tei-nginx.conf)

server {
    listen 80;
    server_name tei.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name tei.example.com;

    ssl_certificate /etc/certs/fullchain.pem;
    ssl_certificate_key /etc/certs/privkey.pem;

    location / {
        if ($http_authorization != "Bearer your_api_key") {
            return 401;
        }
        proxy_pass http://tei:80;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 600s;
        proxy_connect_timeout 600s;
        proxy_send_timeout 600s;
        proxy_buffering off;
        proxy_cache off;
    }
}

Usage

  1. Create a directory for your TEI setup and copy the files above
  2. Update tei.env with your HuggingFace token and desired API key
  3. Start the services:
docker compose --env-file tei.env up -d
  1. Access the embeddings API:
# Direct access to TEI (no auth)
curl http://localhost:8085/embed

# Access through nginx (requires auth)
curl -X POST https://ai-tei.example.com/embed \
  -H "Authorization: Bearer your_api_key" \
  -H "Content-Type: application/json" \
  -d '{"inputs": "Your text here"}'

Integration with Easy AI Chain

Configure the embeddings URL in your chain .env file:

EMBEDDINGS_URL=https://ai-tei.example.com

Replace ai-tei.example.com with the hostname of your TEI deployment.