LLM hardware recommendations¶
For running LLM¶
Gemma 4 26b through vLLM needs a single GPU with enough VRAM for the model. Easy8 runs and tests google/gemma-4-26B-A4B-it.
Minimum required setup¶
| Component | Requirement |
|---|---|
| GPUs | 1 |
| VRAM | 48 GB or more |
| Example GPU | NVIDIA L40S (48 GB) |
Minimal recommended setup¶
| Component | Recommendation |
|---|---|
| GPUs | 1-2 |
| VRAM | 96 GB or more |
| GPU | NVIDIA RTX 6000 Pro (96 GB) |
| GPU | 2x NVIDIA A100, H100 (160 GB) |
vLLM and TEI have also been run together, so the LLM and the embeddings model can share one machine.
Performance notes¶
With a lower-grade GPU (e.g. 48 GB class), inference will work but may be very slow for chatting and interactive use. A minimum-spec setup is better suited for automation with n8n or background tasks where latency is not critical. For responsive interactive chat, use the recommended GPU or better, or use quantized models.
Embeddings (TEI)¶
For TEI hardware requirements, see Embeddings hardware recommendations.