Licensed to be used in conjunction with basebox, only.
// installation
Reference configuration: 3 × NVIDIA RTX PRO 6000
Applies to
Product: Server · Operation: by the customer or by basebox · Audience: Platform Operator · Hardware: 3 × NVIDIA RTX PRO 6000 · Architecture: Single node
What a reference configuration is
A reference configuration documents a concrete, known setup. It does not mean basebox requires this hardware – the GPUs serve the service models and inference. Shared installation steps are not repeated here; they are in the bare-metal installation guide.
A three-GPU workstation-class configuration, offered by FAST LTA with 128 GB or 512 GB RAM. Typical split: one GPU dedicated to the service models, the remaining two GPUs for inference – individually for two models or as a tensor-parallel pair for a larger model. This is the most affordable of the FAST LTA configurations and designed for mid-sized organisations.
Status
Supported – see Status terms. basebox supports this setup; a measured reference workload as for the Validated configurations does not yet exist.
Architecture
Single Kubernetes node with all three GPUs.
GPU
| Physical GPU | Role | Memory (NVIDIA specification) |
|---|---|---|
| RTX PRO 6000 GPU 0 | Inference | 96 GB GDDR7 |
| RTX PRO 6000 GPU 1 | Inference | 96 GB GDDR7 |
| RTX PRO 6000 GPU 2 | Service models | 96 GB GDDR7 |
Two inference GPUs with 96 GB each are in the VRAM class of the 4 × L40S (192 GB total): documented there are e.g. GPT-OSS 120B AWQ (TP across all inference GPUs) or Llama 3.3 70B FP8 with 32–65k context. Take that as orientation, not measurement – interconnect and memory bandwidth differ.
RAM
128 GB or 512 GB (FAST LTA options).
basebox version
Not yet measured by basebox on this configuration. Use the current release (chart 0.3.32 → basebox 1.8.8).
Tested on
No measurement documented.
Component placement
| Component | Placement |
|---|---|
| basebox platform | Node CPU |
| Inference (vLLM) | GPU 0 (+ GPU 1 with TP=2 or as a second instance) |
| GPU RAG, document extraction, OCR, speech-to-text | GPU 2 |
Service model placement
All service models on GPU 2, separate from inference – the Dedicated service GPU pattern. The division of the 96 GB between ragsrv, ragsrv-support and OCR/STT endpoints is pending DevOps confirmation.
Inference configuration
Backend vLLM. Two options: (a) TP=2 across GPU 0 and 1 for a large model with long context; (b) two instances on one GPU each – e.g. a main model and a second one for reasoning or an app – see Multiple inference instances. Guidelines: LLM recommendations.
Tested models
No documented model verification for this configuration.
Deviations from the standard installation
- Inference requests
nvidia.com/gpu: 2withNUM_GPUS: "2"(TP=2) or two deployments withnvidia.com/gpu: 1each. - Service models request GPU resources on GPU 2; assign GPUs by UUID/PCI address.
Known limitations
- Workstation class: no NVLink/NVSwitch as with H100 SXM; tensor parallelism over PCIe is possible but slower in the exchange between shards. Check the peer path with
nvidia-smi topo -m. - Approve the power and thermal budget for three GPUs with the manufacturer or FAST LTA.
- No high availability across several nodes.