Licensed to be used in conjunction with basebox, only.
// installation
Reference configuration: 2 × NVIDIA H200 + 1 × RTX PRO 6000
Applies to
Product: Server · Operation: by the customer or by basebox · Audience: Platform Operator · Hardware: 2 × NVIDIA H200 + 1 × RTX PRO 6000 · Architecture: Single node
What a reference configuration is
A reference configuration documents a concrete, known setup. It does not mean basebox requires this hardware – the GPUs serve the service models and inference. Shared installation steps are not repeated here; they are in the bare-metal installation guide.
Two H200s for inference plus one RTX PRO 6000 dedicated exclusively to the service models (OCR, STT, embeddings, RAG). This is the dedicated service GPU pattern without MIG: inference gets both large GPUs in full, the service models their own physical GPU. Offered by FAST LTA with 128 GB or 512 GB RAM.
Status
Supported – see Status terms. basebox supports this setup; a measured reference workload as for the Validated configurations does not yet exist for this combination.
Architecture
Single Kubernetes node with all three GPUs.
GPU
| Physical GPU | Role | Memory (NVIDIA specification) |
|---|---|---|
| H200 GPU 0 | Inference | 141 GB HBM3e |
| H200 GPU 1 | Inference | 141 GB HBM3e |
| RTX PRO 6000 | Service models | 96 GB GDDR7 |
Inference can use one H200 and keep the second free for another model or more concurrency – or both as a tensor-parallel pair (TP=2) for very large models or long contexts. GPU memory stays local per card; two 141 GB GPUs are not one 282 GB device.
RAM
128 GB or 512 GB (FAST LTA options). 512 GB is recommended when large models are loaded and document processing runs in parallel.
basebox version
Not yet measured by basebox on this configuration. Use the current release (chart 0.3.32 → basebox 1.8.8).
Tested on
No measurement documented.
Component placement
| Component | Placement |
|---|---|
| basebox platform (frontend, AISRV, storesrv, Keycloak, PostgreSQL) | Node CPU |
| Inference (vLLM) | H200 GPU 0 (and GPU 1 with TP=2) |
| GPU RAG, document extraction, OCR, speech-to-text | RTX PRO 6000 |
Service model placement
All service models run on the RTX PRO 6000 and do not touch the inference GPUs. How the 96 GB are divided between ragsrv, ragsrv-support and the OCR/STT endpoints (MIG instances or shared use of the one GPU) is pending DevOps confirmation; the pattern is described under Dedicated service GPU.
Inference configuration
Backend vLLM. Model, quantisation, context and parallelism follow the target model; the 141 GB class per GPU allows e.g. Llama 3.3 70B FP8 with long context on one GPU or GPT-OSS 120B MXFP4 with TP=2. Guidelines: LLM recommendations.
Tested models
No documented model verification exists for this configuration. Tested combinations of other configurations: Tested models.
Deviations from the standard installation
- Inference requests
nvidia.com/gpu: 1(or2withNUM_GPUS: "2"for TP=2). - Service models request GPU resources on the RTX PRO 6000 – as MIG resources or as
nvidia.com/gpuwith node/device selection, depending on the confirmed layout. - Assign GPUs by stable UUID or PCI address (
nvidia-smi -L), not by index numbers.
Known limitations
- High availability across several nodes is not part of this configuration.
- The RTX PRO 6000 is a workstation-class GPU; the server's power and thermal budget must carry all three GPUs at their configured limits – approve with the manufacturer or FAST LTA.
- Capacity depends on model, context length, document workload and traffic pattern; customer-specific sizing is completed in the solution design.