Copyright © 2022-2026 basebox GmbH, all rights reserved.
Licensed to be used in conjunction with basebox, only.

Basebox 2 x NVIDIA H200 141 GB Hardware Profile

Profile overview

Basebox runs on your own hardware. This profile describes a single-node deployment on a customer-owned Kubernetes server with two NVIDIA H200 141 GB GPUs: one complete GPU serves the language model, and the second is divided into four GPU instances that run the retrieval, document, image, and audio services. Your models, documents, and their processed output stay on your infrastructure.

Hardware layout

Item	This profile
GPUs	2 x NVIDIA H200 141 GB in one server
Inference GPU	One complete H200 141 GB GPU, language-model inference only
Service GPU	Four MIG `1g.35gb` instances
Kubernetes GPU resources	`nvidia.com/gpu: 1` and `nvidia.com/mig-1g.35gb: 4`
Kubernetes	A single node holding both GPUs
Storage	Persistent storage for model artifacts, documents, and platform data
Images	Registry access or an internal image mirror

MIG (Multi-Instance GPU) is the NVIDIA feature that divides one physical GPU into independent instances, each with its own memory and compute partition. Each 1g.35gb instance provides approximately 35 GiB of GPU memory.

CPU, system RAM, storage capacity, network, driver, CUDA runtime, and firmware levels are selected together with Basebox, based on your chosen model, document volume, and platform environment.

GPU memory allocation

The profile dedicates one complete GPU to language-model inference. The second GPU is divided into four service instances. Values in this chart are approximate decimal GPU-memory allocations; Kubernetes uses the MIG profile name 1g.35gb.

pie showData
  title Approximate GPU memory allocation across the profile
  "LLM inference" : 141
  "GPU RAG" : 35
  "Document extraction" : 35
  "OCR" : 35
  "Speech-to-text" : 35

Workload allocation

GPU allocation	Basebox workload
Whole H200 141 GB GPU	LLM inference
MIG slice 1	GPU RAG - retrieval over your own documents
MIG slice 2	Document extraction
MIG slice 3	OCR - text from images and scans
MIG slice 4	Speech-to-text

Keeping inference on a whole GPU means the document, image, and audio workloads cannot consume the inference GPU's compute or VRAM allocation. An upload or a transcription is served by the service GPU instead. Host CPU, system RAM, storage, and network remain shared by all workloads, so they are sized for the combined load.

Topology

graph TB
  subgraph SERVER["Customer server - 2 x H200 141 GB"]
    subgraph NODE["Kubernetes node"]
      subgraph G0["Inference GPU - complete, not partitioned"]
        LLM["LLM inference"]
      end
      subgraph G1["Service GPU - 4 x MIG 1g.35gb"]
        S1["slice 1"]
        S2["slice 2"]
        S3["slice 3"]
        S4["slice 4"]
      end
    end
  end
  S1 --> RAG["GPU RAG"]
  S2 --> EXT["Document extraction"]
  S3 --> OCR["OCR"]
  S4 --> STT["Speech-to-text"]

  style LLM fill:#dbeafe,stroke:#1e40af,color:#111827
  style RAG fill:#dcfce7,stroke:#166534,color:#111827
  style EXT fill:#dcfce7,stroke:#166534,color:#111827
  style OCR fill:#dcfce7,stroke:#166534,color:#111827
  style STT fill:#dcfce7,stroke:#166534,color:#111827

Measured reference workload

The profile has been exercised with language-model traffic and all four GPU support services operating on the same server. These figures describe the measured reference workload; customer sizing is based on the selected model, context length, document volume, and expected traffic.

Measurement	Reference result
Continuous mixed-workload duration	60 minutes
Short language-model requests	720/720 completed with the expected result
Long-context language-model requests	12/12 completed with the expected result
Long-context input size	Approximately 258,000 tokens
Combined RAG, extraction, OCR, and speech-to-text service bursts	12/12 completed
Inference workload restarts during the run	0
Document-processing service check	120-page PDF extracted and made searchable in 28.47 seconds
Speech-to-text service check	60-minute synthetic audio fixture transcribed in 31.31 seconds
Inference cold-start allowance	Approximately 6 minutes from restart through warm-up

Document and audio timings are service-level reference measurements. Actual processing time varies with file format, content, model choice, and platform configuration.

flowchart LR
  RUN["60-minute mixed workload"] --> SHORT["720 short LLM requests"]
  RUN --> LONG["12 long-context LLM requests"]
  RUN --> SERVICES["12 combined service bursts"]
  SHORT --> LLMRESULT["732/732 expected LLM results"]
  LONG --> LLMRESULT
  SERVICES --> SERVICERESULT["12/12 RAG, extraction, OCR and STT bursts"]
  LLMRESULT --> STABLE["0 inference restarts"]
  SERVICERESULT --> STABLE

Platform prerequisites

A supported Kubernetes environment with both GPUs on one node
NVIDIA driver and a GPU-enabled container runtime installed on the host
NVIDIA GPU Operator or device plugin configured for mixed MIG resource discovery, so one whole GPU and four MIG instances are advertised together
Elevated host privileges reviewed for the NVIDIA GPU system components, which may require them for mixed MIG discovery
A persistent storage class for model artifacts, documents, and platform data
Basebox container images reachable from a registry, or mirrored internally
Model artifacts available locally or through an approved repository
DNS, TLS, authentication, backup, and monitoring provided through your own platform design

Kubernetes resource contract

The inference workload requests the complete GPU:

resources:
  requests:
    nvidia.com/gpu: 1
  limits:
    nvidia.com/gpu: 1

Each of the four support workloads requests one service instance:

resources:
  requests:
    nvidia.com/mig-1g.35gb: 1
  limits:
    nvidia.com/mig-1g.35gb: 1

The resource names must match what the NVIDIA GPU Operator or device plugin advertises on the node. Basebox maps these resource contracts to inference, GPU RAG, document extraction, OCR, and speech-to-text during deployment.

Deployment sequence

Identify both GPUs by stable UUID or PCI address and record which one takes the inference role and which takes the service role. Do not rely on GPU index numbers, which can change.
Reserve the inference GPU as a complete device with no other GPU workload.
Enable MIG on the service GPU, addressed by its stable identity.
Create four 1g.35gb GPU instances and their compute instances.
Configure mixed MIG resource discovery and refresh the GPU discovery components so the new layout is advertised.
Confirm the node advertises one whole GPU and four MIG resources in both capacity and allocatable before deploying anything.
Deploy Basebox with equal GPU resource requests and limits for each of the five GPU workloads.
Allow the model to warm up after Kubernetes reports readiness, and complete a warm-up generation before admitting users.
Run the acceptance checks below.

Operator verification

Confirm the physical GPUs and MIG instances on the host:

nvidia-smi -L

Inspect the Kubernetes node and verify the GPU resources under both Capacity and Allocatable:

kubectl describe node <node-name>

Expected GPU resource counts:

nvidia.com/gpu:          1
nvidia.com/mig-1g.35gb:  4

Check workload placement, readiness, and restart counts:

kubectl get pods -n <basebox-namespace> -o wide

After the pods are Ready, complete one authenticated chat request, one document ingestion and RAG search, one OCR request, and one speech-to-text request through the Basebox application.

Acceptance checks

Scope

This profile covers a single-node deployment on 2 x NVIDIA H200 141 GB.
Multi-node high availability and disaster recovery require a separate architecture.
Capacity depends on the selected model, context length, document workload, and traffic pattern.
Customer-specific sizing is completed during solution design.
Normal inference and document processing run entirely on your infrastructure.
Installation and updates require registry and model repository access, or mirrored artifacts held locally.