Skip to content
Copyright © 2022-2026 basebox GmbH, all rights reserved.
Licensed to be used in conjunction with basebox, only.

// installation

Reference configuration: Custom hardware

Applies to

Product: Server · Operation: by the customer or by basebox · Audience: Platform Operator · Hardware: Custom hardware

What a reference configuration is

A reference configuration documents a concrete, known setup. It does not mean basebox requires this hardware. Shared installation steps are not repeated here – they are in the bare-metal installation guide.

Customer-specific hardware that matches none of the listed configurations. It requires an individual review with basebox before it can count as supported. This page is also the template: copy it, fill in every section for your setup and assign the status honestly – that produces the documentation basebox uses to review your system and that your operations will need later.

Status

Custom – see Status terms. After a successful review, basebox can classify the setup as Experimental or Supported.

Minimum prerequisites

Regardless of the specific hardware, the following must be met:

Area Requirement
GPU NVIDIA, compute capability 7.0+; driver compatible with CUDA 13.0 (12.9 as fallback); MIG-capable if service models are to run via MIG
VRAM Enough for the target model (weights + KV cache for context × concurrency) plus capacity for the service models, which should not share the inference GPU
CPU / RAM 8 cores / 16 GB minimum; 16+ cores / 32–64 GB+ recommended
Storage 500 GB minimum; 1 TB+ SSD/NVMe recommended; storage class with dynamic provisioning
Operating system Ubuntu 24.04 LTS recommended; 22.04 for existing systems; alternative distributions only after testing
Kubernetes 1.23+ (1.33+ verified), GPU Operator, ingress controller, CloudNativePG
Network Registry access or offline transfer; DNS/TLS path; firewall between AISRV and inference if separated

Details: Requirements · Server Preparation Guide.

Review process with basebox

  1. Record the hardware manifest: server model, GPU SKU and count, VRAM, firmware, CPU, RAM, storage, network, NUMA layout, GPU topology (nvidia-smi -L, nvidia-smi topo -m).
  2. Name target model and workload: model, quantisation, context, expected concurrency, document volume, audio.
  3. Propose the allocation: which GPU(s) inference, which service models (dedicated, MIG or CPU mode).
  4. Install following the bare-metal guide; document deviations.
  5. Run the acceptance checks per Validate installation – including mixed load and reboot.
  6. Send results to basebox (support@basebox.ai) with this completed document; basebox assigns the status.

Template – please fill in

Architecture

Single node or cluster; which nodes carry GPUs.

GPU

Physical GPU Model VRAM Role (inference / service) UUID
GPU 0

RAM

basebox version

Chart and app version of the verified installation (helm list -n basebox).

Tested on

Date, environment, responsible person.

Component placement

Which components run on which node or GPU.

Service model placement

GPU/MIG allocation for GPU RAG, document extraction, OCR, speech-to-text – or CPU mode.

Inference configuration

Backend and version, model, quantisation, context size, tensor parallelism, AISRV_LLM_* values.

Tested models

Per model: version, quantisation, context, concurrency, result.

Deviations from the standard installation

Helm values and configuration that deviate from the bare-metal guide.

Known limitations