Skip to content
Copyright © 2022- basebox GmbH, all rights reserved.
Licensed to be used in conjunction with basebox, only.

Server Preparation Guide

Latest Update: February 03, 2026
Document Version: 2.1

Quick Start Guide

For experienced system administrators preparing a server immediately:

  1. Install Ubuntu 24.04 LTS Server (Download)
  2. Install NVIDIA Drivers: sudo ubuntu-drivers autoinstall && sudo reboot
  3. Install CUDA Toolkit 13.0 (see Step 3 for details)
  4. Install Docker + NVIDIA Container Toolkit (see Steps 4-5)
  5. Install Kubernetes 1.33+ with GPU Operator (see Steps 6-7)

Contact basebox support: support@basebox.ai for deployment assistance

Note: This guide provides the recommended path for new installations. If you have existing infrastructure or different requirements, contact support@basebox.ai

TLDR

Recommended Operating System: Ubuntu 24.04 LTS

Key Requirements:

  • Ubuntu 24.04 LTS Server (the recommended version for new installations)
  • NVIDIA GPU with CUDA support (Compute Capability 7.0+)
  • NVIDIA drivers compatible with CUDA 13.0+
  • CUDA Toolkit 13.0 (always install the latest version)
  • Docker Engine 24.0+ or containerd 2.0+
  • Kubernetes 1.33+ with GPU Operator (required for production deployments)
  • Minimum 8 CPU cores, 16GB RAM, 500GB storage (varies by deployment size)
  • Network access for container image registry during installation phase (air-gapped deployments supported via offline image transfer - contact support@basebox.ai for details)

Note: Ubuntu 22.04 LTS is supported for existing systems. For new installations, always use Ubuntu 24.04 LTS.

Why Ubuntu 24.04 LTS:

  • Best NVIDIA CUDA documentation and support
  • Most comprehensive documentation for GPU workloads
  • Longest support lifecycle (until April 2029)
  • Extensive Docker and Kubernetes community resources
  • Industry standard for GPU-accelerated workloads

Not Recommended: Microsoft Windows (vLLM requires Linux; Windows Server lacks native GPU container support)

Introduction

This guide provides step-by-step instructions for preparing a new server machine for basebox on-premise deployment. The recommendations are based on our experience deploying GPU-accelerated AI inference workloads using vLLM 0.14.0, Docker, and Kubernetes in production environments.

basebox is a Rust-based backend with Vue frontend that requires GPU acceleration for LLM inference. We recommend Kubernetes deployment (with Helm) for all production deployments. This guide focuses on preparing the operating system and foundational software stack.

Documentation Approach:

This guide provides simplified, step-by-step instructions tailored for basebox deployments. While we recommend consulting the official documentation from NVIDIA, Docker, Kubernetes, and Ubuntu for comprehensive details and latest updates, our intention is to streamline the installation process for our customers by consolidating the essential steps and configurations specific to basebox requirements. Each section includes links to the official documentation for reference, which should be consulted first for authoritative information, troubleshooting, and advanced configurations.

Important Notes:

  • Requirements may vary based on your specific deployment configuration
  • Air-gapped deployment is supported - contact support@basebox.ai for offline installation procedures
  • Hardware specifications depend on your GPU models and workload requirements
  • If your environment differs from these recommendations, contact support@basebox.ai

Pre-Installation Checklist

Before beginning installation, verify you have:

Hardware Inventory:

  • NVIDIA GPU(s) installed and recognized by BIOS (run lspci | grep -i nvidia after OS installation)
  • GPU Compute Capability 7.0+ (V100, T4, RTX20xx, A100, L4, H100, etc.)
  • Minimum 8 CPU cores (16+ recommended for production)
  • Minimum 16GB RAM (32GB+ recommended, 64GB+ for large models)
  • Minimum 500GB storage (1TB+ recommended for model storage)
  • Adequate power supply and cooling for GPUs

Network Requirements:

  • Internet connectivity during installation (or offline installation packages prepared)
  • Required ports accessible (22/TCP for SSH, 6443/TCP for K8s API if applicable)
  • Access credentials for server management

Access and Security:

  • Physical or remote access to server (IPMI, iLO, or similar)
  • BeyondTrust PRA or similar privileged access management configured (if applicable)
  • Backup plan for existing data (if not a fresh installation)

Installation Media:

  • Ubuntu 24.04 LTS Server ISO downloaded
  • USB drive or DVD for installation (if not using remote installation)

Verification Commands (run after OS installation): 

# Check CPU cores
lscpu | grep "^CPU(s):"

# Check RAM
free -h

# Check storage
df -h

# Check GPU detection
lspci | grep -i nvidia

Issues or Different Requirements? Contact support@basebox.ai before proceeding.

Ubuntu 24.04 LTS

Ubuntu 24.04 LTS (Noble Numbat)

  • Release: April 2024
  • Support until: April 2029
  • Download: Ubuntu 24.04 LTS Server
  • Architecture: x86_64
  • Latest LTS release with extended support period
  • Best NVIDIA CUDA support and documentation
  • Most comprehensive ecosystem for GPU workloads

Note: Ubuntu 22.04 LTS is supported for existing systems that are already running basebox. For all new installations, use Ubuntu 24.04 LTS.

Step-by-Step Installation Guide

Note: Each step below includes official documentation links at the beginning. We recommend reviewing the official documentation first for comprehensive information, then using our simplified steps for basebox-specific installation. Our guide consolidates essential steps to streamline the process for basebox customers.

Step 1: Install Ubuntu Server

  1. Download Ubuntu Server ISO

  2. Create Installation Media

    • Write ISO to USB drive using tools like Rufus (Windows) or dd (Linux)
    • Boot from installation media

  3. Installation Options

    • Select "Ubuntu Server" (minimal installation)
    • Choose "Use an entire disk" for automatic partitioning (or custom if needed)
    • Set up user account and hostname
    • Enable OpenSSH server during installation (recommended for remote access)

  4. Post-Installation

    # Update package lists
sudo apt update && sudo apt upgrade -y

# Reboot if kernel was updated
sudo reboot

Official Documentation: Ubuntu Server Installation Guide

Step 2: Install NVIDIA GPU Drivers

Prerequisites:

  • NVIDIA GPU installed and detected: lspci | grep -i nvidia
  • Internet connection for driver downloads

NVIDIA Driver Installation (Recommended Method):

# Install latest NVIDIA drivers automatically
sudo ubuntu-drivers autoinstall

# Reboot to load drivers
sudo reboot

# Verify installation
nvidia-smi

Verification:

# Check driver version (should be 550+ for CUDA 13.0 support)
nvidia-smi

# Expected output shows:
# - Driver Version: 550.xx.xx or higher
# - CUDA Version: 13.0+
# - GPU(s) listed with memory information

Official Documentation:

Step 3: Install CUDA Toolkit

Install CUDA Toolkit 13.0:

Always install CUDA 13.0 - the latest version. CUDA 12.9 is supported as fallback if 13.0 is unavailable.

# Add NVIDIA CUDA repository
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update

# Install CUDA Toolkit 13.0
sudo apt-get install -y cuda-toolkit-13-0

# Set environment variables (add to ~/.bashrc)
echo 'export PATH=/usr/local/cuda-13.0/bin${PATH:+:${PATH}}' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-13.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc
source ~/.bashrc

# Verify installation
nvcc --version

Fallback Option (CUDA 12.9):

Use CUDA 12.9 if CUDA 13.0 is unavailable in your repository:

sudo apt-get install -y cuda-toolkit-12-9
echo 'export PATH=/usr/local/cuda-12.9/bin${PATH:+:${PATH}}' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.9/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc
source ~/.bashrc

Official Documentation:

Step 4: Install Docker Engine

Requirements: Docker Engine 24.0+ (with containerd 2.0+ for Kubernetes 1.35+)

# Remove old versions
sudo apt-get remove docker docker-engine docker.io containerd runc

# Add Docker's official GPG key
sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add Docker repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Start and enable Docker
sudo systemctl start docker
sudo systemctl enable docker

# Add current user to docker group (optional, for non-root access)
sudo usermod -aG docker $USER
# Log out and back in for group changes to take effect

# Verify installation
docker --version
sudo docker run hello-world

Official Documentation: - Docker Engine Installation for Ubuntu

Step 5: Install NVIDIA Container Toolkit

Required for GPU access in Docker containers:

# Configure the production repository
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install NVIDIA Container Toolkit
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configure Docker to use NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Verify GPU access in containers
sudo docker run --rm --gpus all nvidia/cuda:12.9.0-base-ubuntu22.04 nvidia-smi

Official Documentation: - NVIDIA Container Toolkit Installation Guide

Step 6: Install Kubernetes (Required for Production)

Install Kubernetes 1.33+:

Kubernetes is required for all production deployments. Install Kubernetes 1.33 or later for optimal stability and feature support.

# Install required packages
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

# Add Kubernetes repository (for 1.33)
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

# Install Kubernetes components
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

# Initialize Kubernetes cluster (on master node)
sudo kubeadm init --pod-network-cidr=10.244.0.0/16

# Set up kubeconfig for regular user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Install network plugin (Calico example)
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

# Install NVIDIA GPU Operator
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm install --wait --generate-name \
  -n gpu-operator --create-namespace \
  nvidia/gpu-operator

Official Documentation:

Step 7: Install Helm (Required for Kubernetes deployment)

Install Helm (Required for Kubernetes deployment):

# Download Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# Verify installation
helm version

Official Documentation:

Step 8: System Configuration

Configure system settings for optimal performance:

# Increase shared memory size (required for model loading)
sudo mkdir -p /dev/shm
echo 'tmpfs /dev/shm tmpfs defaults,size=64g 0 0' | sudo tee -a /etc/fstab
sudo mount -o remount /dev/shm

# Disable swap (required for Kubernetes)
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

# Set kernel parameters for Kubernetes
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

# Configure firewall (if using UFW)
sudo ufw allow 22/tcp   # SSH
sudo ufw allow 6443/tcp # Kubernetes API (if master)
sudo ufw allow 10250/tcp # Kubelet API
sudo ufw enable

Step 9: Verification Checklist

Verify all components are installed correctly:

# Check Ubuntu version
lsb_release -a

# Check NVIDIA driver
nvidia-smi

# Check CUDA toolkit
nvcc --version

# Check Docker
docker --version
sudo docker run --rm --gpus all nvidia/cuda:12.9.0-base-ubuntu22.04 nvidia-smi

# Check Kubernetes (if installed)
kubectl version --client
kubectl get nodes

# Check GPU Operator (if Kubernetes installed)
kubectl get pods -n gpu-operator

# Check system resources
free -h
df -h
lscpu

All checks passing? Proceed to deployment. Issues? See Troubleshooting section or contact support@basebox.ai

Alternative Operating Systems

Ubuntu 24.04 LTS is the recommended operating system for basebox. The following table summarizes our evaluation of alternative operating systems for basebox deployments:

Operating System Description Advantages Comparison with Ubuntu Recommendation
Ubuntu 24.04 LTS Latest LTS release (April 2024) Best NVIDIA CUDA documentation and support
Most comprehensive GPU workload documentation
Longest support lifecycle (until April 2029)
Extensive Docker/Kubernetes community resources
Industry standard for GPU-accelerated workloads		 N/A (baseline) ✅ Recommended - Use for all new installations
Ubuntu 22.04 LTS Previous LTS release (April 2022) Stable, well-tested release
Good NVIDIA CUDA support
Support until April 2027		 Older release with shorter support lifecycle
Less recent CUDA toolkit packages
Fewer community resources for latest features		 ⚠️ Supported - Only for existing systems already running basebox
Debian Stable Ubuntu's parent distribution More conservative package versions
Smaller default footprint
Strong security focus
No commercial ties		 Less NVIDIA-specific documentation
Older CUDA toolkit packages
Smaller GPU workload community
Fewer Docker/Kubernetes tutorials		 ⚠️ Supported - Requires additional configuration. Not officially tested. Contact support if needed
Red Hat Enterprise Linux (RHEL) Enterprise Linux distribution Commercial support from Red Hat
Security hardening and compliance certifications
10-year support lifecycle
Red Hat OpenShift integration
Partner-validated GPU configurations		 Licensing costs
Less accessible documentation
More complex installation
Fewer NVIDIA driver packages		 ⚠️ May work - Consider if you require commercial support contracts or have existing RHEL infrastructure
SUSE Linux Enterprise Server (SLES) Enterprise Linux distribution Enterprise support available
Strong security features
Long-term support		 Less NVIDIA ecosystem support
Smaller community resources
Higher cost
More complex setup		 ⚠️ May work - Consider only if you have specific SLES requirements or existing SLES infrastructure
Rocky Linux / AlmaLinux RHEL-compatible alternatives Community-supported RHEL alternatives
No licensing costs
RHEL compatibility		 Less NVIDIA-specific documentation
Limited GPU workload community
Not extensively tested with basebox		 ⚠️ May work - Not officially tested. Requires thorough testing in non-production environment
Microsoft Windows Server Windows-based server OS Enterprise Windows integration
Familiar Windows administration		 vLLM requires Linux (not supported on Windows)
Limited GPU container passthrough
Docker uses Hyper-V virtualization overhead
Reduced Kubernetes GPU support		 ❌ Not Supported - vLLM requires Linux. Use Ubuntu 24.04 LTS instead

Notes:

  • Ubuntu 24.04 LTS is the officially supported and recommended distribution for new installations
  • Alternative Linux distributions may work but require additional configuration, testing, and troubleshooting

  • If you must use an alternative distribution, follow these steps:

    1. Verify NVIDIA CUDA Toolkit support for your distribution version
    2. Ensure Docker and Kubernetes packages are available
    3. Test thoroughly in a non-production environment first
    4. Contact support@basebox.ai for assistance with non-standard configurations

  • Microsoft Windows Server is not supported due to vLLM's Linux requirement

Troubleshooting Decision Tree

Problem: GPU Not Detected

START: Run `nvidia-smi`
│
├─ ERROR: "command not found"
│  └─ SOLUTION: NVIDIA drivers not installed
│     └─ ACTION: Go to Step 2 and install drivers
│
├─ ERROR: "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver"
│  └─ SOLUTION: Driver installation incomplete or incorrect
│     └─ ACTION:
│        1. Run: sudo apt-get purge nvidia-*
│        2. Reinstall: sudo ubuntu-drivers autoinstall
│        3. Reboot: sudo reboot
│
├─ ERROR: "No devices were found"
│  └─ SOLUTION: GPU not recognized by system
│     └─ ACTION:
│        1. Check physical connection
│        2. Verify in BIOS: GPU should be visible
│        3. Check: lspci | grep -i nvidia
│        4. If not visible, hardware issue - check power/seating
│
└─ SUCCESS: GPU information displayed
   └─ NEXT: Verify CUDA version is 13.0 (or 12.9 as fallback)

Problem: CUDA Version Mismatch

START: Run `nvcc --version`
│
├─ ERROR: "command not found"
│  └─ SOLUTION: CUDA Toolkit not installed
│     └─ ACTION: Go to Step 3 and install CUDA Toolkit
│
├─ VERSION: CUDA 11.x or 12.0-12.8
│  └─ SOLUTION: CUDA version too old for vLLM 0.14.0
│     └─ ACTION:
│        1. Remove old CUDA: sudo apt-get purge cuda-*
│        2. Install CUDA 13.0 (see Step 3)
│        3. Update environment variables
│        4. Verify: nvcc --version
│
├─ VERSION: CUDA 13.0
│  └─ SUCCESS: Recommended version installed
│
├─ VERSION: CUDA 12.9
│  └─ ACCEPTABLE: Fallback version (use if 13.0 unavailable)
│
└─ ERROR: CUDA version doesn't match nvidia-smi
   └─ SOLUTION: Driver/toolkit mismatch
      └─ ACTION:
         1. Check driver supports CUDA version: nvidia-smi
         2. Install compatible driver (550+): see Step 2
         3. Reboot

Problem: Docker Can't Access GPU

START: Run `sudo docker run --rm --gpus all nvidia/cuda:12.9.0-base-ubuntu22.04 nvidia-smi`
│
├─ ERROR: "docker: Error response from daemon: could not select device driver"
│  └─ SOLUTION: NVIDIA Container Toolkit not installed
│     └─ ACTION: Go to Step 5 and install toolkit
│
├─ ERROR: "docker: unknown flag: --gpus"
│  └─ SOLUTION: Docker version too old
│     └─ ACTION:
│        1. Check version: docker --version
│        2. Upgrade Docker: sudo apt-get install --upgrade docker-ce
│        3. Restart: sudo systemctl restart docker
│
├─ ERROR: Permission denied or similar
│  └─ SOLUTION: NVIDIA runtime not configured
│     └─ ACTION:
│        1. Configure runtime: sudo nvidia-ctk runtime configure --runtime=docker
│        2. Restart Docker: sudo systemctl restart docker
│        3. Verify: cat /etc/docker/daemon.json
│
└─ SUCCESS: nvidia-smi output displayed in container
   └─ NEXT: Proceed with basebox deployment

Problem: Kubernetes GPU Not Available

START: Run `kubectl get nodes -o json | jq '.items[].status.allocatable'`
│
├─ NO GPU RESOURCES SHOWN
│  └─ SOLUTION: GPU Operator not installed or not working
│     └─ ACTION:
│        1. Check pods: kubectl get pods -n gpu-operator
│        2. If missing: Install GPU Operator (see Step 6)
│        3. Check logs: kubectl logs -n gpu-operator <pod-name>
│        4. Verify node labels: kubectl get nodes --show-labels
│
├─ ERROR: Pods in CrashLoopBackOff
│  └─ SOLUTION: Configuration issue
│     └─ ACTION:
│        1. Check driver version: nvidia-smi
│        2. Verify containerd config: containerd config dump | grep nvidia
│        3. Check GPU Operator docs for compatibility
│        4. Contact support: support@basebox.ai
│
└─ SUCCESS: nvidia.com/gpu resources shown
   └─ NEXT: Deploy basebox workloads

Problem: Network/Connectivity Issues

START: Basic connectivity check
│
├─ Can't access package repositories
│  └─ ACTION:
│     1. Check internet: ping -c 4 8.8.8.8
│     2. Check DNS: nslookup ubuntu.com
│     3. Check firewall: sudo ufw status
│     4. For air-gapped: Contact support@basebox.ai for offline setup
│
├─ Can't pull Docker images
│  └─ ACTION:
│     1. Check Docker daemon: systemctl status docker
│     2. Check proxy settings if behind corporate proxy
│     3. Verify network access to registry
│     4. For air-gapped: Contact support@basebox.ai
│
└─ Kubernetes nodes not communicating
   └─ ACTION:
      1. Check firewall rules for K8s ports
      2. Verify pod network: kubectl get pods -A
      3. Check CNI plugin: kubectl get pods -n kube-system

When to Contact Support

Contact support@basebox.ai if:

  • Installation fails after following troubleshooting steps
  • Your hardware configuration is non-standard
  • You need air-gapped/offline installation assistance
  • You encounter errors not covered in this guide
  • You need guidance on custom configurations
  • Your organization has specific security or compliance requirements

Provide the following information when contacting support:

# System Information
uname -a
lsb_release -a
lscpu | grep "Model name"
free -h
df -h

# GPU Information
lspci | grep -i nvidia
nvidia-smi

# Software Versions
nvcc --version
docker --version
kubectl version --client (if applicable)

# Logs (if applicable)
sudo journalctl -u docker -n 100
kubectl logs -n gpu-operator <pod-name> (if applicable)

Next Steps: Deployment

After completing server preparation, proceed with basebox deployment:

For all production deployments, use Kubernetes:

Kubernetes deployment is covered in the Kubernetes/Helm Installation Guide. That guide provides detailed instructions for:

  • Using Helm charts to deploy all basebox services
  • Configuring GPU resource allocation per service
  • Setting up ingress, TLS certificates, and networking
  • Production-grade monitoring and logging

Prerequisites:

  • Kubernetes cluster (1.33+) with GPU Operator installed
  • Helm 3.x installed
  • Access to basebox container image registry during installation
  • For air-gapped deployments: Contact support@basebox.ai for offline image transfer procedures
  • Storage class configured for persistent volumes

Benefits:

  • Automatic scaling and load balancing
  • Rolling updates with zero downtime
  • Resource isolation and security
  • Production-grade monitoring and logging
  • Multi-node cluster support

Docker-Based Deployment

For development/testing:

  • Contact support@basebox.ai for Docker-based deployment guidance
  • Suitable for single-server development environments
  • Not recommended for production workloads

GPU Configuration Examples

basebox supports various GPU configurations. After server preparation, you'll configure GPU allocation based on your hardware:

Single GPU (1x H100, 1x L4, etc.)

  • Allocate GPU to inference service
  • Run RAG server in CPU mode
  • Suitable for: Testing, low-volume production

Multi-GPU (4x H100, 4x L40S, 2x RTX 4090, 2x L40)

  • Allocate all GPUs to inference service for tensor parallelism
  • Run RAG server in CPU mode
  • Suitable for: Production workloads, high concurrency

Mixed Configuration

  • Dedicate primary GPUs to inference
  • Optionally allocate one GPU to RAG for OCR workloads
  • Suitable for: Production with document processing requirements

Note: Specific GPU allocation and model selection will be covered in the basebox deployment guide. Contact support@basebox.ai for guidance on optimal configuration for your hardware.

See: LLM recommendations for further details.

Security Considerations

Initial Server Hardening:

# Configure automatic security updates
sudo apt-get install -y unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

# Configure firewall
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw enable

# Disable root login via SSH (edit /etc/ssh/sshd_config)
# Set: PermitRootLogin no
sudo nano /etc/ssh/sshd_config
sudo systemctl restart sshd

# Set up SSH key authentication (disable password auth if desired)
# Set in /etc/ssh/sshd_config: PasswordAuthentication no

# Install fail2ban for intrusion prevention
sudo apt-get install -y fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Container Security:

  • Use non-root users in containers
  • Scan container images for vulnerabilities
  • Limit container capabilities
  • Use read-only filesystems where possible
  • Implement network policies in Kubernetes

Access Control:

  • Use BeyondTrust PRA or similar privileged access management
  • Implement least-privilege access principles
  • Enable audit logging
  • Regular security updates

Additional Security Resources:

Maintenance and Updates

Regular Maintenance Tasks:

# Update system packages (monthly or as needed)
sudo apt update && sudo apt upgrade -y

# Update NVIDIA drivers (when needed, test in non-production first)
sudo apt update
sudo apt install --upgrade nvidia-driver-550

# Update Docker (when new versions available)
sudo apt update
sudo apt install --upgrade docker-ce docker-ce-cli containerd.io

# Update Kubernetes (if installed, follow upgrade path carefully)
# See: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/

# Clean up unused packages and images
sudo apt autoremove -y
docker system prune -a --volumes  # Use with caution - removes unused data

Monitoring:

  • Monitor GPU temperature and utilization: nvidia-smi -l 1
  • Check system resources: htop, df -h, free -h
  • Monitor Docker containers: docker stats
  • Monitor Kubernetes (if used): kubectl top nodes, kubectl top pods

Monitoring Tools:

Support and Resources

Official Documentation:

basebox Support:

  • Email: support@basebox.ai
  • For: Deployment assistance, air-gapped installations, custom configurations, troubleshooting

Community Support:

Conclusion

This guide provides the recommended path for preparing a server for basebox deployment. Ubuntu 24.04 LTS is the recommended operating system for new installations, offering the best balance of NVIDIA CUDA support, Docker/Kubernetes compatibility, security, and maintainability for GPU-accelerated AI inference workloads.

Ubuntu 22.04 LTS is supported for existing systems. Alternative Linux distributions may work but require additional configuration and are not officially supported. Microsoft Windows Server is not suitable for basebox due to vLLM's Linux requirement and limited GPU container support.

Key Takeaways:

  • Ubuntu 24.04 LTS is the recommended OS for new installations
  • CUDA 13.0 required (12.9 fallback)
  • Kubernetes 1.33+ required for production deployments
  • GPU Compute Capability 7.0+ required
  • Air-gapped deployments supported with assistance

Ready to Deploy?

After completing server preparation, contact support@basebox.ai to proceed with basebox deployment. Our team will guide you through the Kubernetes installation process (the recommended production method).

Document Version History:

  • v2.1 (February 03, 2026): Reviewed, cleaned, simplified and polished
  • v2.0 (February 03, 2026): Updated CUDA requirements, Kubernetes versions, added Quick Start and Troubleshooting sections
  • v1.0 (Initial release): Basic server preparation guide

Feedback: Send documentation feedback to support@basebox.ai