Licensed to be used in conjunction with basebox, only.
// installation
Deploy Whisper speech-to-text with Helm
Applies to
Product: basebox Server · Audience: Platform Operator · Reference: basebox 1.9.3, chart 0.3.35
Speech-to-text (STT) requires an accessible Whisper transcription endpoint. On Kubernetes, the service can run as a separate Deployment and Service defined through Helm extraObjects. AISRV sends audio to the HTTP endpoint; the frontend enables microphone input separately.
This guide documents a configuration pattern verified in a deployment using basebox 1.9.3 and inference image v1.9.4. It does not guarantee compatibility with other chart or image versions. Verify the deployed release and Helm templates before adapting it.
Check the prerequisites before deployment
- Kubernetes, Helm, and a suitable NVIDIA GPU with a functioning container runtime are available.
- The
openai/whisper-large-v3-turbomodel files are present in a model cache accessible to the pod, verified against the approved package checksum. - The required inference image is available in the cluster; disconnected environments require a verified offline image import.
- An existing Kubernetes Secret provides the inference API key. Use Secret references only; never commit plaintext keys to Helm values.
- The intended GPU node, GPU/MIG allocation, and existing model-cache PVC are known.
- Production changes have current protected Helm values, backups, a rollback plan, and approval.
kubectl -n basebox get deployments,services,pvc
kubectl -n basebox get secrets
kubectl get runtimeclass
kubectl get nodes -o wide
# On the approved GPU host (not inside a Kubernetes manifest):
nvidia-smi -L
GPU allocation varies by cluster
The reference below uses nvidia-legacy and an explicit NVIDIA_VISIBLE_DEVICES UUID without a Kubernetes GPU resource request. This was used in the verified installation but is not a general GPU scheduling pattern. For another cluster, agree on GPU Operator/MIG resource allocation with the platform team. Never copy GPU UUIDs or node names from a customer installation.
Connect AISRV and the frontend to Whisper
Add these settings as an overlay on top of the existing approved Helm values. The Secret name is a placeholder; its key must match the provisioned Secret.
aisrv:
env:
AISRV_WHISPER_URL: http://whisper:8081/v1/audio/transcriptions
AISRV_WHISPER_MODEL: openai/whisper-large-v3-turbo
AISRV_WHISPER_API_KEY:
valueFrom:
secretKeyRef:
name: <EXISTING_INFERENCE_API_SECRET>
key: inference_api_key
frontend:
env:
VITE_BB_STT_ENABLED: "true"
AISRV_WHISPER_URL points to Kubernetes Service whisper, port 8081, with the OpenAI-compatible route /v1/audio/transcriptions. VITE_BB_STT_ENABLED enables STT in the frontend. Verify both settings reach the running Deployments.
Define the Whisper Deployment and Service
The extraObjects example below follows the structure validated in the reference installation. Replace every <...> placeholder. Check the model path, inference image, RuntimeClass, GPU selection, Secret mount, resources, and probes against the target environment before use.
extraObjects:
whisper-deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: whisper
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: whisper
template:
metadata:
labels:
app.kubernetes.io/name: whisper
app.kubernetes.io/component: stt
spec:
# Example: explicit GPU selection with a tested legacy runtime.
# Replace both values and verify node/GPU pairing.
runtimeClassName: nvidia-legacy
nodeSelector:
kubernetes.io/hostname: <APPROVED_GPU_NODE>
imagePullSecrets:
- name: <EXISTING_IMAGE_PULL_SECRET>
containers:
- name: whisper
image: gitea.basebox.health/basebox-distribution/inference:v1.9.4
imagePullPolicy: IfNotPresent
args:
- /model-cache/models/openai/whisper-large-v3-turbo
- --host
- 0.0.0.0
- --port
- "8081"
- --served-model-name
- openai/whisper-large-v3-turbo
- --dtype
- auto
- --gpu-memory-utilization
- "0.05"
- --trust-remote-code
env:
- name: NVIDIA_VISIBLE_DEVICES
value: "<APPROVED_GPU_UUID>"
- name: NVIDIA_DRIVER_CAPABILITIES
value: compute,utility
- name: HF_HOME
value: /model-cache
- name: VLLM_NO_USAGE_STATS
value: "1"
- name: LD_LIBRARY_PATH
value: /usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/lib/x86_64-linux-gnu
ports:
- name: http
containerPort: 8081
resources:
requests:
cpu: "500m"
memory: 4Gi
limits:
cpu: "8"
memory: 32Gi
startupProbe:
httpGet: {path: /health, port: http}
periodSeconds: 10
failureThreshold: 180
readinessProbe:
httpGet: {path: /health, port: http}
periodSeconds: 10
livenessProbe:
httpGet: {path: /health, port: http}
periodSeconds: 30
volumeMounts:
- name: model-cache
mountPath: /model-cache
- name: inference-api-key
mountPath: /run/secrets/inference_api_key
subPath: inference_api_key
readOnly: true
- name: dshm
mountPath: /dev/shm
volumes:
- name: model-cache
persistentVolumeClaim:
claimName: <EXISTING_MODEL_CACHE_PVC>
- name: inference-api-key
secret:
secretName: <EXISTING_INFERENCE_API_SECRET>
items:
- key: inference_api_key
path: inference_api_key
- name: dshm
emptyDir:
medium: Memory
sizeLimit: 16Gi
whisper-service:
apiVersion: v1
kind: Service
metadata:
name: whisper
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: whisper
ports:
- name: http
port: 8081
targetPort: http
Security and compatibility review
--trust-remote-code was present in the historical reference. Only retain it when the model source and code have been reviewed and approved. Model files, container image, and GPU allocation must be compatible before production deployment.
Validate the Helm changes before upgrading
Use the current chart and existing customer values. Keep live values and rendered manifests in a protected directory outside Git: they may contain sensitive information.
umask 077
PRIVATE_DIR="$HOME/private/basebox-stt-review"
mkdir -p "$PRIVATE_DIR"
chmod 700 "$PRIVATE_DIR"
helm get values basebox -n basebox -o yaml > "$PRIVATE_DIR/current-values.yaml"
# CHART references the verified chart package for the target release.
# STT_OVERLAY references the adapted and approved YAML file.
helm lint "$CHART" -f "$PRIVATE_DIR/current-values.yaml" -f "$STT_OVERLAY"
helm template basebox "$CHART" -n basebox \
-f "$PRIVATE_DIR/current-values.yaml" -f "$STT_OVERLAY" \
> "$PRIVATE_DIR/rendered.yaml"
Inspect the rendered output: whisper must appear as a Deployment and Service, the Service must expose port 8081, image and Secret references must match, and existing workloads must not be unintentionally modified. Only then proceed with the approved deployment process in Updates. Do not rerun historical production scripts without review.
Verify Whisper after deployment
kubectl -n basebox rollout status deployment/whisper --timeout=20m
kubectl -n basebox get deployment whisper
kubectl -n basebox get service whisper
kubectl -n basebox get endpoints whisper
kubectl -n basebox get pods -l app.kubernetes.io/name=whisper
Review pod events and model initialization. Then transcribe a permitted non-sensitive audio sample. Test the frontend microphone separately in a browser with microphone permission enabled.
- Ready Deployment and Service, loaded model: required for backend transcription.
- Successful backend transcription response: backend capability verified.
- Successful browser microphone recording and text insertion: end-to-end behavior verified.
In the reference installation, the Deployment, Service, and automated backend transcription test passed. The browser microphone test was not completed because the remote client did not expose microphone permission.
Troubleshoot startup and transcription problems
| Symptom | Check |
|---|---|
Pod stays Pending |
GPU node, RuntimeClass, PVC, resource requests |
ImagePullBackOff |
Image availability and registry access |
| Pod starts but never becomes Ready | Model path, model cache mount, GPU memory, health endpoint, startup time |
| AISRV cannot transcribe | Service DNS, port 8081, route, API-key Secret reference |
| Microphone option missing | Actual frontend VITE_BB_STT_ENABLED setting, browser permissions |
| Other GPU services are affected | Shared GPU utilization, memory reserve, explicit device selection |
Further reading: Deploy service models · AISRV · Inference Server · Updates · Using speech-to-text