Licensed to be used in conjunction with basebox, only.
// admin
Model configuration
Overview
Settings around a model from the application's perspective: context size, output limits, temperature, reasoning default. Connecting the inference itself – URL, model identifier, credentials – is the Platform Operator's job; this page is about what you as administrator and app creator can influence and what you should know about the configuration.
What it is for
The same model behaves differently depending on configuration: terse or verbose, sober or creative, fast or thorough. Through configuration you adapt it to an app's purpose – and cap consumption.
Two levels of configuration
| Level | Who | What |
|---|---|---|
| Inference connection (Server) | Platform Operator, via Helm values | Endpoint, model identifier, context size, output limit, defaults for temperature, top-p, repetition penalty |
| Application | Administrator and app creators, in the interface | Default model, model per app, thinking effort per app, temperature per app, instructions |
In basebox Cloud, basebox handles the inference level entirely.
The most important parameters
Context size – how many tokens a chat can hold in total (questions, answers, documents). It is a property of the connected model and is set on the server via AISRV_LLM_CONTEXT_SIZE; the "About" page shows the active value. For users: Context window.
Output limit – the maximum length of an answer in tokens (AISRV_LLM_MAX_TOKENS, default 8000). Long answers are no longer cut short; the limit is an upper bound, not a target.
Temperature – controls the variability of answers (0 = focused and reproducible, 1 = creative). The server default is in AISRV_LLM_TEMPERATURE; per app the temperature can be adjusted in the app editor. For knowledge-base apps and factual answers a low value is recommended.
Thinking effort (reasoning) – with models that support reasoning: Low · Medium · High · Very high · Maximum per app under Advanced fields. Users additionally choose Low/Medium/High in the chat or switch reasoning off. More effort = better results on complex tasks, but longer response time and more tokens. See Default models.
Repetition penalty / top-p – fine-tuning of the inference (AISRV_LLM_REPETITION_PENALTY, AISRV_LLM_TOP_P), set by the Platform Operator. Relevant for models that tend to repeat themselves.
Where to find the setting
- Per app: "All apps" → Edit app → Advanced fields tab (model, thinking effort) or the temperature control.
- Organisation: default model in Model selection.
- Inference (Server): the Platform Operator's Helm values – see Configure models and AISRV → LLM configuration.
Step by step: tuning an app to its purpose
- Open the app in the app editor.
- Model: leave "Use default" unless the app needs a specific model.
- Thinking effort: Low for uniform tasks (rephrasing, translating), High or above for analyses and multi-step tasks.
- Temperature: low for facts and knowledge-base answers, higher for drafts and ideas.
- App instructions: specify answer length, format and tone – this often has more effect than any parameter.
- Save and test in the preview.
Notes
Note
- Configuration is no substitute for instructions. "Answer in at most five sentences" in the app instructions is more effective than a low output limit.
- The organisation's system prompt applies in every app as well – see System prompt.
- With an external inference endpoint, the model identifier configured in AISRV must exactly match what the endpoint reports under
/v1/models. - If the Platform Operator changes the context size, this affects all chats; the "About" page shows the new value.
Frequently asked questions
Can I increase the context size as an administrator? No. It is a property of the connected model and is set on the server by the Platform Operator – bounded by model and GPU memory.
Why do I get different answers to the same question? Language models do not answer deterministically; a lower temperature makes answers more reproducible.
Where do I set the reasoning default for the whole installation? Administrators set the default per app and for the whole installation.
Does the app temperature also apply to the API?
No. Via the API, developers set temperature per request themselves.
Need help? Contact support