Skip to content
Copyright © 2022-2026 basebox GmbH, all rights reserved.
Licensed to be used in conjunction with basebox, only.

// ai chat

What is RAG?

Overview

RAG (Retrieval-Augmented Generation) is the method basebox uses to produce answers from your own documents: first search, then answer. Together with your question, the language model receives the matching passages from the knowledge base and answers on that basis – with citations. This page explains what happens and why answers from a knowledge base behave differently from the normal chat.

The principle in one sentence

A language model knows what was in its training data. Your guidelines, contracts and manuals were not in it. RAG closes that gap by presenting the model with the relevant sections of your documents at run time – as if you laid the right pages open in front of a colleague before asking.

What happens when you upload

  1. Extraction – text is read from PDF, Word, e-mail, images (via OCR) and other formats.
  2. Splitting into sections – the document is divided into text sections. You see their number later in the citation.
  3. Embedding – each section is converted into a numerical representation that captures its meaning. Similar content ends up close together.
  4. Storage – sections and embeddings are stored in the app's knowledge base.

Depending on size this takes a few seconds to minutes; the app shows a processing status per file.

What happens when you ask

  1. Your question is converted into an embedding as well.
  2. basebox searches the entire knowledge base – not just the first hits – for the sections whose meaning is closest to your question.
  3. The 20 most relevant sections are selected and passed to the language model together with your question.
  4. The model formulates the answer from those sections and backs up statements with numbered source chips.

Below the answer you see how many documents were searched. A click on a chip shows the passage and offers a download of the original. Details: Knowledge bases & sources.

What this means for your questions

  • Wording matters. The search finds what resembles your question in meaning. Use the terms from the documents – "return" finds different things than "refund".
  • Narrower is better. A specific question shifts the selection of the 20 sections to the area you care about. "What is in the manual?" returns anything; "How do I reset the device?" returns the right page.
  • Only what is in there. If a piece of information is missing from the documents, RAG cannot deliver it. A good model will say so; check the sources when in doubt.
  • The answer is sourced, not guaranteed correct. The chip shows what the model relies on – whether it read the passage correctly you only see by reading it.

RAG, chat upload and web search compared

Knowledge base (RAG) File in the chat Web search
Where the knowledge comes from The app's documents The attached file The public internet
Volume Very large; searched Limited by the context window Current, but unverified
Citations Yes, down to the passage No Links to web pages
Permanent Yes Only in this conversation No
Leaves the environment No No Yes, the search query

A small file for a one-off question belongs in the chat. Anything you or your team need repeatedly belongs in a knowledge base.

Notes

Note

  • For Platform Operators: processing is handled by the services ragsrv and ragsrv-support; the vectors live in PostgreSQL with pgvector. Without a GPU, RAG runs in CPU mode – functional but slower. See Service models.
  • The documents of a knowledge base do not leave your basebox environment. Only the selected sections are passed – inside the environment – to the language model. The exception is the Demo: there these sections go to Claude at Google Vertex AI.

Frequently asked questions

Does the model learn from my documents? No. RAG does not change the model. The documents are searched afresh with every question; remove a document and its content is gone immediately.

Why does the answer cite only 20 sources when more might match? The 20 most relevant sections are shown. Everything is searched. Ask more narrowly to bring other sections to the front.

Can I use RAG in the normal chat? Only via an app with a knowledge base. The normal chat has no knowledge base and therefore no citations.

How to create a knowledge base: Create a new app