Retrieval and prompt injection

How Aimogen Pro retrieves relevant chunks from the embeddings index and injects them into prompts, and how to tune the number and quality of results.

Retrieval is the read side of the index: turning a question into the handful of chunks that answer it.

What happens on a request#

1. The query text is embedded with the same model used for indexing.
2. The vector store returns the most similar chunks from the configured namespace.
3. Chunks below the score threshold are discarded, where the store supports one.
4. The surviving chunks are injected into the prompt.
5. The model answers using them.

Steps 2 and 3 are the ones you tune.

How many chunks#

The trade-off:

Too few and the answer is missing information that was in the index. The Pinecone default of 1 is often too few — a single chunk rarely contains a complete answer.

Too many and three things get worse: the prompt gets expensive, irrelevant chunks distract the model, and long context degrades attention.

Three to five is a good starting point for most knowledge bases. Raise it only if you can see answers missing information that is indexed.

Score thresholds#

OpenAI Vector Store Minimum Score (%) discards weak matches. 0 disables it; the settings help text recommends 20.

This is more valuable than it looks. Without a threshold, a question your index cannot answer still retrieves the three least-bad chunks, and the model — being told these are relevant — builds an answer from them. With a threshold, nothing is retrieved and the model can honestly say it does not know.

If your chatbot confidently answers questions outside its knowledge base, set a threshold first.

Namespaces#

The namespace decides which slice of the index is searched. Three places set it, in increasing precedence:

1. The per-context namespace field under Settings › Embeddings › Enable Embeddings For
2. The embeddings_namespace shortcode attribute
3. The aiomatic_embedding_namespace filter

// Search a namespace matching the current post category
add_filter( 'aiomatic_embedding_namespace', function ( $namespace ) {
    if ( is_singular( 'post' ) ) {
        $cats = get_the_category();
        if ( ! empty( $cats ) ) {
            return 'cat-' . $cats[0]->slug;
        }
    }
    return $namespace;
} );

Injection into prompts#

Retrieved content is added to the prompt automatically. It is not something you place with a placeholder — enabling retrieval for a context is what puts it there.

That means the surrounding prompt should expect it:

Answer the question using only the reference information provided to you.

If the reference information does not contain the answer, say so and suggest
the visitor contacts support@example.com. Do not use general knowledge to fill
gaps, and do not guess.

When you use reference information, stay within what it actually says.

Improving retrieval quality#

Index the right unit of text. A chunk that answers a question is better than a chunk that mentions the topic. Documentation split by section retrieves better than whole pages.

Include titles and context in the embedding template. A bare paragraph is hard to match; the same paragraph with its title and category is much easier. See Adding content.

Consider AI rewriting before indexing. Optimize The %%post_content%% Shortcode Using AI condenses a post to its factual content, removing navigation and marketing noise that dilutes the vector.

Use namespaces. Searching a smaller, relevant set beats searching everything.

Keep the index current. Stale chunks produce confident wrong answers.

Retrieval and cost#

Every retrieved chunk is input tokens on every affected request. In a chatbot conversation with 5 chunks of 500 tokens each, that is 2,500 extra input tokens per message, on top of the conversation history.

This is usually the largest hidden cost of a RAG chatbot. Reduce it by lowering the chunk count and raising the score threshold, so that only genuinely relevant material is injected.

See Controlling cost.

Retrieval as a security surface#

Related#

• Retrieval troubleshooting
• Knowledge and context
• Controlling cost