Search and SERP providers
How AI Internet Access works in Aimogen Pro, the six SERP APIs it supports, the free scraping fallbacks, and where search results are injected into prompts.
Aimogen Pro can give any model current information by running a web search, scraping the results and injecting them into the prompt. This is the AI Internet Access feature, configured under Settings › AI Internet Access, with credentials on Settings › API Keys.
How it works#
- A search query is derived from your topic. The Keyword Extractor Prompt controls how.
- The query is sent to whichever search backend is configured.
- The top N results are fetched, where N is
Number Of Search Results To Add, default
3, maximum10. - Results are formatted with the Template For the
%%web_results%%Shortcode. - The formatted block replaces
%%web_results%%inside Modify Prompt For the AI Writer Using Internet Search Results. - That combined prompt goes to the model.
Because it works by prompt injection, internet access works with every provider, including local Ollama models.
Backends#
Aimogen Pro supports six paid SERP APIs and two free scraping fallbacks. All credentials go on Settings › API Keys.
| Backend | Field | Notes |
|---|---|---|
| Google Custom Search | Google SERP API Key plus Google SERP API Search Engine ID (CX Value) | Official Google. Requires the Custom Search API enabled and a search engine configured to search the whole web |
| SerpAPI | SerpAPI API Key | Also used for Related Headings |
| ValueSERP | ValueSERP API Key | Also used for Related Headings |
| SpaceSERP | SpaceSERP API Key | Internet Access only |
| Serper.dev | Serper.dev API Key | Also used for Related Headings |
| Tavily | Tavily API Key | Built for AI retrieval; returns cleaned content |
| Bing scraping | Use Bing Search Results Scraping | No key. The default fallback |
| DuckDuckGo scraping | Use DuckDuckGo Search Results Scraping | No key |
Configuring Google Custom Search#
Two values are needed and both are easy to get wrong:
- API key from Google Cloud Console credentials, with the Custom Search API enabled for the project.
- Search engine ID (CX) from Programmable Search Engine.
By default a Programmable Search Engine only searches sites you list. To search the whole web you must change that in its configuration — the Knowledge Base has a walkthrough, How to create a Google Custom Search Engine that searches the entire web, linked from the settings help text.
Source ordering and rotation#
If several backends are configured, they are tried in the order shown on the settings screen.
Randomize the Internet Access API sources order shuffles the order on every run instead. That spreads load across providers and helps stay inside free-tier quotas, at the cost of predictable result quality.
Where internet access applies#
Internet access is opt-in per context, so you can enable it for the chatbot but not for bulk runs, or the reverse. Each of these has its own checkbox under Settings › AI Internet Access:
| Context | Setting |
|---|---|
| Single post creator | internet_single |
| Bulk post creator (whole post) | internet_bulk |
| Bulk: title | internet_bulk_title |
| Bulk: sections | internet_bulk_sections |
| Bulk: introduction | internet_bulk_intro |
| Bulk: main content | internet_bulk_content |
| Bulk: Q&A | internet_bulk_qa |
| Bulk: conclusion | internet_bulk_outro |
| Bulk: excerpt | internet_bulk_excerpt |
| AI Content Editor | internet_edit |
| Chatbot | internet_chat_short |
[aimogen-article] shortcode | internet_article_short |
| Editing shortcodes | internet_edit_short |
| Related content | internet_related |
| Assistants | internet_assistant |
| AI Forms | internet_forms |
| OmniBlocks | internet_omni |
Prompt templates#
Modify Prompt For the AI Writer Using Internet Search Results is the wrapper template. Available placeholders:
| Placeholder | Expands to |
|---|---|
%%original_query%% | The original search query |
%%current_date%% | The current date, so the model knows what "recent" means |
%%web_results%% | The formatted search results |
Template For the %%web_results%% Shortcode formats each individual result, with:
| Placeholder | Expands to |
|---|---|
%%result_title%% | Result title |
%%result_link%% | Result URL |
%%result_snippet%% | Result snippet |
%%result_counter%% | Position in the list |
Location targeting#
Search Results Location (Optional) restricts results to a country. The format is country plus a
two-letter code, for example countryDE for Germany. This matters most for non-US English-language
sites, where unrestricted results skew heavily American.
Related Headings is a separate feature#
Do not confuse the two. Related Headings scrapes the "people also ask" style questions for a keyword
and offers them as %%related_questions_KEYWORD%% in prompts. It uses SerpAPI, ValueSERP or Serper.dev
where configured, and falls back to Bing scraping.
Internet Access injects page content. Related Headings injects question phrasings. They are configured in different places and can be used independently.
The front-end shortcode#
[aimogen-internet-search keyword="..."] performs a search and renders the results directly. See the
shortcode reference.
Common problems#
No results injected No backend is reachable. Check that at least one API key is valid, or that Bing scraping is enabled.
Google returns results only from a few sites Your Programmable Search Engine is not configured to search the whole web.
Generation became very slow Each enabled context adds a search plus page fetches. Reduce the number of results, or narrow which contexts have internet access enabled.
Results are stale or irrelevant Tune the Keyword Extractor Prompt — the model derives the query from your topic, and a vague topic produces a vague query.
Related#
- Perplexity — models that search as part of generating
- Data and utility providers — scraping configuration and proxies
- Placeholder reference
Still stuck? Open a support ticket and include the diagnostics from Aimogen Pro › System & Logs › System Info.