@@ -5,7 +5,7 @@ description: FastGPT AI settings explained
...
@@ -5,7 +5,7 @@ description: FastGPT AI settings explained
import { Alert } from '@/components/docs/Alert';
import { Alert } from '@/components/docs/Alert';
AI settings control how AI Chat nodes behave in apps and Workflows, including model selection, response length, image recognition, response format, and reasoning display. This guide explains what each option in the settings modal means and how to choose values for common scenarios.
AI settings control how AI Chat nodes behave in apps and Workflows, including model selection, response length, multimodal recognition, response format, and reasoning display. This guide explains what each option in the settings modal means and how to choose values for common scenarios.
## Where to Find It
## Where to Find It
...
@@ -14,28 +14,30 @@ In the app editor, find the **AI Settings** section, select an AI model, and cli
...
@@ -14,28 +14,30 @@ In the app editor, find the **AI Settings** section, select an AI model, and cli
In a Workflow, click the AI model configuration for the **AI Chat** node. You can open the same settings modal from the settings button on the right.
In a Workflow, click the AI model configuration for the **AI Chat** node. You can open the same settings modal from the settings button on the right.
<Alert icon="🤖" context="success">
<Alert icon="🤖" context="success">
If you do not have specific requirements, selecting a suitable AI model and keeping the other settings at their defaults is usually enough.
If you do not have specific requirements, selecting a suitable AI model and keeping the other
Not every option is always shown. The modal only displays settings supported by the selected model. For example, if a model does not support image recognition, the image recognition switch is hidden. If a model does not support reasoning settings, those options are hidden as well.
Not every option is always shown. The modal only displays settings supported by the selected model. For example, if a model does not support multimodal recognition, multimodal options are hidden. If a model does not support reasoning settings, those options are hidden as well.
## Basic Settings
## Basic Settings
### AI Model
### AI Model
Select the AI model used by the current app or node. Different models vary in response quality, cost, context length, and tool calling capability.
Select the AI model used by the current app or node. Different models vary in response quality, cost, context length, tool calling capability, and multimodal capability.
The model section displays three types of information:
The model section displays several types of information:
- **Credit cost**: A reference cost for model calls. Input content and model output are usually priced separately.
- **Credit cost**: A reference cost for model calls. Input content and model output are usually priced separately.
- **Max context**: The amount of content the model can reference in one request. A larger context window is better for long documents and long conversations.
- **Max context**: The amount of content the model can reference in one request. A larger context window is better for long documents and long conversations.
- **Tool calling**: If supported, the model can use selected app tools to query data, run calculations, or call external capabilities.
- **Tool calling**: If supported, the model can use selected app tools to query data, run calculations, or call external capabilities.
- **Multimodal capability**: If the model supports image, audio, or video input, you can enable the corresponding multimodal recognition capability in AI Settings. Different models may support different media types. Use the capabilities shown in the settings modal as the source of truth.
### Max Histories
### Max Histories
...
@@ -85,13 +87,24 @@ For regular chat, customer support, and Knowledge Base Q&A, keep the default. Ch
...
@@ -85,13 +87,24 @@ For regular chat, customer support, and Knowledge Base Q&A, keep the default. Ch
If you select `json_schema`, you also need to provide the corresponding schema. This option is suitable when the model must return content in a fixed structure.
If you select `json_schema`, you also need to provide the corresponding schema. This option is suitable when the model must return content in a fixed structure.
### Image Recognition
### Multimodal Recognition
If the selected model supports image recognition, this setting controls whether the AI can read images.
If the selected model is configured with multimodal capability, this setting controls whether the AI can read images, audio, or video from user input.
When enabled, the AI can read user-uploaded images or image content from file links. For example, if a user uploads a screenshot, poster, or table image, the model can answer based on the image content.
The available types depend on the model itself. If a model only supports images, only image recognition can be enabled. If it supports images, audio, or video, you can select the needed types.
If the modal shows that the selected model does not support image recognition, switch to a model that supports it.
When enabled, the AI Chat node converts matching uploaded files, or matching media links in the user's question, into model-readable input before sending the request. For example:
- Image recognition: for screenshots, table images, product images, posters, and similar image content.
- Audio recognition: for models that can understand uploaded audio content.
- Video recognition: for models that can understand uploaded video content.
Keep these limits in mind:
1. Even after a type is enabled, the request is filtered again by the actual model capability before it is sent. Unsupported media types are not sent to the model.
2. Media links in the user's question are only parsed when "Extract multimodal files from links" is enabled. Currently, extraction is attempted only when the user's question is under 500 characters, with at most 4 media links processed at a time.
3. Regular document files are not sent directly to the LLM as multimodal input. Documents still need to be parsed into text first.
4. Multimodal recognition depends on the model's own capability. If the modal says the model does not support multimodal recognition, switch to a model that supports the needed media type.
### Hide AI Output
### Hide AI Output
...
@@ -115,7 +128,7 @@ Reasoning effort follows OpenAI's `reasoning_effort` convention, with ai-proxy a
...
@@ -115,7 +128,7 @@ Reasoning effort follows OpenAI's `reasoning_effort` convention, with ai-proxy a
<summary>OpenAI-Compatible Enum and Default Budget Mapping</summary>
<summary>OpenAI-Compatible Enum and Default Budget Mapping</summary>
| FastGPT option | OpenAI-compatible value | Default budget |
| FastGPT option | OpenAI-compatible value | Default budget |
@@ -146,7 +159,7 @@ OpenAI Chat / Completions only parses `reasoning_effort`. When Gemini, Claude, o
...
@@ -146,7 +159,7 @@ OpenAI Chat / Completions only parses `reasoning_effort`. When Gemini, Claude, o
Gemini native requests are parsed from `generationConfig.thinkingConfig`, including `thinkingLevel`, `thinkingBudget`, and `includeThoughts`. When writing to Gemini upstreams, ai-proxy chooses either `thinkingLevel` or `thinkingBudget` based on the model family.
Gemini native requests are parsed from `generationConfig.thinkingConfig`, including `thinkingLevel`, `thinkingBudget`, and `includeThoughts`. When writing to Gemini upstreams, ai-proxy chooses either `thinkingLevel` or `thinkingBudget` based on the model family.
| OpenAI-compatible value | Gemini 3+ Pro | Gemini 3+ non-Pro | gemini-2.5-pro | gemini-2.5-flash | gemini-2.5-flash-lite |
| OpenAI-compatible value | Gemini 3+ Pro | Gemini 3+ non-Pro | gemini-2.5-pro | gemini-2.5-flash | gemini-2.5-flash-lite |
@@ -164,7 +177,7 @@ Gemini 2.5 models clamp the budget to the model's supported range. Some Gemini m
...
@@ -164,7 +177,7 @@ Gemini 2.5 models clamp the budget to the model's supported range. Some Gemini m
Claude native requests are parsed from `thinking` and `output_config`. When writing to Anthropic, AWS Bedrock Claude, or Vertex AI Claude, the payload still follows Claude's thinking format.
Claude native requests are parsed from `thinking` and `output_config`. When writing to Anthropic, AWS Bedrock Claude, or Vertex AI Claude, the payload still follows Claude's thinking format.
Starting from version 4.8.9, FastGPT supports configuring file and image uploads in both `Basic Mode` and `Workflows`. This guide covers how to use the file input feature and explains how document parsing works under the hood.
Starting from version 4.8.9, FastGPT supports configuring file uploads in both `Basic Mode` and `Workflows`. This guide covers how to use file input and explains the difference between document parsing and multimodal file handling.
## Using in Basic Mode
## Using in Basic Mode
...
@@ -31,7 +30,7 @@ In Workflows, find the `File Input` option in the system configuration panel and
...
@@ -31,7 +30,7 @@ In Workflows, find the `File Input` option in the system configuration panel and
There are many ways to use files in Workflows. The simplest approach, shown below, connects document parsing via tool calling — achieving the same result as Basic Mode.
There are many ways to use files in Workflows. The simplest approach, shown below, connects document parsing via tool calling — achieving the same result as Basic Mode.
You can also use Workflows to extract or analyze document content, then pass the results to HTTP requests or other modules to build a document processing pipeline.
You can also use Workflows to extract or analyze document content, then pass the results to HTTP requests or other modules to build a document processing pipeline.
...
@@ -40,7 +39,7 @@ You can also use Workflows to extract or analyze document content, then pass the
...
@@ -40,7 +39,7 @@ You can also use Workflows to extract or analyze document content, then pass the
## How Document Parsing Works
## How Document Parsing Works
Unlike image recognition, LLMs currently cannot parse documents directly. All document "understanding" is achieved by converting documents to text and injecting it into the prompt. The following FAQs explain how this works — understanding the mechanics helps you use document parsing more effectively in Workflows.
Unlike multimodal recognition, LLMs currently cannot parse regular documents directly. All document "understanding" is achieved by converting documents to text and injecting it into the prompt. The following FAQs explain how this works — understanding the mechanics helps you use document parsing more effectively in Workflows.
### How are uploaded files stored in the database?
### How are uploaded files stored in the database?
...
@@ -48,31 +47,37 @@ In FastGPT's chat history, messages with role=user store their value in this str
...
@@ -48,31 +47,37 @@ In FastGPT's chat history, messages with role=user store their value in this str
```ts
```ts
type UserChatItemValueItemType = {
type UserChatItemValueItemType = {
type: 'text' | 'file'
type: 'text' | 'file';
text?: {
text?: {
content: string;
content: string;
};
};
file?: {
file?: {
type: 'img' | 'doc'
type: 'image' | 'audio' | 'video' | 'file';
name?: string;
name?: string;
key?: string;
url: string;
url: string;
};
};
};
};
```
```
Uploaded images and documents are stored as URLs — the parsed document content is not stored.
Uploaded files are stored as URLs — parsed document content is not stored.
### How are images, audio, and video handled?
The document parsing node does not parse multimodal files such as images, audio, or video. These files should be handled by an LLM that supports the corresponding multimodal capability, with multimodal recognition enabled in [AI Settings](./ai_settings).
### How are images handled?
In practice, file input has two different handling paths:
The document parsing node does not process images. Image URLs are filtered out. For image recognition, use an LLM that supports vision.
1. Document parsing: handles document files such as PDF, Word, Excel, Markdown, and HTML, converts their content to text, and provides that text to the AI.
2. Multimodal recognition: handles media files such as images, audio, and video. FastGPT converts them into model-readable input, and a model with the corresponding capability reads them.
### How does the document parsing node work?
### How does the document parsing node work?
The document parsing node accepts an `array<string>` input (file URLs) and outputs a `string` (the parsed content).
The document parsing node accepts an `array<string>` input (file URLs) and outputs a `string` (the parsed content).
* The node only parses URLs with document-type file extensions. If you upload both documents and images, images are ignored.
- The node only parses URLs with document-type file extensions. If you upload both documents and multimodal files, multimodal files are ignored.
* **The document parsing node only processes files from the current workflow run, not files from chat history.**
- **The document parsing node only processes files from the current workflow run, not files from chat history.**
* How multiple documents are concatenated:
- How multiple documents are concatenated:
Multiple files are concatenated using the following template — filename + content, separated by `\n******\n`:
Multiple files are concatenated using the following template — filename + content, separated by `\n******\n`:
title: Knowledge Base Search Methods and Parameters
title: Knowledge Base Search Methods and Parameters
description:
description: This section covers FastGPT's knowledge base architecture, including its QA
This section covers FastGPT's knowledge base architecture, including its QA
storage format and multi-vector mapping, to help you build better knowledge bases. It also explains each search parameter. This guide focuses on practical usage rather than in-depth theory.
storage format and multi-vector mapping, to help you build better knowledge bases. It also explains each search parameter. This guide focuses on practical usage rather than in-depth theory.
---
---
...
@@ -9,9 +8,9 @@ description:
...
@@ -9,9 +8,9 @@ description:
FastGPT uses an Embedding-based RAG approach for its knowledge base. To use FastGPT effectively, you need a basic understanding of how `Embedding` vectors work and their characteristics.
FastGPT uses an Embedding-based RAG approach for its knowledge base. To use FastGPT effectively, you need a basic understanding of how `Embedding` vectors work and their characteristics.
Human text, images, videos, and other media cannot be directly understood by computers. To determine whether two pieces of text are similar or related, they typically need to be converted into a computer-readable format — vectors are one such method.
Human text, images, and other media cannot be directly understood by computers. To determine whether two pieces of text are similar or related, they typically need to be converted into a computer-readable format — vectors are one such method.
A vector is essentially an array of numbers. The "distance" between two vectors can be calculated using mathematical formulas — the smaller the distance, the more similar the vectors. This maps back to text, images, videos, and other media to measure similarity between them. Vector search leverages this principle.
A vector is essentially an array of numbers. The "distance" between two vectors can be calculated using mathematical formulas — the smaller the distance, the more similar the vectors. This maps back to text, images, and other media to measure similarity between them. Vector search leverages this principle.
Since text comes in many types with countless combinations, exact matching is hard to guarantee when converting to vectors for similarity comparison. In vector-based knowledge bases, a `top-k` recall approach is typically used — finding the top `k` most similar results and passing them to an LLM for further `semantic evaluation`, `logical reasoning`, and `summarization`, enabling knowledge base Q&A. This makes vector search the most critical step in the process.
Since text comes in many types with countless combinations, exact matching is hard to guarantee when converting to vectors for similarity comparison. In vector-based knowledge bases, a `top-k` recall approach is typically used — finding the top `k` most similar results and passing them to an LLM for further `semantic evaluation`, `logical reasoning`, and `summarization`, enabling knowledge base Q&A. This makes vector search the most critical step in the process.
...
@@ -19,7 +18,6 @@ Many factors affect vector search accuracy, including: vector model quality, dat
...
@@ -19,7 +18,6 @@ Many factors affect vector search accuracy, including: vector model quality, dat
Retriever precision is relatively straightforward to address, and training vector models is more complex, so optimizing data and query quality becomes a key focus.
Retriever precision is relatively straightforward to address, and training vector models is more complex, so optimizing data and query quality becomes a key focus.
### Improving Vector Search Accuracy
### Improving Vector Search Accuracy
1. Better tokenization and chunking: When a text segment has complete and singular structure and semantics, accuracy improves. Many systems optimize their tokenizers to preserve data completeness.
1. Better tokenization and chunking: When a text segment has complete and singular structure and semantics, accuracy improves. Many systems optimize their tokenizers to preserve data completeness.
...
@@ -40,7 +38,7 @@ In FastGPT, a knowledge base consists of three parts: libraries, collections, an
...
@@ -40,7 +38,7 @@ In FastGPT, a knowledge base consists of three parts: libraries, collections, an
FastGPT uses `PostgreSQL`'s `PG Vector` extension as the vector retriever, with `HNSW` indexing. `PostgreSQL` is used solely for vector search (this engine can be swapped for other databases), while `MongoDB` handles all other data storage.
FastGPT uses `PostgreSQL`'s `PG Vector` extension as the vector retriever, with `HNSW` indexing. `PostgreSQL` is used solely for vector search (this engine can be swapped for other databases), while `MongoDB` handles all other data storage.
In `MongoDB`'s `dataset.datas` collection, vector source data is stored along with an `indexes` field that records corresponding vector IDs. This is an array, meaning a single data entry can map to multiple vectors.
In `MongoDB`'s `dataset.datas` collection, vector source data is stored along with an `indexes` field that records corresponding vector IDs. This is an array, meaning a single data entry can map to multiple vectors. In addition to default text indexes, image content can also generate image description indexes or image vector indexes when the configured models support it.
In `PostgreSQL`, a `vector` field stores the vectors. During search, vectors are recalled first, then their IDs are used to look up the original data in `MongoDB`. If multiple vectors map to the same source data, they are merged and the highest vector score is used.
In `PostgreSQL`, a `vector` field stores the vectors. During search, vectors are recalled first, then their IDs are used to look up the original data in `MongoDB`. If multiple vectors map to the same source data, they are merged and the highest vector score is used.
...
@@ -54,20 +52,68 @@ You can add multiple vectors to a longer text so that if any one vector is match
...
@@ -54,20 +52,68 @@ You can add multiple vectors to a longer text so that if any one vector is match
This means you can continuously improve data chunk accuracy through annotation.
This means you can continuously improve data chunk accuracy through annotation.
### Search Pipeline
### Overall Search Strategy
A Knowledge Base search is not simply "user question -> vector database -> result." Depending on the input and search parameters, FastGPT combines text, images, semantic recall, full-text recall, query optimization, and reranking, then fuses multiple result paths into the final quoted content.
1. Use `Query Optimization` for coreference resolution and query expansion, improving multi-turn conversation search capability and semantic richness.
1. Use `Query Optimization` for coreference resolution and query expansion, improving multi-turn conversation search capability and semantic richness.
2. Use `Concat Query` to improve `Rerank` accuracy during multi-turn conversations.
2. Use `Semantic Search`, `Full-Text Search`, or `Hybrid Search` to recall candidate content.
3. Use `RRF` (Reciprocal Rank Fusion) to merge results from multiple search channels.
3. If the input contains images, use image description search or image vector search depending on model capability.
4. Use `Rerank` for secondary sorting to improve precision.
4. Use `RRF` (Reciprocal Rank Fusion) to merge results from multiple search channels.
5. Use `Rerank` for secondary sorting to improve text result relevance.
6. Apply similarity filtering and the reference limit to produce the final quoted content sent to the model.
### Image Search Method
In Knowledge Base search, images can participate in retrieval in addition to text questions. FastGPT handles images differently depending on the configured model capabilities.
Image search mainly works in two ways:
1. Image description search: If an available vision model is configured, the system can understand the image first, generate a text description, and use that description in regular text retrieval.
2. Image vector search: If the selected embedding model supports image input, the system can generate vectors for images directly and match them against image vectors in the Knowledge Base.
Image search is not a separate system outside the Knowledge Base. It adds an image-input path to the existing Knowledge Base search pipeline.
Common usage patterns include:
- Text-to-image search: enter text to find semantically related image content.
- Image-to-image search: enter an image to find visually or semantically similar image content.
- Text + image search: enter both text and an image, using the text question as an additional constraint on image search results.
Image search quality usually depends on image clarity, whether the image content is easy for the model to understand, whether a vision model is configured, and whether the embedding model supports image vectors.
Whether an image can be retrieved does not only depend on uploading an image at search time. It also depends on which indexes were created during ingestion:
| Knowledge Base capability | Text-only query | Image-only query | Text + image query |
| Regular embedding model, no vision model | Normal text retrieval | Usually unavailable | Mainly uses the text part |
| Regular embedding model with a vision model | Normal text retrieval | Converts the image into a description, then uses text retrieval | Text + image description participate in retrieval |
| Image-capable embedding model, no vision model | Normal text retrieval | Image vector retrieval | Text retrieval + image vector retrieval |
| Image-capable embedding model with a vision model | Text retrieval, including image descriptions | Image description + image vector retrieval | Text + image description + image vector retrieval |
So when image-to-image search performs poorly, do not only adjust search parameters. Also check whether the Knowledge Base is configured with a vision model or an image-capable embedding model, and whether valid image indexes were generated during ingestion.
### Result Ranking and Fusion
FastGPT fuses results from different recall paths instead of using only one path. Common paths include text vector recall, full-text recall, image description recall, image vector recall, and reranked results.
Keep these points in mind:
1. `Semantic Search` relies more on vector similarity and is better for natural-language questions and semantically related content.
2. `Full-Text Search` relies more on keyword matches and is better for IDs, model numbers, proper nouns, error codes, and other exact queries.
3. `Hybrid Search` uses both semantic recall and full-text recall, then merges the results with `RRF`.
4. `Rerank` re-sorts candidate text results and works best when the question is clear and there are enough candidates.
5. Image search adds image description or image vector results, which are then fused with text-side results.
This means final quoted content may not be strictly sorted by a single vector similarity score. Content matched by multiple recall paths is usually more likely to rank higher.
@@ -76,11 +122,13 @@ This means you can continuously improve data chunk accuracy through annotation.
...
@@ -76,11 +122,13 @@ This means you can continuously improve data chunk accuracy through annotation.
Semantic search calculates the vector distance between the user's query and knowledge base content to determine "similarity" — mathematical similarity, not linguistic.
Semantic search calculates the vector distance between the user's query and knowledge base content to determine "similarity" — mathematical similarity, not linguistic.
Pros:
Pros:
- Understands similar semantics
- Understands similar semantics
- Cross-language understanding (e.g., Chinese query matching English content)
- Cross-language understanding (e.g., Chinese query matching English content)
- Multimodal understanding (text, images, etc., depending on model capability)
Cons:
Cons:
- Depends on model training quality
- Depends on model training quality
- Inconsistent accuracy
- Inconsistent accuracy
- Affected by keywords and sentence completeness
- Affected by keywords and sentence completeness
...
@@ -115,6 +163,8 @@ A value between `0-1` that filters out low-relevance search results.
...
@@ -115,6 +163,8 @@ A value between `0-1` that filters out low-relevance search results.
This only takes effect when using `Semantic Search` or `Result Reranking`.
This only takes effect when using `Semantic Search` or `Result Reranking`.
Note that minimum relevance is a filtering threshold, not the final sorting rule. After query optimization, hybrid search, image search, or result reranking is enabled, final results may be fused from multiple recall paths and may not be strictly sorted by a single vector similarity score.
### Query Optimization
### Query Optimization
#### Background
#### Background
...
@@ -132,3 +182,20 @@ When the user asks "What's the second point?", the system searches for "What's t
...
@@ -132,3 +182,20 @@ When the user asks "What's the second point?", the system searches for "What's t
#### How It Works
#### How It Works
Before performing `data retrieval`, the model first performs `coreference resolution` and `query expansion`. This resolves ambiguous references and enriches the query's semantic content. You can view the optimized query in the conversation details after each interaction.
Before performing `data retrieval`, the model first performs `coreference resolution` and `query expansion`. This resolves ambiguous references and enriches the query's semantic content. You can view the optimized query in the conversation details after each interaction.
Query Optimization adds an extra model call before the actual search. It often improves retrieval in multi-turn conversations, but it also increases total latency. If the current question is already clear, or response speed is more important, decide whether to enable it based on actual results.
### Common Tuning Tips
If search results are not as expected, start from the symptom. Avoid changing every parameter at once.
| No results found | Confirm the data has finished training; lower the minimum relevance; increase the reference limit; check whether the question is too short or missing a subject |
| Results are too broad or off-topic | Raise the minimum relevance; reduce the reference limit; improve chunking; check whether recalled content contains too many unrelated chunks |
| IDs, model numbers, or proper nouns are inaccurate | Use full-text or hybrid search; reduce semantic search weight; avoid overusing query optimization for exact ID queries |
| Natural-language questions do not retrieve well | Use semantic or hybrid search; enable query optimization; add more accurate indexes to the data |
| Query optimization makes search slower | Query optimization adds an extra model call. Use a faster optimization model, or enable it only for follow-up questions and short queries |
| Rerank still gives poor ordering | Check whether the user question is complete; make sure enough candidates are recalled; adjust minimum relevance and reference limit |
| Image-to-image search is weak | Confirm the embedding model supports image input; confirm image vector indexes were generated during ingestion; check whether the image is clear and has an obvious subject |
| Text + image search is unstable | Clarify whether text or image should be more important; if you only want visual similarity, reduce extra text constraints |
