Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit c483a563 by Archer Committed by GitHub
Options

sandbox tool inject (#6836) 

* sandbox tool inject

* perf: read file prompt

* submodule
parent 83c02c97
Showing with 527 additions and 153 deletions
.claude/design/bug/chat-file-remap-功能开发文档.md
# 功能开发文档 # 功能开发文档
...@@ -38,7 +38,7 @@ ...@@ -38,7 +38,7 @@
| `packages/service/core/workflow/dispatch/ai/chat.ts` | 修改 | 并行处理 human messages，逐条重写 user query，文件内容不进 system | `Promise.all(...rewriteUserQueryWithFileContent(...))` | T2/T3 | | `packages/service/core/workflow/dispatch/ai/chat.ts` | 修改 | 并行处理 human messages，逐条重写 user query，文件内容不进 system | `Promise.all(...rewriteUserQueryWithFileContent(...))` | T2/T3 |
| `packages/service/core/workflow/dispatch/ai/tool/index.ts` | 修改 | Tool LLM messages 同步并行重写；保留 `hasReadFilesTool` skip | `skip: hasReadFilesTool` | T2/T4 | | `packages/service/core/workflow/dispatch/ai/tool/index.ts` | 修改 | Tool LLM messages 同步并行重写；保留 `hasReadFilesTool` skip | `skip: hasReadFilesTool` | T2/T4 |
| `packages/service/core/workflow/utils/context.ts` | 修改/复用 | 承载单条 user query 文件内容重写 helper | `rewriteUserQueryWithFileContent(...)` | T2 | | `packages/service/core/workflow/utils/context.ts` | 修改/复用 | 承载单条 user query 文件内容重写 helper | `rewriteUserQueryWithFileContent(...)` | T2 |
| `packages/service/core/workflow/dispatch/tools/readFiles.ts` | 修改/复用 | 保留可读文件 URL 标准化、读文件与解析文件能力，供 readFiles tool 和重写 helper 复用 | `normalizeReadableFileUrl(...)` / `getFileContentFromLinks(...)` | T2 | | `packages/service/core/workflow/dispatch/tools/readFiles.ts` | 修改/复用 | 保留可读文件 URL 标准化、读文件与解析文件能力，供 readFiles tool 和重写 helper 复用 | `normalizeReadableFileUrl(...)` / `parseFileContentFromUrls(...)` | T2 |
| `packages/service/core/ai/llm/utils.ts` | 修改/测试驱动 | 保持 `file_url` 过滤，确保同条 text 保留 | 不改协议行为 | T5 | | `packages/service/core/ai/llm/utils.ts` | 修改/测试驱动 | 保持 `file_url` 过滤，确保同条 text 保留 | 不改协议行为 | T5 |
| `test/cases/...` | 修改/新增 | 替换保存前增强测试，新增运行时逐条注入测试 | 当前轮/历史/Tool/maxFiles | T5 | | `test/cases/...` | 修改/新增 | 替换保存前增强测试，新增运行时逐条注入测试 | 当前轮/历史/Tool/maxFiles | T5 |
...@@ -65,7 +65,7 @@ const userMessages = await Promise.all( ...@@ -65,7 +65,7 @@ const userMessages = await Promise.all(
requestOrigin, requestOrigin,
maxFiles, maxFiles,
customPdfParse, customPdfParse,
getFileContentFromLinks, parseFileContentFromUrls,
teamId, teamId,
tmbId tmbId
}) })
...@@ -95,11 +95,11 @@ N/A（无对外接口结构变化）。 ...@@ -95,11 +95,11 @@ N/A（无对外接口结构变化）。
| 模块 | 函数/类型 | 具体改动 | 依赖关系 | | 模块 | 函数/类型 | 具体改动 | 依赖关系 |
|---|---|---|---| |---|---|---|---|
| `packages/service/core/workflow/dispatch/ai/chat.ts` | `getChatMessages` 附近 | 构造 LLM messages 前，对历史 human 与当前轮 user 做文件内容注入 | 依赖 `getFileContentFromLinks` | | `packages/service/core/workflow/dispatch/ai/chat.ts` | `getChatMessages` 附近 | 构造 LLM messages 前，对历史 human 与当前轮 user 做文件内容注入 | 依赖 `parseFileContentFromUrls` |
| `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput` | 不再把文件正文作为 system quote；当前轮文件参与逐条注入 | 与 token 裁剪链路协同 | | `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput` | 不再把文件正文作为 system quote；当前轮文件参与逐条注入 | 与 token 裁剪链路协同 |
| `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `dispatchRunTools` | 与 Chat 路径一致；无 `readFiles` tool 时注入，有则跳过 | 避免与 readFiles tool 重复预解析 | | `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `dispatchRunTools` | 与 Chat 路径一致；无 `readFiles` tool 时注入，有则跳过 | 避免与 readFiles tool 重复预解析 |
| `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 单条 user query 重写 `<FilesContent>`，外层负责并行处理 history/current messages | 通过入参复用 `getFileContentFromLinks` | | `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 单条 user query 重写 `<FilesContent>`，外层负责并行处理 history/current messages | 通过入参复用 `parseFileContentFromUrls` |
| `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `getFileContentFromLinks` | 统一负责 URL 标准化、过滤、文件读取与解析；按单条 query URL 顺序与 `maxFiles` 控制解析量 | 保持现有错误兜底 | | `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `parseFileContentFromUrls` | 统一负责 URL 标准化、过滤、文件读取与解析；按单条 query URL 顺序与 `maxFiles` 控制解析量 | 保持现有错误兜底 |
| `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤；回归验证 text part 不丢 | 最终模型请求安全过滤 | | `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤；回归验证 text part 不丢 | 最终模型请求安全过滤 |
### 3.3 运行时注入算法 ### 3.3 运行时注入算法
...@@ -108,7 +108,7 @@ N/A（无对外接口结构变化）。 ...@@ -108,7 +108,7 @@ N/A（无对外接口结构变化）。
2. Chat/Tool 外层通过 `Promise.all` 并行处理运行时 messages。 2. Chat/Tool 外层通过 `Promise.all` 并行处理运行时 messages。
3. 非 human message 原样返回；human message 调用 `rewriteUserQueryWithFileContent`。 3. 非 human message 原样返回；human message 调用 `rewriteUserQueryWithFileContent`。
4. 单条 user query 内只收集本条 `file.url`，不做跨 message URL 去重或共享缓存。 4. 单条 user query 内只收集本条 `file.url`，不做跨 message URL 去重或共享缓存。
5. 调用 `getFileContentFromLinks` 统一完成 URL 标准化、过滤、`maxFiles` 截断与文件解析。 5. 调用 `parseFileContentFromUrls` 统一完成 URL 标准化、过滤、`maxFiles` 截断与文件解析。
6. 将解析结果回填到当前 user query： 6. 将解析结果回填到当前 user query：
- message 原本有 text：追加分隔符和 `<FilesContent>`。 - message 原本有 text：追加分隔符和 `<FilesContent>`。
- message 原本无 text：新增 text part。 - message 原本无 text：新增 text part。
...@@ -144,7 +144,7 @@ const userMessages = await Promise.all( ...@@ -144,7 +144,7 @@ const userMessages = await Promise.all(
maxFiles, maxFiles,
requestOrigin, requestOrigin,
customPdfParse, customPdfParse,
getFileContentFromLinks, parseFileContentFromUrls,
teamId, teamId,
tmbId tmbId
}) })
......
.claude/design/bug/chat-file-remap-需求设计文档.md
# 需求设计文档 # 需求设计文档
...@@ -129,7 +129,7 @@ ...@@ -129,7 +129,7 @@
| `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput/getChatMessages` | 构造 LLM messages 前增强运行时副本：历史和当前轮每条 user message 注入自己的文件内容；文件内容不进 system | Chat node 满足历史逐条注入 | | `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput/getChatMessages` | 构造 LLM messages 前增强运行时副本：历史和当前轮每条 user message 注入自己的文件内容；文件内容不进 system | Chat node 满足历史逐条注入 |
| `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `getMultiInput/dispatchRunTools` | 无 `readFiles` tool 时同 Chat；有 `readFiles` tool 时跳过预解析 | 避免与 readFiles tool 职责冲突 | | `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `getMultiInput/dispatchRunTools` | 无 `readFiles` tool 时同 Chat；有 `readFiles` tool 时跳过预解析 | 避免与 readFiles tool 职责冲突 |
| `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 承载单条 user query 的文件内容重写逻辑，外层并行处理 history/current messages | 不污染 readFiles tool 职责 | | `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 承载单条 user query 的文件内容重写逻辑，外层并行处理 history/current messages | 不污染 readFiles tool 职责 |
| `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `getFileContentFromLinks` | `getFileContentFromLinks` 统一负责 URL 标准化、过滤、文件读取与解析；`normalizeReadableFileUrl` 仅作为底层清洗工具 | 不改对外 API | | `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `parseFileContentFromUrls` | `parseFileContentFromUrls` 统一负责 URL 标准化、过滤、文件读取与解析；`normalizeReadableFileUrl` 仅作为底层清洗工具 | 不改对外 API |
| `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤逻辑；确保同条消息 text 不被过滤 | 回归保障 | | `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤逻辑；确保同条消息 text 不被过滤 | 回归保障 |
### 6.4 运行时注入规则 ### 6.4 运行时注入规则
...@@ -137,7 +137,7 @@ ...@@ -137,7 +137,7 @@
1. 使用消息副本，不修改 `histories`、`query`、`userQuestion` 原对象。 1. 使用消息副本，不修改 `histories`、`query`、`userQuestion` 原对象。
2. Chat/Tool 外层用 `Promise.all` 并行处理运行时 messages。 2. Chat/Tool 外层用 `Promise.all` 并行处理运行时 messages。
3. 单条 user query 只收集本条 `file.url`；不做跨 message URL 去重，不共享解析缓存。 3. 单条 user query 只收集本条 `file.url`；不做跨 message URL 去重，不共享解析缓存。
4. `getFileContentFromLinks` 负责 URL 标准化、过滤、`maxFiles` 截断和文件解析。 4. `parseFileContentFromUrls` 负责 URL 标准化、过滤、`maxFiles` 截断和文件解析。
5. 文件解析结果回填到原本所属的 user message： 5. 文件解析结果回填到原本所属的 user message：
- 原 message 已有 text：追加 `\n\n===---===---===\n\n<FilesContent>...`。 - 原 message 已有 text：追加 `\n\n===---===---===\n\n<FilesContent>...`。
- 原 message 只有 file：新增一个 text part 存放 `<FilesContent>`。 - 原 message 只有 file：新增一个 text part 存放 `<FilesContent>`。
......
.codex/design/bug/chat-file-remap-功能开发文档.md
# 功能开发文档 # 功能开发文档
...@@ -38,7 +38,7 @@ ...@@ -38,7 +38,7 @@
| `packages/service/core/workflow/dispatch/ai/chat.ts` | 修改 | 并行处理 human messages，逐条重写 user query，文件内容不进 system | `Promise.all(...rewriteUserQueryWithFileContent(...))` | T2/T3 | | `packages/service/core/workflow/dispatch/ai/chat.ts` | 修改 | 并行处理 human messages，逐条重写 user query，文件内容不进 system | `Promise.all(...rewriteUserQueryWithFileContent(...))` | T2/T3 |
| `packages/service/core/workflow/dispatch/ai/tool/index.ts` | 修改 | Tool LLM messages 同步并行重写；保留 `hasReadFilesTool` skip | `skip: hasReadFilesTool` | T2/T4 | | `packages/service/core/workflow/dispatch/ai/tool/index.ts` | 修改 | Tool LLM messages 同步并行重写；保留 `hasReadFilesTool` skip | `skip: hasReadFilesTool` | T2/T4 |
| `packages/service/core/workflow/utils/context.ts` | 修改/复用 | 承载单条 user query 文件内容重写 helper | `rewriteUserQueryWithFileContent(...)` | T2 | | `packages/service/core/workflow/utils/context.ts` | 修改/复用 | 承载单条 user query 文件内容重写 helper | `rewriteUserQueryWithFileContent(...)` | T2 |
| `packages/service/core/workflow/dispatch/tools/readFiles.ts` | 修改/复用 | 保留可读文件 URL 标准化、读文件与解析文件能力，供 readFiles tool 和重写 helper 复用 | `normalizeReadableFileUrl(...)` / `getFileContentFromLinks(...)` | T2 | | `packages/service/core/workflow/dispatch/tools/readFiles.ts` | 修改/复用 | 保留可读文件 URL 标准化、读文件与解析文件能力，供 readFiles tool 和重写 helper 复用 | `normalizeReadableFileUrl(...)` / `parseFileContentFromUrls(...)` | T2 |
| `packages/service/core/ai/llm/utils.ts` | 修改/测试驱动 | 保持 `file_url` 过滤，确保同条 text 保留 | 不改协议行为 | T5 | | `packages/service/core/ai/llm/utils.ts` | 修改/测试驱动 | 保持 `file_url` 过滤，确保同条 text 保留 | 不改协议行为 | T5 |
| `test/cases/...` | 修改/新增 | 替换保存前增强测试，新增运行时逐条注入测试 | 当前轮/历史/Tool/maxFiles | T5 | | `test/cases/...` | 修改/新增 | 替换保存前增强测试，新增运行时逐条注入测试 | 当前轮/历史/Tool/maxFiles | T5 |
...@@ -65,7 +65,7 @@ const userMessages = await Promise.all( ...@@ -65,7 +65,7 @@ const userMessages = await Promise.all(
requestOrigin, requestOrigin,
maxFiles, maxFiles,
customPdfParse, customPdfParse,
getFileContentFromLinks, parseFileContentFromUrls,
teamId, teamId,
tmbId tmbId
}) })
...@@ -95,11 +95,11 @@ N/A（无对外接口结构变化）。 ...@@ -95,11 +95,11 @@ N/A（无对外接口结构变化）。
| 模块 | 函数/类型 | 具体改动 | 依赖关系 | | 模块 | 函数/类型 | 具体改动 | 依赖关系 |
|---|---|---|---| |---|---|---|---|
| `packages/service/core/workflow/dispatch/ai/chat.ts` | `getChatMessages` 附近 | 构造 LLM messages 前，对历史 human 与当前轮 user 做文件内容注入 | 依赖 `getFileContentFromLinks` | | `packages/service/core/workflow/dispatch/ai/chat.ts` | `getChatMessages` 附近 | 构造 LLM messages 前，对历史 human 与当前轮 user 做文件内容注入 | 依赖 `parseFileContentFromUrls` |
| `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput` | 不再把文件正文作为 system quote；当前轮文件参与逐条注入 | 与 token 裁剪链路协同 | | `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput` | 不再把文件正文作为 system quote；当前轮文件参与逐条注入 | 与 token 裁剪链路协同 |
| `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `dispatchRunTools` | 与 Chat 路径一致；无 `readFiles` tool 时注入，有则跳过 | 避免与 readFiles tool 重复预解析 | | `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `dispatchRunTools` | 与 Chat 路径一致；无 `readFiles` tool 时注入，有则跳过 | 避免与 readFiles tool 重复预解析 |
| `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 单条 user query 重写 `<FilesContent>`，外层负责并行处理 history/current messages | 通过入参复用 `getFileContentFromLinks` | | `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 单条 user query 重写 `<FilesContent>`，外层负责并行处理 history/current messages | 通过入参复用 `parseFileContentFromUrls` |
| `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `getFileContentFromLinks` | 统一负责 URL 标准化、过滤、文件读取与解析；按单条 query URL 顺序与 `maxFiles` 控制解析量 | 保持现有错误兜底 | | `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `parseFileContentFromUrls` | 统一负责 URL 标准化、过滤、文件读取与解析；按单条 query URL 顺序与 `maxFiles` 控制解析量 | 保持现有错误兜底 |
| `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤；回归验证 text part 不丢 | 最终模型请求安全过滤 | | `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤；回归验证 text part 不丢 | 最终模型请求安全过滤 |
### 3.3 运行时注入算法 ### 3.3 运行时注入算法
...@@ -108,7 +108,7 @@ N/A（无对外接口结构变化）。 ...@@ -108,7 +108,7 @@ N/A（无对外接口结构变化）。
2. Chat/Tool 外层通过 `Promise.all` 并行处理运行时 messages。 2. Chat/Tool 外层通过 `Promise.all` 并行处理运行时 messages。
3. 非 human message 原样返回；human message 调用 `rewriteUserQueryWithFileContent`。 3. 非 human message 原样返回；human message 调用 `rewriteUserQueryWithFileContent`。
4. 单条 user query 内只收集本条 `file.url`，不做跨 message URL 去重或共享缓存。 4. 单条 user query 内只收集本条 `file.url`，不做跨 message URL 去重或共享缓存。
5. 调用 `getFileContentFromLinks` 统一完成 URL 标准化、过滤、`maxFiles` 截断与文件解析。 5. 调用 `parseFileContentFromUrls` 统一完成 URL 标准化、过滤、`maxFiles` 截断与文件解析。
6. 将解析结果回填到当前 user query： 6. 将解析结果回填到当前 user query：
- message 原本有 text：追加分隔符和 `<FilesContent>`。 - message 原本有 text：追加分隔符和 `<FilesContent>`。
- message 原本无 text：新增 text part。 - message 原本无 text：新增 text part。
...@@ -144,7 +144,7 @@ const userMessages = await Promise.all( ...@@ -144,7 +144,7 @@ const userMessages = await Promise.all(
maxFiles, maxFiles,
requestOrigin, requestOrigin,
customPdfParse, customPdfParse,
getFileContentFromLinks, parseFileContentFromUrls,
teamId, teamId,
tmbId tmbId
}) })
......
.codex/design/bug/chat-file-remap-需求设计文档.md
# 需求设计文档 # 需求设计文档
...@@ -129,7 +129,7 @@ ...@@ -129,7 +129,7 @@
| `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput/getChatMessages` | 构造 LLM messages 前增强运行时副本：历史和当前轮每条 user message 注入自己的文件内容；文件内容不进 system | Chat node 满足历史逐条注入 | | `packages/service/core/workflow/dispatch/ai/chat.ts` | `getMultiInput/getChatMessages` | 构造 LLM messages 前增强运行时副本：历史和当前轮每条 user message 注入自己的文件内容；文件内容不进 system | Chat node 满足历史逐条注入 |
| `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `getMultiInput/dispatchRunTools` | 无 `readFiles` tool 时同 Chat；有 `readFiles` tool 时跳过预解析 | 避免与 readFiles tool 职责冲突 | | `packages/service/core/workflow/dispatch/ai/tool/index.ts` | `getMultiInput/dispatchRunTools` | 无 `readFiles` tool 时同 Chat；有 `readFiles` tool 时跳过预解析 | 避免与 readFiles tool 职责冲突 |
| `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 承载单条 user query 的文件内容重写逻辑，外层并行处理 history/current messages | 不污染 readFiles tool 职责 | | `packages/service/core/workflow/utils/context.ts` | `rewriteUserQueryWithFileContent` | 承载单条 user query 的文件内容重写逻辑，外层并行处理 history/current messages | 不污染 readFiles tool 职责 |
| `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `getFileContentFromLinks` | `getFileContentFromLinks` 统一负责 URL 标准化、过滤、文件读取与解析；`normalizeReadableFileUrl` 仅作为底层清洗工具 | 不改对外 API | | `packages/service/core/workflow/dispatch/tools/readFiles.ts` | `normalizeReadableFileUrl` / `parseFileContentFromUrls` | `parseFileContentFromUrls` 统一负责 URL 标准化、过滤、文件读取与解析；`normalizeReadableFileUrl` 仅作为底层清洗工具 | 不改对外 API |
| `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤逻辑；确保同条消息 text 不被过滤 | 回归保障 | | `packages/service/core/ai/llm/utils.ts` | `loadRequestMessages` | 保持 `file_url` 过滤逻辑；确保同条消息 text 不被过滤 | 回归保障 |
### 6.4 运行时注入规则 ### 6.4 运行时注入规则
...@@ -137,7 +137,7 @@ ...@@ -137,7 +137,7 @@
1. 使用消息副本，不修改 `histories`、`query`、`userQuestion` 原对象。 1. 使用消息副本，不修改 `histories`、`query`、`userQuestion` 原对象。
2. Chat/Tool 外层用 `Promise.all` 并行处理运行时 messages。 2. Chat/Tool 外层用 `Promise.all` 并行处理运行时 messages。
3. 单条 user query 只收集本条 `file.url`；不做跨 message URL 去重，不共享解析缓存。 3. 单条 user query 只收集本条 `file.url`；不做跨 message URL 去重，不共享解析缓存。
4. `getFileContentFromLinks` 负责 URL 标准化、过滤、`maxFiles` 截断和文件解析。 4. `parseFileContentFromUrls` 负责 URL 标准化、过滤、`maxFiles` 截断和文件解析。
5. 文件解析结果回填到原本所属的 user message： 5. 文件解析结果回填到原本所属的 user message：
- 原 message 已有 text：追加 `\n\n===---===---===\n\n<FilesContent>...`。 - 原 message 已有 text：追加 `\n\n===---===---===\n\n<FilesContent>...`。
- 原 message 只有 file：新增一个 text part 存放 `<FilesContent>`。 - 原 message 只有 file：新增一个 text part 存放 `<FilesContent>`。
......
document/content/self-host/upgrading/4-14/41417.mdx 0 → 100644
---
title: 'V4.14.17(处理中)'
description: 'FastGPT V4.14.17 更新说明'
---
## 升级指南
### 1. 更新镜像 tag
- 更新 fastgpt-app(fastgpt 主服务) 镜像 tag: v4.14.17
- 更新 fastgpt-pro(fastgpt 商业版) 镜像 tag: v4.14.17
## 🐛 修复
1. API 知识库 parentId 类型校验错误。
2. 门户页对话无法上传文件。
3. 商业版未包含内部文件解析接口，如果未配置 S3 External Endpoint，会导致文件解析失败。
\ No newline at end of file
document/content/self-host/upgrading/4-14/meta.en.json
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
"title": "4.14.x", "title": "4.14.x",
"description": "", "description": "",
"pages": [ "pages": [
"41417",
"41416", "41416",
"41415", "41415",
"41414", "41414",
......
document/content/self-host/upgrading/4-14/meta.json
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
"title": "4.14.x", "title": "4.14.x",
"description": "", "description": "",
"pages": [ "pages": [
"41417",
"41416", "41416",
"41415", "41415",
"41414", "41414",
......
document/content/self-host/upgrading/4-15/4150.mdx
...@@ -7,6 +7,7 @@ description: 'FastGPT V4.15.0 更新说明' ...@@ -7,6 +7,7 @@ description: 'FastGPT V4.15.0 更新说明'
1. 新增循环节点，弃用旧的批量执行。 1. 新增循环节点，弃用旧的批量执行。
2. 全局变量输入框支持输入 object 类型数据。 2. 全局变量输入框支持输入 object 类型数据。
3. 工具调用模式下，如果开启了虚拟机功能，用户对话框上传的文件会直接注入到虚拟机中。
## ⚙️ 优化 ## ⚙️ 优化
......
document/content/toc.mdx
...@@ -118,6 +118,7 @@ description: FastGPT 文档目录 ...@@ -118,6 +118,7 @@ description: FastGPT 文档目录
- [/self-host/upgrading/4-14/41414](/self-host/upgrading/4-14/41414) - [/self-host/upgrading/4-14/41414](/self-host/upgrading/4-14/41414)
- [/self-host/upgrading/4-14/41415](/self-host/upgrading/4-14/41415) - [/self-host/upgrading/4-14/41415](/self-host/upgrading/4-14/41415)
- [/self-host/upgrading/4-14/41416](/self-host/upgrading/4-14/41416) - [/self-host/upgrading/4-14/41416](/self-host/upgrading/4-14/41416)
- [/self-host/upgrading/4-14/41417](/self-host/upgrading/4-14/41417)
- [/self-host/upgrading/4-14/4142](/self-host/upgrading/4-14/4142) - [/self-host/upgrading/4-14/4142](/self-host/upgrading/4-14/4142)
- [/self-host/upgrading/4-14/4143](/self-host/upgrading/4-14/4143) - [/self-host/upgrading/4-14/4143](/self-host/upgrading/4-14/4143)
- [/self-host/upgrading/4-14/4144](/self-host/upgrading/4-14/4144) - [/self-host/upgrading/4-14/4144](/self-host/upgrading/4-14/4144)
......
document/data/doc-last-modified.json
...@@ -251,7 +251,7 @@ ...@@ -251,7 +251,7 @@
"content/self-host/upgrading/4-14/41481.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/41481.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/4149.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4149.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/4149.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4149.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-15/4150.mdx": "2026-04-28T13:31:00+08:00", "content/self-host/upgrading/4-15/4150.mdx": "2026-04-28T15:10:52+08:00",
"content/self-host/upgrading/outdated/40.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/40.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/40.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/40.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/41.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/41.en.mdx": "2026-04-26T21:08:47+08:00",
......
packages/global/core/ai/sandbox/constants.ts
...@@ -77,11 +77,14 @@ export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = { ...@@ -77,11 +77,14 @@ export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = {
}; };
// Prompt // Prompt
export const SANDBOX_SYSTEM_PROMPT = `你拥有一个独立的 Linux 沙盒环境（Ubuntu 22.04），可通过 ${SANDBOX_TOOL_NAME} 工具执行命令： export const SANDBOX_USER_FILES_PATH = 'user_files/';
- 预装：bash / python3 / node / bun / git / curl export const SANDBOX_SYSTEM_PROMPT = `## 沙盒能力
你拥有一个独立的 Linux 沙盒环境（Ubuntu 22.04），可通过 ${SANDBOX_TOOL_NAME} 工具执行命令。
- 系统预装：bash / python3 / node / bun / git / curl
- 可自行安装软件包（apt / pip / npm） - 可自行安装软件包（apt / pip / npm）
- 生成的文件内容都保存在当前目录下即可 - 生成的文件内容都保存在当前目录下即可
- 若需要将生成的文件分享给用户，可使用 ${SANDBOX_GET_FILE_URL_TOOL_NAME} 工具获取文件的临时访问链接`; - 用户主动上传的文件存储在 ${SANDBOX_USER_FILES_PATH} 目录下
- 若需要将生成的文件链接，可使用 ${SANDBOX_GET_FILE_URL_TOOL_NAME} 工具获取文件的临时访问链接`;
// 聚合 // 聚合
export const sandboxToolMap: Record< export const sandboxToolMap: Record<
......
packages/service/core/ai/llm/agentLoop/prompt.ts
...@@ -46,16 +46,18 @@ ${list} ...@@ -46,16 +46,18 @@ ${list}
/* ===== Inject user query ===== */ /* ===== Inject user query ===== */
export const getUserFilesPrompt = ( export const getUserFilesPrompt = (
files: { id: string; name: string; content?: string }[] = [] files: { id?: string; name: string; sandboxPath?: string; content?: string }[] = []
) => { ) => {
if (files.length === 0) return ''; if (files.length === 0) return '';
return `# Input Files return `# Input Files
本次用户上传的文件： 用户本次上传的文件：
${files ${files
.map((file) => .map((file) =>
`<file> `<file>
${file.id ? `<id>${file.id}</id>` : ''}
<name>${file.name}</name> <name>${file.name}</name>
${file.sandboxPath ? `<sandboxPath>${file.sandboxPath}</sandboxPath>` : ''}
${file.content ? `<content>${file.content}</content>` : ''} ${file.content ? `<content>${file.content}</content>` : ''}
</file>`.trim() </file>`.trim()
) )
......
packages/service/core/ai/sandbox/toolCall/index.ts
...@@ -6,6 +6,9 @@ import { toolMap as getFileUrlToolMap } from './getFileUrl.tool'; ...@@ -6,6 +6,9 @@ import { toolMap as getFileUrlToolMap } from './getFileUrl.tool';
import { toolMap as shellToolMap } from './shell.tool'; import { toolMap as shellToolMap } from './shell.tool';
import { getSandboxClient } from '../controller'; import { getSandboxClient } from '../controller';
import { parseJsonArgs } from '../../utils'; import { parseJsonArgs } from '../../utils';
import { axios } from '../../../../common/api/axios';
import { serverRequestBaseUrl } from '../../../../common/api/serverRequest';
import type { FileWriteEntry } from '@fastgpt-sdk/sandbox-adapter';
const ToolMap = { const ToolMap = {
...getFileUrlToolMap, ...getFileUrlToolMap,
...@@ -74,6 +77,39 @@ export const runSandboxTools = async ({ ...@@ -74,6 +77,39 @@ export const runSandboxTools = async ({
}; };
}; };
export const injectSandboxFiles = async ({
appId,
userId,
chatId,
files
}: {
appId: string;
userId: string;
chatId: string;
files: { path: string; url: string }[];
}) => {
const instance = await getSandboxClient({ appId, userId, chatId });
await instance.ensureAvailable();
const writeFilesData = await Promise.all(
files
.filter((file) => file.path)
.map(async ({ path, url }): Promise<FileWriteEntry> => {
const response = await axios.get<ArrayBuffer>(url, {
baseURL: serverRequestBaseUrl,
responseType: 'arraybuffer'
});
return {
path,
data: response.data
};
})
);
await instance.provider.writeFiles(writeFilesData);
};
export const getSandboxToolInfo = (name: string, lang: localeType = LangEnum.en) => { export const getSandboxToolInfo = (name: string, lang: localeType = LangEnum.en) => {
if (name in sandboxToolMap) { if (name in sandboxToolMap) {
const info = sandboxToolMap[name]; const info = sandboxToolMap[name];
......
packages/service/core/workflow/dispatch/ai/chat.ts
...@@ -30,9 +30,9 @@ import { getHistoryPreview } from '@fastgpt/global/core/chat/utils'; ...@@ -30,9 +30,9 @@ import { getHistoryPreview } from '@fastgpt/global/core/chat/utils';
import { computedMaxToken } from '../../../ai/utils'; import { computedMaxToken } from '../../../ai/utils';
import { formatTime2YMDHM } from '@fastgpt/global/common/string/time'; import { formatTime2YMDHM } from '@fastgpt/global/common/string/time';
import type { AiChatQuoteRoleType } from '@fastgpt/global/core/workflow/template/system/aiChat/type'; import type { AiChatQuoteRoleType } from '@fastgpt/global/core/workflow/template/system/aiChat/type';
import { getFileContentFromLinks } from '../../utils/file'; import { parseFileContentFromUrls } from '../../utils/file';
import { parseUrlToFileType } from '../../utils/context'; import { parseUrlToFileType } from '../../utils/context';
import { rewriteUserQueryWithFiles } from '../../utils/file'; import { formatUserQueryWithFiles } from '../../utils/file';
import { i18nT } from '../../../../../web/i18n/utils'; import { i18nT } from '../../../../../web/i18n/utils';
import { postTextCensor } from '../../../chat/postTextCensor'; import { postTextCensor } from '../../../chat/postTextCensor';
import { createLLMResponse } from '../../../ai/llm/request'; import { createLLMResponse } from '../../../ai/llm/request';
...@@ -167,9 +167,6 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp ...@@ -167,9 +167,6 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
})() })()
]); ]);
console.log(111111);
console.dir(filterMessages, { depth: null });
const { const {
completeMessages, completeMessages,
reasoningText, reasoningText,
...@@ -422,18 +419,29 @@ const getChatMessages = async ({ ...@@ -422,18 +419,29 @@ const getChatMessages = async ({
return message; return message;
} }
return { const query = await formatUserQueryWithFiles({
...message,
value: await rewriteUserQueryWithFiles({
queryId: message.dataId || `${index}`,
userQuery: message.value, userQuery: message.value,
parseFileFn: async (urls) => {
const files = await parseFileContentFromUrls({
urls,
requestOrigin, requestOrigin,
maxFiles, maxFiles,
customPdfParse,
usageId,
teamId: runningUserInfo.teamId, teamId: runningUserInfo.teamId,
tmbId: runningUserInfo.tmbId tmbId: runningUserInfo.tmbId,
}) customPdfParse,
usageId
});
return files.map((file) => ({
name: file.name,
content: file.content
}));
}
});
return {
...message,
value: query
}; };
}) })
); );
......
packages/service/core/workflow/dispatch/ai/toolcall/constants.ts
import { replaceVariable } from '@fastgpt/global/common/string/tools';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { getNanoid } from '@fastgpt/global/common/string/tools'; import { getNanoid } from '@fastgpt/global/common/string/tools';
import type { ChildResponseItemType } from './type'; import type { ChildResponseItemType } from './type';
......
packages/service/core/workflow/dispatch/ai/toolcall/index.ts
...@@ -4,6 +4,7 @@ import type { DispatchNodeResultType } from '@fastgpt/global/core/workflow/runti ...@@ -4,6 +4,7 @@ import type { DispatchNodeResultType } from '@fastgpt/global/core/workflow/runti
import { getLLMModel } from '../../../../ai/model'; import { getLLMModel } from '../../../../ai/model';
import { filterToolNodeIdByEdges, getNodeErrResponse, getHistories } from '../../utils'; import { filterToolNodeIdByEdges, getNodeErrResponse, getHistories } from '../../utils';
import { runToolCall } from './toolCall'; import { runToolCall } from './toolCall';
import type { FileInputType } from './type';
import { type DispatchToolModuleProps, type ToolNodeItemType } from './type'; import { type DispatchToolModuleProps, type ToolNodeItemType } from './type';
import type { UserChatItemFileItemType, ChatItemMiniType } from '@fastgpt/global/core/chat/type'; import type { UserChatItemFileItemType, ChatItemMiniType } from '@fastgpt/global/core/chat/type';
import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants'; import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
...@@ -16,11 +17,12 @@ import { ...@@ -16,11 +17,12 @@ import {
import { getHistoryPreview } from '@fastgpt/global/core/chat/utils'; import { getHistoryPreview } from '@fastgpt/global/core/chat/utils';
import { filterToolResponseToPreview } from './utils'; import { filterToolResponseToPreview } from './utils';
import { parseUrlToFileType } from '../../../utils/context'; import { parseUrlToFileType } from '../../../utils/context';
import { rewriteUserQueryWithFiles } from '../../../utils/file'; import { formatUserQueryWithFiles, parseFileInfoFromUrls } from '../../../utils/file';
import { postTextCensor } from '../../../../chat/postTextCensor'; import { postTextCensor } from '../../../../chat/postTextCensor';
import type { FlowNodeInputItemType } from '@fastgpt/global/core/workflow/type/io'; import type { FlowNodeInputItemType } from '@fastgpt/global/core/workflow/type/io';
import type { McpToolDataType } from '@fastgpt/global/core/app/tool/mcpTool/type'; import type { McpToolDataType } from '@fastgpt/global/core/app/tool/mcpTool/type';
import { getToolConfigStatus } from '@fastgpt/global/core/app/formEdit/utils'; import { getToolConfigStatus } from '@fastgpt/global/core/app/formEdit/utils';
import { SANDBOX_USER_FILES_PATH } from '@fastgpt/global/core/ai/sandbox/constants';
type Response = DispatchNodeResultType<{ type Response = DispatchNodeResultType<{
[NodeOutputKeyEnum.answerText]: string; [NodeOutputKeyEnum.answerText]: string;
...@@ -38,6 +40,7 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise< ...@@ -38,6 +40,7 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise<
runningUserInfo, runningUserInfo,
externalProvider, externalProvider,
usageId, usageId,
responseChatItemId,
params: { params: {
model, model,
systemPrompt, systemPrompt,
...@@ -46,10 +49,13 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise< ...@@ -46,10 +49,13 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise<
fileUrlList: fileLinks, fileUrlList: fileLinks,
aiChatVision, aiChatVision,
aiChatReasoning, aiChatReasoning,
isResponseAnswerText = true isResponseAnswerText = true,
useAgentSandbox
} }
} = props; } = props;
const useSandbox = !!useAgentSandbox && !!global.feConfigs?.show_agent_sandbox;
try { try {
const toolModel = getLLMModel(model); const toolModel = getLLMModel(model);
const useVision = aiChatVision && toolModel.vision; const useVision = aiChatVision && toolModel.vision;
...@@ -120,11 +126,14 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise< ...@@ -120,11 +126,14 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise<
.filter(Boolean) .filter(Boolean)
.join('\n\n-----\n\n'); .join('\n\n-----\n\n');
const allFiles = new Map<string, FileInputType>();
const currentInputFiles: FileInputType[] = [];
const messages = await (async () => { const messages = await (async () => {
const value: ChatItemMiniType[] = [ const value: ChatItemMiniType[] = [
...getSystemPrompt_ChatItemType(concatenateSystemPrompt), ...getSystemPrompt_ChatItemType(concatenateSystemPrompt),
...chatHistories, ...chatHistories,
{ {
dataId: responseChatItemId,
obj: ChatRoleEnum.Human, obj: ChatRoleEnum.Human,
value: runtimePrompt2ChatsValue({ value: runtimePrompt2ChatsValue({
text: userChatInput, text: userChatInput,
...@@ -142,18 +151,40 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise< ...@@ -142,18 +151,40 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise<
return message; return message;
} }
return { const prefixId = message.dataId || `${index}`;
...message, const query = await formatUserQueryWithFiles({
value: await rewriteUserQueryWithFiles({
queryId: message.dataId || `${index}`,
userQuery: message.value, userQuery: message.value,
parseFileFn: async (urls) => {
const files = await parseFileInfoFromUrls({
urls,
requestOrigin, requestOrigin,
maxFiles, maxFiles,
customPdfParse: chatConfig?.fileSelectConfig?.customPdfParse, teamId: runningUserInfo.teamId
usageId, }).then((res) =>
teamId: runningUserInfo.teamId, res
tmbId: runningUserInfo.tmbId .filter((item) => item.success)
}) .map((item, index) => ({
id: `${prefixId}-${index}`,
name: item.name,
url: item.url,
sandboxPath: useSandbox ? `${SANDBOX_USER_FILES_PATH}${item.name}` : undefined
}))
);
files.forEach((file) => {
allFiles.set(file.id, file);
});
if (index === runtimeMessages.length - 1) {
currentInputFiles.push(...files);
}
return files;
}
});
return {
...message,
value: query
}; };
}) })
); );
...@@ -188,6 +219,8 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise< ...@@ -188,6 +219,8 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise<
return runToolCall({ return runToolCall({
...props, ...props,
allFiles,
currentInputFiles,
runtimeNodes, runtimeNodes,
runtimeEdges, runtimeEdges,
toolNodes, toolNodes,
......
packages/service/core/workflow/dispatch/ai/toolcall/toolCall.ts
...@@ -19,7 +19,18 @@ import type { ToolCallChildrenInteractive } from '@fastgpt/global/core/workflow/ ...@@ -19,7 +19,18 @@ import type { ToolCallChildrenInteractive } from '@fastgpt/global/core/workflow/
import type { JsonSchemaPropertiesItemType } from '@fastgpt/global/core/app/jsonschema'; import type { JsonSchemaPropertiesItemType } from '@fastgpt/global/core/app/jsonschema';
import { SANDBOX_SYSTEM_PROMPT, SANDBOX_TOOLS } from '@fastgpt/global/core/ai/sandbox/constants'; import { SANDBOX_SYSTEM_PROMPT, SANDBOX_TOOLS } from '@fastgpt/global/core/ai/sandbox/constants';
import { getSandboxToolWorkflowResponse } from './constants'; import { getSandboxToolWorkflowResponse } from './constants';
import { getSandboxToolInfo, runSandboxTools } from '../../../../ai/sandbox/toolCall'; import {
getSandboxToolInfo,
injectSandboxFiles,
runSandboxTools
} from '../../../../ai/sandbox/toolCall';
import {
dispatchReadFileTool,
ReadFileTooData,
ReadFileToolParamsSchema,
ReadFileToolSchema
} from './tools/file';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
type ResponseType = { type ResponseType = {
requestIds: string[]; requestIds: string[];
...@@ -40,6 +51,8 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -40,6 +51,8 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
toolNodes, toolNodes,
toolModel, toolModel,
childrenInteractiveParams, childrenInteractiveParams,
allFiles,
currentInputFiles,
...workflowProps ...workflowProps
} = props; } = props;
...@@ -114,7 +127,12 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -114,7 +127,12 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
}; };
}); });
// 注入 sandbox 提示 // 注入 readFile tool
if (allFiles.size > 0) {
tools.push(ReadFileToolSchema);
}
// 注入 sandbox tool
if (useAgentSandbox && global.feConfigs?.show_agent_sandbox) { if (useAgentSandbox && global.feConfigs?.show_agent_sandbox) {
// 注入 sandbox_shell 工具 // 注入 sandbox_shell 工具
tools.push(...SANDBOX_TOOLS); tools.push(...SANDBOX_TOOLS);
...@@ -128,9 +146,27 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -128,9 +146,27 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
} else { } else {
finalMessages = [{ role: 'system', content: SANDBOX_SYSTEM_PROMPT }, ...messages]; finalMessages = [{ role: 'system', content: SANDBOX_SYSTEM_PROMPT }, ...messages];
} }
// 注入文件到沙盒里
await injectSandboxFiles({
appId: workflowProps.runningAppInfo.id,
userId: workflowProps.uid,
chatId: workflowProps.chatId,
files: currentInputFiles.map((file) => ({
path: file.sandboxPath!,
url: file.url
}))
});
} }
const getToolInfo = (name: string) => { const getToolInfo = (name: string) => {
if (name === ReadFileTooData.id) {
return {
type: 'file' as const,
name: parseI18nString(ReadFileTooData.name, workflowProps.lang),
avatar: ReadFileTooData.avatar
};
}
const sandboxToolInfo = getSandboxToolInfo(name, workflowProps.lang); const sandboxToolInfo = getSandboxToolInfo(name, workflowProps.lang);
if (sandboxToolInfo) { if (sandboxToolInfo) {
return { return {
...@@ -276,6 +312,20 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -276,6 +312,20 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
}); });
return { response, flowResponse }; return { response, flowResponse };
} else if (toolInfo.type === 'file') {
const { ids } = ReadFileToolParamsSchema.parse(parseJsonArgs(call.function.arguments));
const { response, usages, nodeResponse } = await dispatchReadFileTool({
files: ids.map((id) => ({ id, url: allFiles.get(id)?.url! })),
teamId: workflowProps.runningUserInfo.teamId,
tmbId: workflowProps.runningUserInfo.tmbId,
customPdfParse: workflowProps.chatConfig?.fileSelectConfig?.customPdfParse,
usageId: workflowProps.usageId
});
return {
response,
usages,
nodeResponse
};
} else { } else {
const toolNode = toolInfo.rawData; const toolNode = toolInfo.rawData;
......
packages/service/core/workflow/dispatch/ai/toolcall/tools/file.ts 0 → 100644
import type { ChatCompletionTool } from '@fastgpt/global/core/ai/llm/type';
import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import { getFileContentByUrl } from '../../../../utils/file';
import { getErrText } from '@fastgpt/global/common/error/utils';
import { getLogger } from '@fastgpt-sdk/otel/logger';
import { LogCategories } from '../../../../../../common/logger';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { i18nT } from '../../../../../../../web/i18n/utils';
import z from 'zod';
const logger = getLogger(LogCategories.MODULE.AI.TOOL_CALL);
export const ReadFileTooData = {
id: 'read_files',
name: {
'zh-CN': '文件解析',
en: 'File parse',
'zh-Hant': '文件解析'
},
avatar: 'core/workflow/template/readFiles'
};
export const ReadFileToolSchema: ChatCompletionTool = {
type: 'function',
function: {
name: ReadFileTooData.id,
description: '解析文件内容，获取文本。',
parameters: {
type: 'object',
properties: {
ids: { type: 'array', items: { type: 'string' } }
},
required: ['ids']
}
}
};
export const ReadFileToolParamsSchema = z.object({
ids: z.array(z.string())
});
type FileReadParams = {
files: { id: string; url: string }[];
teamId: string;
tmbId: string;
customPdfParse?: boolean;
usageId?: string;
};
export const dispatchReadFileTool = async ({
files,
teamId,
tmbId,
customPdfParse,
usageId
}: FileReadParams) => {
try {
const usages: ChatNodeUsageType[] = [];
const readFilesResult = await Promise.all(
files.map(async ({ url, id }) => {
try {
const { name, content } = await getFileContentByUrl({
url,
teamId,
tmbId,
customPdfParse,
usageId
});
return {
id,
name,
content
};
} catch (error) {
return {
id,
name: url,
content: getErrText(error, 'Load file error')
};
}
})
);
// Stringify the result
const response = readFilesResult
.map(
(file) => `<file>
<id>${file.id}</id>
<content>${file.content}</content>
</file>`
)
.join('\n');
return {
response,
usages,
nodeResponse: {
moduleType: FlowNodeTypeEnum.readFiles,
moduleName: i18nT('chat:read_file')
}
};
} catch (error) {
logger.error('[File Read] Compression failed, using original content', { error });
return {
response: `Failed to read file: ${getErrText(error)}`,
usages: [],
nodeResponse: {
moduleType: FlowNodeTypeEnum.readFiles,
moduleName: i18nT('chat:read_file'),
errorText: `Failed to read file: ${getErrText(error)}`
}
};
}
};
packages/service/core/workflow/dispatch/ai/toolcall/type.ts
...@@ -30,6 +30,8 @@ export type DispatchToolModuleProps = ModuleDispatchProps<{ ...@@ -30,6 +30,8 @@ export type DispatchToolModuleProps = ModuleDispatchProps<{
toolNodes: ToolNodeItemType[]; toolNodes: ToolNodeItemType[];
toolModel: LLMModelItemType; toolModel: LLMModelItemType;
childrenInteractiveParams?: ToolCallChildrenInteractive['params']; childrenInteractiveParams?: ToolCallChildrenInteractive['params'];
allFiles: Map<string, FileInputType>;
currentInputFiles: FileInputType[];
}; };
export type ToolNodeItemType = { export type ToolNodeItemType = {
...@@ -49,3 +51,10 @@ export type ChildResponseItemType = { ...@@ -49,3 +51,10 @@ export type ChildResponseItemType = {
runTimes: DispatchFlowResponse['runTimes']; runTimes: DispatchFlowResponse['runTimes'];
flowUsages: DispatchFlowResponse['flowUsages']; flowUsages: DispatchFlowResponse['flowUsages'];
}; };
export type FileInputType = {
id: string;
name: string;
url: string;
sandboxPath?: string;
};
packages/service/core/workflow/dispatch/tools/readFiles.ts
...@@ -6,7 +6,7 @@ import { type DispatchNodeResultType } from '@fastgpt/global/core/workflow/runti ...@@ -6,7 +6,7 @@ import { type DispatchNodeResultType } from '@fastgpt/global/core/workflow/runti
import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants'; import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
import { type ChatItemMiniType } from '@fastgpt/global/core/chat/type'; import { type ChatItemMiniType } from '@fastgpt/global/core/chat/type';
import { getNodeErrResponse } from '../utils'; import { getNodeErrResponse } from '../utils';
import { getFileContentFromLinks } from '../../utils/file'; import { parseFileContentFromUrls } from '../../utils/file';
import { getUserFilesPrompt } from '../../../ai/llm/agentLoop/prompt'; import { getUserFilesPrompt } from '../../../ai/llm/agentLoop/prompt';
import { sliceStrStartEnd } from '@fastgpt/global/common/string/tools'; import { sliceStrStartEnd } from '@fastgpt/global/common/string/tools';
...@@ -35,7 +35,7 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => { ...@@ -35,7 +35,7 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => {
const filesFromHistories = version !== '489' ? [] : getHistoryFileLinks(histories); const filesFromHistories = version !== '489' ? [] : getHistoryFileLinks(histories);
try { try {
const readFilesResult = await getFileContentFromLinks({ const readFilesResult = await parseFileContentFromUrls({
// Concat fileUrlList and filesFromHistories; remove not supported files // Concat fileUrlList and filesFromHistories; remove not supported files
urls: [...fileUrlList, ...filesFromHistories], urls: [...fileUrlList, ...filesFromHistories],
requestOrigin, requestOrigin,
...@@ -47,7 +47,7 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => { ...@@ -47,7 +47,7 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => {
}); });
const files = readFilesResult.map((item, index) => ({ const files = readFilesResult.map((item, index) => ({
id: `${index}`, id: `${index}`,
name: item.filename, name: item.name,
content: item.content content: item.content
})); }));
...@@ -61,14 +61,14 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => { ...@@ -61,14 +61,14 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => {
data: { data: {
[NodeOutputKeyEnum.text]: text, [NodeOutputKeyEnum.text]: text,
[NodeOutputKeyEnum.rawResponse]: readFilesResult.map((item) => ({ [NodeOutputKeyEnum.rawResponse]: readFilesResult.map((item) => ({
filename: item.filename, filename: item.name,
url: item.url, url: item.url,
text: item.content text: item.content
})) }))
}, },
[DispatchNodeResponseKeyEnum.nodeResponse]: { [DispatchNodeResponseKeyEnum.nodeResponse]: {
readFiles: readFilesResult.map((item) => ({ readFiles: readFilesResult.map((item) => ({
name: item.filename, name: item.name,
url: item.url url: item.url
})), })),
readFilesResult: getPreviewResponse readFilesResult: getPreviewResponse
......
packages/service/core/workflow/utils/file.ts
...@@ -29,19 +29,20 @@ type GetFileProps = { ...@@ -29,19 +29,20 @@ type GetFileProps = {
usageId?: string; usageId?: string;
}; };
export const rewriteUserQueryWithFiles = async ({ export const formatUserQueryWithFiles = async ({
queryId,
userQuery, userQuery,
requestOrigin, parseFileFn
maxFiles, }: {
customPdfParse,
teamId,
tmbId,
usageId
}: GetFileProps & {
queryId: string;
userQuery: UserChatItemValueItemType[]; userQuery: UserChatItemValueItemType[];
}) => { parseFileFn: (urls: string[]) => Promise<
{
id?: string;
name: string;
sandboxPath?: string;
content?: string;
}[]
>;
}): Promise<UserChatItemValueItemType[]> => {
const urls = userQuery const urls = userQuery
.map((item) => (item.file?.type === ChatFileTypeEnum.file ? item.file.url : '')) .map((item) => (item.file?.type === ChatFileTypeEnum.file ? item.file.url : ''))
.filter(Boolean); .filter(Boolean);
...@@ -50,29 +51,15 @@ export const rewriteUserQueryWithFiles = async ({ ...@@ -50,29 +51,15 @@ export const rewriteUserQueryWithFiles = async ({
return userQuery; return userQuery;
} }
const readFilesResult = await getFileContentFromLinks({ const readFilesResult = await parseFileFn(urls);
urls,
requestOrigin,
maxFiles,
teamId,
tmbId,
customPdfParse,
usageId
});
if (readFilesResult.length === 0) { if (readFilesResult.length === 0) {
return userQuery; return userQuery;
} }
const files = readFilesResult.map((item, index) => ({
id: `${queryId}-${index}`,
name: item.filename,
content: item.content
}));
// 把 file 和 text 合并成一个 text(实际上应该只会有一个 text+多个 files) // 把 file 和 text 合并成一个 text(实际上应该只会有一个 text+多个 files)
const text = userQuery.find((item) => item.text?.content)?.text?.content; const text = userQuery.find((item) => item.text?.content)?.text?.content;
const fileQuery = getUserFilesPrompt(files); const fileQuery = getUserFilesPrompt(readFilesResult);
const finalQuery = injectUserQueryPrompt({ const finalQuery = injectUserQueryPrompt({
query: text, query: text,
...@@ -124,56 +111,15 @@ export const normalizeReadableFileUrl = ({ ...@@ -124,56 +111,15 @@ export const normalizeReadableFileUrl = ({
} }
}; };
export const getFileContentFromLinks = async ({ export const getFileInfoFromUrl = async ({ teamId, url }: { teamId: string; url: string }) => {
urls,
requestOrigin,
maxFiles,
teamId,
tmbId,
customPdfParse,
usageId
}: GetFileProps & {
urls: string[];
}) => {
const parseUrlList = urls
.map((url) => normalizeReadableFileUrl({ url, requestOrigin }))
.filter(Boolean)
.slice(0, maxFiles);
const readFilesResult = await Promise.all(
parseUrlList
.map(async (url) => {
// Get from buffer
const rawTextBuffer = await getS3RawTextSource().getRawTextBuffer({
sourceId: url,
customPdfParse
});
if (rawTextBuffer) {
return {
success: true,
filename: rawTextBuffer.filename,
url,
content: rawTextBuffer.text
};
}
try {
if (await isInternalAddress(url)) {
return Promise.reject(PRIVATE_URL_TEXT);
}
// Get file buffer data // Get file buffer data
const response = await axios.get(url, { const response = await axios.get(url, {
baseURL: serverRequestBaseUrl, baseURL: serverRequestBaseUrl,
responseType: 'arraybuffer' responseType: 'arraybuffer'
}); });
const buffer = Buffer.from(response.data, 'binary');
const urlObj = new URL(url, 'http://localhost:3000'); const urlObj = new URL(url, 'http://localhost:3000');
const isChatExternalUrl = !urlObj.pathname.startsWith( const isChatExternalUrl = !urlObj.pathname.startsWith(`/${S3Buckets.private}/${S3Sources.chat}/`);
`/${S3Buckets.private}/${S3Sources.chat}/`
);
// Get file name // Get file name
const { filename, extension, imageParsePrefix } = (() => { const { filename, extension, imageParsePrefix } = (() => {
...@@ -193,9 +139,48 @@ export const getFileContentFromLinks = async ({ ...@@ -193,9 +139,48 @@ export const getFileContentFromLinks = async ({
return S3ChatSource.parseChatUrl(url); return S3ChatSource.parseChatUrl(url);
})(); })();
return {
isChatExternalUrl,
filename,
extension,
imageParsePrefix,
contentType: response.headers['content-type'],
stream: response.data
};
};
export const getFileContentByUrl = async ({
url,
teamId,
tmbId,
customPdfParse,
usageId
}: {
url: string;
teamId: string;
tmbId: string;
customPdfParse?: boolean;
usageId?: string;
}) => {
// Get from buffer
const rawTextBuffer = await getS3RawTextSource().getRawTextBuffer({
sourceId: url,
customPdfParse
});
if (rawTextBuffer) {
return {
name: rawTextBuffer.filename,
url,
content: rawTextBuffer.text
};
}
const { isChatExternalUrl, filename, extension, imageParsePrefix, contentType, stream } =
await getFileInfoFromUrl({ teamId, url });
const buffer = Buffer.from(stream, 'binary');
// Get encoding // Get encoding
const encoding = (() => { const encoding = (() => {
const contentType = response.headers['content-type'];
if (contentType) { if (contentType) {
const charsetRegex = /charset=([^;]*)/; const charsetRegex = /charset=([^;]*)/;
const matches = charsetRegex.exec(contentType); const matches = charsetRegex.exec(contentType);
...@@ -235,11 +220,61 @@ export const getFileContentFromLinks = async ({ ...@@ -235,11 +220,61 @@ export const getFileContentFromLinks = async ({
customPdfParse customPdfParse
}); });
return { success: true, filename, url, content: replacedText }; return {
name: filename,
url,
content: replacedText
};
};
export const parseFileContentFromUrls = async ({
urls,
requestOrigin,
maxFiles,
teamId,
tmbId,
customPdfParse,
usageId
}: GetFileProps & {
urls: string[];
}): Promise<
{
success: boolean;
name: string;
url: string;
content: string;
}[]
> => {
const parseUrlList = urls
.map((url) => normalizeReadableFileUrl({ url, requestOrigin }))
.filter(Boolean)
.slice(0, maxFiles);
const readFilesResult = await Promise.all(
parseUrlList
.map(async (url) => {
try {
if (await isInternalAddress(url)) {
return {
success: false,
name: '',
url,
content: PRIVATE_URL_TEXT
};
}
const { name, content } = await getFileContentByUrl({
url,
teamId,
tmbId,
customPdfParse,
usageId
});
return { success: true, name, url, content: content };
} catch (error) { } catch (error) {
return { return {
success: false, success: false,
filename: '', name: '',
url, url,
content: getErrText(error, 'Load file error') content: getErrText(error, 'Load file error')
}; };
...@@ -250,3 +285,66 @@ export const getFileContentFromLinks = async ({ ...@@ -250,3 +285,66 @@ export const getFileContentFromLinks = async ({
return readFilesResult; return readFilesResult;
}; };
export const parseFileInfoFromUrls = async ({
urls,
requestOrigin,
maxFiles,
teamId
}: {
requestOrigin?: string;
maxFiles: number;
teamId: string;
urls: string[];
}): Promise<
{
success: boolean;
name: string;
url: string;
}[]
> => {
const parseUrlList = urls
.map((url) => normalizeReadableFileUrl({ url, requestOrigin }))
.filter(Boolean)
.slice(0, maxFiles);
const readFilesResult = await Promise.all(
parseUrlList
.map(async (url) => {
// Get from buffer
const rawTextBuffer = await getS3RawTextSource().getRawTextBuffer({
sourceId: url,
customPdfParse: false
});
if (rawTextBuffer) {
return {
success: true,
name: rawTextBuffer.filename,
url
};
}
try {
if (await isInternalAddress(url)) {
return {
success: false,
name: '',
url
};
}
const { filename } = await getFileInfoFromUrl({ teamId, url });
return { success: true, name: filename, url };
} catch (error) {
return {
success: false,
name: '',
url
};
}
})
.filter(Boolean)
);
return readFilesResult;
};
packages/service/test/core/workflow/dispatch/tools/readFiles.test.ts
...@@ -4,10 +4,10 @@ import type { ChatItemMiniType } from '@fastgpt/global/core/chat/type'; ...@@ -4,10 +4,10 @@ import type { ChatItemMiniType } from '@fastgpt/global/core/chat/type';
import { NodeOutputKeyEnum } from '@fastgpt/global/core/workflow/constants'; import { NodeOutputKeyEnum } from '@fastgpt/global/core/workflow/constants';
import { DispatchNodeResponseKeyEnum } from '@fastgpt/global/core/workflow/runtime/constants'; import { DispatchNodeResponseKeyEnum } from '@fastgpt/global/core/workflow/runtime/constants';
const mockGetFileContentFromLinks = vi.hoisted(() => vi.fn()); const mockparseFileContentFromUrls = vi.hoisted(() => vi.fn());
vi.mock('@fastgpt/service/core/workflow/utils/file', () => ({ vi.mock('@fastgpt/service/core/workflow/utils/file', () => ({
getFileContentFromLinks: mockGetFileContentFromLinks parseFileContentFromUrls: mockparseFileContentFromUrls
})); }));
import { import {
...@@ -28,13 +28,13 @@ const baseProps = { ...@@ -28,13 +28,13 @@ const baseProps = {
describe('dispatchReadFiles', () => { describe('dispatchReadFiles', () => {
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks(); vi.clearAllMocks();
mockGetFileContentFromLinks.mockResolvedValue([]); mockparseFileContentFromUrls.mockResolvedValue([]);
}); });
it('成功读取并返回文本/原始响应/节点响应/工具响应结构', async () => { it('成功读取并返回文本/原始响应/节点响应/工具响应结构', async () => {
mockGetFileContentFromLinks.mockResolvedValue([ mockparseFileContentFromUrls.mockResolvedValue([
{ success: true, filename: 'a.pdf', url: '/a.pdf', content: 'Alpha' }, { success: true, name: 'a.pdf', url: '/a.pdf', content: 'Alpha' },
{ success: true, filename: 'b.pdf', url: '/b.pdf', content: 'Beta' } { success: true, name: 'b.pdf', url: '/b.pdf', content: 'Beta' }
]); ]);
const result = await dispatchReadFiles({ const result = await dispatchReadFiles({
...@@ -42,7 +42,7 @@ describe('dispatchReadFiles', () => { ...@@ -42,7 +42,7 @@ describe('dispatchReadFiles', () => {
params: { fileUrlList: ['/a.pdf', '/b.pdf'] } params: { fileUrlList: ['/a.pdf', '/b.pdf'] }
}); });
expect(mockGetFileContentFromLinks).toHaveBeenCalledWith({ expect(mockparseFileContentFromUrls).toHaveBeenCalledWith({
urls: ['/a.pdf', '/b.pdf'], urls: ['/a.pdf', '/b.pdf'],
requestOrigin: 'http://localhost:3000', requestOrigin: 'http://localhost:3000',
maxFiles: 20, maxFiles: 20,
...@@ -90,7 +90,7 @@ describe('dispatchReadFiles', () => { ...@@ -90,7 +90,7 @@ describe('dispatchReadFiles', () => {
params: { fileUrlList: ['/a.pdf'] } params: { fileUrlList: ['/a.pdf'] }
}); });
expect(mockGetFileContentFromLinks).toHaveBeenCalledWith( expect(mockparseFileContentFromUrls).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
maxFiles: 5, maxFiles: 5,
customPdfParse: true customPdfParse: true
...@@ -105,7 +105,7 @@ describe('dispatchReadFiles', () => { ...@@ -105,7 +105,7 @@ describe('dispatchReadFiles', () => {
params: { fileUrlList: ['/a.pdf'] } params: { fileUrlList: ['/a.pdf'] }
}); });
expect(mockGetFileContentFromLinks).toHaveBeenCalledWith( expect(mockparseFileContentFromUrls).toHaveBeenCalledWith(
expect.objectContaining({ maxFiles: 20, customPdfParse: false }) expect.objectContaining({ maxFiles: 20, customPdfParse: false })
); );
}); });
...@@ -117,7 +117,7 @@ describe('dispatchReadFiles', () => { ...@@ -117,7 +117,7 @@ describe('dispatchReadFiles', () => {
params: { fileUrlList: ['/a.pdf'] } params: { fileUrlList: ['/a.pdf'] }
}); });
expect(mockGetFileContentFromLinks).toHaveBeenCalledWith( expect(mockparseFileContentFromUrls).toHaveBeenCalledWith(
expect.objectContaining({ maxFiles: 20 }) expect.objectContaining({ maxFiles: 20 })
); );
}); });
...@@ -145,7 +145,7 @@ describe('dispatchReadFiles', () => { ...@@ -145,7 +145,7 @@ describe('dispatchReadFiles', () => {
params: { fileUrlList: ['/current.pdf'] } params: { fileUrlList: ['/current.pdf'] }
}); });
expect(mockGetFileContentFromLinks).toHaveBeenCalledWith( expect(mockparseFileContentFromUrls).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
urls: ['/current.pdf', '/history.pdf'] urls: ['/current.pdf', '/history.pdf']
}) })
...@@ -175,7 +175,7 @@ describe('dispatchReadFiles', () => { ...@@ -175,7 +175,7 @@ describe('dispatchReadFiles', () => {
params: { fileUrlList: ['/current.pdf'] } params: { fileUrlList: ['/current.pdf'] }
}); });
expect(mockGetFileContentFromLinks).toHaveBeenCalledWith( expect(mockparseFileContentFromUrls).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
urls: ['/current.pdf'] urls: ['/current.pdf']
}) })
...@@ -188,11 +188,13 @@ describe('dispatchReadFiles', () => { ...@@ -188,11 +188,13 @@ describe('dispatchReadFiles', () => {
params: {} params: {}
}); });
expect(mockGetFileContentFromLinks).toHaveBeenCalledWith(expect.objectContaining({ urls: [] })); expect(mockparseFileContentFromUrls).toHaveBeenCalledWith(
expect.objectContaining({ urls: [] })
);
}); });
it('空文件结果返回空文本和空数组结构', async () => { it('空文件结果返回空文本和空数组结构', async () => {
mockGetFileContentFromLinks.mockResolvedValue([]); mockparseFileContentFromUrls.mockResolvedValue([]);
const result = await dispatchReadFiles({ const result = await dispatchReadFiles({
...baseProps, ...baseProps,
...@@ -209,8 +211,8 @@ describe('dispatchReadFiles', () => { ...@@ -209,8 +211,8 @@ describe('dispatchReadFiles', () => {
it('超大内容下预览仍按 sliceStrStartEnd 截断 (start/end 各 1000)', async () => { it('超大内容下预览仍按 sliceStrStartEnd 截断 (start/end 各 1000)', async () => {
const huge = 'x'.repeat(5000); const huge = 'x'.repeat(5000);
mockGetFileContentFromLinks.mockResolvedValue([ mockparseFileContentFromUrls.mockResolvedValue([
{ success: true, filename: 'big.txt', url: '/big.txt', content: huge } { success: true, name: 'big.txt', url: '/big.txt', content: huge }
]); ]);
const result = await dispatchReadFiles({ const result = await dispatchReadFiles({
...@@ -226,8 +228,8 @@ describe('dispatchReadFiles', () => { ...@@ -226,8 +228,8 @@ describe('dispatchReadFiles', () => {
expect(preview).toContain('## big.txt'); expect(preview).toContain('## big.txt');
}); });
it('getFileContentFromLinks 抛错时通过 getNodeErrResponse 返回错误结构', async () => { it('parseFileContentFromUrls 抛错时通过 getNodeErrResponse 返回错误结构', async () => {
mockGetFileContentFromLinks.mockRejectedValue(new Error('boom')); mockparseFileContentFromUrls.mockRejectedValue(new Error('boom'));
const result = await dispatchReadFiles({ const result = await dispatchReadFiles({
...baseProps, ...baseProps,
......
packages/service/type/env.ts
...@@ -7,7 +7,6 @@ declare global { ...@@ -7,7 +7,6 @@ declare global {
PRO_URL: string; PRO_URL: string;
LOG_DEPTH: string; LOG_DEPTH: string;
DB_MAX_LINK: string; DB_MAX_LINK: string;
FILE_TOKEN_KEY: string;
STORAGE_VENDOR?: 'minio' | 'aws-s3' | 'cos' | 'oss'; STORAGE_VENDOR?: 'minio' | 'aws-s3' | 'cos' | 'oss';
STORAGE_PUBLIC_BUCKET?: string; STORAGE_PUBLIC_BUCKET?: string;
......
pro @ 41720ca1
Subproject commit 1d38337167baeed33ece061772c84b0ccce71333 Subproject commit 41720ca13d5c9c85a6f135bb8c71567308ef3d96
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment