Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit b2cfada9 by Archer Committed by GitHub
Options

perf: agent tool code (#6798) 

* perf: agent tool code

* fix: review
parent 2fe03ec0
Showing with 554 additions and 525 deletions
.claude/design/core/ai/gradient-pricing-fix.md
...@@ -65,12 +65,12 @@ const { totalPoints: modelTotalPoints } = formatModelChars2Points({ ...@@ -65,12 +65,12 @@ const { totalPoints: modelTotalPoints } = formatModelChars2Points({
}); });
``` ```
`runToolCall` 调用 `runAgentCall` 时**不传 `usagePush`**，所以单次计价全部丢失，只依赖这里的累加计算 → **实际计费错误**。 `runToolCall` 调用 `runAgentLoop` 时**不传 `usagePush`**，所以单次计价全部丢失，只依赖这里的累加计算 → **实际计费错误**。
### 3. `packages/service/core/workflow/dispatch/ai/agent/master/call.ts` (masterCall) — **展示 BUG** ### 3. `packages/service/core/workflow/dispatch/ai/agent/master/call.ts` (masterCall) — **展示 BUG**
```ts ```ts
// inputTokens = runAgentCall 返回的累加值 // inputTokens = runAgentLoop 返回的累加值
const llmUsage = formatModelChars2Points({ const llmUsage = formatModelChars2Points({
inputTokens, // ❌ 累加值 inputTokens, // ❌ 累加值
outputTokens outputTokens
...@@ -101,15 +101,15 @@ const { totalPoints } = formatModelChars2Points({ ...@@ -101,15 +101,15 @@ const { totalPoints } = formatModelChars2Points({
**不应用累加的 token 数计算价格，而应该每次 LLM 调用单独计价，再累加价格。** **不应用累加的 token 数计算价格，而应该每次 LLM 调用单独计价，再累加价格。**
### 方案：`runAgentCall` 返回预计算的 `llmTotalPoints` ### 方案：`runAgentLoop` 返回预计算的 `llmTotalPoints`
在 `runAgentCall` 的 while 循环中，每次 LLM 调用后立即计算该次的价格，并累加到 `llmTotalPoints`，最终将其作为返回值之一。调用方直接使用该预计算值，而不再重复调用 `formatModelChars2Points(累加 tokens)`。 在 `runAgentLoop` 的 while 循环中，每次 LLM 调用后立即计算该次的价格，并累加到 `llmTotalPoints`，最终将其作为返回值之一。调用方直接使用该预计算值，而不再重复调用 `formatModelChars2Points(累加 tokens)`。
--- ---
## 具体修改 ## 具体修改
### 修改 1：`runAgentCall` — 增加 `llmTotalPoints` 返回值 ### 修改 1：`runAgentLoop` — 增加 `llmTotalPoints` 返回值
**文件**：`packages/service/core/ai/llm/agentCall/index.ts` **文件**：`packages/service/core/ai/llm/agentCall/index.ts`
...@@ -155,8 +155,8 @@ type ResponseType = { ...@@ -155,8 +155,8 @@ type ResponseType = {
toolCallOutputTokens: number; // 保留展示用 toolCallOutputTokens: number; // 保留展示用
}; };
// runAgentCall 返回后 // runAgentLoop 返回后
const { inputTokens, outputTokens, llmTotalPoints, ... } = await runAgentCall(...); const { inputTokens, outputTokens, llmTotalPoints, ... } = await runAgentLoop(...);
return { return {
... ...
...@@ -189,8 +189,8 @@ const modelTotalPoints = toolCallTotalPoints; // 直接使用预计算值， ...@@ -189,8 +189,8 @@ const modelTotalPoints = toolCallTotalPoints; // 直接使用预计算值，
**文件**：`packages/service/core/workflow/dispatch/ai/agent/master/call.ts` **文件**：`packages/service/core/workflow/dispatch/ai/agent/master/call.ts`
```ts ```ts
// runAgentCall 返回 llmTotalPoints // runAgentLoop 返回 llmTotalPoints
const { inputTokens, outputTokens, llmTotalPoints, childrenUsages, ... } = await runAgentCall(...); const { inputTokens, outputTokens, llmTotalPoints, childrenUsages, ... } = await runAgentLoop(...);
// 修改前（❌） // 修改前（❌）
const llmUsage = formatModelChars2Points({ model: agentModel, inputTokens, outputTokens }); const llmUsage = formatModelChars2Points({ model: agentModel, inputTokens, outputTokens });
...@@ -265,7 +265,7 @@ usage.outputTokens += regenResult.usage.outputTokens; ...@@ -265,7 +265,7 @@ usage.outputTokens += regenResult.usage.outputTokens;
## TODO ## TODO
- [ ] 修改 `runAgentCall` 返回类型，新增 `llmTotalPoints` - [ ] 修改 `runAgentLoop` 返回类型，新增 `llmTotalPoints`
- [ ] 修改 `runToolCall` 返回类型，新增 `toolCallTotalPoints` - [ ] 修改 `runToolCall` 返回类型，新增 `toolCallTotalPoints`
- [ ] 修改 `dispatchRunTools` 使用预计算值 - [ ] 修改 `dispatchRunTools` 使用预计算值
- [ ] 修改 `masterCall` 使用预计算值（修正展示） - [ ] 修改 `masterCall` 使用预计算值（修正展示）
......
.claude/design/core/ai/sandbox/get-file-url.md
...@@ -200,7 +200,7 @@ export const dispatchSandboxGetFileUrl = async ({ ...@@ -200,7 +200,7 @@ export const dispatchSandboxGetFileUrl = async ({
#### 3.5.1 普通工作流：toolCall.ts #### 3.5.1 普通工作流：toolCall.ts
`handleToolResponse` 中合并 `SANDBOX_TOOL_NAME` 和 `SANDBOX_GET_FILE_URL_TOOL_NAME` 到同一拦截块： `onRunTool` 中合并 `SANDBOX_TOOL_NAME` 和 `SANDBOX_GET_FILE_URL_TOOL_NAME` 到同一拦截块：
```typescript ```typescript
if ( if (
......
document/content/docs/self-host/upgrading/4-15/4150.mdx 0 → 100644
---
title: 'V4.15.0(进行中)'
description: 'FastGPT V4.15.0 更新说明'
---
## 🚀 新增内容
## ⚙️ 优化
## 🐛 修复
## 代码优化
1. 优化 Agent tool 声明和运行，统一所有 tool 的声明和运行方式。
\ No newline at end of file
document/content/docs/self-host/upgrading/4-15/meta.en.json 0 → 100644
{
"title": "4.15.x",
"description": "",
"pages": ["4150"]
}
document/content/docs/self-host/upgrading/4-15/meta.json 0 → 100644
{
"title": "4.15.x",
"description": "",
"pages": ["4150"]
}
document/content/docs/self-host/upgrading/meta.en.json
{ {
"title": "Version History", "title": "Version History",
"description": "FastGPT version history", "description": "FastGPT version history",
"pages": ["4-14", "4-13", "4-12", "outdated"] "pages": ["4-15", "4-14", "4-13", "4-12", "outdated"]
} }
document/content/docs/self-host/upgrading/meta.json
{ {
"title": "版本列表", "title": "版本列表",
"description": "FastGPT 版本列表", "description": "FastGPT 版本列表",
"pages": ["4-14", "4-13", "4-12", "outdated"] "pages": ["4-15", "4-14", "4-13", "4-12", "outdated"]
} }
document/content/docs/toc.mdx
...@@ -127,6 +127,7 @@ description: FastGPT 文档目录 ...@@ -127,6 +127,7 @@ description: FastGPT 文档目录
- [/docs/self-host/upgrading/4-14/4148](/docs/self-host/upgrading/4-14/4148) - [/docs/self-host/upgrading/4-14/4148](/docs/self-host/upgrading/4-14/4148)
- [/docs/self-host/upgrading/4-14/41481](/docs/self-host/upgrading/4-14/41481) - [/docs/self-host/upgrading/4-14/41481](/docs/self-host/upgrading/4-14/41481)
- [/docs/self-host/upgrading/4-14/4149](/docs/self-host/upgrading/4-14/4149) - [/docs/self-host/upgrading/4-14/4149](/docs/self-host/upgrading/4-14/4149)
- [/docs/self-host/upgrading/4-15/4150](/docs/self-host/upgrading/4-15/4150)
- [/docs/self-host/upgrading/outdated/40](/docs/self-host/upgrading/outdated/40) - [/docs/self-host/upgrading/outdated/40](/docs/self-host/upgrading/outdated/40)
- [/docs/self-host/upgrading/outdated/41](/docs/self-host/upgrading/outdated/41) - [/docs/self-host/upgrading/outdated/41](/docs/self-host/upgrading/outdated/41)
- [/docs/self-host/upgrading/outdated/4100](/docs/self-host/upgrading/outdated/4100) - [/docs/self-host/upgrading/outdated/4100](/docs/self-host/upgrading/outdated/4100)
......
document/data/doc-last-modified.json
...@@ -224,8 +224,12 @@ ...@@ -224,8 +224,12 @@
"document/content/docs/self-host/upgrading/4-14/4141.mdx": "2026-03-03T17:39:47+08:00", "document/content/docs/self-host/upgrading/4-14/4141.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/4-14/41410.en.mdx": "2026-03-31T23:15:29+08:00", "document/content/docs/self-host/upgrading/4-14/41410.en.mdx": "2026-03-31T23:15:29+08:00",
"document/content/docs/self-host/upgrading/4-14/41410.mdx": "2026-04-18T20:47:39+08:00", "document/content/docs/self-host/upgrading/4-14/41410.mdx": "2026-04-18T20:47:39+08:00",
"document/content/docs/self-host/upgrading/4-14/41411.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41411.mdx": "2026-04-20T20:18:35+08:00", "document/content/docs/self-host/upgrading/4-14/41411.mdx": "2026-04-20T20:18:35+08:00",
"document/content/docs/self-host/upgrading/4-14/41412.mdx": "2026-04-20T20:18:35+08:00", "document/content/docs/self-host/upgrading/4-14/41412.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41412.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41413.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41413.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/4142.en.mdx": "2026-03-03T17:39:47+08:00", "document/content/docs/self-host/upgrading/4-14/4142.en.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/4-14/4142.mdx": "2026-03-03T17:39:47+08:00", "document/content/docs/self-host/upgrading/4-14/4142.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/4-14/4143.en.mdx": "2026-03-03T17:39:47+08:00", "document/content/docs/self-host/upgrading/4-14/4143.en.mdx": "2026-03-03T17:39:47+08:00",
...@@ -386,8 +390,8 @@ ...@@ -386,8 +390,8 @@
"document/content/docs/self-host/upgrading/outdated/499.mdx": "2026-03-03T17:39:47+08:00", "document/content/docs/self-host/upgrading/outdated/499.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/upgrade-intruction.en.mdx": "2026-03-03T17:39:47+08:00", "document/content/docs/self-host/upgrading/upgrade-intruction.en.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/upgrade-intruction.mdx": "2026-04-20T13:51:34+08:00", "document/content/docs/self-host/upgrading/upgrade-intruction.mdx": "2026-04-20T13:51:34+08:00",
"document/content/docs/toc.en.mdx": "2026-04-17T23:28:43+08:00", "document/content/docs/toc.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/toc.mdx": "2026-04-20T17:45:22+08:00", "document/content/docs/toc.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/use-cases/app-cases/dalle3.en.mdx": "2026-02-26T22:14:30+08:00", "document/content/docs/use-cases/app-cases/dalle3.en.mdx": "2026-02-26T22:14:30+08:00",
"document/content/docs/use-cases/app-cases/dalle3.mdx": "2025-07-23T21:35:03+08:00", "document/content/docs/use-cases/app-cases/dalle3.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/use-cases/app-cases/english_essay_correction_bot.en.mdx": "2026-02-26T22:14:30+08:00", "document/content/docs/use-cases/app-cases/english_essay_correction_bot.en.mdx": "2026-02-26T22:14:30+08:00",
......
packages/global/core/ai/sandbox/constants.ts
...@@ -15,10 +15,10 @@ export const SANDBOX_SUSPEND_MINUTES = 5; ...@@ -15,10 +15,10 @@ export const SANDBOX_SUSPEND_MINUTES = 5;
// ---- sandboxId 生成 ---- // ---- sandboxId 生成 ----
export const generateSandboxId = (appId: string, userId: string, chatId: string): string => { export const generateSandboxId = (appId: string, userId: string, chatId: string): string => {
return hashStr(`${appId}-${userId}-${chatId}`).slice(0, 16); return hashStr(`${String(appId)}-${String(userId)}-${String(chatId)}`).slice(0, 16);
}; };
// Tool // Shell Tool
export const SANDBOX_NAME: I18nStringType = { export const SANDBOX_NAME: I18nStringType = {
'zh-CN': '虚拟机', 'zh-CN': '虚拟机',
'zh-Hant': '虛擬機', 'zh-Hant': '虛擬機',
...@@ -26,10 +26,6 @@ export const SANDBOX_NAME: I18nStringType = { ...@@ -26,10 +26,6 @@ export const SANDBOX_NAME: I18nStringType = {
}; };
export const SANDBOX_ICON = 'core/app/sandbox/sandbox' as const; export const SANDBOX_ICON = 'core/app/sandbox/sandbox' as const;
export const SANDBOX_TOOL_NAME = 'sandbox_shell'; export const SANDBOX_TOOL_NAME = 'sandbox_shell';
export const SandboxShellToolSchema = z.object({
command: z.string(),
timeout: z.number().optional()
});
export const SANDBOX_SHELL_TOOL: ChatCompletionTool = { export const SANDBOX_SHELL_TOOL: ChatCompletionTool = {
type: 'function', type: 'function',
function: { function: {
...@@ -51,15 +47,13 @@ export const SANDBOX_SHELL_TOOL: ChatCompletionTool = { ...@@ -51,15 +47,13 @@ export const SANDBOX_SHELL_TOOL: ChatCompletionTool = {
} }
}; };
// Get File URL Tool
export const SANDBOX_READ_FILE_TOOL_NAME: I18nStringType = { export const SANDBOX_READ_FILE_TOOL_NAME: I18nStringType = {
'zh-CN': '虚拟机/获取文件链接', 'zh-CN': '虚拟机/获取文件链接',
'zh-Hant': '虛擬機/獲取文件鏈接', 'zh-Hant': '虛擬機/獲取文件鏈接',
en: 'Sandbox/Get File URL' en: 'Sandbox/Get File URL'
}; };
export const SANDBOX_GET_FILE_URL_TOOL_NAME = 'sandbox_get_file_url'; export const SANDBOX_GET_FILE_URL_TOOL_NAME = 'sandbox_get_file_url';
export const SandboxGetFileUrlToolSchema = z.object({
paths: z.array(z.string())
});
export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = { export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = {
type: 'function', type: 'function',
function: { function: {
...@@ -82,10 +76,30 @@ export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = { ...@@ -82,10 +76,30 @@ export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = {
} }
}; };
export const SANDBOX_TOOLS: ChatCompletionTool[] = [SANDBOX_SHELL_TOOL, SANDBOX_GET_FILE_URL_TOOL]; // Prompt
export const SANDBOX_SYSTEM_PROMPT = `你拥有一个独立的 Linux 沙盒环境（Ubuntu 22.04），可通过 ${SANDBOX_TOOL_NAME} 工具执行命令： export const SANDBOX_SYSTEM_PROMPT = `你拥有一个独立的 Linux 沙盒环境（Ubuntu 22.04），可通过 ${SANDBOX_TOOL_NAME} 工具执行命令：
- 预装：bash / python3 / node / bun / git / curl - 预装：bash / python3 / node / bun / git / curl
- 可自行安装软件包（apt / pip / npm） - 可自行安装软件包（apt / pip / npm）
- 生成的文件内容都保存在当前目录下即可 - 生成的文件内容都保存在当前目录下即可
- 若需要将生成的文件分享给用户，可使用 ${SANDBOX_GET_FILE_URL_TOOL_NAME} 工具获取文件的临时访问链接`; - 若需要将生成的文件分享给用户，可使用 ${SANDBOX_GET_FILE_URL_TOOL_NAME} 工具获取文件的临时访问链接`;
// 聚合
export const sandboxToolMap: Record<
string,
{ schema: ChatCompletionTool; name: I18nStringType; avatar: string; toolDescription: string }
> = {
[SANDBOX_TOOL_NAME]: {
schema: SANDBOX_SHELL_TOOL,
name: SANDBOX_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_SHELL_TOOL.function.description!
},
[SANDBOX_GET_FILE_URL_TOOL_NAME]: {
schema: SANDBOX_GET_FILE_URL_TOOL,
name: SANDBOX_READ_FILE_TOOL_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_GET_FILE_URL_TOOL.function.description!
}
};
export const SANDBOX_TOOLS = Object.values(sandboxToolMap).map((item) => item.schema);
packages/global/core/chat/adapt.ts
...@@ -217,7 +217,7 @@ export const GPTMessages2Chats = ({ ...@@ -217,7 +217,7 @@ export const GPTMessages2Chats = ({
messages: ChatCompletionMessageParam[]; messages: ChatCompletionMessageParam[];
reserveTool?: boolean; reserveTool?: boolean;
reserveReason?: boolean; reserveReason?: boolean;
getToolInfo?: (name: string) => { name: string; avatar: string }; getToolInfo?: (name: string) => { name: string; avatar?: string } | undefined;
}): ChatItemMiniType[] => { }): ChatItemMiniType[] => {
const chatMessages = messages const chatMessages = messages
.map((item) => { .map((item) => {
......
packages/global/core/workflow/node/agent/constants.ts
import { import type { I18nStringType, localeType } from '../../../../common/i18n/type';
SANDBOX_GET_FILE_URL_TOOL, import { sandboxToolMap } from '../../../ai/sandbox/constants';
SANDBOX_ICON,
SANDBOX_NAME,
SANDBOX_READ_FILE_TOOL_NAME,
SANDBOX_SHELL_TOOL
} from '../../../ai/sandbox/constants';
import type { I18nStringType } from '../../../../common/i18n/type';
import { skillToolsMap } from './skillTools'; import { skillToolsMap } from './skillTools';
import { parseI18nString } from '../../../../common/i18n/utils';
export enum SubAppIds { export enum SubAppIds {
plan = 'plan_agent', plan = 'plan_agent',
ask = 'ask_agent', ask = 'ask_agent',
model = 'model_agent', model = 'model_agent',
fileRead = 'file_read', fileRead = 'file_read',
datasetSearch = 'dataset_search', datasetSearch = 'dataset_search'
sandboxTool = 'sandbox_shell',
sandboxGetFileUrl = 'sandbox_get_file_url'
} }
export const systemSubInfo: Record< export const systemSubInfo: Record<
string, string,
{ name: I18nStringType; avatar: string; toolDescription: string } { name: I18nStringType; avatar: string; toolDescription: string }
> = { > = {
[SubAppIds.sandboxTool]: {
name: SANDBOX_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_SHELL_TOOL.function.description!
},
[SubAppIds.sandboxGetFileUrl]: {
name: SANDBOX_READ_FILE_TOOL_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_GET_FILE_URL_TOOL.function.description!
},
[SubAppIds.plan]: { [SubAppIds.plan]: {
name: { name: {
'zh-CN': '规划Agent', 'zh-CN': '规划Agent',
...@@ -78,5 +61,16 @@ export const systemSubInfo: Record< ...@@ -78,5 +61,16 @@ export const systemSubInfo: Record<
avatar: 'core/workflow/template/agent', avatar: 'core/workflow/template/agent',
toolDescription: '调用 LLM 模型完成一些通用任务。' toolDescription: '调用 LLM 模型完成一些通用任务。'
}, },
...sandboxToolMap,
...skillToolsMap ...skillToolsMap
}; };
export const getSystemToolInfo = (id: string, lang: localeType = 'en') => {
if (id in systemSubInfo) {
const info = systemSubInfo[id];
return {
name: parseI18nString(info.name, lang),
avatar: info.avatar,
toolDescription: info.toolDescription
};
}
};
packages/global/core/workflow/node/agent/skillTools.ts
import z from 'zod'; import z from 'zod';
import type { ChatCompletionTool } from '../../../ai/llm/type'; import type { ChatCompletionTool } from '../../../ai/llm/type';
import type { I18nStringType, localeType } from '../../../../common/i18n/type';
import { parseI18nString } from '../../../../common/i18n/utils';
export enum SandboxToolIds { export enum SandboxToolIds {
readFile = 'sandbox_read_file', readFile = 'sandbox_read_file',
...@@ -10,7 +12,10 @@ export enum SandboxToolIds { ...@@ -10,7 +12,10 @@ export enum SandboxToolIds {
fetchUserFile = 'sandbox_fetch_user_file' fetchUserFile = 'sandbox_fetch_user_file'
} }
export const skillToolsMap = { export const skillToolsMap: Record<
string,
{ name: I18nStringType; avatar: string; toolDescription: string }
> = {
// Sandbox tools // Sandbox tools
[SandboxToolIds.readFile]: { [SandboxToolIds.readFile]: {
name: { name: {
...@@ -73,6 +78,19 @@ export const skillToolsMap = { ...@@ -73,6 +78,19 @@ export const skillToolsMap = {
'Download a user-uploaded file (document or image) from the conversation and write it as a binary file into the sandbox filesystem. Use this when a skill script needs to process a raw file. Workflow: call this tool first to place the file at target_path (relative to workspace), then run skill scripts that read from that path.' 'Download a user-uploaded file (document or image) from the conversation and write it as a binary file into the sandbox filesystem. Use this when a skill script needs to process a raw file. Workflow: call this tool first to place the file at target_path (relative to workspace), then run skill scripts that read from that path.'
} }
}; };
export const getSkillToolInfo = (
id: string,
lang: localeType = 'en'
): { name: string; avatar: string; toolDescription: string } | undefined => {
const toolInfo = skillToolsMap[id];
if (toolInfo) {
return {
name: parseI18nString(toolInfo.name, lang),
avatar: toolInfo.avatar,
toolDescription: toolInfo.toolDescription
};
}
};
// Zod parameter schemas (runtime validation) // Zod parameter schemas (runtime validation)
export const SandboxReadFileSchema = z.object({ export const SandboxReadFileSchema = z.object({
......
packages/service/common/logger/categories.ts
...@@ -76,6 +76,7 @@ export const LogCategories = { ...@@ -76,6 +76,7 @@ export const LogCategories = {
}), }),
AI: Object.assign(['ai'], { AI: Object.assign(['ai'], {
AGENT: ['ai', 'agent'], AGENT: ['ai', 'agent'],
TOOL_CALL: ['ai', 'tool-call'],
HELPERBOT: ['ai', 'helperbot'], HELPERBOT: ['ai', 'helperbot'],
CONFIG: ['ai', 'config'], CONFIG: ['ai', 'config'],
EMBEDDING: ['ai', 'embedding'], EMBEDDING: ['ai', 'embedding'],
......
packages/service/core/ai/llm/request.ts
...@@ -31,6 +31,7 @@ import { getErrText } from '@fastgpt/global/common/error/utils'; ...@@ -31,6 +31,7 @@ import { getErrText } from '@fastgpt/global/common/error/utils';
import json5 from 'json5'; import json5 from 'json5';
import { getLogger, LogCategories } from '../../../common/logger'; import { getLogger, LogCategories } from '../../../common/logger';
import { saveLLMRequestRecord } from '../record/controller'; import { saveLLMRequestRecord } from '../record/controller';
import type { ToolCallEventType } from './toolCall/type';
const getRequestId = () => { const getRequestId = () => {
return customNanoid('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890_-', 16); return customNanoid('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890_-', 16);
...@@ -38,11 +39,9 @@ const getRequestId = () => { ...@@ -38,11 +39,9 @@ const getRequestId = () => {
const logger = getLogger(LogCategories.MODULE.AI.LLM); const logger = getLogger(LogCategories.MODULE.AI.LLM);
export type ResponseEvents = { export type ResponseEvents = ToolCallEventType & {
onStreaming?: (e: { text: string }) => void; onStreaming?: (e: { text: string }) => void;
onReasoning?: (e: { text: string }) => void; onReasoning?: (e: { text: string }) => void;
onToolCall?: (e: { call: ChatCompletionMessageToolCall }) => void;
onToolParam?: (e: { tool: ChatCompletionMessageToolCall; params: string }) => void;
}; };
export type CreateLLMResponseProps< export type CreateLLMResponseProps<
...@@ -459,7 +458,7 @@ export const createStreamResponse = async ({ ...@@ -459,7 +458,7 @@ export const createStreamResponse = async ({
if (currentTool && arg) { if (currentTool && arg) {
currentTool.function.arguments += arg; currentTool.function.arguments += arg;
onToolParam?.({ tool: currentTool, params: arg }); onToolParam?.({ call: currentTool, argsDelta: arg });
} }
} }
}); });
......
packages/service/core/ai/llm/toolCall/type.ts 0 → 100644
import type { ChatCompletionMessageToolCall } from '@fastgpt/global/core/ai/llm/type';
export type ToolCallEventType = {
onToolCall?: (e: { call: ChatCompletionMessageToolCall }) => void;
onToolParam?: (e: { call: ChatCompletionMessageToolCall; argsDelta: string }) => void;
// 工具执行完成后的生命周期钩子（含未找到 / parseParams 失败 / execute 抛错的兜底）
onAfterToolCall?: (e: {
success: boolean;
call: ChatCompletionMessageToolCall;
response?: string;
errorMessage?: string;
}) => void;
// 工具压缩后回调
onAfterToolResponseCompress?: (e: {
call: ChatCompletionMessageToolCall;
response: string;
usage: {
inputTokens: number;
outputTokens: number;
totalPoints: number;
};
}) => void;
};
packages/service/core/ai/sandbox/toolCall.ts deleted 100644 → 0
import {
SANDBOX_TOOL_NAME,
SANDBOX_GET_FILE_URL_TOOL_NAME,
SandboxShellToolSchema,
SandboxGetFileUrlToolSchema
} from '@fastgpt/global/core/ai/sandbox/constants';
import { getErrText } from '@fastgpt/global/common/error/utils';
import { parseJsonArgs } from '../utils';
import { getSandboxClient } from './controller';
import { getS3ChatSource } from '../../../common/s3/sources/chat';
import path from 'path';
import { jwtSignS3ObjectKey } from '../../../common/s3/utils';
import { addHours } from 'date-fns';
import { Readable } from 'stream';
import { getLogger } from '@fastgpt-sdk/otel';
import { LogCategories } from '../../../common/logger';
type SandboxToolCallParams = {
toolName: string;
rawArgs: string;
appId: string;
userId: string;
chatId: string;
};
export type SandboxToolCallResult = {
input: Record<string, any>;
response: string;
durationSeconds: number;
};
/**
* 纯沙盒工具执行层。
* 只负责调用沙盒、上传 S3 等底层操作，返回统一的执行结果，不绑定任何业务响应格式。
*/
export const callSandboxTool = async ({
toolName,
rawArgs,
appId,
userId,
chatId
}: SandboxToolCallParams): Promise<SandboxToolCallResult> => {
const startTime = Date.now();
const getDuration = () => +((Date.now() - startTime) / 1000).toFixed(2);
if (toolName === SANDBOX_TOOL_NAME) {
const parsed = SandboxShellToolSchema.safeParse(parseJsonArgs(rawArgs));
if (!parsed.success) {
return { input: {}, response: parsed.error.message, durationSeconds: getDuration() };
}
const { command, timeout } = parsed.data;
try {
const instance = await getSandboxClient({ appId, userId, chatId });
const result = await instance.exec(command, timeout);
return {
input: { command, timeout },
response: JSON.stringify({
stdout: result.stdout,
stderr: result.stderr,
exitCode: result.exitCode
}),
durationSeconds: getDuration()
};
} catch (error: any) {
getLogger(LogCategories.MODULE.AI.AGENT).error('[Sandbox Shell] Execution failed', { error });
return {
input: { command, timeout },
response: getErrText(error),
durationSeconds: getDuration()
};
}
}
if (toolName === SANDBOX_GET_FILE_URL_TOOL_NAME) {
const parsed = SandboxGetFileUrlToolSchema.safeParse(parseJsonArgs(rawArgs));
if (!parsed.success) {
return { input: {}, response: parsed.error.message, durationSeconds: getDuration() };
}
const { paths } = parsed.data;
try {
const instance = await getSandboxClient({ appId, userId, chatId });
const result = await Promise.all(
paths.map(async (url) => {
const filename = path.basename(url);
const stream = instance.provider.readFileStream(url);
const readable = Readable.from(stream); // AsyncIterable<Uint8Array> → Readable
const chatBucket = getS3ChatSource();
const expiredTime = addHours(new Date(), 2);
const { key } = await chatBucket.uploadChatFile({
appId,
chatId,
uId: userId,
filename,
body: readable,
expiredTime: expiredTime
});
const fileUrl = jwtSignS3ObjectKey(key, expiredTime);
return {
fileUrl,
filename
};
})
);
return {
input: { paths },
response: JSON.stringify(result),
durationSeconds: getDuration()
};
} catch (error) {
getLogger(LogCategories.MODULE.AI.AGENT).error('[Sandbox Get File URL] failed', { error });
return {
input: { paths },
response: `Get file URL error: ${getErrText(error)}`,
durationSeconds: getDuration()
};
}
}
return {
input: {},
response: `Unknown sandbox tool: ${toolName}`,
durationSeconds: getDuration()
};
};
packages/service/core/ai/sandbox/toolCall/getFileUrl.tool.ts 0 → 100644
import z from 'zod';
import path from 'path';
import { Readable } from 'stream';
import { addHours } from 'date-fns';
import { defineTool } from './type';
import { getS3ChatSource } from '../../../../common/s3/sources/chat';
import { jwtSignS3ObjectKey } from '../../../../common/s3/utils';
import { SANDBOX_GET_FILE_URL_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const SandboxGetFileUrlToolSchema = z.object({
paths: z.array(z.string())
});
export const sandboxGetFileUrlTool = defineTool({
zodSchema: SandboxGetFileUrlToolSchema,
execute: async ({ appId, userId, chatId, sandboxInstance, params }) => {
const result = await Promise.all(
params.paths.map(async (filePath) => {
const filename = path.basename(filePath);
const stream = sandboxInstance.provider.readFileStream(filePath);
const readable = Readable.from(stream);
const chatBucket = getS3ChatSource();
const expiredTime = addHours(new Date(), 2);
const { key } = await chatBucket.uploadChatFile({
appId,
chatId,
uId: userId,
filename,
body: readable,
expiredTime
});
const fileUrl = jwtSignS3ObjectKey(key, expiredTime);
return { fileUrl, filename };
})
);
return { response: JSON.stringify(result) };
}
});
export const toolMap = {
[SANDBOX_GET_FILE_URL_TOOL_NAME]: sandboxGetFileUrlTool
};
packages/service/core/ai/sandbox/toolCall/index.ts 0 → 100644
import { sandboxToolMap } from '@fastgpt/global/core/ai/sandbox/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import type { localeType } from '@fastgpt/global/common/i18n/type';
import { LangEnum } from '@fastgpt/global/common/i18n/type';
import { toolMap as getFileUrlToolMap } from './getFileUrl.tool';
import { toolMap as shellToolMap } from './shell.tool';
import { getSandboxClient } from '../controller';
import { parseJsonArgs } from '../../utils';
const ToolMap = {
...getFileUrlToolMap,
...shellToolMap
};
export type SandboxToolCallResult = {
success: boolean;
input: Record<string, any>;
response: string;
durationSeconds: number;
};
export const runSandboxTools = async ({
appId,
userId,
chatId,
toolName,
args
}: {
appId: string;
userId: string;
chatId: string;
toolName: string;
args: string;
}): Promise<SandboxToolCallResult> => {
const startTime = Date.now();
const getDuration = () => +((Date.now() - startTime) / 1000).toFixed(2);
const tool = ToolMap[toolName as keyof typeof ToolMap];
if (!tool) {
return {
success: false,
input: {},
response: `Unknown sandbox tool: ${toolName}`,
durationSeconds: getDuration()
};
}
// Parse args
const parsedArgs = tool.zodSchema.safeParse(parseJsonArgs(args));
if (!parsedArgs.success) {
return {
success: false,
input: {},
response: parsedArgs.error.message,
durationSeconds: getDuration()
};
}
const instance = await getSandboxClient({ appId, userId, chatId });
const result = await tool.execute({
appId,
userId,
chatId,
sandboxInstance: instance,
params: parsedArgs.data as any
});
return {
success: true,
input: parsedArgs.data,
response: result.response,
durationSeconds: getDuration()
};
};
export const getSandboxToolInfo = (name: string, lang: localeType = LangEnum.en) => {
if (name in sandboxToolMap) {
const info = sandboxToolMap[name];
return {
name: parseI18nString(info.name, lang),
avatar: info.avatar,
toolDescription: info.toolDescription
};
}
};
packages/service/core/ai/sandbox/toolCall/shell.tool.ts 0 → 100644
import z from 'zod';
import { defineTool } from './type';
import { SANDBOX_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const SandboxShellToolSchema = z.object({
command: z.string(),
timeout: z.number().optional()
});
export const sandboxShellTool = defineTool({
zodSchema: SandboxShellToolSchema,
execute: async ({ sandboxInstance, params }) => {
const result = await sandboxInstance.exec(params.command, params.timeout);
return {
response: JSON.stringify({
stdout: result.stdout,
stderr: result.stderr,
exitCode: result.exitCode
})
};
}
});
export const toolMap = {
[SANDBOX_TOOL_NAME]: sandboxShellTool
};
packages/service/core/ai/sandbox/toolCall/type.ts 0 → 100644
import type { z } from 'zod';
import type { SandboxClient } from '../controller';
type ToolExecuteContext<P> = {
appId: string;
userId: string;
chatId: string;
sandboxInstance: SandboxClient;
params: P;
};
// 声明式工具定义
export type ToolDefinition<S extends z.ZodTypeAny = z.ZodTypeAny> = {
zodSchema: S;
execute: (ctx: ToolExecuteContext<z.infer<S>>) => Promise<{ response: string }>;
};
export const defineTool = <S extends z.ZodTypeAny>(def: ToolDefinition<S>): ToolDefinition<S> =>
def;
packages/service/core/chat/HelperBot/dispatch/topAgent/utils.ts
...@@ -8,6 +8,7 @@ import { MongoResourcePermission } from '../../../../../support/permission/schem ...@@ -8,6 +8,7 @@ import { MongoResourcePermission } from '../../../../../support/permission/schem
import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant'; import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant';
import { getGroupsByTmbId } from '../../../../../support/permission/memberGroup/controllers'; import { getGroupsByTmbId } from '../../../../../support/permission/memberGroup/controllers';
import { getOrgIdSetWithParentByTmbId } from '../../../../../support/permission/org/controllers'; import { getOrgIdSetWithParentByTmbId } from '../../../../../support/permission/org/controllers';
import { SANDBOX_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const getAccessibleDatasets = async ({ teamId, tmbId }: { teamId: string; tmbId: string }) => { const getAccessibleDatasets = async ({ teamId, tmbId }: { teamId: string; tmbId: string }) => {
const [roleList, myGroupMap, myOrgSet] = await Promise.all([ const [roleList, myGroupMap, myOrgSet] = await Promise.all([
...@@ -110,7 +111,7 @@ ${dataset} ...@@ -110,7 +111,7 @@ ${dataset}
}) })
]); ]);
const builtinTools = [SubAppIds.fileRead, SubAppIds.sandboxTool].map((id) => { const builtinTools = [SubAppIds.fileRead, SANDBOX_TOOL_NAME].map((id) => {
const info = systemSubInfo[id]; const info = systemSubInfo[id];
return `- **${id}** [工具]: ${parseI18nString(info.name, lang)} - ${info.toolDescription}`; return `- **${id}** [工具]: ${parseI18nString(info.name, lang)} - ${info.toolDescription}`;
}); });
......
packages/service/core/dataset/search/controller.ts
...@@ -52,7 +52,7 @@ export type SearchDatasetDataProps = { ...@@ -52,7 +52,7 @@ export type SearchDatasetDataProps = {
[NodeInputKeyEnum.datasetSimilarity]?: number; // min distance [NodeInputKeyEnum.datasetSimilarity]?: number; // min distance
[NodeInputKeyEnum.datasetMaxTokens]: number; // max Token limit [NodeInputKeyEnum.datasetMaxTokens]: number; // max Token limit
[NodeInputKeyEnum.datasetSearchMode]?: `${DatasetSearchModeEnum}`; [NodeInputKeyEnum.datasetSearchMode]?: DatasetSearchModeEnum;
[NodeInputKeyEnum.datasetSearchEmbeddingWeight]?: number; [NodeInputKeyEnum.datasetSearchEmbeddingWeight]?: number;
[NodeInputKeyEnum.datasetSearchUsingReRank]?: boolean; [NodeInputKeyEnum.datasetSearchUsingReRank]?: boolean;
......
packages/service/core/workflow/dispatch/ai/agent/index.ts
...@@ -22,8 +22,7 @@ import { ...@@ -22,8 +22,7 @@ import {
} from '@fastgpt/global/core/chat/adapt'; } from '@fastgpt/global/core/chat/adapt';
import { getPlanCallResponseText } from '@fastgpt/global/core/chat/utils'; import { getPlanCallResponseText } from '@fastgpt/global/core/chat/utils';
import { filterMemoryMessages } from '../utils'; import { filterMemoryMessages } from '../utils';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils'; import { getSystemToolInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import type { DispatchPlanAgentResponse } from './sub/plan'; import type { DispatchPlanAgentResponse } from './sub/plan';
import { dispatchPlanAgent } from './sub/plan'; import { dispatchPlanAgent } from './sub/plan';
...@@ -272,13 +271,12 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise ...@@ -272,13 +271,12 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise
}; };
} }
const systemToolNode = systemSubInfo[id] || systemSubInfo[formatId]; const systemToolNode = getSystemToolInfo(id, lang) || getSystemToolInfo(formatId, lang);
const systemDisplayName = parseI18nString(systemToolNode?.name, lang);
return { return {
name: systemDisplayName || '', name: systemToolNode?.name || '',
avatar: systemToolNode?.avatar || '', avatar: systemToolNode?.avatar || '',
toolDescription: systemToolNode?.toolDescription || systemDisplayName || '' toolDescription: systemToolNode?.toolDescription || systemToolNode?.name || ''
}; };
}; };
const getSubApp = (id: string) => { const getSubApp = (id: string) => {
......
packages/service/core/workflow/dispatch/ai/agent/piAgent/index.ts
...@@ -15,12 +15,13 @@ import { formatFileInput } from '../sub/file/utils'; ...@@ -15,12 +15,13 @@ import { formatFileInput } from '../sub/file/utils';
import { normalizeSkillIds } from '@fastgpt/global/core/app/formEdit/type'; import { normalizeSkillIds } from '@fastgpt/global/core/app/formEdit/type';
import { systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants'; import { systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils'; import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import type { ToolDispatchContext } from '../utils';
import { getSubapps } from '../utils'; import { getSubapps } from '../utils';
import { createCapabilityToolCallHandler, type AgentCapability } from '../capability/type'; import { createCapabilityToolCallHandler, type AgentCapability } from '../capability/type';
import { createSandboxSkillsCapability } from '../capability/sandboxSkills'; import { createSandboxSkillsCapability } from '../capability/sandboxSkills';
import { textAdaptGptResponse } from '@fastgpt/global/core/workflow/runtime/utils'; import { textAdaptGptResponse } from '@fastgpt/global/core/workflow/runtime/utils';
import { buildPiModel, getModelApiKey } from './modelBridge'; import { buildPiModel, getModelApiKey } from './modelBridge';
import { buildAgentTools, type ToolDispatchContext } from './toolAdapter'; import { buildAgentTools } from './toolAdapter';
import { getLogger, LogCategories } from '../../../../../../common/logger'; import { getLogger, LogCategories } from '../../../../../../common/logger';
import { env } from '../../../../../../env'; import { env } from '../../../../../../env';
import type { DispatchAgentModuleProps } from '..'; import type { DispatchAgentModuleProps } from '..';
...@@ -166,35 +167,19 @@ export const dispatchPiAgent = async (props: DispatchAgentModuleProps): Promise< ...@@ -166,35 +167,19 @@ export const dispatchPiAgent = async (props: DispatchAgentModuleProps): Promise<
const apiKey = getModelApiKey(model); const apiKey = getModelApiKey(model);
const toolCtx: ToolDispatchContext = { const toolCtx: ToolDispatchContext = {
checkIsStopping, ...props,
chatConfig, streamResponseFn: workflowStreamResponse,
runningUserInfo: props.runningUserInfo, getSubAppInfo,
runningAppInfo, getSubApp,
chatId, completionTools: agentCompletionTools,
uid: props.uid, filesMap,
variables: props.variables, capabilityToolCallHandler
externalProvider: props.externalProvider,
workflowStreamResponse,
lang,
requestOrigin,
mode,
timezone: props.timezone,
retainDatasetCite: props.retainDatasetCite,
maxRunTimes: props.maxRunTimes,
workflowDispatchDeep: props.workflowDispatchDeep,
usagePush,
model,
datasetParams
}; };
const piTools = await buildAgentTools({ const piTools = await buildAgentTools({
completionTools: agentCompletionTools,
ctx: toolCtx, ctx: toolCtx,
filesMap, nodeResponses,
getSubApp, usagePush
getSubAppInfo,
capabilityToolCallHandler,
nodeResponses
}); });
/* ===== Restore session messages from last AI history ===== */ /* ===== Restore session messages from last AI history ===== */
......
packages/service/core/workflow/dispatch/ai/agent/sub/app/index.ts
...@@ -37,7 +37,11 @@ type Props = Pick< ...@@ -37,7 +37,11 @@ type Props = Pick<
| 'responseDetail' | 'responseDetail'
| 'variables' | 'variables'
> & { > & {
appId: string; app: {
name: string;
avatar?: string;
id: string;
};
userChatInput: string; userChatInput: string;
customAppVariables: Record<string, any>; customAppVariables: Record<string, any>;
}; };
...@@ -46,25 +50,21 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse> ...@@ -46,25 +50,21 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse>
const { const {
runningAppInfo, runningAppInfo,
runningUserInfo, runningUserInfo,
appId, app,
variables, variables,
customAppVariables, customAppVariables,
userChatInput, userChatInput,
...data ...data
} = props; } = props;
if (!appId) {
return Promise.reject(new Error('AppId is empty'));
}
// Auth the app by tmbId(Not the user, but the workflow user) // Auth the app by tmbId(Not the user, but the workflow user)
const { app: appData } = await authAppByTmbId({ const { app: appData } = await authAppByTmbId({
appId, appId: app.id,
tmbId: runningAppInfo.tmbId, tmbId: runningAppInfo.tmbId,
per: ReadPermissionVal per: ReadPermissionVal
}); });
const { nodes, edges, chatConfig } = await getAppVersionById({ const { nodes, edges, chatConfig } = await getAppVersionById({
appId, appId: app.id,
app: appData app: appData
}); });
...@@ -86,7 +86,7 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse> ...@@ -86,7 +86,7 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse>
); );
const runtimeEdges = storeEdges2RuntimeEdges(edges); const runtimeEdges = storeEdges2RuntimeEdges(edges);
const { assistantResponses, flowUsages, runTimes } = await runWorkflow({ const { assistantResponses, flowUsages } = await runWorkflow({
...data, ...data,
uid: variables.userId, uid: variables.userId,
chatId: variables.chatId, chatId: variables.chatId,
...@@ -119,9 +119,17 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse> ...@@ -119,9 +119,17 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse>
return { return {
response: text, response: text,
result: {}, usages: flowUsages,
runningTime: runTimes || 0, nodeResponse: {
usages: flowUsages moduleType: FlowNodeTypeEnum.appModule,
moduleName: app.name,
moduleLogo: app.avatar,
toolInput: {
userChatInput,
...customAppVariables
},
toolRes: text
}
}; };
}; };
...@@ -129,25 +137,21 @@ export const dispatchPlugin = async (props: Props): Promise<DispatchSubAppRespon ...@@ -129,25 +137,21 @@ export const dispatchPlugin = async (props: Props): Promise<DispatchSubAppRespon
const { const {
runningAppInfo, runningAppInfo,
runningUserInfo, runningUserInfo,
appId, app,
variables, variables,
customAppVariables, customAppVariables,
userChatInput, userChatInput,
...data ...data
} = props; } = props;
if (!appId) {
return Promise.reject(new Error('AppId is empty'));
}
// Auth the app by tmbId(Not the user, but the workflow user) // Auth the app by tmbId(Not the user, but the workflow user)
const { app: appData } = await authAppByTmbId({ const { app: appData } = await authAppByTmbId({
appId, appId: app.id,
tmbId: runningAppInfo.tmbId, tmbId: runningAppInfo.tmbId,
per: ReadPermissionVal per: ReadPermissionVal
}); });
const { nodes, edges, chatConfig } = await getAppVersionById({ const { nodes, edges, chatConfig } = await getAppVersionById({
appId, appId: app.id,
app: appData app: appData
}); });
...@@ -248,8 +252,13 @@ export const dispatchPlugin = async (props: Props): Promise<DispatchSubAppRespon ...@@ -248,8 +252,13 @@ export const dispatchPlugin = async (props: Props): Promise<DispatchSubAppRespon
return { return {
response, response,
result: output?.pluginOutput || {}, usages: flowUsages,
runningTime: runTimes || 0, nodeResponse: {
usages: flowUsages moduleType: FlowNodeTypeEnum.pluginModule,
moduleName: app.name,
moduleLogo: app.avatar,
toolInput: customAppVariables,
toolRes: output?.pluginOutput || {}
}
}; };
}; };
packages/service/core/workflow/dispatch/ai/agent/sub/dataset/index.ts
...@@ -7,7 +7,6 @@ import { countPromptTokens } from '../../../../../../../common/string/tiktoken/i ...@@ -7,7 +7,6 @@ import { countPromptTokens } from '../../../../../../../common/string/tiktoken/i
import { calculateCompressionThresholds } from '../../../../../../ai/llm/compress/constants'; import { calculateCompressionThresholds } from '../../../../../../ai/llm/compress/constants';
import { formatModelChars2Points } from '../../../../../../../support/wallet/usage/utils'; import { formatModelChars2Points } from '../../../../../../../support/wallet/usage/utils';
import { i18nT } from '../../../../../../../../web/i18n/utils'; import { i18nT } from '../../../../../../../../web/i18n/utils';
import type { SelectedDatasetType } from '@fastgpt/global/core/workflow/type/io';
import { DatasetSearchModeEnum } from '@fastgpt/global/core/dataset/constants'; import { DatasetSearchModeEnum } from '@fastgpt/global/core/dataset/constants';
import { MongoDataset } from '../../../../../../dataset/schema'; import { MongoDataset } from '../../../../../../dataset/schema';
import { import {
...@@ -15,29 +14,19 @@ import { ...@@ -15,29 +14,19 @@ import {
type DefaultSearchDatasetDataProps type DefaultSearchDatasetDataProps
} from '../../../../../../dataset/search/controller'; } from '../../../../../../dataset/search/controller';
import { getErrText } from '@fastgpt/global/common/error/utils'; import { getErrText } from '@fastgpt/global/common/error/utils';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { getLogger, LogCategories } from '../../../../../../../common/logger'; import { getLogger, LogCategories } from '../../../../../../../common/logger';
import type { DispatchSubAppResponse } from '../../type';
import type { AppFormEditFormType } from '@fastgpt/global/core/app/formEdit/type';
import { DatasetSearchToolSchema } from './utils';
import { parseJsonArgs } from '../../../../../../ai/utils';
const logger = getLogger(LogCategories.MODULE.AI.AGENT);
type DatasetSearchParams = { type DatasetSearchParams = {
teamId: string; teamId: string;
tmbId: string; tmbId: string;
query: string; args: string;
llmModel: string; llmModel: string;
config: { datasetParams?: AppFormEditFormType['dataset'];
datasets: SelectedDatasetType[];
similarity: number;
maxTokens: number;
searchMode: DatasetSearchModeEnum;
embeddingWeight?: number;
usingReRank: boolean;
rerankModel?: string;
rerankWeight?: number;
usingExtensionQuery: boolean;
extensionModel?: string;
extensionBg?: string;
collectionFilterMatch?: string;
};
}; };
/** /**
...@@ -157,38 +146,41 @@ ${chunkSummaries} ...@@ -157,38 +146,41 @@ ${chunkSummaries}
}; };
export const dispatchAgentDatasetSearch = async ({ export const dispatchAgentDatasetSearch = async ({
query, args,
config, datasetParams,
teamId, teamId,
tmbId, tmbId,
llmModel llmModel
}: DatasetSearchParams): Promise<{ }: DatasetSearchParams): Promise<DispatchSubAppResponse> => {
response: string; if (!datasetParams || datasetParams.datasets.length === 0) {
usages: ChatNodeUsageType[]; return {
nodeResponse?: ChatHistoryItemResType; response: 'No dataset selected'
}> => { };
const startTime = Date.now(); }
getLogger(LogCategories.MODULE.AI.AGENT).debug('[Agent Dataset Search] Starting', {
const toolParams = DatasetSearchToolSchema.safeParse(parseJsonArgs(args));
if (!toolParams.success) {
return {
response: toolParams.error.message
};
}
const query = toolParams.data.query;
logger.debug('[Agent Dataset Search] Starting', {
query, query,
config datasetParams
}); });
try { try {
const datasetIds = await Promise.resolve(config.datasets.map((item) => item.datasetId)); const datasetIds = await Promise.resolve(datasetParams.datasets.map((item) => item.datasetId));
if (datasetIds.length === 0) {
return {
response: 'No dataset selected',
usages: []
};
}
// Get vector model // Get vector model
const vectorModel = getEmbeddingModel( const vectorModel = getEmbeddingModel(
(await MongoDataset.findById(datasetIds[0], 'vectorModel').lean())?.vectorModel (await MongoDataset.findById(datasetIds[0], 'vectorModel').lean())?.vectorModel
); );
// Get Rerank Model // Get Rerank Model
const rerankModelData = getRerankModel(config.rerankModel); const rerankModelData = getRerankModel(datasetParams.rerankModel);
const searchData: DefaultSearchDatasetDataProps = { const searchData: DefaultSearchDatasetDataProps = {
histories: [], histories: [],
...@@ -196,17 +188,17 @@ export const dispatchAgentDatasetSearch = async ({ ...@@ -196,17 +188,17 @@ export const dispatchAgentDatasetSearch = async ({
reRankQuery: query, reRankQuery: query,
queries: [query], queries: [query],
model: vectorModel.model, model: vectorModel.model,
similarity: config.similarity, similarity: datasetParams.similarity ?? 0.4,
limit: config.maxTokens, limit: datasetParams.limit || 5000,
datasetIds, datasetIds,
searchMode: config.searchMode, searchMode: datasetParams.searchMode,
embeddingWeight: config.embeddingWeight, embeddingWeight: datasetParams.embeddingWeight,
usingReRank: config.usingReRank, usingReRank: datasetParams.usingReRank,
rerankModel: rerankModelData, rerankModel: rerankModelData,
rerankWeight: config.rerankWeight, rerankWeight: datasetParams.rerankWeight ?? 0.5,
datasetSearchUsingExtensionQuery: config.usingExtensionQuery, datasetSearchUsingExtensionQuery: datasetParams.datasetSearchUsingExtensionQuery ?? false,
datasetSearchExtensionModel: config.extensionModel, datasetSearchExtensionModel: datasetParams.datasetSearchExtensionModel,
datasetSearchExtensionBg: config.extensionBg datasetSearchExtensionBg: datasetParams.datasetSearchExtensionBg
}; };
const { const {
searchRes, searchRes,
...@@ -295,29 +287,24 @@ export const dispatchAgentDatasetSearch = async ({ ...@@ -295,29 +287,24 @@ export const dispatchAgentDatasetSearch = async ({
}); });
} }
} }
const totalPoints = usages.reduce((acc, item) => acc + item.totalPoints, 0);
const id = getNanoid(6); const nodeResponse: DispatchSubAppResponse['nodeResponse'] = {
const nodeResponse: ChatHistoryItemResType = {
nodeId: id,
id: id,
moduleType: FlowNodeTypeEnum.datasetSearchNode, moduleType: FlowNodeTypeEnum.datasetSearchNode,
moduleName: i18nT('chat:dataset_search'), moduleName: i18nT('chat:dataset_search'),
totalPoints,
query, query,
embeddingModel: vectorModel.name, embeddingModel: vectorModel.name,
embeddingTokens, embeddingTokens,
similarity: usingSimilarityFilter ? config.similarity : undefined, similarity: usingSimilarityFilter ? searchData.similarity : undefined,
limit: config.maxTokens, limit: searchData.limit,
searchMode: config.searchMode, searchMode: searchData.searchMode,
embeddingWeight: embeddingWeight:
config.searchMode === DatasetSearchModeEnum.mixedRecall searchData.searchMode === DatasetSearchModeEnum.mixedRecall
? config.embeddingWeight ? searchData.embeddingWeight
: undefined, : undefined,
// Rerank // Rerank
...(searchUsingReRank && { ...(searchUsingReRank && {
rerankModel: rerankModelData?.name, rerankModel: rerankModelData?.name,
rerankWeight: config.rerankWeight, rerankWeight: searchData.rerankWeight,
reRankInputTokens reRankInputTokens
}), }),
searchUsingReRank, searchUsingReRank,
...@@ -330,8 +317,7 @@ export const dispatchAgentDatasetSearch = async ({ ...@@ -330,8 +317,7 @@ export const dispatchAgentDatasetSearch = async ({
} }
: undefined, : undefined,
// Results // Results
quoteList: searchResults, quoteList: searchResults
runningTime: +((Date.now() - startTime) / 1000).toFixed(2)
}; };
return { return {
...@@ -340,10 +326,9 @@ export const dispatchAgentDatasetSearch = async ({ ...@@ -340,10 +326,9 @@ export const dispatchAgentDatasetSearch = async ({
nodeResponse nodeResponse
}; };
} catch (error) { } catch (error) {
getLogger(LogCategories.MODULE.AI.AGENT).error('[Agent Dataset Search] Failed', { error }); logger.error('[Agent Dataset Search] Failed', { error });
return { return {
response: `Failed to search dataset: ${getErrText(error)}`, response: `Failed to search dataset: ${getErrText(error)}`
usages: []
}; };
} }
}; };
packages/service/core/workflow/dispatch/ai/agent/sub/file/index.ts
...@@ -13,12 +13,11 @@ import { getLLMModel } from '../../../../../../ai/model'; ...@@ -13,12 +13,11 @@ import { getLLMModel } from '../../../../../../ai/model';
import { compressLargeContent } from '../../../../../../ai/llm/compress'; import { compressLargeContent } from '../../../../../../ai/llm/compress';
import { calculateCompressionThresholds } from '../../../../../../ai/llm/compress/constants'; import { calculateCompressionThresholds } from '../../../../../../ai/llm/compress/constants';
import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type'; import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { i18nT } from '../../../../../../../../web/i18n/utils'; import { i18nT } from '../../../../../../../../web/i18n/utils';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type';
import { getLogger, LogCategories } from '../../../../../../../common/logger'; import { getLogger, LogCategories } from '../../../../../../../common/logger';
import type { OpenaiAccountType } from '@fastgpt/global/support/user/team/type'; import type { OpenaiAccountType } from '@fastgpt/global/support/user/team/type';
import type { DispatchSubAppResponse } from '../../type';
type FileReadParams = { type FileReadParams = {
files: { index: string; url: string }[]; files: { index: string; url: string }[];
...@@ -37,12 +36,7 @@ export const dispatchFileRead = async ({ ...@@ -37,12 +36,7 @@ export const dispatchFileRead = async ({
customPdfParse, customPdfParse,
model, model,
userKey userKey
}: FileReadParams): Promise<{ }: FileReadParams): Promise<DispatchSubAppResponse> => {
response: string;
usages: ChatNodeUsageType[];
nodeResponse?: ChatHistoryItemResType;
}> => {
const startTime = Date.now();
try { try {
const usages: ChatNodeUsageType[] = []; const usages: ChatNodeUsageType[] = [];
const readFilesResult = await Promise.all( const readFilesResult = await Promise.all(
...@@ -162,12 +156,8 @@ export const dispatchFileRead = async ({ ...@@ -162,12 +156,8 @@ export const dispatchFileRead = async ({
response: responseText, response: responseText,
usages, usages,
nodeResponse: { nodeResponse: {
nodeId: getNanoid(6),
id: getNanoid(6),
moduleType: FlowNodeTypeEnum.readFiles, moduleType: FlowNodeTypeEnum.readFiles,
moduleName: i18nT('chat:read_file'), moduleName: i18nT('chat:read_file'),
totalPoints: usages.reduce((acc, item) => acc + item.totalPoints, 0),
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
compressTextAgent: result.usage compressTextAgent: result.usage
? { ? {
inputTokens: result.usage.inputTokens || 0, inputTokens: result.usage.inputTokens || 0,
......
packages/service/core/workflow/dispatch/ai/agent/sub/sandbox/index.ts
import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type'; import { SANDBOX_ICON, SANDBOX_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
import {
SANDBOX_ICON,
SANDBOX_NAME,
SANDBOX_TOOL_NAME,
SANDBOX_GET_FILE_URL_TOOL_NAME
} from '@fastgpt/global/core/ai/sandbox/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils'; import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import type { localeType } from '@fastgpt/global/common/i18n/type'; import type { localeType } from '@fastgpt/global/common/i18n/type';
import { callSandboxTool } from '../../../../../../ai/sandbox/toolCall'; import { runSandboxTools } from '../../../../../../ai/sandbox/toolCall';
import type { DispatchSubAppResponse } from '../../type';
type SandboxDispatchParams = { export const dispatchSandboxTool = async ({
appId: string; toolName,
userId: string; rawArgs,
chatId: string;
lang?: localeType;
};
type SandboxDispatchResult = {
response: string;
usages: ChatNodeUsageType[];
nodeResponse: ChatHistoryItemResType;
};
const buildNodeResponse = ({
toolId,
input,
response,
durationSeconds,
lang
}: {
toolId: string;
input: Record<string, any>;
response: string;
durationSeconds: number;
lang?: localeType;
}): ChatHistoryItemResType => {
const nodeId = getNanoid(6);
return {
nodeId,
id: nodeId,
moduleType: FlowNodeTypeEnum.tool,
moduleName: parseI18nString(SANDBOX_NAME, lang),
moduleLogo: SANDBOX_ICON,
toolId,
toolInput: input,
toolRes: response,
totalPoints: 0,
runningTime: durationSeconds
};
};
export const dispatchSandboxShell = async ({
command,
timeout,
appId,
userId,
chatId,
lang
}: SandboxDispatchParams & {
command: string;
timeout?: number;
}): Promise<SandboxDispatchResult> => {
const { input, response, durationSeconds } = await callSandboxTool({
toolName: SANDBOX_TOOL_NAME,
rawArgs: JSON.stringify({ command, timeout }),
appId,
userId,
chatId
});
return {
response,
usages: [],
nodeResponse: buildNodeResponse({
toolId: SANDBOX_TOOL_NAME,
input,
response,
durationSeconds,
lang
})
};
};
export const dispatchSandboxGetFileUrl = async ({
paths,
appId, appId,
userId, userId,
chatId, chatId,
lang lang
}: SandboxDispatchParams & { }: {
paths: string[]; toolName: string;
}): Promise<SandboxDispatchResult> => { rawArgs: string;
const { input, response, durationSeconds } = await callSandboxTool({ appId: string;
toolName: SANDBOX_GET_FILE_URL_TOOL_NAME, userId: string;
rawArgs: JSON.stringify({ paths }), chatId: string;
lang?: localeType;
}): Promise<DispatchSubAppResponse> => {
const { input, response } = await runSandboxTools({
toolName,
args: rawArgs,
appId, appId,
userId, userId,
chatId chatId
...@@ -104,14 +30,14 @@ export const dispatchSandboxGetFileUrl = async ({ ...@@ -104,14 +30,14 @@ export const dispatchSandboxGetFileUrl = async ({
return { return {
response, response,
usages: [], nodeResponse: {
nodeResponse: buildNodeResponse({ moduleType: FlowNodeTypeEnum.tool,
toolId: SANDBOX_GET_FILE_URL_TOOL_NAME, moduleName: parseI18nString(SANDBOX_NAME, lang),
input, moduleLogo: SANDBOX_ICON,
response, toolId: toolName,
durationSeconds, toolInput: input,
lang toolRes: response
}) }
}; };
}; };
......
packages/service/core/workflow/dispatch/ai/agent/sub/tool/index.ts
import type { StoreSecretValueType } from '@fastgpt/global/common/secret/type'; import type { StoreSecretValueType } from '@fastgpt/global/common/secret/type';
import { SystemToolSecretInputTypeEnum } from '@fastgpt/global/core/app/tool/systemTool/constants'; import { SystemToolSecretInputTypeEnum } from '@fastgpt/global/core/app/tool/systemTool/constants';
import type { DispatchSubAppResponse } from '../../type'; import type { DispatchSubAppResponse } from '../../type';
import { splitCombineToolId } from '@fastgpt/global/core/app/tool/utils';
import { getSystemToolById } from '../../../../../../app/tool/controller'; import { getSystemToolById } from '../../../../../../app/tool/controller';
import { getSecretValue } from '../../../../../../../common/secret/utils'; import { getSecretValue } from '../../../../../../../common/secret/utils';
import { MongoSystemTool } from '../../../../../../plugin/tool/systemToolSchema'; import { MongoSystemTool } from '../../../../../../plugin/tool/systemToolSchema';
...@@ -21,6 +20,9 @@ import { MCPClient } from '../../../../../../app/mcp'; ...@@ -21,6 +20,9 @@ import { MCPClient } from '../../../../../../app/mcp';
import { runHTTPTool } from '../../../../../../app/http'; import { runHTTPTool } from '../../../../../../app/http';
import { getS3ChatSource } from '../../../../../../../common/s3/sources/chat'; import { getS3ChatSource } from '../../../../../../../common/s3/sources/chat';
import { parseToolId } from '../../../../child/runTool'; import { parseToolId } from '../../../../child/runTool';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
type SystemInputConfigType = { type SystemInputConfigType = {
type: SystemToolSecretInputTypeEnum; type: SystemToolSecretInputTypeEnum;
...@@ -29,6 +31,7 @@ type SystemInputConfigType = { ...@@ -29,6 +31,7 @@ type SystemInputConfigType = {
export type Props = { export type Props = {
tool: { tool: {
name: string; name: string;
avatar?: string;
version?: string; version?: string;
toolConfig: RuntimeNodeItemType['toolConfig']; toolConfig: RuntimeNodeItemType['toolConfig'];
}; };
...@@ -45,7 +48,7 @@ export type Props = { ...@@ -45,7 +48,7 @@ export type Props = {
}; };
export const dispatchTool = async ({ export const dispatchTool = async ({
tool: { name, version, toolConfig }, tool: { name, avatar, version, toolConfig },
params: { system_input_config, ...params }, params: { system_input_config, ...params },
runningUserInfo, runningUserInfo,
runningAppInfo, runningAppInfo,
...@@ -53,19 +56,29 @@ export const dispatchTool = async ({ ...@@ -53,19 +56,29 @@ export const dispatchTool = async ({
uid, uid,
variables, variables,
workflowStreamResponse workflowStreamResponse
}: Props): Promise< }: Props): Promise<DispatchSubAppResponse> => {
DispatchSubAppResponse & { const getNodeResponse = ({
toolParams: Record<string, any>; result,
} response
> => { }: RequireOnlyOne<{
const startTime = Date.now(); result?: any;
response?: string;
const getErrResponse = (error: any) => { }>): DispatchSubAppResponse['nodeResponse'] => {
return {
moduleType: FlowNodeTypeEnum.tool,
moduleName: name,
moduleLogo: avatar,
toolInput: params,
toolRes: result || response
};
};
const getErrResponse = (error: any): DispatchSubAppResponse => {
const response = getErrText(error, 'Call tool error');
return { return {
toolParams: params, response,
runningTime: +((Date.now() - startTime) / 1000).toFixed(2), nodeResponse: getNodeResponse({
response: getErrText(error, 'Call tool error'), response
usages: [] })
}; };
}; };
...@@ -165,9 +178,9 @@ export const dispatchTool = async ({ ...@@ -165,9 +178,9 @@ export const dispatchTool = async ({
return { return {
response: JSON.stringify(result), response: JSON.stringify(result),
toolParams: params, nodeResponse: getNodeResponse({
result, result
runningTime: +((Date.now() - startTime) / 1000).toFixed(2), }),
usages: [ usages: [
{ {
moduleName: name, moduleName: name,
...@@ -196,11 +209,10 @@ export const dispatchTool = async ({ ...@@ -196,11 +209,10 @@ export const dispatchTool = async ({
params params
}); });
return { return {
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
response: JSON.stringify(result), response: JSON.stringify(result),
toolParams: params, nodeResponse: getNodeResponse({
result, result: result
usages: [] })
}; };
} else if (toolConfig?.httpTool?.toolId) { } else if (toolConfig?.httpTool?.toolId) {
const { parentId, toolName } = parseToolId(toolConfig.httpTool.toolId); const { parentId, toolName } = parseToolId(toolConfig.httpTool.toolId);
...@@ -242,19 +254,18 @@ export const dispatchTool = async ({ ...@@ -242,19 +254,18 @@ export const dispatchTool = async ({
if (errorMsg) { if (errorMsg) {
return { return {
toolParams: params, nodeResponse: getNodeResponse({
runningTime: +((Date.now() - startTime) / 1000).toFixed(2), response: errorMsg
response: errorMsg, }),
usages: [] response: errorMsg
}; };
} }
return { return {
toolParams: params, nodeResponse: getNodeResponse({
runningTime: +((Date.now() - startTime) / 1000).toFixed(2), result: data
response: typeof data === 'object' ? JSON.stringify(data) : data, }),
result: data, response: typeof data === 'object' ? JSON.stringify(data) : data
usages: []
}; };
} else { } else {
return getErrResponse("Can't find the tool"); return getErrResponse("Can't find the tool");
......
packages/service/core/workflow/dispatch/ai/agent/type.ts
...@@ -3,6 +3,7 @@ import type { JSONSchemaInputType } from '@fastgpt/global/core/app/jsonschema'; ...@@ -3,6 +3,7 @@ import type { JSONSchemaInputType } from '@fastgpt/global/core/app/jsonschema';
import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type'; import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import z from 'zod'; import z from 'zod';
import { NodeToolConfigTypeSchema } from '@fastgpt/global/core/workflow/type/node'; import { NodeToolConfigTypeSchema } from '@fastgpt/global/core/workflow/type/node';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type';
export type ToolNodeItemType = RuntimeNodeItemType & { export type ToolNodeItemType = RuntimeNodeItemType & {
toolParams: RuntimeNodeItemType['inputs']; toolParams: RuntimeNodeItemType['inputs'];
...@@ -10,10 +11,9 @@ export type ToolNodeItemType = RuntimeNodeItemType & { ...@@ -10,10 +11,9 @@ export type ToolNodeItemType = RuntimeNodeItemType & {
}; };
export type DispatchSubAppResponse = { export type DispatchSubAppResponse = {
response: string; response: string; // 返回给 LLM 的响应
result?: any;
runningTime: number;
usages?: ChatNodeUsageType[]; usages?: ChatNodeUsageType[];
nodeResponse?: Omit<ChatHistoryItemResType, 'runningTime' | 'totalPoints' | 'id' | 'nodeId'>; // 部分字段外层会自动根据 usages 计算。
}; };
export const SubAppRuntimeSchema = z.object({ export const SubAppRuntimeSchema = z.object({
......
packages/service/core/workflow/dispatch/ai/tool/toolCall.ts
...@@ -14,20 +14,12 @@ import { parseJsonArgs } from '../../../../ai/utils'; ...@@ -14,20 +14,12 @@ import { parseJsonArgs } from '../../../../ai/utils';
import { sliceStrStartEnd } from '@fastgpt/global/common/string/tools'; import { sliceStrStartEnd } from '@fastgpt/global/common/string/tools';
import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants'; import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
import { toolValueTypeList, valueTypeJsonSchemaMap } from '@fastgpt/global/core/workflow/constants'; import { toolValueTypeList, valueTypeJsonSchemaMap } from '@fastgpt/global/core/workflow/constants';
import { runAgentCall } from '../../../../ai/llm/agentCall'; import { runAgentLoop } from '../../../../ai/llm/agentLoop';
import type { ToolCallChildrenInteractive } from '@fastgpt/global/core/workflow/template/system/interactive/type'; import type { ToolCallChildrenInteractive } from '@fastgpt/global/core/workflow/template/system/interactive/type';
import type { JsonSchemaPropertiesItemType } from '@fastgpt/global/core/app/jsonschema'; import type { JsonSchemaPropertiesItemType } from '@fastgpt/global/core/app/jsonschema';
import { import { SANDBOX_SYSTEM_PROMPT, SANDBOX_TOOLS } from '@fastgpt/global/core/ai/sandbox/constants';
SANDBOX_SYSTEM_PROMPT,
SANDBOX_ICON,
SANDBOX_TOOL_NAME,
SANDBOX_GET_FILE_URL_TOOL_NAME,
SANDBOX_TOOLS
} from '@fastgpt/global/core/ai/sandbox/constants';
import { getSandboxToolWorkflowResponse } from './constants'; import { getSandboxToolWorkflowResponse } from './constants';
import { callSandboxTool } from '../../../../ai/sandbox/toolCall'; import { getSandboxToolInfo, runSandboxTools } from '../../../../ai/sandbox/toolCall';
import { systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
type ResponseType = { type ResponseType = {
requestIds: string[]; requestIds: string[];
...@@ -75,6 +67,11 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -75,6 +67,11 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
} }
} = workflowProps; } = workflowProps;
// 注入 sandbox_shell 工具和提示词
let finalMessages = messages;
// 工具响应原始值
const toolRunResponses: ChildResponseItemType[] = [];
// 构建 tools 参数 // 构建 tools 参数
const toolNodesMap = new Map<string, ToolNodeItemType>(); const toolNodesMap = new Map<string, ToolNodeItemType>();
const tools: ChatCompletionTool[] = toolNodes.map((item) => { const tools: ChatCompletionTool[] = toolNodes.map((item) => {
...@@ -117,8 +114,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -117,8 +114,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
}; };
}); });
// 注入 sandbox_shell 工具和提示词 // 注入 sandbox 提示
let finalMessages = messages;
if (useAgentSandbox && global.feConfigs?.show_agent_sandbox) { if (useAgentSandbox && global.feConfigs?.show_agent_sandbox) {
// 注入 sandbox_shell 工具 // 注入 sandbox_shell 工具
tools.push(...SANDBOX_TOOLS); tools.push(...SANDBOX_TOOLS);
...@@ -135,25 +131,26 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -135,25 +131,26 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
} }
const getToolInfo = (name: string) => { const getToolInfo = (name: string) => {
const systemTool = systemSubInfo[name]; const sandboxToolInfo = getSandboxToolInfo(name, workflowProps.lang);
if (systemTool) { if (sandboxToolInfo) {
return { return {
name: parseI18nString(systemTool.name, workflowProps.lang), type: 'sandbox' as const,
avatar: systemTool.avatar name: sandboxToolInfo.name,
avatar: sandboxToolInfo.avatar
}; };
} }
const toolNode = toolNodesMap.get(name); const toolNode = toolNodesMap.get(name);
return { if (toolNode) {
name: toolNode?.name || '', return {
avatar: toolNode?.avatar || '', type: 'user' as const,
rawData: toolNode name: toolNode.name,
}; avatar: toolNode.avatar,
rawData: toolNode
};
}
}; };
// 工具响应原始值
const toolRunResponses: ChildResponseItemType[] = [];
const { const {
inputTokens, inputTokens,
outputTokens, outputTokens,
...@@ -164,7 +161,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -164,7 +161,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
finish_reason, finish_reason,
error, error,
requestIds requestIds
} = await runAgentCall({ } = await runAgentLoop({
maxRunAgentTimes: 50, maxRunAgentTimes: 50,
body: { body: {
messages: finalMessages, messages: finalMessages,
...@@ -224,23 +221,32 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -224,23 +221,32 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
}); });
} }
}, },
onToolParam({ tool, params }) { onToolParam({ call, argsDelta }) {
if (!isResponseAnswerText) return; if (!isResponseAnswerText) return;
workflowStreamResponse?.({ workflowStreamResponse?.({
id: tool.id, id: call.id,
event: SseResponseEventEnum.toolParams, event: SseResponseEventEnum.toolParams,
data: { data: {
tool: { tool: {
id: tool.id, id: call.id,
toolName: '', toolName: '',
toolAvatar: '', toolAvatar: '',
params params: argsDelta
} }
} }
}); });
}, },
handleToolResponse: async ({ call, messages }) => { onRunTool: async ({ call }) => {
const tool = getToolInfo(call.function?.name); const toolInfo = getToolInfo(call.function?.name);
if (!toolInfo) {
return {
response: 'Call tool not found',
assistantMessages: [],
usages: [],
interactive: undefined,
stop: false
};
}
const { const {
response, response,
...@@ -251,21 +257,18 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -251,21 +257,18 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
stop stop
} = await (async () => { } = await (async () => {
// 拦截 sandbox 工具调用 // 拦截 sandbox 工具调用
if ( if (toolInfo.type === 'sandbox') {
call.function?.name === SANDBOX_TOOL_NAME || const { input, response, durationSeconds } = await runSandboxTools({
call.function?.name === SANDBOX_GET_FILE_URL_TOOL_NAME
) {
const { input, response, durationSeconds } = await callSandboxTool({
toolName: call.function.name, toolName: call.function.name,
rawArgs: call.function.arguments ?? '', args: call.function.arguments ?? '',
appId: String(workflowProps.runningAppInfo.id), appId: workflowProps.runningAppInfo.id,
userId: String(workflowProps.uid), userId: workflowProps.uid,
chatId: workflowProps.chatId chatId: workflowProps.chatId
}); });
const flowResponse = getSandboxToolWorkflowResponse({ const flowResponse = getSandboxToolWorkflowResponse({
name: tool.name, name: toolInfo.name,
logo: SANDBOX_ICON, logo: toolInfo.avatar,
toolId: call.function.name, toolId: call.function.name,
input, input,
response, response,
...@@ -274,13 +277,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -274,13 +277,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
return { response, flowResponse }; return { response, flowResponse };
} else { } else {
const toolNode = tool?.rawData; const toolNode = toolInfo.rawData;
if (!toolNode) {
return {
response: 'Call tool not found'
};
}
// Init tool params and run // Init tool params and run
const startParams = parseJsonArgs(call.function.arguments); const startParams = parseJsonArgs(call.function.arguments);
...@@ -315,24 +312,26 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -315,24 +312,26 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
} }
})(); })();
if (isResponseAnswerText) { // 推送存储数据，与 tool 逻辑无关
workflowStreamResponse?.({ {
id: call.id, if (isResponseAnswerText) {
event: SseResponseEventEnum.toolResponse, workflowStreamResponse?.({
data: { id: call.id,
tool: { event: SseResponseEventEnum.toolResponse,
id: call.id, data: {
toolName: '', tool: {
toolAvatar: '', id: call.id,
params: '', toolName: '',
response: sliceStrStartEnd(response, 5000, 5000) toolAvatar: '',
params: '',
response: sliceStrStartEnd(response, 5000, 5000)
}
} }
} });
}); }
} if (flowResponse) {
toolRunResponses.push(flowResponse);
if (flowResponse) { }
toolRunResponses.push(flowResponse);
} }
return { return {
...@@ -343,7 +342,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo ...@@ -343,7 +342,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
stop stop
}; };
}, },
handleInteractiveTool: async ({ childrenResponse, toolParams }) => { onRunInteractiveTool: async ({ childrenResponse, toolParams }) => {
initToolNodes(runtimeNodes, childrenResponse.entryNodeIds); initToolNodes(runtimeNodes, childrenResponse.entryNodeIds);
initToolCallEdges(runtimeEdges, childrenResponse.entryNodeIds); initToolCallEdges(runtimeEdges, childrenResponse.entryNodeIds);
......
projects/app/package.json
{ {
"name": "app", "name": "app",
"version": "4.14.11", "version": "4.14.13",
"private": false, "private": false,
"scripts": { "scripts": {
"dev": "NODE_OPTIONS='--max-old-space-size=8192' npm run build:workers && next dev", "dev": "NODE_OPTIONS='--max-old-space-size=8192' npm run build:workers && next dev",
......
projects/app/src/pageComponents/app/detail/Edit/ChatAgent/hooks/useSkillManager.tsx
...@@ -31,6 +31,7 @@ import { ...@@ -31,6 +31,7 @@ import {
import { useLatest } from 'ahooks'; import { useLatest } from 'ahooks';
import { SubAppIds, systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants'; import { SubAppIds, systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils'; import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import { SANDBOX_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const ConfigToolModal = dynamic(() => import('../../component/ConfigToolModal')); const ConfigToolModal = dynamic(() => import('../../component/ConfigToolModal'));
...@@ -111,10 +112,10 @@ export const useSkillManager = ({ ...@@ -111,10 +112,10 @@ export const useSkillManager = ({
}); });
} }
const sandboxToolInfo = systemSubInfo[SubAppIds.sandboxTool]; const sandboxToolInfo = systemSubInfo[SANDBOX_TOOL_NAME];
if (sandboxToolInfo) { if (sandboxToolInfo) {
apiTools.unshift({ apiTools.unshift({
id: SubAppIds.sandboxTool, id: SANDBOX_TOOL_NAME,
label: parseI18nString(sandboxToolInfo.name, i18n.language), label: parseI18nString(sandboxToolInfo.name, i18n.language),
icon: sandboxToolInfo.avatar, icon: sandboxToolInfo.avatar,
description: sandboxToolInfo.toolDescription, description: sandboxToolInfo.toolDescription,
...@@ -338,11 +339,11 @@ export const useSkillManager = ({ ...@@ -338,11 +339,11 @@ export const useSkillManager = ({
} }
// Merge sandbox tool // Merge sandbox tool
const sandboxToolInfo = systemSubInfo[SubAppIds.sandboxTool]; const sandboxToolInfo = systemSubInfo[SANDBOX_TOOL_NAME];
if (sandboxToolInfo) { if (sandboxToolInfo) {
tools.push({ tools.push({
id: SubAppIds.sandboxTool, id: SANDBOX_TOOL_NAME,
pluginId: SubAppIds.sandboxTool, pluginId: SANDBOX_TOOL_NAME,
name: parseI18nString(sandboxToolInfo.name, i18n.language), name: parseI18nString(sandboxToolInfo.name, i18n.language),
avatar: sandboxToolInfo.avatar, avatar: sandboxToolInfo.avatar,
intro: sandboxToolInfo.toolDescription, intro: sandboxToolInfo.toolDescription,
......
test/cases/service/core/ai/llm/request.test.ts
...@@ -747,8 +747,8 @@ describe('createLLMResponse', () => { ...@@ -747,8 +747,8 @@ describe('createLLMResponse', () => {
onToolCall: ({ call }) => { onToolCall: ({ call }) => {
toolCallResults.push(call); toolCallResults.push(call);
}, },
onToolParam: ({ params }) => { onToolParam: ({ argsDelta }) => {
toolParamResults.push(params); toolParamResults.push(argsDelta);
} }
}); });
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment