Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit b2cfada9 by Archer Committed by GitHub
Options

perf: agent tool code (#6798) 

* perf: agent tool code

* fix: review
parent 2fe03ec0
Showing with 554 additions and 525 deletions
.claude/design/core/ai/gradient-pricing-fix.md
......@@ -65,12 +65,12 @@ const { totalPoints: modelTotalPoints } = formatModelChars2Points({
});
```
`runToolCall` 调用 `runAgentCall` 时**不传 `usagePush`**，所以单次计价全部丢失，只依赖这里的累加计算 → **实际计费错误**。
`runToolCall` 调用 `runAgentLoop` 时**不传 `usagePush`**，所以单次计价全部丢失，只依赖这里的累加计算 → **实际计费错误**。
### 3. `packages/service/core/workflow/dispatch/ai/agent/master/call.ts` (masterCall) — **展示 BUG**
```ts
// inputTokens = runAgentCall 返回的累加值
// inputTokens = runAgentLoop 返回的累加值
const llmUsage = formatModelChars2Points({
inputTokens, // ❌ 累加值
outputTokens
......@@ -101,15 +101,15 @@ const { totalPoints } = formatModelChars2Points({
**不应用累加的 token 数计算价格，而应该每次 LLM 调用单独计价，再累加价格。**
### 方案：`runAgentCall` 返回预计算的 `llmTotalPoints`
### 方案：`runAgentLoop` 返回预计算的 `llmTotalPoints`
在 `runAgentCall` 的 while 循环中，每次 LLM 调用后立即计算该次的价格，并累加到 `llmTotalPoints`，最终将其作为返回值之一。调用方直接使用该预计算值，而不再重复调用 `formatModelChars2Points(累加 tokens)`。
在 `runAgentLoop` 的 while 循环中，每次 LLM 调用后立即计算该次的价格，并累加到 `llmTotalPoints`，最终将其作为返回值之一。调用方直接使用该预计算值，而不再重复调用 `formatModelChars2Points(累加 tokens)`。
---
## 具体修改
### 修改 1：`runAgentCall` — 增加 `llmTotalPoints` 返回值
### 修改 1：`runAgentLoop` — 增加 `llmTotalPoints` 返回值
**文件**：`packages/service/core/ai/llm/agentCall/index.ts`
......@@ -155,8 +155,8 @@ type ResponseType = {
toolCallOutputTokens: number; // 保留展示用
};
// runAgentCall 返回后
const { inputTokens, outputTokens, llmTotalPoints, ... } = await runAgentCall(...);
// runAgentLoop 返回后
const { inputTokens, outputTokens, llmTotalPoints, ... } = await runAgentLoop(...);
return {
...
......@@ -189,8 +189,8 @@ const modelTotalPoints = toolCallTotalPoints; // 直接使用预计算值，
**文件**：`packages/service/core/workflow/dispatch/ai/agent/master/call.ts`
```ts
// runAgentCall 返回 llmTotalPoints
const { inputTokens, outputTokens, llmTotalPoints, childrenUsages, ... } = await runAgentCall(...);
// runAgentLoop 返回 llmTotalPoints
const { inputTokens, outputTokens, llmTotalPoints, childrenUsages, ... } = await runAgentLoop(...);
// 修改前（❌）
const llmUsage = formatModelChars2Points({ model: agentModel, inputTokens, outputTokens });
......@@ -265,7 +265,7 @@ usage.outputTokens += regenResult.usage.outputTokens;
## TODO
- [ ] 修改 `runAgentCall` 返回类型，新增 `llmTotalPoints`
- [ ] 修改 `runAgentLoop` 返回类型，新增 `llmTotalPoints`
- [ ] 修改 `runToolCall` 返回类型，新增 `toolCallTotalPoints`
- [ ] 修改 `dispatchRunTools` 使用预计算值
- [ ] 修改 `masterCall` 使用预计算值（修正展示）
......
.claude/design/core/ai/sandbox/get-file-url.md
......@@ -200,7 +200,7 @@ export const dispatchSandboxGetFileUrl = async ({
#### 3.5.1 普通工作流：toolCall.ts
`handleToolResponse` 中合并 `SANDBOX_TOOL_NAME` 和 `SANDBOX_GET_FILE_URL_TOOL_NAME` 到同一拦截块：
`onRunTool` 中合并 `SANDBOX_TOOL_NAME` 和 `SANDBOX_GET_FILE_URL_TOOL_NAME` 到同一拦截块：
```typescript
if (
......
document/content/docs/self-host/upgrading/4-15/4150.mdx 0 → 100644
---
title: 'V4.15.0(进行中)'
description: 'FastGPT V4.15.0 更新说明'
---
## 🚀 新增内容
## ⚙️ 优化
## 🐛 修复
## 代码优化
1. 优化 Agent tool 声明和运行，统一所有 tool 的声明和运行方式。
\ No newline at end of file
document/content/docs/self-host/upgrading/4-15/meta.en.json 0 → 100644
{
"title": "4.15.x",
"description": "",
"pages": ["4150"]
}
document/content/docs/self-host/upgrading/4-15/meta.json 0 → 100644
{
"title": "4.15.x",
"description": "",
"pages": ["4150"]
}
document/content/docs/self-host/upgrading/meta.en.json
{
"title": "Version History",
"description": "FastGPT version history",
"pages": ["4-14", "4-13", "4-12", "outdated"]
"pages": ["4-15", "4-14", "4-13", "4-12", "outdated"]
}
document/content/docs/self-host/upgrading/meta.json
{
"title": "版本列表",
"description": "FastGPT 版本列表",
"pages": ["4-14", "4-13", "4-12", "outdated"]
"pages": ["4-15", "4-14", "4-13", "4-12", "outdated"]
}
document/content/docs/toc.mdx
......@@ -127,6 +127,7 @@ description: FastGPT 文档目录
- [/docs/self-host/upgrading/4-14/4148](/docs/self-host/upgrading/4-14/4148)
- [/docs/self-host/upgrading/4-14/41481](/docs/self-host/upgrading/4-14/41481)
- [/docs/self-host/upgrading/4-14/4149](/docs/self-host/upgrading/4-14/4149)
- [/docs/self-host/upgrading/4-15/4150](/docs/self-host/upgrading/4-15/4150)
- [/docs/self-host/upgrading/outdated/40](/docs/self-host/upgrading/outdated/40)
- [/docs/self-host/upgrading/outdated/41](/docs/self-host/upgrading/outdated/41)
- [/docs/self-host/upgrading/outdated/4100](/docs/self-host/upgrading/outdated/4100)
......
document/data/doc-last-modified.json
......@@ -224,8 +224,12 @@
"document/content/docs/self-host/upgrading/4-14/4141.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/4-14/41410.en.mdx": "2026-03-31T23:15:29+08:00",
"document/content/docs/self-host/upgrading/4-14/41410.mdx": "2026-04-18T20:47:39+08:00",
"document/content/docs/self-host/upgrading/4-14/41411.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41411.mdx": "2026-04-20T20:18:35+08:00",
"document/content/docs/self-host/upgrading/4-14/41412.mdx": "2026-04-20T20:18:35+08:00",
"document/content/docs/self-host/upgrading/4-14/41412.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41412.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41413.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/41413.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/self-host/upgrading/4-14/4142.en.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/4-14/4142.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/4-14/4143.en.mdx": "2026-03-03T17:39:47+08:00",
......@@ -386,8 +390,8 @@
"document/content/docs/self-host/upgrading/outdated/499.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/upgrade-intruction.en.mdx": "2026-03-03T17:39:47+08:00",
"document/content/docs/self-host/upgrading/upgrade-intruction.mdx": "2026-04-20T13:51:34+08:00",
"document/content/docs/toc.en.mdx": "2026-04-17T23:28:43+08:00",
"document/content/docs/toc.mdx": "2026-04-20T17:45:22+08:00",
"document/content/docs/toc.en.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/toc.mdx": "2026-04-21T23:04:26+08:00",
"document/content/docs/use-cases/app-cases/dalle3.en.mdx": "2026-02-26T22:14:30+08:00",
"document/content/docs/use-cases/app-cases/dalle3.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/use-cases/app-cases/english_essay_correction_bot.en.mdx": "2026-02-26T22:14:30+08:00",
......
packages/global/core/ai/sandbox/constants.ts
......@@ -15,10 +15,10 @@ export const SANDBOX_SUSPEND_MINUTES = 5;
// ---- sandboxId 生成 ----
export const generateSandboxId = (appId: string, userId: string, chatId: string): string => {
return hashStr(`${appId}-${userId}-${chatId}`).slice(0, 16);
return hashStr(`${String(appId)}-${String(userId)}-${String(chatId)}`).slice(0, 16);
};
// Tool
// Shell Tool
export const SANDBOX_NAME: I18nStringType = {
'zh-CN': '虚拟机',
'zh-Hant': '虛擬機',
......@@ -26,10 +26,6 @@ export const SANDBOX_NAME: I18nStringType = {
};
export const SANDBOX_ICON = 'core/app/sandbox/sandbox' as const;
export const SANDBOX_TOOL_NAME = 'sandbox_shell';
export const SandboxShellToolSchema = z.object({
command: z.string(),
timeout: z.number().optional()
});
export const SANDBOX_SHELL_TOOL: ChatCompletionTool = {
type: 'function',
function: {
......@@ -51,15 +47,13 @@ export const SANDBOX_SHELL_TOOL: ChatCompletionTool = {
}
};
// Get File URL Tool
export const SANDBOX_READ_FILE_TOOL_NAME: I18nStringType = {
'zh-CN': '虚拟机/获取文件链接',
'zh-Hant': '虛擬機/獲取文件鏈接',
en: 'Sandbox/Get File URL'
};
export const SANDBOX_GET_FILE_URL_TOOL_NAME = 'sandbox_get_file_url';
export const SandboxGetFileUrlToolSchema = z.object({
paths: z.array(z.string())
});
export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = {
type: 'function',
function: {
......@@ -82,10 +76,30 @@ export const SANDBOX_GET_FILE_URL_TOOL: ChatCompletionTool = {
}
};
export const SANDBOX_TOOLS: ChatCompletionTool[] = [SANDBOX_SHELL_TOOL, SANDBOX_GET_FILE_URL_TOOL];
// Prompt
export const SANDBOX_SYSTEM_PROMPT = `你拥有一个独立的 Linux 沙盒环境（Ubuntu 22.04），可通过 ${SANDBOX_TOOL_NAME} 工具执行命令：
- 预装：bash / python3 / node / bun / git / curl
- 可自行安装软件包（apt / pip / npm）
- 生成的文件内容都保存在当前目录下即可
- 若需要将生成的文件分享给用户，可使用 ${SANDBOX_GET_FILE_URL_TOOL_NAME} 工具获取文件的临时访问链接`;
// 聚合
export const sandboxToolMap: Record<
string,
{ schema: ChatCompletionTool; name: I18nStringType; avatar: string; toolDescription: string }
> = {
[SANDBOX_TOOL_NAME]: {
schema: SANDBOX_SHELL_TOOL,
name: SANDBOX_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_SHELL_TOOL.function.description!
},
[SANDBOX_GET_FILE_URL_TOOL_NAME]: {
schema: SANDBOX_GET_FILE_URL_TOOL,
name: SANDBOX_READ_FILE_TOOL_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_GET_FILE_URL_TOOL.function.description!
}
};
export const SANDBOX_TOOLS = Object.values(sandboxToolMap).map((item) => item.schema);
packages/global/core/chat/adapt.ts
......@@ -217,7 +217,7 @@ export const GPTMessages2Chats = ({
messages: ChatCompletionMessageParam[];
reserveTool?: boolean;
reserveReason?: boolean;
getToolInfo?: (name: string) => { name: string; avatar: string };
getToolInfo?: (name: string) => { name: string; avatar?: string } | undefined;
}): ChatItemMiniType[] => {
const chatMessages = messages
.map((item) => {
......
packages/global/core/workflow/node/agent/constants.ts
import {
SANDBOX_GET_FILE_URL_TOOL,
SANDBOX_ICON,
SANDBOX_NAME,
SANDBOX_READ_FILE_TOOL_NAME,
SANDBOX_SHELL_TOOL
} from '../../../ai/sandbox/constants';
import type { I18nStringType } from '../../../../common/i18n/type';
import type { I18nStringType, localeType } from '../../../../common/i18n/type';
import { sandboxToolMap } from '../../../ai/sandbox/constants';
import { skillToolsMap } from './skillTools';
import { parseI18nString } from '../../../../common/i18n/utils';
export enum SubAppIds {
plan = 'plan_agent',
ask = 'ask_agent',
model = 'model_agent',
fileRead = 'file_read',
datasetSearch = 'dataset_search',
sandboxTool = 'sandbox_shell',
sandboxGetFileUrl = 'sandbox_get_file_url'
datasetSearch = 'dataset_search'
}
export const systemSubInfo: Record<
string,
{ name: I18nStringType; avatar: string; toolDescription: string }
> = {
[SubAppIds.sandboxTool]: {
name: SANDBOX_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_SHELL_TOOL.function.description!
},
[SubAppIds.sandboxGetFileUrl]: {
name: SANDBOX_READ_FILE_TOOL_NAME,
avatar: SANDBOX_ICON,
toolDescription: SANDBOX_GET_FILE_URL_TOOL.function.description!
},
[SubAppIds.plan]: {
name: {
'zh-CN': '规划Agent',
......@@ -78,5 +61,16 @@ export const systemSubInfo: Record<
avatar: 'core/workflow/template/agent',
toolDescription: '调用 LLM 模型完成一些通用任务。'
},
...sandboxToolMap,
...skillToolsMap
};
export const getSystemToolInfo = (id: string, lang: localeType = 'en') => {
if (id in systemSubInfo) {
const info = systemSubInfo[id];
return {
name: parseI18nString(info.name, lang),
avatar: info.avatar,
toolDescription: info.toolDescription
};
}
};
packages/global/core/workflow/node/agent/skillTools.ts
import z from 'zod';
import type { ChatCompletionTool } from '../../../ai/llm/type';
import type { I18nStringType, localeType } from '../../../../common/i18n/type';
import { parseI18nString } from '../../../../common/i18n/utils';
export enum SandboxToolIds {
readFile = 'sandbox_read_file',
......@@ -10,7 +12,10 @@ export enum SandboxToolIds {
fetchUserFile = 'sandbox_fetch_user_file'
}
export const skillToolsMap = {
export const skillToolsMap: Record<
string,
{ name: I18nStringType; avatar: string; toolDescription: string }
> = {
// Sandbox tools
[SandboxToolIds.readFile]: {
name: {
......@@ -73,6 +78,19 @@ export const skillToolsMap = {
'Download a user-uploaded file (document or image) from the conversation and write it as a binary file into the sandbox filesystem. Use this when a skill script needs to process a raw file. Workflow: call this tool first to place the file at target_path (relative to workspace), then run skill scripts that read from that path.'
}
};
export const getSkillToolInfo = (
id: string,
lang: localeType = 'en'
): { name: string; avatar: string; toolDescription: string } | undefined => {
const toolInfo = skillToolsMap[id];
if (toolInfo) {
return {
name: parseI18nString(toolInfo.name, lang),
avatar: toolInfo.avatar,
toolDescription: toolInfo.toolDescription
};
}
};
// Zod parameter schemas (runtime validation)
export const SandboxReadFileSchema = z.object({
......
packages/service/common/logger/categories.ts
......@@ -76,6 +76,7 @@ export const LogCategories = {
}),
AI: Object.assign(['ai'], {
AGENT: ['ai', 'agent'],
TOOL_CALL: ['ai', 'tool-call'],
HELPERBOT: ['ai', 'helperbot'],
CONFIG: ['ai', 'config'],
EMBEDDING: ['ai', 'embedding'],
......
packages/service/core/ai/llm/request.ts
......@@ -31,6 +31,7 @@ import { getErrText } from '@fastgpt/global/common/error/utils';
import json5 from 'json5';
import { getLogger, LogCategories } from '../../../common/logger';
import { saveLLMRequestRecord } from '../record/controller';
import type { ToolCallEventType } from './toolCall/type';
const getRequestId = () => {
return customNanoid('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890_-', 16);
......@@ -38,11 +39,9 @@ const getRequestId = () => {
const logger = getLogger(LogCategories.MODULE.AI.LLM);
export type ResponseEvents = {
export type ResponseEvents = ToolCallEventType & {
onStreaming?: (e: { text: string }) => void;
onReasoning?: (e: { text: string }) => void;
onToolCall?: (e: { call: ChatCompletionMessageToolCall }) => void;
onToolParam?: (e: { tool: ChatCompletionMessageToolCall; params: string }) => void;
};
export type CreateLLMResponseProps<
......@@ -459,7 +458,7 @@ export const createStreamResponse = async ({
if (currentTool && arg) {
currentTool.function.arguments += arg;
onToolParam?.({ tool: currentTool, params: arg });
onToolParam?.({ call: currentTool, argsDelta: arg });
}
}
});
......
packages/service/core/ai/llm/toolCall/type.ts 0 → 100644
import type { ChatCompletionMessageToolCall } from '@fastgpt/global/core/ai/llm/type';
export type ToolCallEventType = {
onToolCall?: (e: { call: ChatCompletionMessageToolCall }) => void;
onToolParam?: (e: { call: ChatCompletionMessageToolCall; argsDelta: string }) => void;
// 工具执行完成后的生命周期钩子（含未找到 / parseParams 失败 / execute 抛错的兜底）
onAfterToolCall?: (e: {
success: boolean;
call: ChatCompletionMessageToolCall;
response?: string;
errorMessage?: string;
}) => void;
// 工具压缩后回调
onAfterToolResponseCompress?: (e: {
call: ChatCompletionMessageToolCall;
response: string;
usage: {
inputTokens: number;
outputTokens: number;
totalPoints: number;
};
}) => void;
};
packages/service/core/ai/sandbox/toolCall.ts deleted 100644 → 0
import {
SANDBOX_TOOL_NAME,
SANDBOX_GET_FILE_URL_TOOL_NAME,
SandboxShellToolSchema,
SandboxGetFileUrlToolSchema
} from '@fastgpt/global/core/ai/sandbox/constants';
import { getErrText } from '@fastgpt/global/common/error/utils';
import { parseJsonArgs } from '../utils';
import { getSandboxClient } from './controller';
import { getS3ChatSource } from '../../../common/s3/sources/chat';
import path from 'path';
import { jwtSignS3ObjectKey } from '../../../common/s3/utils';
import { addHours } from 'date-fns';
import { Readable } from 'stream';
import { getLogger } from '@fastgpt-sdk/otel';
import { LogCategories } from '../../../common/logger';
type SandboxToolCallParams = {
toolName: string;
rawArgs: string;
appId: string;
userId: string;
chatId: string;
};
export type SandboxToolCallResult = {
input: Record<string, any>;
response: string;
durationSeconds: number;
};
/**
* 纯沙盒工具执行层。
* 只负责调用沙盒、上传 S3 等底层操作，返回统一的执行结果，不绑定任何业务响应格式。
*/
export const callSandboxTool = async ({
toolName,
rawArgs,
appId,
userId,
chatId
}: SandboxToolCallParams): Promise<SandboxToolCallResult> => {
const startTime = Date.now();
const getDuration = () => +((Date.now() - startTime) / 1000).toFixed(2);
if (toolName === SANDBOX_TOOL_NAME) {
const parsed = SandboxShellToolSchema.safeParse(parseJsonArgs(rawArgs));
if (!parsed.success) {
return { input: {}, response: parsed.error.message, durationSeconds: getDuration() };
}
const { command, timeout } = parsed.data;
try {
const instance = await getSandboxClient({ appId, userId, chatId });
const result = await instance.exec(command, timeout);
return {
input: { command, timeout },
response: JSON.stringify({
stdout: result.stdout,
stderr: result.stderr,
exitCode: result.exitCode
}),
durationSeconds: getDuration()
};
} catch (error: any) {
getLogger(LogCategories.MODULE.AI.AGENT).error('[Sandbox Shell] Execution failed', { error });
return {
input: { command, timeout },
response: getErrText(error),
durationSeconds: getDuration()
};
}
}
if (toolName === SANDBOX_GET_FILE_URL_TOOL_NAME) {
const parsed = SandboxGetFileUrlToolSchema.safeParse(parseJsonArgs(rawArgs));
if (!parsed.success) {
return { input: {}, response: parsed.error.message, durationSeconds: getDuration() };
}
const { paths } = parsed.data;
try {
const instance = await getSandboxClient({ appId, userId, chatId });
const result = await Promise.all(
paths.map(async (url) => {
const filename = path.basename(url);
const stream = instance.provider.readFileStream(url);
const readable = Readable.from(stream); // AsyncIterable<Uint8Array> → Readable
const chatBucket = getS3ChatSource();
const expiredTime = addHours(new Date(), 2);
const { key } = await chatBucket.uploadChatFile({
appId,
chatId,
uId: userId,
filename,
body: readable,
expiredTime: expiredTime
});
const fileUrl = jwtSignS3ObjectKey(key, expiredTime);
return {
fileUrl,
filename
};
})
);
return {
input: { paths },
response: JSON.stringify(result),
durationSeconds: getDuration()
};
} catch (error) {
getLogger(LogCategories.MODULE.AI.AGENT).error('[Sandbox Get File URL] failed', { error });
return {
input: { paths },
response: `Get file URL error: ${getErrText(error)}`,
durationSeconds: getDuration()
};
}
}
return {
input: {},
response: `Unknown sandbox tool: ${toolName}`,
durationSeconds: getDuration()
};
};
packages/service/core/ai/sandbox/toolCall/getFileUrl.tool.ts 0 → 100644
import z from 'zod';
import path from 'path';
import { Readable } from 'stream';
import { addHours } from 'date-fns';
import { defineTool } from './type';
import { getS3ChatSource } from '../../../../common/s3/sources/chat';
import { jwtSignS3ObjectKey } from '../../../../common/s3/utils';
import { SANDBOX_GET_FILE_URL_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const SandboxGetFileUrlToolSchema = z.object({
paths: z.array(z.string())
});
export const sandboxGetFileUrlTool = defineTool({
zodSchema: SandboxGetFileUrlToolSchema,
execute: async ({ appId, userId, chatId, sandboxInstance, params }) => {
const result = await Promise.all(
params.paths.map(async (filePath) => {
const filename = path.basename(filePath);
const stream = sandboxInstance.provider.readFileStream(filePath);
const readable = Readable.from(stream);
const chatBucket = getS3ChatSource();
const expiredTime = addHours(new Date(), 2);
const { key } = await chatBucket.uploadChatFile({
appId,
chatId,
uId: userId,
filename,
body: readable,
expiredTime
});
const fileUrl = jwtSignS3ObjectKey(key, expiredTime);
return { fileUrl, filename };
})
);
return { response: JSON.stringify(result) };
}
});
export const toolMap = {
[SANDBOX_GET_FILE_URL_TOOL_NAME]: sandboxGetFileUrlTool
};
packages/service/core/ai/sandbox/toolCall/index.ts 0 → 100644
import { sandboxToolMap } from '@fastgpt/global/core/ai/sandbox/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import type { localeType } from '@fastgpt/global/common/i18n/type';
import { LangEnum } from '@fastgpt/global/common/i18n/type';
import { toolMap as getFileUrlToolMap } from './getFileUrl.tool';
import { toolMap as shellToolMap } from './shell.tool';
import { getSandboxClient } from '../controller';
import { parseJsonArgs } from '../../utils';
const ToolMap = {
...getFileUrlToolMap,
...shellToolMap
};
export type SandboxToolCallResult = {
success: boolean;
input: Record<string, any>;
response: string;
durationSeconds: number;
};
export const runSandboxTools = async ({
appId,
userId,
chatId,
toolName,
args
}: {
appId: string;
userId: string;
chatId: string;
toolName: string;
args: string;
}): Promise<SandboxToolCallResult> => {
const startTime = Date.now();
const getDuration = () => +((Date.now() - startTime) / 1000).toFixed(2);
const tool = ToolMap[toolName as keyof typeof ToolMap];
if (!tool) {
return {
success: false,
input: {},
response: `Unknown sandbox tool: ${toolName}`,
durationSeconds: getDuration()
};
}
// Parse args
const parsedArgs = tool.zodSchema.safeParse(parseJsonArgs(args));
if (!parsedArgs.success) {
return {
success: false,
input: {},
response: parsedArgs.error.message,
durationSeconds: getDuration()
};
}
const instance = await getSandboxClient({ appId, userId, chatId });
const result = await tool.execute({
appId,
userId,
chatId,
sandboxInstance: instance,
params: parsedArgs.data as any
});
return {
success: true,
input: parsedArgs.data,
response: result.response,
durationSeconds: getDuration()
};
};
export const getSandboxToolInfo = (name: string, lang: localeType = LangEnum.en) => {
if (name in sandboxToolMap) {
const info = sandboxToolMap[name];
return {
name: parseI18nString(info.name, lang),
avatar: info.avatar,
toolDescription: info.toolDescription
};
}
};
packages/service/core/ai/sandbox/toolCall/shell.tool.ts 0 → 100644
import z from 'zod';
import { defineTool } from './type';
import { SANDBOX_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const SandboxShellToolSchema = z.object({
command: z.string(),
timeout: z.number().optional()
});
export const sandboxShellTool = defineTool({
zodSchema: SandboxShellToolSchema,
execute: async ({ sandboxInstance, params }) => {
const result = await sandboxInstance.exec(params.command, params.timeout);
return {
response: JSON.stringify({
stdout: result.stdout,
stderr: result.stderr,
exitCode: result.exitCode
})
};
}
});
export const toolMap = {
[SANDBOX_TOOL_NAME]: sandboxShellTool
};
packages/service/core/ai/sandbox/toolCall/type.ts 0 → 100644
import type { z } from 'zod';
import type { SandboxClient } from '../controller';
type ToolExecuteContext<P> = {
appId: string;
userId: string;
chatId: string;
sandboxInstance: SandboxClient;
params: P;
};
// 声明式工具定义
export type ToolDefinition<S extends z.ZodTypeAny = z.ZodTypeAny> = {
zodSchema: S;
execute: (ctx: ToolExecuteContext<z.infer<S>>) => Promise<{ response: string }>;
};
export const defineTool = <S extends z.ZodTypeAny>(def: ToolDefinition<S>): ToolDefinition<S> =>
def;
packages/service/core/chat/HelperBot/dispatch/topAgent/utils.ts
......@@ -8,6 +8,7 @@ import { MongoResourcePermission } from '../../../../../support/permission/schem
import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant';
import { getGroupsByTmbId } from '../../../../../support/permission/memberGroup/controllers';
import { getOrgIdSetWithParentByTmbId } from '../../../../../support/permission/org/controllers';
import { SANDBOX_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const getAccessibleDatasets = async ({ teamId, tmbId }: { teamId: string; tmbId: string }) => {
const [roleList, myGroupMap, myOrgSet] = await Promise.all([
......@@ -110,7 +111,7 @@ ${dataset}
})
]);
const builtinTools = [SubAppIds.fileRead, SubAppIds.sandboxTool].map((id) => {
const builtinTools = [SubAppIds.fileRead, SANDBOX_TOOL_NAME].map((id) => {
const info = systemSubInfo[id];
return `- **${id}** [工具]: ${parseI18nString(info.name, lang)} - ${info.toolDescription}`;
});
......
packages/service/core/dataset/search/controller.ts
......@@ -52,7 +52,7 @@ export type SearchDatasetDataProps = {
[NodeInputKeyEnum.datasetSimilarity]?: number; // min distance
[NodeInputKeyEnum.datasetMaxTokens]: number; // max Token limit
[NodeInputKeyEnum.datasetSearchMode]?: `${DatasetSearchModeEnum}`;
[NodeInputKeyEnum.datasetSearchMode]?: DatasetSearchModeEnum;
[NodeInputKeyEnum.datasetSearchEmbeddingWeight]?: number;
[NodeInputKeyEnum.datasetSearchUsingReRank]?: boolean;
......
packages/service/core/workflow/dispatch/ai/agent/index.ts
......@@ -22,8 +22,7 @@ import {
} from '@fastgpt/global/core/chat/adapt';
import { getPlanCallResponseText } from '@fastgpt/global/core/chat/utils';
import { filterMemoryMessages } from '../utils';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import { systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { getSystemToolInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import type { DispatchPlanAgentResponse } from './sub/plan';
import { dispatchPlanAgent } from './sub/plan';
......@@ -272,13 +271,12 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise
};
}
const systemToolNode = systemSubInfo[id] || systemSubInfo[formatId];
const systemDisplayName = parseI18nString(systemToolNode?.name, lang);
const systemToolNode = getSystemToolInfo(id, lang) || getSystemToolInfo(formatId, lang);
return {
name: systemDisplayName || '',
name: systemToolNode?.name || '',
avatar: systemToolNode?.avatar || '',
toolDescription: systemToolNode?.toolDescription || systemDisplayName || ''
toolDescription: systemToolNode?.toolDescription || systemToolNode?.name || ''
};
};
const getSubApp = (id: string) => {
......
packages/service/core/workflow/dispatch/ai/agent/piAgent/index.ts
......@@ -15,12 +15,13 @@ import { formatFileInput } from '../sub/file/utils';
import { normalizeSkillIds } from '@fastgpt/global/core/app/formEdit/type';
import { systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import type { ToolDispatchContext } from '../utils';
import { getSubapps } from '../utils';
import { createCapabilityToolCallHandler, type AgentCapability } from '../capability/type';
import { createSandboxSkillsCapability } from '../capability/sandboxSkills';
import { textAdaptGptResponse } from '@fastgpt/global/core/workflow/runtime/utils';
import { buildPiModel, getModelApiKey } from './modelBridge';
import { buildAgentTools, type ToolDispatchContext } from './toolAdapter';
import { buildAgentTools } from './toolAdapter';
import { getLogger, LogCategories } from '../../../../../../common/logger';
import { env } from '../../../../../../env';
import type { DispatchAgentModuleProps } from '..';
......@@ -166,35 +167,19 @@ export const dispatchPiAgent = async (props: DispatchAgentModuleProps): Promise<
const apiKey = getModelApiKey(model);
const toolCtx: ToolDispatchContext = {
checkIsStopping,
chatConfig,
runningUserInfo: props.runningUserInfo,
runningAppInfo,
chatId,
uid: props.uid,
variables: props.variables,
externalProvider: props.externalProvider,
workflowStreamResponse,
lang,
requestOrigin,
mode,
timezone: props.timezone,
retainDatasetCite: props.retainDatasetCite,
maxRunTimes: props.maxRunTimes,
workflowDispatchDeep: props.workflowDispatchDeep,
usagePush,
model,
datasetParams
...props,
streamResponseFn: workflowStreamResponse,
getSubAppInfo,
getSubApp,
completionTools: agentCompletionTools,
filesMap,
capabilityToolCallHandler
};
const piTools = await buildAgentTools({
completionTools: agentCompletionTools,
ctx: toolCtx,
filesMap,
getSubApp,
getSubAppInfo,
capabilityToolCallHandler,
nodeResponses
nodeResponses,
usagePush
});
/* ===== Restore session messages from last AI history ===== */
......
packages/service/core/workflow/dispatch/ai/agent/sub/app/index.ts
......@@ -37,7 +37,11 @@ type Props = Pick<
| 'responseDetail'
| 'variables'
> & {
appId: string;
app: {
name: string;
avatar?: string;
id: string;
};
userChatInput: string;
customAppVariables: Record<string, any>;
};
......@@ -46,25 +50,21 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse>
const {
runningAppInfo,
runningUserInfo,
appId,
app,
variables,
customAppVariables,
userChatInput,
...data
} = props;
if (!appId) {
return Promise.reject(new Error('AppId is empty'));
}
// Auth the app by tmbId(Not the user, but the workflow user)
const { app: appData } = await authAppByTmbId({
appId,
appId: app.id,
tmbId: runningAppInfo.tmbId,
per: ReadPermissionVal
});
const { nodes, edges, chatConfig } = await getAppVersionById({
appId,
appId: app.id,
app: appData
});
......@@ -86,7 +86,7 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse>
);
const runtimeEdges = storeEdges2RuntimeEdges(edges);
const { assistantResponses, flowUsages, runTimes } = await runWorkflow({
const { assistantResponses, flowUsages } = await runWorkflow({
...data,
uid: variables.userId,
chatId: variables.chatId,
......@@ -119,9 +119,17 @@ export const dispatchApp = async (props: Props): Promise<DispatchSubAppResponse>
return {
response: text,
result: {},
runningTime: runTimes || 0,
usages: flowUsages
usages: flowUsages,
nodeResponse: {
moduleType: FlowNodeTypeEnum.appModule,
moduleName: app.name,
moduleLogo: app.avatar,
toolInput: {
userChatInput,
...customAppVariables
},
toolRes: text
}
};
};
......@@ -129,25 +137,21 @@ export const dispatchPlugin = async (props: Props): Promise<DispatchSubAppRespon
const {
runningAppInfo,
runningUserInfo,
appId,
app,
variables,
customAppVariables,
userChatInput,
...data
} = props;
if (!appId) {
return Promise.reject(new Error('AppId is empty'));
}
// Auth the app by tmbId(Not the user, but the workflow user)
const { app: appData } = await authAppByTmbId({
appId,
appId: app.id,
tmbId: runningAppInfo.tmbId,
per: ReadPermissionVal
});
const { nodes, edges, chatConfig } = await getAppVersionById({
appId,
appId: app.id,
app: appData
});
......@@ -248,8 +252,13 @@ export const dispatchPlugin = async (props: Props): Promise<DispatchSubAppRespon
return {
response,
result: output?.pluginOutput || {},
runningTime: runTimes || 0,
usages: flowUsages
usages: flowUsages,
nodeResponse: {
moduleType: FlowNodeTypeEnum.pluginModule,
moduleName: app.name,
moduleLogo: app.avatar,
toolInput: customAppVariables,
toolRes: output?.pluginOutput || {}
}
};
};
packages/service/core/workflow/dispatch/ai/agent/sub/dataset/index.ts
......@@ -7,7 +7,6 @@ import { countPromptTokens } from '../../../../../../../common/string/tiktoken/i
import { calculateCompressionThresholds } from '../../../../../../ai/llm/compress/constants';
import { formatModelChars2Points } from '../../../../../../../support/wallet/usage/utils';
import { i18nT } from '../../../../../../../../web/i18n/utils';
import type { SelectedDatasetType } from '@fastgpt/global/core/workflow/type/io';
import { DatasetSearchModeEnum } from '@fastgpt/global/core/dataset/constants';
import { MongoDataset } from '../../../../../../dataset/schema';
import {
......@@ -15,29 +14,19 @@ import {
type DefaultSearchDatasetDataProps
} from '../../../../../../dataset/search/controller';
import { getErrText } from '@fastgpt/global/common/error/utils';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { getLogger, LogCategories } from '../../../../../../../common/logger';
import type { DispatchSubAppResponse } from '../../type';
import type { AppFormEditFormType } from '@fastgpt/global/core/app/formEdit/type';
import { DatasetSearchToolSchema } from './utils';
import { parseJsonArgs } from '../../../../../../ai/utils';
const logger = getLogger(LogCategories.MODULE.AI.AGENT);
type DatasetSearchParams = {
teamId: string;
tmbId: string;
query: string;
args: string;
llmModel: string;
config: {
datasets: SelectedDatasetType[];
similarity: number;
maxTokens: number;
searchMode: DatasetSearchModeEnum;
embeddingWeight?: number;
usingReRank: boolean;
rerankModel?: string;
rerankWeight?: number;
usingExtensionQuery: boolean;
extensionModel?: string;
extensionBg?: string;
collectionFilterMatch?: string;
};
datasetParams?: AppFormEditFormType['dataset'];
};
/**
......@@ -157,38 +146,41 @@ ${chunkSummaries}
};
export const dispatchAgentDatasetSearch = async ({
query,
config,
args,
datasetParams,
teamId,
tmbId,
llmModel
}: DatasetSearchParams): Promise<{
response: string;
usages: ChatNodeUsageType[];
nodeResponse?: ChatHistoryItemResType;
}> => {
const startTime = Date.now();
getLogger(LogCategories.MODULE.AI.AGENT).debug('[Agent Dataset Search] Starting', {
}: DatasetSearchParams): Promise<DispatchSubAppResponse> => {
if (!datasetParams || datasetParams.datasets.length === 0) {
return {
response: 'No dataset selected'
};
}
const toolParams = DatasetSearchToolSchema.safeParse(parseJsonArgs(args));
if (!toolParams.success) {
return {
response: toolParams.error.message
};
}
const query = toolParams.data.query;
logger.debug('[Agent Dataset Search] Starting', {
query,
config
datasetParams
});
try {
const datasetIds = await Promise.resolve(config.datasets.map((item) => item.datasetId));
if (datasetIds.length === 0) {
return {
response: 'No dataset selected',
usages: []
};
}
const datasetIds = await Promise.resolve(datasetParams.datasets.map((item) => item.datasetId));
// Get vector model
const vectorModel = getEmbeddingModel(
(await MongoDataset.findById(datasetIds[0], 'vectorModel').lean())?.vectorModel
);
// Get Rerank Model
const rerankModelData = getRerankModel(config.rerankModel);
const rerankModelData = getRerankModel(datasetParams.rerankModel);
const searchData: DefaultSearchDatasetDataProps = {
histories: [],
......@@ -196,17 +188,17 @@ export const dispatchAgentDatasetSearch = async ({
reRankQuery: query,
queries: [query],
model: vectorModel.model,
similarity: config.similarity,
limit: config.maxTokens,
similarity: datasetParams.similarity ?? 0.4,
limit: datasetParams.limit || 5000,
datasetIds,
searchMode: config.searchMode,
embeddingWeight: config.embeddingWeight,
usingReRank: config.usingReRank,
searchMode: datasetParams.searchMode,
embeddingWeight: datasetParams.embeddingWeight,
usingReRank: datasetParams.usingReRank,
rerankModel: rerankModelData,
rerankWeight: config.rerankWeight,
datasetSearchUsingExtensionQuery: config.usingExtensionQuery,
datasetSearchExtensionModel: config.extensionModel,
datasetSearchExtensionBg: config.extensionBg
rerankWeight: datasetParams.rerankWeight ?? 0.5,
datasetSearchUsingExtensionQuery: datasetParams.datasetSearchUsingExtensionQuery ?? false,
datasetSearchExtensionModel: datasetParams.datasetSearchExtensionModel,
datasetSearchExtensionBg: datasetParams.datasetSearchExtensionBg
};
const {
searchRes,
......@@ -295,29 +287,24 @@ export const dispatchAgentDatasetSearch = async ({
});
}
}
const totalPoints = usages.reduce((acc, item) => acc + item.totalPoints, 0);
const id = getNanoid(6);
const nodeResponse: ChatHistoryItemResType = {
nodeId: id,
id: id,
const nodeResponse: DispatchSubAppResponse['nodeResponse'] = {
moduleType: FlowNodeTypeEnum.datasetSearchNode,
moduleName: i18nT('chat:dataset_search'),
totalPoints,
query,
embeddingModel: vectorModel.name,
embeddingTokens,
similarity: usingSimilarityFilter ? config.similarity : undefined,
limit: config.maxTokens,
searchMode: config.searchMode,
similarity: usingSimilarityFilter ? searchData.similarity : undefined,
limit: searchData.limit,
searchMode: searchData.searchMode,
embeddingWeight:
config.searchMode === DatasetSearchModeEnum.mixedRecall
? config.embeddingWeight
searchData.searchMode === DatasetSearchModeEnum.mixedRecall
? searchData.embeddingWeight
: undefined,
// Rerank
...(searchUsingReRank && {
rerankModel: rerankModelData?.name,
rerankWeight: config.rerankWeight,
rerankWeight: searchData.rerankWeight,
reRankInputTokens
}),
searchUsingReRank,
......@@ -330,8 +317,7 @@ export const dispatchAgentDatasetSearch = async ({
}
: undefined,
// Results
quoteList: searchResults,
runningTime: +((Date.now() - startTime) / 1000).toFixed(2)
quoteList: searchResults
};
return {
......@@ -340,10 +326,9 @@ export const dispatchAgentDatasetSearch = async ({
nodeResponse
};
} catch (error) {
getLogger(LogCategories.MODULE.AI.AGENT).error('[Agent Dataset Search] Failed', { error });
logger.error('[Agent Dataset Search] Failed', { error });
return {
response: `Failed to search dataset: ${getErrText(error)}`,
usages: []
response: `Failed to search dataset: ${getErrText(error)}`
};
}
};
packages/service/core/workflow/dispatch/ai/agent/sub/file/index.ts
......@@ -13,12 +13,11 @@ import { getLLMModel } from '../../../../../../ai/model';
import { compressLargeContent } from '../../../../../../ai/llm/compress';
import { calculateCompressionThresholds } from '../../../../../../ai/llm/compress/constants';
import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { i18nT } from '../../../../../../../../web/i18n/utils';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type';
import { getLogger, LogCategories } from '../../../../../../../common/logger';
import type { OpenaiAccountType } from '@fastgpt/global/support/user/team/type';
import type { DispatchSubAppResponse } from '../../type';
type FileReadParams = {
files: { index: string; url: string }[];
......@@ -37,12 +36,7 @@ export const dispatchFileRead = async ({
customPdfParse,
model,
userKey
}: FileReadParams): Promise<{
response: string;
usages: ChatNodeUsageType[];
nodeResponse?: ChatHistoryItemResType;
}> => {
const startTime = Date.now();
}: FileReadParams): Promise<DispatchSubAppResponse> => {
try {
const usages: ChatNodeUsageType[] = [];
const readFilesResult = await Promise.all(
......@@ -162,12 +156,8 @@ export const dispatchFileRead = async ({
response: responseText,
usages,
nodeResponse: {
nodeId: getNanoid(6),
id: getNanoid(6),
moduleType: FlowNodeTypeEnum.readFiles,
moduleName: i18nT('chat:read_file'),
totalPoints: usages.reduce((acc, item) => acc + item.totalPoints, 0),
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
compressTextAgent: result.usage
? {
inputTokens: result.usage.inputTokens || 0,
......
packages/service/core/workflow/dispatch/ai/agent/sub/sandbox/index.ts
import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type';
import {
SANDBOX_ICON,
SANDBOX_NAME,
SANDBOX_TOOL_NAME,
SANDBOX_GET_FILE_URL_TOOL_NAME
} from '@fastgpt/global/core/ai/sandbox/constants';
import { SANDBOX_ICON, SANDBOX_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import type { localeType } from '@fastgpt/global/common/i18n/type';
import { callSandboxTool } from '../../../../../../ai/sandbox/toolCall';
import { runSandboxTools } from '../../../../../../ai/sandbox/toolCall';
import type { DispatchSubAppResponse } from '../../type';
type SandboxDispatchParams = {
appId: string;
userId: string;
chatId: string;
lang?: localeType;
};
type SandboxDispatchResult = {
response: string;
usages: ChatNodeUsageType[];
nodeResponse: ChatHistoryItemResType;
};
const buildNodeResponse = ({
toolId,
input,
response,
durationSeconds,
lang
}: {
toolId: string;
input: Record<string, any>;
response: string;
durationSeconds: number;
lang?: localeType;
}): ChatHistoryItemResType => {
const nodeId = getNanoid(6);
return {
nodeId,
id: nodeId,
moduleType: FlowNodeTypeEnum.tool,
moduleName: parseI18nString(SANDBOX_NAME, lang),
moduleLogo: SANDBOX_ICON,
toolId,
toolInput: input,
toolRes: response,
totalPoints: 0,
runningTime: durationSeconds
};
};
export const dispatchSandboxShell = async ({
command,
timeout,
appId,
userId,
chatId,
lang
}: SandboxDispatchParams & {
command: string;
timeout?: number;
}): Promise<SandboxDispatchResult> => {
const { input, response, durationSeconds } = await callSandboxTool({
toolName: SANDBOX_TOOL_NAME,
rawArgs: JSON.stringify({ command, timeout }),
appId,
userId,
chatId
});
return {
response,
usages: [],
nodeResponse: buildNodeResponse({
toolId: SANDBOX_TOOL_NAME,
input,
response,
durationSeconds,
lang
})
};
};
export const dispatchSandboxGetFileUrl = async ({
paths,
export const dispatchSandboxTool = async ({
toolName,
rawArgs,
appId,
userId,
chatId,
lang
}: SandboxDispatchParams & {
paths: string[];
}): Promise<SandboxDispatchResult> => {
const { input, response, durationSeconds } = await callSandboxTool({
toolName: SANDBOX_GET_FILE_URL_TOOL_NAME,
rawArgs: JSON.stringify({ paths }),
}: {
toolName: string;
rawArgs: string;
appId: string;
userId: string;
chatId: string;
lang?: localeType;
}): Promise<DispatchSubAppResponse> => {
const { input, response } = await runSandboxTools({
toolName,
args: rawArgs,
appId,
userId,
chatId
......@@ -104,14 +30,14 @@ export const dispatchSandboxGetFileUrl = async ({
return {
response,
usages: [],
nodeResponse: buildNodeResponse({
toolId: SANDBOX_GET_FILE_URL_TOOL_NAME,
input,
response,
durationSeconds,
lang
})
nodeResponse: {
moduleType: FlowNodeTypeEnum.tool,
moduleName: parseI18nString(SANDBOX_NAME, lang),
moduleLogo: SANDBOX_ICON,
toolId: toolName,
toolInput: input,
toolRes: response
}
};
};
......
packages/service/core/workflow/dispatch/ai/agent/sub/tool/index.ts
import type { StoreSecretValueType } from '@fastgpt/global/common/secret/type';
import { SystemToolSecretInputTypeEnum } from '@fastgpt/global/core/app/tool/systemTool/constants';
import type { DispatchSubAppResponse } from '../../type';
import { splitCombineToolId } from '@fastgpt/global/core/app/tool/utils';
import { getSystemToolById } from '../../../../../../app/tool/controller';
import { getSecretValue } from '../../../../../../../common/secret/utils';
import { MongoSystemTool } from '../../../../../../plugin/tool/systemToolSchema';
......@@ -21,6 +20,9 @@ import { MCPClient } from '../../../../../../app/mcp';
import { runHTTPTool } from '../../../../../../app/http';
import { getS3ChatSource } from '../../../../../../../common/s3/sources/chat';
import { parseToolId } from '../../../../child/runTool';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
type SystemInputConfigType = {
type: SystemToolSecretInputTypeEnum;
......@@ -29,6 +31,7 @@ type SystemInputConfigType = {
export type Props = {
tool: {
name: string;
avatar?: string;
version?: string;
toolConfig: RuntimeNodeItemType['toolConfig'];
};
......@@ -45,7 +48,7 @@ export type Props = {
};
export const dispatchTool = async ({
tool: { name, version, toolConfig },
tool: { name, avatar, version, toolConfig },
params: { system_input_config, ...params },
runningUserInfo,
runningAppInfo,
......@@ -53,19 +56,29 @@ export const dispatchTool = async ({
uid,
variables,
workflowStreamResponse
}: Props): Promise<
DispatchSubAppResponse & {
toolParams: Record<string, any>;
}
> => {
const startTime = Date.now();
const getErrResponse = (error: any) => {
}: Props): Promise<DispatchSubAppResponse> => {
const getNodeResponse = ({
result,
response
}: RequireOnlyOne<{
result?: any;
response?: string;
}>): DispatchSubAppResponse['nodeResponse'] => {
return {
moduleType: FlowNodeTypeEnum.tool,
moduleName: name,
moduleLogo: avatar,
toolInput: params,
toolRes: result || response
};
};
const getErrResponse = (error: any): DispatchSubAppResponse => {
const response = getErrText(error, 'Call tool error');
return {
toolParams: params,
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
response: getErrText(error, 'Call tool error'),
usages: []
response,
nodeResponse: getNodeResponse({
response
})
};
};
......@@ -165,9 +178,9 @@ export const dispatchTool = async ({
return {
response: JSON.stringify(result),
toolParams: params,
result,
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
nodeResponse: getNodeResponse({
result
}),
usages: [
{
moduleName: name,
......@@ -196,11 +209,10 @@ export const dispatchTool = async ({
params
});
return {
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
response: JSON.stringify(result),
toolParams: params,
result,
usages: []
nodeResponse: getNodeResponse({
result: result
})
};
} else if (toolConfig?.httpTool?.toolId) {
const { parentId, toolName } = parseToolId(toolConfig.httpTool.toolId);
......@@ -242,19 +254,18 @@ export const dispatchTool = async ({
if (errorMsg) {
return {
toolParams: params,
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
response: errorMsg,
usages: []
nodeResponse: getNodeResponse({
response: errorMsg
}),
response: errorMsg
};
}
return {
toolParams: params,
runningTime: +((Date.now() - startTime) / 1000).toFixed(2),
response: typeof data === 'object' ? JSON.stringify(data) : data,
result: data,
usages: []
nodeResponse: getNodeResponse({
result: data
}),
response: typeof data === 'object' ? JSON.stringify(data) : data
};
} else {
return getErrResponse("Can't find the tool");
......
packages/service/core/workflow/dispatch/ai/agent/type.ts
......@@ -3,6 +3,7 @@ import type { JSONSchemaInputType } from '@fastgpt/global/core/app/jsonschema';
import type { ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
import z from 'zod';
import { NodeToolConfigTypeSchema } from '@fastgpt/global/core/workflow/type/node';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type';
export type ToolNodeItemType = RuntimeNodeItemType & {
toolParams: RuntimeNodeItemType['inputs'];
......@@ -10,10 +11,9 @@ export type ToolNodeItemType = RuntimeNodeItemType & {
};
export type DispatchSubAppResponse = {
response: string;
result?: any;
runningTime: number;
response: string; // 返回给 LLM 的响应
usages?: ChatNodeUsageType[];
nodeResponse?: Omit<ChatHistoryItemResType, 'runningTime' | 'totalPoints' | 'id' | 'nodeId'>; // 部分字段外层会自动根据 usages 计算。
};
export const SubAppRuntimeSchema = z.object({
......
packages/service/core/workflow/dispatch/ai/tool/toolCall.ts
......@@ -14,20 +14,12 @@ import { parseJsonArgs } from '../../../../ai/utils';
import { sliceStrStartEnd } from '@fastgpt/global/common/string/tools';
import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
import { toolValueTypeList, valueTypeJsonSchemaMap } from '@fastgpt/global/core/workflow/constants';
import { runAgentCall } from '../../../../ai/llm/agentCall';
import { runAgentLoop } from '../../../../ai/llm/agentLoop';
import type { ToolCallChildrenInteractive } from '@fastgpt/global/core/workflow/template/system/interactive/type';
import type { JsonSchemaPropertiesItemType } from '@fastgpt/global/core/app/jsonschema';
import {
SANDBOX_SYSTEM_PROMPT,
SANDBOX_ICON,
SANDBOX_TOOL_NAME,
SANDBOX_GET_FILE_URL_TOOL_NAME,
SANDBOX_TOOLS
} from '@fastgpt/global/core/ai/sandbox/constants';
import { SANDBOX_SYSTEM_PROMPT, SANDBOX_TOOLS } from '@fastgpt/global/core/ai/sandbox/constants';
import { getSandboxToolWorkflowResponse } from './constants';
import { callSandboxTool } from '../../../../ai/sandbox/toolCall';
import { systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import { getSandboxToolInfo, runSandboxTools } from '../../../../ai/sandbox/toolCall';
type ResponseType = {
requestIds: string[];
......@@ -75,6 +67,11 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
}
} = workflowProps;
// 注入 sandbox_shell 工具和提示词
let finalMessages = messages;
// 工具响应原始值
const toolRunResponses: ChildResponseItemType[] = [];
// 构建 tools 参数
const toolNodesMap = new Map<string, ToolNodeItemType>();
const tools: ChatCompletionTool[] = toolNodes.map((item) => {
......@@ -117,8 +114,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
};
});
// 注入 sandbox_shell 工具和提示词
let finalMessages = messages;
// 注入 sandbox 提示
if (useAgentSandbox && global.feConfigs?.show_agent_sandbox) {
// 注入 sandbox_shell 工具
tools.push(...SANDBOX_TOOLS);
......@@ -135,25 +131,26 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
}
const getToolInfo = (name: string) => {
const systemTool = systemSubInfo[name];
if (systemTool) {
const sandboxToolInfo = getSandboxToolInfo(name, workflowProps.lang);
if (sandboxToolInfo) {
return {
name: parseI18nString(systemTool.name, workflowProps.lang),
avatar: systemTool.avatar
type: 'sandbox' as const,
name: sandboxToolInfo.name,
avatar: sandboxToolInfo.avatar
};
}
const toolNode = toolNodesMap.get(name);
return {
name: toolNode?.name || '',
avatar: toolNode?.avatar || '',
rawData: toolNode
};
if (toolNode) {
return {
type: 'user' as const,
name: toolNode.name,
avatar: toolNode.avatar,
rawData: toolNode
};
}
};
// 工具响应原始值
const toolRunResponses: ChildResponseItemType[] = [];
const {
inputTokens,
outputTokens,
......@@ -164,7 +161,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
finish_reason,
error,
requestIds
} = await runAgentCall({
} = await runAgentLoop({
maxRunAgentTimes: 50,
body: {
messages: finalMessages,
......@@ -224,23 +221,32 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
});
}
},
onToolParam({ tool, params }) {
onToolParam({ call, argsDelta }) {
if (!isResponseAnswerText) return;
workflowStreamResponse?.({
id: tool.id,
id: call.id,
event: SseResponseEventEnum.toolParams,
data: {
tool: {
id: tool.id,
id: call.id,
toolName: '',
toolAvatar: '',
params
params: argsDelta
}
}
});
},
handleToolResponse: async ({ call, messages }) => {
const tool = getToolInfo(call.function?.name);
onRunTool: async ({ call }) => {
const toolInfo = getToolInfo(call.function?.name);
if (!toolInfo) {
return {
response: 'Call tool not found',
assistantMessages: [],
usages: [],
interactive: undefined,
stop: false
};
}
const {
response,
......@@ -251,21 +257,18 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
stop
} = await (async () => {
// 拦截 sandbox 工具调用
if (
call.function?.name === SANDBOX_TOOL_NAME ||
call.function?.name === SANDBOX_GET_FILE_URL_TOOL_NAME
) {
const { input, response, durationSeconds } = await callSandboxTool({
if (toolInfo.type === 'sandbox') {
const { input, response, durationSeconds } = await runSandboxTools({
toolName: call.function.name,
rawArgs: call.function.arguments ?? '',
appId: String(workflowProps.runningAppInfo.id),
userId: String(workflowProps.uid),
args: call.function.arguments ?? '',
appId: workflowProps.runningAppInfo.id,
userId: workflowProps.uid,
chatId: workflowProps.chatId
});
const flowResponse = getSandboxToolWorkflowResponse({
name: tool.name,
logo: SANDBOX_ICON,
name: toolInfo.name,
logo: toolInfo.avatar,
toolId: call.function.name,
input,
response,
......@@ -274,13 +277,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
return { response, flowResponse };
} else {
const toolNode = tool?.rawData;
if (!toolNode) {
return {
response: 'Call tool not found'
};
}
const toolNode = toolInfo.rawData;
// Init tool params and run
const startParams = parseJsonArgs(call.function.arguments);
......@@ -315,24 +312,26 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
}
})();
if (isResponseAnswerText) {
workflowStreamResponse?.({
id: call.id,
event: SseResponseEventEnum.toolResponse,
data: {
tool: {
id: call.id,
toolName: '',
toolAvatar: '',
params: '',
response: sliceStrStartEnd(response, 5000, 5000)
// 推送存储数据，与 tool 逻辑无关
{
if (isResponseAnswerText) {
workflowStreamResponse?.({
id: call.id,
event: SseResponseEventEnum.toolResponse,
data: {
tool: {
id: call.id,
toolName: '',
toolAvatar: '',
params: '',
response: sliceStrStartEnd(response, 5000, 5000)
}
}
}
});
}
if (flowResponse) {
toolRunResponses.push(flowResponse);
});
}
if (flowResponse) {
toolRunResponses.push(flowResponse);
}
}
return {
......@@ -343,7 +342,7 @@ export const runToolCall = async (props: DispatchToolModuleProps): Promise<Respo
stop
};
},
handleInteractiveTool: async ({ childrenResponse, toolParams }) => {
onRunInteractiveTool: async ({ childrenResponse, toolParams }) => {
initToolNodes(runtimeNodes, childrenResponse.entryNodeIds);
initToolCallEdges(runtimeEdges, childrenResponse.entryNodeIds);
......
projects/app/package.json
{
"name": "app",
"version": "4.14.11",
"version": "4.14.13",
"private": false,
"scripts": {
"dev": "NODE_OPTIONS='--max-old-space-size=8192' npm run build:workers && next dev",
......
projects/app/src/pageComponents/app/detail/Edit/ChatAgent/hooks/useSkillManager.tsx
......@@ -31,6 +31,7 @@ import {
import { useLatest } from 'ahooks';
import { SubAppIds, systemSubInfo } from '@fastgpt/global/core/workflow/node/agent/constants';
import { parseI18nString } from '@fastgpt/global/common/i18n/utils';
import { SANDBOX_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/constants';
const ConfigToolModal = dynamic(() => import('../../component/ConfigToolModal'));
......@@ -111,10 +112,10 @@ export const useSkillManager = ({
});
}
const sandboxToolInfo = systemSubInfo[SubAppIds.sandboxTool];
const sandboxToolInfo = systemSubInfo[SANDBOX_TOOL_NAME];
if (sandboxToolInfo) {
apiTools.unshift({
id: SubAppIds.sandboxTool,
id: SANDBOX_TOOL_NAME,
label: parseI18nString(sandboxToolInfo.name, i18n.language),
icon: sandboxToolInfo.avatar,
description: sandboxToolInfo.toolDescription,
......@@ -338,11 +339,11 @@ export const useSkillManager = ({
}
// Merge sandbox tool
const sandboxToolInfo = systemSubInfo[SubAppIds.sandboxTool];
const sandboxToolInfo = systemSubInfo[SANDBOX_TOOL_NAME];
if (sandboxToolInfo) {
tools.push({
id: SubAppIds.sandboxTool,
pluginId: SubAppIds.sandboxTool,
id: SANDBOX_TOOL_NAME,
pluginId: SANDBOX_TOOL_NAME,
name: parseI18nString(sandboxToolInfo.name, i18n.language),
avatar: sandboxToolInfo.avatar,
intro: sandboxToolInfo.toolDescription,
......
test/cases/service/core/ai/llm/request.test.ts
......@@ -747,8 +747,8 @@ describe('createLLMResponse', () => {
onToolCall: ({ call }) => {
toolCallResults.push(call);
},
onToolParam: ({ params }) => {
toolParamResults.push(params);
onToolParam: ({ argsDelta }) => {
toolParamResults.push(argsDelta);
}
});
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment