Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit 42b661ca by DigHuang Committed by GitHub
Options

refactor(sandbox): unify runtime preparation flow and support package mirrors (#7184) 

* refactor(sandbox): unify runtime preparation flow and support package mirrors

* feat: pypi trust url
parent 3fa6b366
Showing with 849 additions and 433 deletions
document/content/self-host/config/env.en.mdx
...@@ -75,6 +75,8 @@ These variables are mainly validated by `packages/service/env.ts` and apply to ` ...@@ -75,6 +75,8 @@ These variables are mainly validated by `packages/service/env.ts` and apply to `
| `AGENT_SANDBOX_SKILL_MAX_SIZE` | `10` | Maximum Skill sandbox package size, in MB. Used for Skill package upload, download, and publishing checks. | | `AGENT_SANDBOX_SKILL_MAX_SIZE` | `10` | Maximum Skill sandbox package size, in MB. Used for Skill package upload, download, and publishing checks. |
| `AGENT_SANDBOX_MAX_FILE_SIZE` | `10` | Maximum single-file size for Agent sandbox IDE reads, writes, and uploads, in MB. | | `AGENT_SANDBOX_MAX_FILE_SIZE` | `10` | Maximum single-file size for Agent sandbox IDE reads, writes, and uploads, in MB. |
| `AGENT_SANDBOX_MAX_EDIT_DEBUG` | `100` | Limit for Agent edit/debug sandboxes. | | `AGENT_SANDBOX_MAX_EDIT_DEBUG` | `100` | Limit for Agent edit/debug sandboxes. |
| `AGENT_SANDBOX_NPM_REGISTRY` | Empty | npm registry used by npm, yarn, pnpm, and bun inside Agent sandboxes. |
| `AGENT_SANDBOX_PYPI_INDEX_URL` | Empty | PyPI index URL used by pip, `python -m pip`, and uv inside Agent sandboxes. |
### Databases, Cache, and Vector Stores ### Databases, Cache, and Vector Stores
......
document/content/self-host/config/env.mdx
...@@ -75,6 +75,8 @@ description: projects/app、projects/code-sandbox 与 pro/admin 环境变量说 ...@@ -75,6 +75,8 @@ description: projects/app、projects/code-sandbox 与 pro/admin 环境变量说
| `AGENT_SANDBOX_SKILL_MAX_SIZE` | `10` | Skill sandbox 包大小上限，单位 MB；用于 Skill 包上传、下载和打包发布校验。 | | `AGENT_SANDBOX_SKILL_MAX_SIZE` | `10` | Skill sandbox 包大小上限，单位 MB；用于 Skill 包上传、下载和打包发布校验。 |
| `AGENT_SANDBOX_MAX_FILE_SIZE` | `10` | Agent 沙箱 IDE 单文件读写和上传大小上限，单位 MB。 | | `AGENT_SANDBOX_MAX_FILE_SIZE` | `10` | Agent 沙箱 IDE 单文件读写和上传大小上限，单位 MB。 |
| `AGENT_SANDBOX_MAX_EDIT_DEBUG` | `100` | Agent 编辑/调试沙箱数量限制。 | | `AGENT_SANDBOX_MAX_EDIT_DEBUG` | `100` | Agent 编辑/调试沙箱数量限制。 |
| `AGENT_SANDBOX_NPM_REGISTRY` | 空 | Agent 沙箱内 npm、yarn、pnpm、bun 使用的 npm registry。 |
| `AGENT_SANDBOX_PYPI_INDEX_URL` | 空 | Agent 沙箱内 pip、`python -m pip`、uv 使用的 PyPI index URL。 |
### 数据库、缓存与向量库 ### 数据库、缓存与向量库
......
document/data/doc-last-modified.json
...@@ -167,8 +167,8 @@ ...@@ -167,8 +167,8 @@
"content/plugin/model-presets.mdx": "2026-06-04T16:10:15+08:00", "content/plugin/model-presets.mdx": "2026-06-04T16:10:15+08:00",
"content/plugin/system-tool-development.en.mdx": "2026-06-09T16:03:58+08:00", "content/plugin/system-tool-development.en.mdx": "2026-06-09T16:03:58+08:00",
"content/plugin/system-tool-development.mdx": "2026-06-09T16:03:58+08:00", "content/plugin/system-tool-development.mdx": "2026-06-09T16:03:58+08:00",
"content/self-host/config/env.en.mdx": "2026-06-25T11:22:10+08:00", "content/self-host/config/env.en.mdx": "2026-06-25T14:51:00+08:00",
"content/self-host/config/env.mdx": "2026-06-25T11:22:10+08:00", "content/self-host/config/env.mdx": "2026-06-25T14:51:00+08:00",
"content/self-host/config/json.en.mdx": "2026-06-22T11:01:59+08:00", "content/self-host/config/json.en.mdx": "2026-06-22T11:01:59+08:00",
"content/self-host/config/json.mdx": "2026-06-22T11:01:59+08:00", "content/self-host/config/json.mdx": "2026-06-22T11:01:59+08:00",
"content/self-host/config/model/intro.en.mdx": "2026-06-04T16:10:15+08:00", "content/self-host/config/model/intro.en.mdx": "2026-06-04T16:10:15+08:00",
...@@ -291,8 +291,8 @@ ...@@ -291,8 +291,8 @@
"content/self-host/upgrading/4-14/41481.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/41481.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/4149.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4149.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/4149.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4149.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-15/41500.en.mdx": "2026-06-25T11:22:10+08:00", "content/self-host/upgrading/4-15/41500.en.mdx": "2026-06-25T14:51:00+08:00",
"content/self-host/upgrading/4-15/41500.mdx": "2026-06-25T11:22:10+08:00", "content/self-host/upgrading/4-15/41500.mdx": "2026-06-25T14:51:00+08:00",
"content/self-host/upgrading/4-15/41501.mdx": "2026-06-23T21:09:39+08:00", "content/self-host/upgrading/4-15/41501.mdx": "2026-06-23T21:09:39+08:00",
"content/self-host/upgrading/4-15/41502.en.mdx": "2026-05-25T11:21:30+08:00", "content/self-host/upgrading/4-15/41502.en.mdx": "2026-05-25T11:21:30+08:00",
"content/self-host/upgrading/4-15/41502.mdx": "2026-06-23T13:54:06+08:00", "content/self-host/upgrading/4-15/41502.mdx": "2026-06-23T13:54:06+08:00",
...@@ -302,8 +302,8 @@ ...@@ -302,8 +302,8 @@
"content/self-host/upgrading/4-15/41504.mdx": "2026-06-15T23:34:43+08:00", "content/self-host/upgrading/4-15/41504.mdx": "2026-06-15T23:34:43+08:00",
"content/self-host/upgrading/4-15/41505.en.mdx": "2026-06-23T13:54:06+08:00", "content/self-host/upgrading/4-15/41505.en.mdx": "2026-06-23T13:54:06+08:00",
"content/self-host/upgrading/4-15/41505.mdx": "2026-06-24T13:53:05+08:00", "content/self-host/upgrading/4-15/41505.mdx": "2026-06-24T13:53:05+08:00",
"content/self-host/upgrading/4-15/41506.en.mdx": "2026-06-25T11:22:10+08:00", "content/self-host/upgrading/4-15/41506.en.mdx": "2026-06-25T14:51:00+08:00",
"content/self-host/upgrading/4-15/41506.mdx": "2026-06-25T11:22:10+08:00", "content/self-host/upgrading/4-15/41506.mdx": "2026-06-25T14:51:00+08:00",
"content/self-host/upgrading/outdated/40.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/40.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/40.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/40.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/41.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/41.en.mdx": "2026-04-26T21:08:47+08:00",
...@@ -444,6 +444,6 @@ ...@@ -444,6 +444,6 @@
"content/self-host/upgrading/outdated/499.mdx": "2026-05-07T15:06:40+08:00", "content/self-host/upgrading/outdated/499.mdx": "2026-05-07T15:06:40+08:00",
"content/self-host/upgrading/upgrade-intruction.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/upgrade-intruction.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/upgrade-intruction.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/upgrade-intruction.mdx": "2026-04-26T21:08:47+08:00",
"content/toc.en.mdx": "2026-06-25T11:22:10+08:00", "content/toc.en.mdx": "2026-06-25T14:51:00+08:00",
"content/toc.mdx": "2026-06-25T11:22:10+08:00" "content/toc.mdx": "2026-06-25T14:51:00+08:00"
} }
\ No newline at end of file
packages/service/core/ai/sandbox/runtime/entrypoint.ts
...@@ -3,11 +3,12 @@ import { getLogger, LogCategories } from '../../../../common/logger'; ...@@ -3,11 +3,12 @@ import { getLogger, LogCategories } from '../../../../common/logger';
import { serviceEnv } from '../../../../env'; import { serviceEnv } from '../../../../env';
import { isRedisLeaseError, withRedisLease } from '../../../../common/redis/lock'; import { isRedisLeaseError, withRedisLease } from '../../../../common/redis/lock';
import { createAgentSandboxInitializingError } from '../error'; import { createAgentSandboxInitializingError } from '../error';
import type { SandboxPrepareContext, SandboxPrepareStep } from './prepare';
import { buildRuntimeHash, shellQuote } from './utils'; import { buildRuntimeHash, shellQuote } from './utils';
import { import {
getRuntimeStateHash, getRuntimeStateValue,
readSandboxRuntimeState, readSandboxRuntimeState,
setRuntimeStateHash, setRuntimeStateValue,
writeSandboxRuntimeState writeSandboxRuntimeState
} from './state'; } from './state';
...@@ -66,7 +67,7 @@ export const runAgentSandboxEntrypoint = async ({ ...@@ -66,7 +67,7 @@ export const runAgentSandboxEntrypoint = async ({
const stateContext = await readSandboxRuntimeState({ sandbox }); const stateContext = await readSandboxRuntimeState({ sandbox });
const scriptHash = buildRuntimeHash(script); const scriptHash = buildRuntimeHash(script);
if (getRuntimeStateHash(stateContext.state, SANDBOX_ENTRYPOINT_STATE_HASH_KEY) === scriptHash) { if (getRuntimeStateValue(stateContext.state, SANDBOX_ENTRYPOINT_STATE_HASH_KEY) === scriptHash) {
return; return;
} }
...@@ -79,10 +80,26 @@ export const runAgentSandboxEntrypoint = async ({ ...@@ -79,10 +80,26 @@ export const runAgentSandboxEntrypoint = async ({
if (!result) return; if (!result) return;
setRuntimeStateHash(stateContext.state, SANDBOX_ENTRYPOINT_STATE_HASH_KEY, scriptHash); setRuntimeStateValue(stateContext.state, SANDBOX_ENTRYPOINT_STATE_HASH_KEY, scriptHash);
await writeSandboxRuntimeState(sandbox, stateContext); await writeSandboxRuntimeState(sandbox, stateContext);
}; };
/** 将 sandbox entrypoint 包装为 prepare step，便于和文件注入、skill 扫描按顺序编排。 */
export const runSandboxEntrypoint =
<Context extends SandboxPrepareContext>({
sandboxEntrypoint
}: {
sandboxEntrypoint?: string;
}): SandboxPrepareStep<Context> =>
async (context) => {
await runAgentSandboxEntrypoint({
sandbox: context.sandbox,
sandboxEntrypoint,
workDirectory: context.workDirectory
});
return context;
};
export const executeEntrypointCommand = async ({ export const executeEntrypointCommand = async ({
sandbox, sandbox,
command, command,
......
packages/service/core/ai/sandbox/runtime/files.ts
import type { FileWriteEntry, ISandbox } from '@fastgpt-sdk/sandbox-adapter'; import type { FileWriteEntry, ISandbox } from '@fastgpt-sdk/sandbox-adapter';
import { SANDBOX_USER_FILES_PATH } from '@fastgpt/global/core/ai/sandbox/constants'; import { SANDBOX_USER_FILES_PATH } from '@fastgpt/global/core/ai/sandbox/constants';
import { pickOutboundAxios } from '../../../../common/api/axios'; import { pickOutboundAxios } from '../../../../common/api/axios';
import { getSafeAgentInputFilename } from '../../../workflow/dispatch/ai/agent/adapter/fileName'; import { getSafeSandboxInputFilename } from './utils';
export type SandboxInputFile = { export type SandboxInputFile = {
name: string; name: string;
...@@ -41,7 +41,7 @@ export const injectInputFilesToSandbox = async (sandbox: ISandbox, files: Sandbo ...@@ -41,7 +41,7 @@ export const injectInputFilesToSandbox = async (sandbox: ISandbox, files: Sandbo
const usedNames = new Map<string, number>(); const usedNames = new Map<string, number>();
for (const [index, file] of files.entries()) { for (const [index, file] of files.entries()) {
const filename = getSafeAgentInputFilename(file.name, index, usedNames); const filename = getSafeSandboxInputFilename(file.name, index, usedNames);
const path = `${SANDBOX_USER_FILES_PATH}${filename}`; const path = `${SANDBOX_USER_FILES_PATH}${filename}`;
writeFileTasks.push( writeFileTasks.push(
pickOutboundAxios(file.url) pickOutboundAxios(file.url)
......
packages/service/core/ai/sandbox/runtime/index.ts
...@@ -19,18 +19,14 @@ export async function prepareAgentSandboxRuntime({ ...@@ -19,18 +19,14 @@ export async function prepareAgentSandboxRuntime({
userId, userId,
chatId, chatId,
sandboxId, sandboxId,
teamId, teamId
needSandboxRuntime
}: { }: {
appId: string; appId: string;
userId: string; userId: string;
chatId: string; chatId: string;
sandboxId?: string; sandboxId?: string;
teamId: string; teamId: string;
needSandboxRuntime: boolean; }): Promise<AgentSandboxRuntimeContext> {
}): Promise<AgentSandboxRuntimeContext | undefined> {
if (!needSandboxRuntime) return;
try { try {
await checkTeamSandboxPermission(teamId); await checkTeamSandboxPermission(teamId);
} catch { } catch {
......
packages/service/core/ai/sandbox/runtime/mirrors.ts 0 → 100644
import type { ISandbox } from '@fastgpt-sdk/sandbox-adapter';
import { getLogger, LogCategories } from '../../../../common/logger';
import { serviceEnv } from '../../../../env';
import { buildRuntimeHash, joinSandboxPath, shellQuote } from './utils';
import {
getRuntimeStateValue,
readSandboxRuntimeState,
setRuntimeStateValue,
writeSandboxRuntimeState
} from './state';
import { resolveSandboxHome } from './home';
const logger = getLogger(LogCategories.MODULE.AI.SANDBOX);
const SANDBOX_MIRRORS_STATE_HASH_KEY = 'sandboxPackageMirrors';
export type SandboxRuntimeMirrorsConfig = {
npmRegistry?: string;
pypiIndexUrl?: string;
};
export const getSandboxRuntimeMirrorsConfig = (): SandboxRuntimeMirrorsConfig =>
normalizeMirrorsConfig({
npmRegistry: serviceEnv.AGENT_SANDBOX_NPM_REGISTRY,
pypiIndexUrl: serviceEnv.AGENT_SANDBOX_PYPI_INDEX_URL
});
export const prepareSandboxRuntimeMirrors = async ({
sandbox,
config = getSandboxRuntimeMirrorsConfig()
}: {
sandbox: ISandbox;
config?: SandboxRuntimeMirrorsConfig;
}): Promise<void> => {
const files = buildSandboxRuntimeMirrorFiles(config);
if (files.length === 0) return;
const homeDirectory = await resolveSandboxHome(sandbox);
if (!homeDirectory) return;
const stateContext = await readSandboxRuntimeState({ sandbox, homeDirectory });
if (!stateContext.statePath) return;
const mirrorScript = buildSandboxRuntimeMirrorScript({ homeDirectory, files });
const scriptHash = buildRuntimeHash(mirrorScript);
if (getRuntimeStateValue(stateContext.state, SANDBOX_MIRRORS_STATE_HASH_KEY) === scriptHash) {
return;
}
const result = await sandbox
.execute(mirrorScript, {
timeoutMs: 5_000,
maxOutputBytes: 1024
})
.catch((error) => {
logger.warn('[Sandbox Runtime] Failed to execute mirror config script', { error });
return undefined;
});
if (!result || result.exitCode !== 0) {
logger.warn('[Sandbox Runtime] Failed to execute mirror config script', {
exitCode: result?.exitCode,
stderr: result?.stderr
});
return;
}
setRuntimeStateValue(stateContext.state, SANDBOX_MIRRORS_STATE_HASH_KEY, scriptHash);
await writeSandboxRuntimeState(sandbox, stateContext);
};
const buildSandboxRuntimeMirrorFiles = (config: SandboxRuntimeMirrorsConfig) => {
const normalized = normalizeMirrorsConfig(config);
const files: Array<{ path: string; content: string }> = [];
if (normalized.npmRegistry) {
files.push({
path: '.npmrc',
content: `registry=${normalized.npmRegistry}\n`
});
files.push({
path: '.yarnrc',
content: `registry "${normalized.npmRegistry}"\n`
});
files.push({
path: '.yarnrc.yml',
content: `npmRegistryServer: "${escapeYamlString(normalized.npmRegistry)}"\n`
});
files.push({
path: '.bunfig.toml',
content: `[install]\nregistry = "${escapeTomlString(normalized.npmRegistry)}"\n`
});
}
let pypiTrustedHost: string | undefined;
if (normalized.pypiIndexUrl) {
try {
pypiTrustedHost = new URL(normalized.pypiIndexUrl).host || undefined;
} catch {
pypiTrustedHost = undefined;
}
}
const pipConfig = [
'[global]',
...(normalized.pypiIndexUrl ? [`index-url = ${normalized.pypiIndexUrl}`] : []),
...(pypiTrustedHost ? [`trusted-host = ${pypiTrustedHost}`] : [])
];
if (pipConfig.length > 1) {
files.push({
path: '.pip/pip.conf',
content: `${pipConfig.join('\n')}\n`
});
files.push({
path: '.config/pip/pip.conf',
content: `${pipConfig.join('\n')}\n`
});
files.push({
path: '.config/uv/uv.toml',
content: `${[
`default-index = "${escapeTomlString(normalized.pypiIndexUrl!)}"`,
...(pypiTrustedHost
? [`allow-insecure-host = ["${escapeTomlString(pypiTrustedHost)}"]`]
: [])
].join('\n')}\n`
});
}
return files;
};
const normalizeMirrorsConfig = (config: SandboxRuntimeMirrorsConfig): SandboxRuntimeMirrorsConfig =>
Object.fromEntries(
Object.entries(config).flatMap(([key, value]) => {
const trimmed = value?.trim();
return trimmed ? [[key, trimmed]] : [];
})
);
const buildSandboxRuntimeMirrorScript = ({
homeDirectory,
files
}: {
homeDirectory: string;
files: Array<{ path: string; content: string }>;
}) => {
const writeEntries = files.map((file) => ({
path: joinSandboxPath(homeDirectory, file.path),
content: file.content
}));
const prepareDirs = Array.from(
new Set(writeEntries.map((entry) => entry.path.split('/').slice(0, -1).join('/')))
);
return [
`mkdir -p ${prepareDirs.map((dir) => shellQuote(dir)).join(' ')}`,
...writeEntries.map(({ path, content }) => {
const encodedContent = Buffer.from(content, 'utf-8').toString('base64');
return `printf %s ${shellQuote(encodedContent)} | base64 -d > ${shellQuote(path)}`;
})
].join('\n');
};
const escapeTomlString = (value: string) => value.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
const escapeYamlString = (value: string) => value.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
packages/service/core/ai/sandbox/runtime/prepare.ts 0 → 100644
import type { ISandbox } from '@fastgpt-sdk/sandbox-adapter';
import {
injectInputFilesToSandbox,
readSandboxPwd,
type SandboxCommandClient,
type SandboxInputFile
} from './files';
import { prepareSandboxRuntimeMirrors } from './mirrors';
import { shellQuote } from './utils';
export type SandboxPrepareContext = {
sandbox: ISandbox;
sandboxClient?: SandboxCommandClient;
workDirectory: string;
currentWorkingDirectory?: string;
};
export type SandboxPrepareStep<Context extends SandboxPrepareContext> = (
context: Context
) => Promise<Context>;
/** 顺序执行 sandbox prepare steps，每个 step 都返回下一步可继续消费的上下文。 */
export const prepareSandbox = async <Context extends SandboxPrepareContext>(
context: Context,
...steps: SandboxPrepareStep<Context>[]
): Promise<Context> => {
let currentContext = context;
for (const step of steps) {
currentContext = await step(currentContext);
}
return currentContext;
};
/** 在 sandbox 内写入 npm/pnpm/yarn/bun/pip/uv 镜像源配置。 */
export const preparePackageMirrors =
<Context extends SandboxPrepareContext>(): SandboxPrepareStep<Context> =>
async (context) => {
await prepareSandboxRuntimeMirrors({ sandbox: context.sandbox });
return context;
};
/** 确保当前 runtime 工作目录存在，供后续文件注入、entrypoint 和 skill 扫描使用。 */
export const prepareWorkDirectory =
<Context extends SandboxPrepareContext>(): SandboxPrepareStep<Context> =>
async (context) => {
const result = await context.sandbox.execute(`mkdir -p ${shellQuote(context.workDirectory)}`);
if (result.exitCode !== 0) {
throw new Error(`Failed to prepare workspace directory: ${result.stderr}`);
}
return context;
};
/** 检查工作目录是否已有内容，用于 edit-debug 复用旧 sandbox 时判断是否需要重新部署包。 */
export const inspectWorkDirectoryContent =
<
Context extends SandboxPrepareContext & { workspaceHasContent?: boolean }
>(): SandboxPrepareStep<Context> =>
async (context) => {
const quotedWorkDirectory = shellQuote(context.workDirectory);
const result = await context.sandbox.execute(
`mkdir -p ${quotedWorkDirectory} && test -n "$(find ${quotedWorkDirectory} -mindepth 1 -print -quit 2>/dev/null)"`
);
if (result.exitCode === 0) {
return {
...context,
workspaceHasContent: true
};
}
if (result.exitCode === 1) {
return {
...context,
workspaceHasContent: false
};
}
throw new Error(`Failed to inspect workspace content: ${result.stderr || result.stdout}`);
};
/** 清空工作目录内容但保留目录本身，避免 volume 根目录权限和挂载点问题。 */
export const emptyWorkDirectory =
<Context extends SandboxPrepareContext>(): SandboxPrepareStep<Context> =>
async (context) => {
// 已有实例重部署时先清空工作区；保留挂载点本身，避免 volume 根目录权限问题。
const cleanCmd = `find ${shellQuote(context.workDirectory)} -mindepth 1 -delete || (rm -rf ${shellQuote(context.workDirectory)}/* && rm -rf ${shellQuote(context.workDirectory)}/.[!.]*)`;
const result = await context.sandbox.execute(cleanCmd);
if (result.exitCode !== 0) {
throw new Error(`Failed to clean workspace processes and files: ${result.stderr}`);
}
return context;
};
/** 将本轮用户输入文件写入 sandbox 的 user_files 目录。 */
export const injectCurrentInputFiles =
<Context extends SandboxPrepareContext>(
currentFiles: SandboxInputFile[]
): SandboxPrepareStep<Context> =>
async (context) => {
await injectInputFilesToSandbox(context.sandbox, currentFiles);
return context;
};
/** 读取 sandbox 当前目录，失败时返回 undefined，由上层决定是否展示提示。 */
export const readCurrentWorkingDirectory =
<
Context extends SandboxPrepareContext & { sandboxClient: SandboxCommandClient }
>(): SandboxPrepareStep<Context> =>
async (context) => ({
...context,
currentWorkingDirectory: await readSandboxPwd(context.sandboxClient)
});
packages/service/core/ai/sandbox/runtime/state.ts
...@@ -8,9 +8,10 @@ const logger = getLogger(LogCategories.MODULE.AI.AGENT); ...@@ -8,9 +8,10 @@ const logger = getLogger(LogCategories.MODULE.AI.AGENT);
const RUNTIME_STATE_DIR_RELATIVE_PATH = '.fastgpt/runtime'; const RUNTIME_STATE_DIR_RELATIVE_PATH = '.fastgpt/runtime';
const RUNTIME_STATE_FILE_NAME = 'state.json'; const RUNTIME_STATE_FILE_NAME = 'state.json';
type RuntimeStateValue = string | string[];
export type SandboxRuntimeState = { export type SandboxRuntimeState = {
hashes?: Record<string, string>; values?: Record<string, RuntimeStateValue>;
lists?: Record<string, string[]>;
}; };
export type SandboxRuntimeStateContext = { export type SandboxRuntimeStateContext = {
...@@ -119,38 +120,26 @@ export const writeSandboxRuntimeState = async ( ...@@ -119,38 +120,26 @@ export const writeSandboxRuntimeState = async (
} }
}; };
export const getRuntimeStateHash = (state: SandboxRuntimeState, key: string): string | undefined => export const getRuntimeStateValue = (state: SandboxRuntimeState, key: string): RuntimeStateValue =>
state.hashes?.[key]; state.values?.[key] ?? '';
export const setRuntimeStateHash = ( export const setRuntimeStateValue = (
state: SandboxRuntimeState, state: SandboxRuntimeState,
key: string, key: string,
hash: string value: RuntimeStateValue
): void => { ): void => {
state.hashes = { const normalizedValue = Array.isArray(value) ? Array.from(new Set(value)) : value;
...(state.hashes ?? {}), state.values = {
[key]: hash ...(state.values ?? {}),
...(Array.isArray(normalizedValue) && normalizedValue.length === 0
? {}
: { [key]: normalizedValue })
}; };
}; if (Array.isArray(normalizedValue) && normalizedValue.length === 0) {
delete state.values[key];
export const getRuntimeStateList = (state: SandboxRuntimeState, key: string): string[] =>
state.lists?.[key] ?? [];
export const setRuntimeStateList = (
state: SandboxRuntimeState,
key: string,
values: string[]
): void => {
const uniqueValues = Array.from(new Set(values));
state.lists = {
...(state.lists ?? {}),
...(uniqueValues.length > 0 ? { [key]: uniqueValues } : {})
};
if (uniqueValues.length === 0) {
delete state.lists[key];
} }
if (Object.keys(state.lists).length === 0) { if (Object.keys(state.values).length === 0) {
delete state.lists; delete state.values;
} }
}; };
...@@ -193,13 +182,13 @@ const resolveRuntimeStateLocation = async ({ ...@@ -193,13 +182,13 @@ const resolveRuntimeStateLocation = async ({
const normalizeRuntimeState = (value: unknown): SandboxRuntimeState => { const normalizeRuntimeState = (value: unknown): SandboxRuntimeState => {
if (!value || typeof value !== 'object') return {}; if (!value || typeof value !== 'object') return {};
const raw = value as SandboxRuntimeState; const raw = value as SandboxRuntimeState;
const hashes = normalizeStringRecord(raw.hashes); const values = {
const lists = normalizeStringListRecord(raw.lists); ...normalizeStringRecord((raw as { hashes?: unknown }).hashes),
...normalizeStringListRecord((raw as { lists?: unknown }).lists),
return { ...normalizeRuntimeStateValueRecord(raw.values)
...(hashes ? { hashes } : {}),
...(lists ? { lists } : {})
}; };
return Object.keys(values).length > 0 ? { values } : {};
}; };
const normalizeStringRecord = (value: unknown): Record<string, string> | undefined => { const normalizeStringRecord = (value: unknown): Record<string, string> | undefined => {
...@@ -223,3 +212,25 @@ const normalizeStringListRecord = (value: unknown): Record<string, string[]> | u ...@@ -223,3 +212,25 @@ const normalizeStringListRecord = (value: unknown): Record<string, string[]> | u
}); });
return entries.length > 0 ? Object.fromEntries(entries) : undefined; return entries.length > 0 ? Object.fromEntries(entries) : undefined;
}; };
const normalizeRuntimeStateValueRecord = (
value: unknown
): Record<string, RuntimeStateValue> | undefined => {
if (!value || typeof value !== 'object' || Array.isArray(value)) return;
const entries: Array<[string, RuntimeStateValue]> = [];
Object.entries(value).forEach(([key, item]) => {
if (typeof item === 'string') {
entries.push([key, item]);
return;
}
if (!Array.isArray(item)) return;
const values = Array.from(new Set(item.filter((value) => typeof value === 'string')));
if (values.length > 0) {
entries.push([key, values]);
}
});
return entries.length > 0 ? Object.fromEntries(entries) : undefined;
};
packages/service/core/ai/sandbox/runtime/utils.ts
...@@ -16,3 +16,28 @@ export const joinSandboxPath = (basePath: string, path: string) => ...@@ -16,3 +16,28 @@ export const joinSandboxPath = (basePath: string, path: string) =>
/** 构建 runtime 状态和 manifest 统一使用的内容 hash。 */ /** 构建 runtime 状态和 manifest 统一使用的内容 hash。 */
export const buildRuntimeHash = (content: HashContent): string => export const buildRuntimeHash = (content: HashContent): string =>
`sha256:${createHash('sha256').update(content).digest('hex')}`; `sha256:${createHash('sha256').update(content).digest('hex')}`;
/**
* 将外部文件名收敛为可写入 sandbox user_files 的单个 path segment。
* URL query、API body 和模型上下文都可能携带文件名，因此调用方不能信任原始 name。
*/
export const getSafeSandboxInputFilename = (
filename: string,
index: number,
usedNames: Map<string, number>
) => {
const fallbackName = `file-${index}`;
const normalized = filename.replace(/\\/g, '/').split('/').pop()?.trim() || fallbackName;
const withoutControlChars = normalized.replace(/[\u0000-\u001F\u007F]/g, '').trim();
const baseName =
withoutControlChars && withoutControlChars !== '.' && withoutControlChars !== '..'
? withoutControlChars
: fallbackName;
const firstDotIndex = baseName.indexOf('.');
const stem = firstDotIndex > 0 ? baseName.slice(0, firstDotIndex) : baseName;
const extension = firstDotIndex > 0 ? baseName.slice(firstDotIndex) : '';
const count = usedNames.get(baseName) || 0;
usedNames.set(baseName, count + 1);
return count === 0 ? baseName : `${stem}-${count}${extension}`;
};
packages/service/core/ai/sandbox/toolCall/index.ts
...@@ -11,6 +11,8 @@ import { toolMap as writeFileToolMap } from './writeFile.tool'; ...@@ -11,6 +11,8 @@ import { toolMap as writeFileToolMap } from './writeFile.tool';
import { getSandboxClient, type SandboxClient } from '../service/runtime'; import { getSandboxClient, type SandboxClient } from '../service/runtime';
import { parseJsonArgs } from '../../utils'; import { parseJsonArgs } from '../../utils';
import { writeUrlFilesToSandbox } from '../service/file'; import { writeUrlFilesToSandbox } from '../service/file';
import { getSandboxRuntimeProfile } from '../runtime/profile';
import { preparePackageMirrors, prepareSandbox } from '../runtime/prepare';
const ToolMap = { const ToolMap = {
...editFileToolMap, ...editFileToolMap,
...@@ -114,6 +116,15 @@ export const prepareSandboxToolRuntime = async ({ ...@@ -114,6 +116,15 @@ export const prepareSandboxToolRuntime = async ({
files: { path: string; url: string }[]; files: { path: string; url: string }[];
}) => { }) => {
const instance = await getSandboxClient(sandboxId ? { sandboxId } : { appId, userId, chatId }); const instance = await getSandboxClient(sandboxId ? { sandboxId } : { appId, userId, chatId });
const runtimeProfile = getSandboxRuntimeProfile();
await prepareSandbox(
{
sandbox: instance.provider,
sandboxClient: instance,
workDirectory: runtimeProfile.workDirectory
},
preparePackageMirrors()
);
await writeUrlFilesToSandbox(instance.provider, files); await writeUrlFilesToSandbox(instance.provider, files);
return instance; return instance;
}; };
......
packages/service/core/ai/skill/runtime/builtin.ts
...@@ -6,9 +6,9 @@ import type { ...@@ -6,9 +6,9 @@ import type {
import { getSandboxBuiltinSkillsRootPath } from '../../sandbox/runtime/profile/utils'; import { getSandboxBuiltinSkillsRootPath } from '../../sandbox/runtime/profile/utils';
import { buildRuntimeHash, joinSandboxPath, shellQuote } from '../../sandbox/runtime/utils'; import { buildRuntimeHash, joinSandboxPath, shellQuote } from '../../sandbox/runtime/utils';
import { import {
getRuntimeStateHash, getRuntimeStateValue,
readSandboxRuntimeState, readSandboxRuntimeState,
setRuntimeStateHash, setRuntimeStateValue,
writeSandboxRuntimeState writeSandboxRuntimeState
} from '../../sandbox/runtime/state'; } from '../../sandbox/runtime/state';
...@@ -47,7 +47,7 @@ export async function syncBuiltinSkillsToSandbox({ ...@@ -47,7 +47,7 @@ export async function syncBuiltinSkillsToSandbox({
for (const source of syncSources) { for (const source of syncSources) {
const targetDirectory = joinSandboxPath(builtinSkillsRootPath, source.name); const targetDirectory = joinSandboxPath(builtinSkillsRootPath, source.name);
const stateKey = getBuiltinSkillStateHashKey(source.name); const stateKey = getBuiltinSkillStateHashKey(source.name);
if (getRuntimeStateHash(runtimeStateContext.state, stateKey) === source.etag) { if (getRuntimeStateValue(runtimeStateContext.state, stateKey) === source.etag) {
continue; continue;
} }
...@@ -69,7 +69,7 @@ export async function syncBuiltinSkillsToSandbox({ ...@@ -69,7 +69,7 @@ export async function syncBuiltinSkillsToSandbox({
throw new Error(`Failed to write builtin skill files: ${failedWrite.error?.message}`); throw new Error(`Failed to write builtin skill files: ${failedWrite.error?.message}`);
} }
setRuntimeStateHash(runtimeStateContext.state, stateKey, source.etag); setRuntimeStateValue(runtimeStateContext.state, stateKey, source.etag);
await writeSandboxRuntimeState(sandbox, runtimeStateContext); await writeSandboxRuntimeState(sandbox, runtimeStateContext);
} }
} }
......
packages/service/core/ai/skill/runtime/entrypoint.ts
...@@ -7,9 +7,9 @@ import { ...@@ -7,9 +7,9 @@ import {
} from '../../sandbox/runtime/entrypoint'; } from '../../sandbox/runtime/entrypoint';
import { joinSandboxPath, shellQuote } from '../../sandbox/runtime/utils'; import { joinSandboxPath, shellQuote } from '../../sandbox/runtime/utils';
import { import {
getRuntimeStateList, getRuntimeStateValue,
readSandboxRuntimeState, readSandboxRuntimeState,
setRuntimeStateList, setRuntimeStateValue,
writeSandboxRuntimeState writeSandboxRuntimeState
} from '../../sandbox/runtime/state'; } from '../../sandbox/runtime/state';
...@@ -36,14 +36,15 @@ export const runAgentSkillVersionEntrypoints = async ({ ...@@ -36,14 +36,15 @@ export const runAgentSkillVersionEntrypoints = async ({
const stateContext = await readSandboxRuntimeState({ sandbox }); const stateContext = await readSandboxRuntimeState({ sandbox });
const state = stateContext.state; const state = stateContext.state;
const originalSkillEntrypoints = getRuntimeStateList(state, SKILL_ENTRYPOINT_STATE_LIST_KEY); const stateValue = getRuntimeStateValue(state, SKILL_ENTRYPOINT_STATE_LIST_KEY);
const originalSkillEntrypoints = Array.isArray(stateValue) ? stateValue : [];
const selectedVersionIds = new Set(versions.map(({ versionId }) => versionId)); const selectedVersionIds = new Set(versions.map(({ versionId }) => versionId));
const executedVersionIds = new Set( const executedVersionIds = new Set(
originalSkillEntrypoints.filter((versionId) => selectedVersionIds.has(versionId)) originalSkillEntrypoints.filter((versionId) => selectedVersionIds.has(versionId))
); );
let stateDirty = originalSkillEntrypoints.length !== executedVersionIds.size; let stateDirty = originalSkillEntrypoints.length !== executedVersionIds.size;
const writeSkillEntrypointState = async () => { const writeSkillEntrypointState = async () => {
setRuntimeStateList(stateContext.state, SKILL_ENTRYPOINT_STATE_LIST_KEY, [ setRuntimeStateValue(stateContext.state, SKILL_ENTRYPOINT_STATE_LIST_KEY, [
...executedVersionIds ...executedVersionIds
]); ]);
await writeSandboxRuntimeState(sandbox, stateContext); await writeSandboxRuntimeState(sandbox, stateContext);
......
packages/service/core/ai/skill/runtime/index.ts
...@@ -2,3 +2,10 @@ export type { DeployedSkillInfo, DeployedSkillVersion } from './types'; ...@@ -2,3 +2,10 @@ export type { DeployedSkillInfo, DeployedSkillVersion } from './types';
export { getAgentSkillInfos, injectAgentSkillFilesToSandbox } from './core'; export { getAgentSkillInfos, injectAgentSkillFilesToSandbox } from './core';
export { getBuiltinSkillsRootPath, syncBuiltinSkillsToSandbox } from './builtin'; export { getBuiltinSkillsRootPath, syncBuiltinSkillsToSandbox } from './builtin';
export { runAgentSkillVersionEntrypoints } from './entrypoint'; export { runAgentSkillVersionEntrypoints } from './entrypoint';
export {
deployDownloadedSkillPackage,
downloadSkillPackageToContext,
reportSkillPrepareProgress,
type SkillPackagePrepareContext,
type SkillPackagePrepareStep
} from './prepare';
packages/service/core/ai/skill/runtime/prepare.ts 0 → 100644
import type { SandboxStatusPhase } from '@fastgpt/global/core/chat/type';
import type { SandboxPrepareContext, SandboxPrepareStep } from '../../sandbox/runtime/prepare';
import { joinSandboxPath, shellQuote } from '../../sandbox/runtime/utils';
import { serviceEnv } from '../../../../env';
import { DEFAULT_GITIGNORE_CONTENT, downloadSkillPackage } from '../package';
export type SkillPackagePrepareContext = SandboxPrepareContext & {
packageBuffer?: Buffer;
workspaceHasContent?: boolean;
};
export type SkillPackagePrepareStep = SandboxPrepareStep<SkillPackagePrepareContext>;
/** 下载指定 skill 版本包，并挂到 prepare context，供后续部署 step 使用。 */
export const downloadSkillPackageToContext =
({
storageKey,
onProgress
}: {
storageKey: string;
onProgress?: (phase: SandboxStatusPhase) => void;
}): SkillPackagePrepareStep =>
async (context) => {
onProgress?.('downloadingPackage');
return {
...context,
packageBuffer: await downloadSkillPackage({ storageKey })
};
};
/** 在 prepare 链路中显式上报 skill 部署阶段，保持调用处生命周期可读。 */
export const reportSkillPrepareProgress =
({
phase,
onProgress
}: {
phase: SandboxStatusPhase;
onProgress?: (phase: SandboxStatusPhase) => void;
}): SkillPackagePrepareStep =>
async (context) => {
onProgress?.(phase);
return context;
};
/** 将已下载的 skill ZIP 写入 sandbox 并解压到当前工作目录。 */
export const deployDownloadedSkillPackage =
({
skillsRootPath,
onProgress
}: {
skillsRootPath: string;
onProgress?: (phase: SandboxStatusPhase) => void;
}): SkillPackagePrepareStep =>
async (context) => {
if (!context.packageBuffer) {
throw new Error('Skill package buffer is required before deployment');
}
onProgress?.('uploadingPackage');
const prepareSkillsRootResult = await context.sandbox.execute(
`mkdir -p ${shellQuote(skillsRootPath)}`
);
if (prepareSkillsRootResult.exitCode !== 0) {
throw new Error(`Failed to prepare skill directory: ${prepareSkillsRootResult.stderr}`);
}
const zipPath = joinSandboxPath(skillsRootPath, 'package.zip');
const maxPackageBytes = serviceEnv.AGENT_SANDBOX_SKILL_MAX_SIZE * 1024 * 1024;
const writeResults = await context.sandbox.writeFiles([
{
path: zipPath,
data: context.packageBuffer
}
]);
const failedWrite = writeResults.find((result) => result.error);
if (failedWrite) {
throw new Error(`Failed to write skill package ZIP: ${failedWrite.error?.message}`);
}
onProgress?.('extractingPackage');
const unzipCmd = [
`cd ${shellQuote(context.workDirectory)}`,
`unzip -Z -t ${shellQuote(zipPath)} | awk -v max=${maxPackageBytes} 'BEGIN { ok=0 } /uncompressed,/ { ok=(($3 + 0) <= max) } END { exit ok ? 0 : 1 }'`,
`unzip -Z1 ${shellQuote(zipPath)} | awk 'BEGIN { ok=1 } /^\\// || /(^|\\/)\\.\\.($|\\/)/ { ok=0 } END { exit ok ? 0 : 1 }'`,
`unzip -o -q ${shellQuote(zipPath)} -d .`,
`rm -f ${shellQuote(zipPath)}`,
`if [ ! -f .gitignore ]; then echo ${shellQuote(DEFAULT_GITIGNORE_CONTENT)} > .gitignore; fi`
].join(' && ');
const extractResult = await context.sandbox.execute(unzipCmd);
if (extractResult.exitCode !== 0) {
throw new Error(`Failed to decompress package inside sandbox: ${extractResult.stderr}`);
}
return context;
};
packages/service/core/workflow/dispatch/ai/agent/adapter/fileName.ts deleted 100644 → 0
/**
* 将外部文件名收敛为可写入 sandbox user_files 的单个 path segment。
* URL query 和 API body 都可能携带文件名，因此调用方不能信任原始 name。
*/
export const getSafeAgentInputFilename = (
filename: string,
index: number,
usedNames: Map<string, number>
) => {
const fallbackName = `file-${index}`;
const normalized = filename.replace(/\\/g, '/').split('/').pop()?.trim() || fallbackName;
const withoutControlChars = normalized.replace(/[\u0000-\u001F\u007F]/g, '').trim();
const baseName =
withoutControlChars && withoutControlChars !== '.' && withoutControlChars !== '..'
? withoutControlChars
: fallbackName;
const firstDotIndex = baseName.indexOf('.');
const stem = firstDotIndex > 0 ? baseName.slice(0, firstDotIndex) : baseName;
const extension = firstDotIndex > 0 ? baseName.slice(firstDotIndex) : '';
const count = usedNames.get(baseName) || 0;
usedNames.set(baseName, count + 1);
return count === 0 ? baseName : `${stem}-${count}${extension}`;
};
packages/service/core/workflow/dispatch/ai/agent/adapter/userContext.ts
...@@ -12,7 +12,7 @@ import type { DeployedSkillInfo } from '../../../../../ai/skill/runtime/types'; ...@@ -12,7 +12,7 @@ import type { DeployedSkillInfo } from '../../../../../ai/skill/runtime/types';
import { getNanoid } from '@fastgpt/global/common/string/tools'; import { getNanoid } from '@fastgpt/global/common/string/tools';
import { SubAppIds } from '@fastgpt/global/core/workflow/node/agent/constants'; import { SubAppIds } from '@fastgpt/global/core/workflow/node/agent/constants';
import { SANDBOX_READ_FILE_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/tools'; import { SANDBOX_READ_FILE_TOOL_NAME } from '@fastgpt/global/core/ai/sandbox/tools';
import { getSafeAgentInputFilename } from './fileName'; import { getSafeSandboxInputFilename } from '../../../../../ai/sandbox/runtime/utils';
export type AgentInputFile = { export type AgentInputFile = {
id: string; id: string;
...@@ -122,7 +122,7 @@ export function parseAgentInputFiles({ ...@@ -122,7 +122,7 @@ export function parseAgentInputFiles({
return { return {
id: `${prefixId}-${index}`, id: `${prefixId}-${index}`,
name: getSafeAgentInputFilename(file.name || parsedFile.name || url, index, usedNames), name: getSafeSandboxInputFilename(file.name || parsedFile.name || url, index, usedNames),
type: parsedFile.type, type: parsedFile.type,
url: parsedFile.url url: parsedFile.url
}; };
......
packages/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.ts
...@@ -11,19 +11,21 @@ import { ...@@ -11,19 +11,21 @@ import {
import { prepareAgentSandboxRuntime } from '../../../../../../ai/sandbox/runtime'; import { prepareAgentSandboxRuntime } from '../../../../../../ai/sandbox/runtime';
import type { SandboxClient } from '../../../../../../ai/sandbox/service/runtime'; import type { SandboxClient } from '../../../../../../ai/sandbox/service/runtime';
import { import {
injectInputFilesToSandbox, injectCurrentInputFiles,
readSandboxPwd preparePackageMirrors,
} from '../../../../../../ai/sandbox/runtime/files'; prepareSandbox,
readCurrentWorkingDirectory,
type SandboxPrepareContext,
type SandboxPrepareStep
} from '../../../../../../ai/sandbox/runtime/prepare';
import { import {
runAgentSandboxEntrypoint, runSandboxEntrypoint,
withAgentSandboxInitLease withAgentSandboxInitLease
} from '../../../../../../ai/sandbox/runtime/entrypoint'; } from '../../../../../../ai/sandbox/runtime/entrypoint';
import { resolveSandboxHome } from '../../../../../../ai/sandbox/runtime/home'; import { resolveSandboxHome } from '../../../../../../ai/sandbox/runtime/home';
export type AgentSandboxPrepareContext = { export type AgentSandboxPrepareContext = SandboxPrepareContext & {
sandboxClient: SandboxClient; sandboxClient: SandboxClient;
workDirectory: string;
currentWorkingDirectory?: string;
deployedSkillVersions: DeployedSkillVersion[]; deployedSkillVersions: DeployedSkillVersion[];
skillInfos: DeployedSkillInfo[]; skillInfos: DeployedSkillInfo[];
skillScanDirectories: string[]; skillScanDirectories: string[];
...@@ -52,9 +54,7 @@ type EnsureAgentSandboxRuntimeResult = { ...@@ -52,9 +54,7 @@ type EnsureAgentSandboxRuntimeResult = {
currentWorkingDirectory?: string; currentWorkingDirectory?: string;
skillInfos: DeployedSkillInfo[]; skillInfos: DeployedSkillInfo[];
}; };
type AgentSandboxPrepareStep = ( type AgentSandboxPrepareStep = SandboxPrepareStep<AgentSandboxPrepareContext>;
context: AgentSandboxPrepareContext
) => Promise<AgentSandboxPrepareContext>;
/** /**
* 确保 Agent 本轮 sandbox runtime 可用。 * 确保 Agent 本轮 sandbox runtime 可用。
...@@ -75,25 +75,26 @@ export async function ensureAgentSandboxRuntime({ ...@@ -75,25 +75,26 @@ export async function ensureAgentSandboxRuntime({
prepareActions = [], prepareActions = [],
currentFiles currentFiles
}: EnsureAgentSandboxRuntimeParams): Promise<EnsureAgentSandboxRuntimeResult> { }: EnsureAgentSandboxRuntimeParams): Promise<EnsureAgentSandboxRuntimeResult> {
if (!needSandboxRuntime) {
return {
skillInfos: []
};
}
const sandboxContext = await prepareAgentSandboxRuntime({ const sandboxContext = await prepareAgentSandboxRuntime({
appId, appId,
userId, userId,
chatId, chatId,
sandboxId, sandboxId,
teamId, teamId
needSandboxRuntime
}); });
if (!sandboxContext) {
return {
skillInfos: []
};
}
const preparedContext = await withAgentSandboxInitLease({ const preparedContext = await withAgentSandboxInitLease({
sandboxId: sandboxContext.sandboxClient.getSandboxId(), sandboxId: sandboxContext.sandboxClient.getSandboxId(),
fn: () => { fn: () => {
const context = { const context = {
...sandboxContext, ...sandboxContext,
sandbox: sandboxContext.sandboxClient.provider,
deployedSkillVersions: [], deployedSkillVersions: [],
skillInfos: [], skillInfos: [],
skillScanDirectories: [] skillScanDirectories: []
...@@ -102,6 +103,7 @@ export async function ensureAgentSandboxRuntime({ ...@@ -102,6 +103,7 @@ export async function ensureAgentSandboxRuntime({
return editSkillId return editSkillId
? prepareSandbox( ? prepareSandbox(
context, context,
preparePackageMirrors(),
injectCurrentInputFiles(currentFiles), injectCurrentInputFiles(currentFiles),
...prepareActions, ...prepareActions,
readCurrentWorkingDirectory(), readCurrentWorkingDirectory(),
...@@ -109,6 +111,7 @@ export async function ensureAgentSandboxRuntime({ ...@@ -109,6 +111,7 @@ export async function ensureAgentSandboxRuntime({
) )
: prepareSandbox( : prepareSandbox(
context, context,
preparePackageMirrors(),
injectSelectedSkillFiles({ teamId, tmbId, skillIds }), injectSelectedSkillFiles({ teamId, tmbId, skillIds }),
injectCurrentInputFiles(currentFiles), injectCurrentInputFiles(currentFiles),
...prepareActions, ...prepareActions,
...@@ -127,29 +130,6 @@ export async function ensureAgentSandboxRuntime({ ...@@ -127,29 +130,6 @@ export async function ensureAgentSandboxRuntime({
}; };
} }
const prepareSandbox = async (
context: AgentSandboxPrepareContext,
...steps: AgentSandboxPrepareStep[]
): Promise<AgentSandboxPrepareContext> => {
let currentContext = context;
for (const step of steps) {
currentContext = await step(currentContext);
}
return currentContext;
};
const injectCurrentInputFiles =
(currentFiles: AgentInputFile[]): AgentSandboxPrepareStep =>
async (context) => {
await injectInputFilesToSandbox(context.sandboxClient.provider, currentFiles);
return context;
};
const readCurrentWorkingDirectory = (): AgentSandboxPrepareStep => async (context) => ({
...context,
currentWorkingDirectory: await readSandboxPwd(context.sandboxClient)
});
/** /**
* 创建“同步内置 Skill 到当前 sandbox”的 prepare action。 * 创建“同步内置 Skill 到当前 sandbox”的 prepare action。
* *
...@@ -168,13 +148,13 @@ export const createBuiltinSkillPrepareAction = ...@@ -168,13 +148,13 @@ export const createBuiltinSkillPrepareAction =
const sources = await getSources(); const sources = await getSources();
if (sources.length === 0) return context; if (sources.length === 0) return context;
const homeDirectory = await resolveSandboxHome(context.sandboxClient.provider); const homeDirectory = await resolveSandboxHome(context.sandbox);
if (!homeDirectory) { if (!homeDirectory) {
throw new Error('Failed to resolve sandbox HOME for builtin skill sync'); throw new Error('Failed to resolve sandbox HOME for builtin skill sync');
} }
await injectToSandbox({ await injectToSandbox({
sandbox: context.sandboxClient.provider, sandbox: context.sandbox,
homeDirectory, homeDirectory,
sources sources
}); });
...@@ -193,7 +173,7 @@ export const createBuiltinSkillPrepareAction = ...@@ -193,7 +173,7 @@ export const createBuiltinSkillPrepareAction =
const scanEditDebugSkillInfos = (): AgentSandboxPrepareStep => async (context) => ({ const scanEditDebugSkillInfos = (): AgentSandboxPrepareStep => async (context) => ({
...context, ...context,
skillInfos: await getAgentSkillInfos({ skillInfos: await getAgentSkillInfos({
sandbox: context.sandboxClient.provider, sandbox: context.sandbox,
skillDirectories: [context.workDirectory, ...context.skillScanDirectories] skillDirectories: [context.workDirectory, ...context.skillScanDirectories]
}) })
}); });
...@@ -211,7 +191,7 @@ const injectSelectedSkillFiles = ...@@ -211,7 +191,7 @@ const injectSelectedSkillFiles =
async (context) => ({ async (context) => ({
...context, ...context,
deployedSkillVersions: await injectAgentSkillFilesToSandbox({ deployedSkillVersions: await injectAgentSkillFilesToSandbox({
sandbox: context.sandboxClient.provider, sandbox: context.sandbox,
teamId, teamId,
tmbId, tmbId,
skillIds, skillIds,
...@@ -219,21 +199,10 @@ const injectSelectedSkillFiles = ...@@ -219,21 +199,10 @@ const injectSelectedSkillFiles =
}) })
}); });
const runSandboxEntrypoint =
({ sandboxEntrypoint }: { sandboxEntrypoint?: string }): AgentSandboxPrepareStep =>
async (context) => {
await runAgentSandboxEntrypoint({
sandbox: context.sandboxClient.provider,
sandboxEntrypoint,
workDirectory: context.workDirectory
});
return context;
};
const runSelectedSkillEntrypoints = (): AgentSandboxPrepareStep => async (context) => { const runSelectedSkillEntrypoints = (): AgentSandboxPrepareStep => async (context) => {
if (context.deployedSkillVersions.length > 0) { if (context.deployedSkillVersions.length > 0) {
await runAgentSkillVersionEntrypoints({ await runAgentSkillVersionEntrypoints({
sandbox: context.sandboxClient.provider, sandbox: context.sandbox,
versions: context.deployedSkillVersions versions: context.deployedSkillVersions
}); });
} }
...@@ -249,7 +218,7 @@ const scanSelectedSkillInfos = (): AgentSandboxPrepareStep => async (context) => ...@@ -249,7 +218,7 @@ const scanSelectedSkillInfos = (): AgentSandboxPrepareStep => async (context) =>
]; ];
return skillDirectories.length > 0 return skillDirectories.length > 0
? getAgentSkillInfos({ ? getAgentSkillInfos({
sandbox: context.sandboxClient.provider, sandbox: context.sandbox,
skillDirectories skillDirectories
}) })
: Promise.resolve([]); : Promise.resolve([]);
......
packages/service/env.ts
...@@ -10,7 +10,6 @@ const defaultableIntSchema = (defaultValue: number) => ...@@ -10,7 +10,6 @@ const defaultableIntSchema = (defaultValue: number) =>
(value) => (value === '' || value === undefined ? defaultValue : value), (value) => (value === '' || value === undefined ? defaultValue : value),
z.coerce.number<number>().int().nonnegative() z.coerce.number<number>().int().nonnegative()
); );
// 系统最大字符串处理长度 // 系统最大字符串处理长度
const SYSTEM_STRING_LENGTH_UNIT = 1_000_000; const SYSTEM_STRING_LENGTH_UNIT = 1_000_000;
...@@ -108,6 +107,8 @@ export const serviceEnv = createEnv({ ...@@ -108,6 +107,8 @@ export const serviceEnv = createEnv({
AGENT_SANDBOX_ENTRYPOINT_TIMEOUT_SECONDS: IntSchema.min(1).max(600).default(30).meta({ AGENT_SANDBOX_ENTRYPOINT_TIMEOUT_SECONDS: IntSchema.min(1).max(600).default(30).meta({
description: 'Agent sandbox entrypoint 执行超时时间（秒）' description: 'Agent sandbox entrypoint 执行超时时间（秒）'
}), }),
AGENT_SANDBOX_NPM_REGISTRY: z.string().optional(),
AGENT_SANDBOX_PYPI_INDEX_URL: z.string().optional(),
// ==================== 数据库与缓存 ==================== // ==================== 数据库与缓存 ====================
// Redisg // Redisg
......
packages/service/test/core/ai/sandbox/runtime.mirrors.test.ts 0 → 100644
import { describe, expect, it, vi } from 'vitest';
import { prepareSandboxRuntimeMirrors } from '@fastgpt/service/core/ai/sandbox/runtime/mirrors';
import { buildRuntimeHash } from '@fastgpt/service/core/ai/sandbox/runtime/utils';
const createSandbox = () => {
let stateContent: string | undefined;
const executedCommands: string[] = [];
const sandbox = {
execute: vi.fn(async (command: string) => {
if (command === 'printf "%s" "$HOME"') {
return { exitCode: 0, stdout: '/home/test', stderr: '' };
}
if (command.startsWith("mkdir -p '/home/test/.fastgpt/runtime'")) {
return { exitCode: 0, stdout: '', stderr: '' };
}
if (command.includes("base64 -d > '/home/test/")) {
executedCommands.push(command);
return { exitCode: 0, stdout: '', stderr: '' };
}
throw new Error(`Unexpected command: ${command}`);
}),
readFiles: vi.fn(async (paths: string[]) =>
paths.map((path) => ({
path,
content: Buffer.from(stateContent || ''),
error: stateContent ? null : new Error('not found')
}))
),
writeFiles: vi.fn(async (entries: Array<{ path: string; data: string }>) => {
const stateEntry = entries.find((entry) =>
entry.path.endsWith('/.fastgpt/runtime/state.json')
);
if (stateEntry) {
stateContent = stateEntry.data;
}
return entries.map((entry) => ({
path: entry.path,
bytesWritten: entry.data.length,
error: null
}));
}),
getExecutedCommands: () => executedCommands,
getState: () => (stateContent ? JSON.parse(stateContent) : undefined)
};
return sandbox;
};
describe('sandbox runtime mirrors', () => {
it('executes npm, yarn, pnpm, bun, pip and uv mirror script once per hash', async () => {
const sandbox = createSandbox();
const expectedMirrorFiles = [
{
path: '.npmrc',
content: 'registry=https://npm.example.com\n'
},
{
path: '.yarnrc',
content: 'registry "https://npm.example.com"\n'
},
{
path: '.yarnrc.yml',
content: 'npmRegistryServer: "https://npm.example.com"\n'
},
{
path: '.bunfig.toml',
content: '[install]\nregistry = "https://npm.example.com"\n'
},
{
path: '.pip/pip.conf',
content:
'[global]\nindex-url = https://pypi.example.com/simple\ntrusted-host = pypi.example.com\n'
},
{
path: '.config/pip/pip.conf',
content:
'[global]\nindex-url = https://pypi.example.com/simple\ntrusted-host = pypi.example.com\n'
},
{
path: '.config/uv/uv.toml',
content:
'default-index = "https://pypi.example.com/simple"\nallow-insecure-host = ["pypi.example.com"]\n'
}
];
const expectedScript = [
"mkdir -p '/home/test' '/home/test/.pip' '/home/test/.config/pip' '/home/test/.config/uv'",
...expectedMirrorFiles.map(({ path, content }) => {
const encodedContent = Buffer.from(content, 'utf-8').toString('base64');
return `printf %s '${encodedContent}' | base64 -d > '/home/test/${path}'`;
})
].join('\n');
await prepareSandboxRuntimeMirrors({
sandbox: sandbox as any,
config: {
npmRegistry: 'https://npm.example.com',
pypiIndexUrl: 'https://pypi.example.com/simple'
}
});
await prepareSandboxRuntimeMirrors({
sandbox: sandbox as any,
config: {
npmRegistry: 'https://npm.example.com',
pypiIndexUrl: 'https://pypi.example.com/simple'
}
});
expect(sandbox.getExecutedCommands()).toEqual([expectedScript]);
expect(sandbox.getState()?.values?.sandboxPackageMirrors).toBe(
buildRuntimeHash(expectedScript)
);
});
});
packages/service/test/core/ai/sandbox/toolCall/index.test.ts
...@@ -13,6 +13,10 @@ const fileServiceMock = vi.hoisted(() => ({ ...@@ -13,6 +13,10 @@ const fileServiceMock = vi.hoisted(() => ({
writeUrlFilesToSandbox: vi.fn() writeUrlFilesToSandbox: vi.fn()
})); }));
const mirrorMock = vi.hoisted(() => ({
prepareSandboxRuntimeMirrors: vi.fn()
}));
const s3Mock = vi.hoisted(() => ({ const s3Mock = vi.hoisted(() => ({
uploadChatFile: vi.fn(), uploadChatFile: vi.fn(),
jwtSignS3ObjectKey: vi.fn() jwtSignS3ObjectKey: vi.fn()
...@@ -26,6 +30,10 @@ vi.mock('@fastgpt/service/core/ai/sandbox/service/file', () => ({ ...@@ -26,6 +30,10 @@ vi.mock('@fastgpt/service/core/ai/sandbox/service/file', () => ({
writeUrlFilesToSandbox: fileServiceMock.writeUrlFilesToSandbox writeUrlFilesToSandbox: fileServiceMock.writeUrlFilesToSandbox
})); }));
vi.mock('@fastgpt/service/core/ai/sandbox/runtime/mirrors', () => ({
prepareSandboxRuntimeMirrors: mirrorMock.prepareSandboxRuntimeMirrors
}));
vi.mock('@fastgpt/service/common/s3/sources/chat', () => ({ vi.mock('@fastgpt/service/common/s3/sources/chat', () => ({
getS3ChatSource: () => ({ getS3ChatSource: () => ({
uploadChatFile: s3Mock.uploadChatFile uploadChatFile: s3Mock.uploadChatFile
...@@ -147,6 +155,12 @@ describe('sandbox toolCall index', () => { ...@@ -147,6 +155,12 @@ describe('sandbox toolCall index', () => {
).resolves.toBe(sandbox); ).resolves.toBe(sandbox);
expect(sandbox.ensureAvailable).not.toHaveBeenCalled(); expect(sandbox.ensureAvailable).not.toHaveBeenCalled();
expect(mirrorMock.prepareSandboxRuntimeMirrors).toHaveBeenCalledWith({
sandbox: sandbox.provider
});
expect(mirrorMock.prepareSandboxRuntimeMirrors.mock.invocationCallOrder[0]).toBeLessThan(
fileServiceMock.writeUrlFilesToSandbox.mock.invocationCallOrder[0]
);
expect(fileServiceMock.writeUrlFilesToSandbox).toHaveBeenCalledWith(sandbox.provider, files); expect(fileServiceMock.writeUrlFilesToSandbox).toHaveBeenCalledWith(sandbox.provider, files);
}); });
......
packages/service/test/core/ai/skill/builtinRuntime.test.ts
...@@ -62,12 +62,13 @@ describe('builtin skill runtime', () => { ...@@ -62,12 +62,13 @@ describe('builtin skill runtime', () => {
expect(sandbox.writeFiles.mock.calls[1][0]).toEqual([ expect(sandbox.writeFiles.mock.calls[1][0]).toEqual([
expect.objectContaining({ expect.objectContaining({
path: '/home/sandbox/.fastgpt/runtime/state.json', path: '/home/sandbox/.fastgpt/runtime/state.json',
data: expect.stringContaining('builtinSkill:skill-creator') data: expect.stringContaining('"values"')
}) })
]); ]);
expect(sandbox.writeFiles.mock.calls[1][0][0].data).toContain('builtinSkill:skill-creator');
}); });
it('skips writing builtin skill when runtime state etag is current', async () => { it('skips writing builtin skill when legacy runtime hash state is current', async () => {
const sources = [ const sources = [
{ {
name: 'skill-creator', name: 'skill-creator',
......
packages/service/test/core/ai/skill/editSandboxPackage.test.ts
...@@ -4,6 +4,7 @@ const mocks = vi.hoisted(() => ({ ...@@ -4,6 +4,7 @@ const mocks = vi.hoisted(() => ({
connectToSandbox: vi.fn(), connectToSandbox: vi.fn(),
disconnectSandbox: vi.fn(), disconnectSandbox: vi.fn(),
deleteWorkspaceArchive: vi.fn(), deleteWorkspaceArchive: vi.fn(),
prepareSandboxRuntimeMirrors: vi.fn(),
logger: { logger: {
info: vi.fn(), info: vi.fn(),
warn: vi.fn(), warn: vi.fn(),
...@@ -93,6 +94,10 @@ vi.mock('@fastgpt/service/core/ai/sandbox/service/resource', () => ({ ...@@ -93,6 +94,10 @@ vi.mock('@fastgpt/service/core/ai/sandbox/service/resource', () => ({
deleteSandboxResource: vi.fn() deleteSandboxResource: vi.fn()
})); }));
vi.mock('@fastgpt/service/core/ai/sandbox/runtime/mirrors', () => ({
prepareSandboxRuntimeMirrors: mocks.prepareSandboxRuntimeMirrors
}));
vi.mock('@fastgpt/service/common/s3/sources/sandbox', () => ({ vi.mock('@fastgpt/service/common/s3/sources/sandbox', () => ({
getS3SandboxSource: () => ({ getS3SandboxSource: () => ({
deleteWorkspaceArchive: mocks.deleteWorkspaceArchive deleteWorkspaceArchive: mocks.deleteWorkspaceArchive
...@@ -108,18 +113,6 @@ vi.mock('@fastgpt/service/core/ai/sandbox/instance/repository', () => ({ ...@@ -108,18 +113,6 @@ vi.mock('@fastgpt/service/core/ai/sandbox/instance/repository', () => ({
updateSandboxInstanceRecordBySandboxId: vi.fn() updateSandboxInstanceRecordBySandboxId: vi.fn()
})); }));
vi.mock('@fastgpt/service/core/ai/skill/runtime', () => {
const trimSandboxPathRight = (value: string) => (value === '/' ? '' : value.replace(/\/+$/, ''));
return {
getSkillsRootPath: (workDirectory: string) => `${trimSandboxPathRight(workDirectory)}/skills`,
joinSandboxPath: (basePath: string, path: string) =>
`${trimSandboxPathRight(basePath)}/${path}`,
shellQuote: (value: string) => `'${value.replace(/'/g, `'\\''`)}'`,
getSafeSkillDirectoryName: (name: string) => name
};
});
vi.mock('@fastgpt/service/common/logger', () => ({ vi.mock('@fastgpt/service/common/logger', () => ({
getLogger: () => mocks.logger, getLogger: () => mocks.logger,
LogCategories: { LogCategories: {
...@@ -293,12 +286,8 @@ describe('packageSkillInSandbox', () => { ...@@ -293,12 +286,8 @@ describe('packageSkillInSandbox', () => {
expect(mocks.disconnectSandbox).toHaveBeenCalledWith(sandbox); expect(mocks.disconnectSandbox).toHaveBeenCalledWith(sandbox);
}); });
it('reads and parses custom .gitignore files from the sandbox correctly', async () => { it('applies default ignore rules and root workspace gitignore when packaging', async () => {
const gitignoreContent = ` const gitignoreContent = 'dist/\n';
# ignore node and env
my_custom_ignored_dir/
temp_data.csv
`;
const zipContent = new Uint8Array([9, 8, 7]); const zipContent = new Uint8Array([9, 8, 7]);
const sandbox = { const sandbox = {
execute: vi.fn(async (command: string) => { execute: vi.fn(async (command: string) => {
...@@ -332,80 +321,14 @@ temp_data.csv ...@@ -332,80 +321,14 @@ temp_data.csv
Buffer.from(zipContent) Buffer.from(zipContent)
); );
// Verify .gitignore was read
expect(sandbox.readFiles).toHaveBeenCalledWith(['/workspace/.gitignore']); expect(sandbox.readFiles).toHaveBeenCalledWith(['/workspace/.gitignore']);
// Verify zip was called with custom excludes
expect(sandbox.execute).toHaveBeenCalledWith(
expect.stringContaining("-x 'my_custom_ignored_dir/*'")
);
expect(sandbox.execute).toHaveBeenCalledWith(
expect.stringContaining("-x '*/my_custom_ignored_dir/*'")
);
expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x 'temp_data.csv'"));
});
it('applies default ignore rules when packaging without workspace gitignore', async () => {
const zipContent = new Uint8Array([1, 2, 3]);
const sandbox = createSandbox({
readFilesResult: [
{
path: '/workspace/package.zip',
content: zipContent,
error: null
}
]
});
mocks.connectToSandbox.mockResolvedValueOnce(sandbox);
await expect(packageSkillInSandbox({ sandboxId: 'sandbox-1' })).resolves.toEqual(
Buffer.from(zipContent)
);
expect(sandbox.execute).toHaveBeenCalledWith( expect(sandbox.execute).toHaveBeenCalledWith(
expect.stringContaining("-name '.venv' -o -name 'node_modules'") expect.stringContaining("-name '.venv' -o -name 'node_modules'")
); );
expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x '.venv/*'")); expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x '.venv/*'"));
expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x '*/.venv/*'")); expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x '*/.venv/*'"));
});
it('only reads root gitignore when packaging', async () => {
const rootGitignoreContent = 'dist/\n';
const zipContent = new Uint8Array([1, 2, 3]);
const sandbox = {
execute: vi.fn(async (command: string) => {
if (command.startsWith('[ -d ')) {
return { exitCode: 0, stdout: '', stderr: '' };
}
if (command.includes('find . ') && command.includes('-prune')) {
return { exitCode: 0, stdout: '100', stderr: '' };
}
if (command.startsWith('cd ')) {
return { exitCode: 0, stdout: 'zip ok', stderr: '' };
}
if (command.startsWith('rm -f ')) {
return { exitCode: 0, stdout: '', stderr: '' };
}
return { exitCode: 0, stdout: '', stderr: '' };
}),
readFiles: vi.fn(async (paths: string[]) => {
if (paths.includes('/workspace/package.zip')) {
return [{ path: '/workspace/package.zip', content: zipContent, error: null }];
}
if (paths.includes('/workspace/.gitignore')) {
return [{ path: '/workspace/.gitignore', content: rootGitignoreContent, error: null }];
}
return [];
})
};
mocks.connectToSandbox.mockResolvedValueOnce(sandbox);
await expect(packageSkillInSandbox({ sandboxId: 'sandbox-1' })).resolves.toEqual(
Buffer.from(zipContent)
);
expect(sandbox.readFiles).toHaveBeenCalledWith(['/workspace/.gitignore']);
expect(sandbox.readFiles).not.toHaveBeenCalledWith(['/workspace/.venv/.gitignore']);
expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x 'dist/*'")); expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x 'dist/*'"));
expect(sandbox.execute).toHaveBeenCalledWith(expect.stringContaining("-x '*/dist/*'"));
}); });
}); });
...@@ -513,6 +436,9 @@ describe('createEditDebugSandbox', () => { ...@@ -513,6 +436,9 @@ describe('createEditDebugSandbox', () => {
]); ]);
expect(provider.execute).toHaveBeenCalledWith("mkdir -p '/workspace/skills'"); expect(provider.execute).toHaveBeenCalledWith("mkdir -p '/workspace/skills'");
expect(provider.execute).toHaveBeenCalledWith(expect.stringContaining('unzip')); expect(provider.execute).toHaveBeenCalledWith(expect.stringContaining('unzip'));
expect(mocks.prepareSandboxRuntimeMirrors).toHaveBeenCalledWith({
sandbox: provider
});
expect(mocks.disconnectSandbox).toHaveBeenCalledWith(provider); expect(mocks.disconnectSandbox).toHaveBeenCalledWith(provider);
}); });
......
packages/service/test/core/ai/skill/runtime.entrypoint.test.ts
...@@ -93,7 +93,7 @@ describe('runtime entrypoint', () => { ...@@ -93,7 +93,7 @@ describe('runtime entrypoint', () => {
sandbox: sandbox as any, sandbox: sandbox as any,
sandboxEntrypoint: 'echo first' sandboxEntrypoint: 'echo first'
}); });
const firstHash = sandbox.getState()?.hashes?.sandboxEntrypoint; const firstHash = sandbox.getState()?.values?.sandboxEntrypoint;
await runAgentSandboxEntrypoint({ await runAgentSandboxEntrypoint({
sandbox: sandbox as any, sandbox: sandbox as any,
...@@ -109,8 +109,8 @@ describe('runtime entrypoint', () => { ...@@ -109,8 +109,8 @@ describe('runtime entrypoint', () => {
.filter(isSandboxEntrypointCommand); .filter(isSandboxEntrypointCommand);
expect(entrypointCommands).toHaveLength(2); expect(entrypointCommands).toHaveLength(2);
expect(sandbox.getState()?.hashes?.sandboxEntrypoint).toMatch(/^sha256:/); expect(sandbox.getState()?.values?.sandboxEntrypoint).toMatch(/^sha256:/);
expect(sandbox.getState()?.hashes?.sandboxEntrypoint).not.toBe(firstHash); expect(sandbox.getState()?.values?.sandboxEntrypoint).not.toBe(firstHash);
}); });
it('runs sandbox entrypoint from the configured work directory', async () => { it('runs sandbox entrypoint from the configured work directory', async () => {
...@@ -128,27 +128,25 @@ describe('runtime entrypoint', () => { ...@@ -128,27 +128,25 @@ describe('runtime entrypoint', () => {
expect(entrypointCommand).toMatch(/^cd '\/workspace' && \/bin\/bash -c /); expect(entrypointCommand).toMatch(/^cd '\/workspace' && \/bin\/bash -c /);
}); });
it('does not write sandbox entrypoint state when execution fails', async () => { it('does not write sandbox entrypoint state when execution fails or throws', async () => {
const sandbox = createSandbox({ entrypointExitCode: 1 }); const failedSandbox = createSandbox({ entrypointExitCode: 1 });
await runAgentSandboxEntrypoint({ await runAgentSandboxEntrypoint({
sandbox: sandbox as any, sandbox: failedSandbox as any,
sandboxEntrypoint: 'exit 1' sandboxEntrypoint: 'exit 1'
}); });
expect(sandbox.getState()?.hashes?.sandboxEntrypoint).toBeUndefined(); const throwingSandbox = createSandbox({ entrypointThrows: true });
});
it('does not throw or write state when sandbox entrypoint execution throws', async () => {
const sandbox = createSandbox({ entrypointThrows: true });
await expect( await expect(
runAgentSandboxEntrypoint({ runAgentSandboxEntrypoint({
sandbox: sandbox as any, sandbox: throwingSandbox as any,
sandboxEntrypoint: 'echo throw' sandboxEntrypoint: 'echo throw'
}) })
).resolves.toBeUndefined(); ).resolves.toBeUndefined();
expect(sandbox.getState()?.hashes?.sandboxEntrypoint).toBeUndefined();
expect(failedSandbox.getState()?.values?.sandboxEntrypoint).toBeUndefined();
expect(throwingSandbox.getState()?.values?.sandboxEntrypoint).toBeUndefined();
}); });
it('uses skill version state to skip successful skill entrypoints', async () => { it('uses skill version state to skip successful skill entrypoints', async () => {
...@@ -168,7 +166,7 @@ describe('runtime entrypoint', () => { ...@@ -168,7 +166,7 @@ describe('runtime entrypoint', () => {
.filter(isSkillEntrypointCommand); .filter(isSkillEntrypointCommand);
expect(runCommands).toHaveLength(1); expect(runCommands).toHaveLength(1);
expect(sandbox.getState()?.lists?.skillEntrypoints).toEqual(['version-1']); expect(sandbox.getState()?.values?.skillEntrypoints).toEqual(['version-1']);
}); });
it('retries skill entrypoint after a failed run', async () => { it('retries skill entrypoint after a failed run', async () => {
...@@ -188,6 +186,6 @@ describe('runtime entrypoint', () => { ...@@ -188,6 +186,6 @@ describe('runtime entrypoint', () => {
.filter(isSkillEntrypointCommand); .filter(isSkillEntrypointCommand);
expect(runCommands).toHaveLength(2); expect(runCommands).toHaveLength(2);
expect(sandbox.getState()?.lists?.skillEntrypoints).toBeUndefined(); expect(sandbox.getState()?.values?.skillEntrypoints).toBeUndefined();
}); });
}); });
packages/service/test/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.test.ts
...@@ -5,6 +5,7 @@ const { ...@@ -5,6 +5,7 @@ const {
prepareAgentSandboxRuntimeMock, prepareAgentSandboxRuntimeMock,
withAgentSandboxInitLeaseMock, withAgentSandboxInitLeaseMock,
injectInputFilesToSandboxMock, injectInputFilesToSandboxMock,
prepareSandboxRuntimeMirrorsMock,
readSandboxPwdMock, readSandboxPwdMock,
runAgentSandboxEntrypointMock, runAgentSandboxEntrypointMock,
resolveSandboxHomeMock, resolveSandboxHomeMock,
...@@ -18,6 +19,7 @@ const { ...@@ -18,6 +19,7 @@ const {
prepareAgentSandboxRuntimeMock: vi.fn(), prepareAgentSandboxRuntimeMock: vi.fn(),
withAgentSandboxInitLeaseMock: vi.fn(async ({ fn }: { fn: () => Promise<unknown> }) => fn()), withAgentSandboxInitLeaseMock: vi.fn(async ({ fn }: { fn: () => Promise<unknown> }) => fn()),
injectInputFilesToSandboxMock: vi.fn(), injectInputFilesToSandboxMock: vi.fn(),
prepareSandboxRuntimeMirrorsMock: vi.fn(),
readSandboxPwdMock: vi.fn(), readSandboxPwdMock: vi.fn(),
runAgentSandboxEntrypointMock: vi.fn(), runAgentSandboxEntrypointMock: vi.fn(),
resolveSandboxHomeMock: vi.fn(), resolveSandboxHomeMock: vi.fn(),
...@@ -45,6 +47,16 @@ vi.mock('@fastgpt/service/core/ai/sandbox/runtime/files', () => ({ ...@@ -45,6 +47,16 @@ vi.mock('@fastgpt/service/core/ai/sandbox/runtime/files', () => ({
vi.mock('@fastgpt/service/core/ai/sandbox/runtime/entrypoint', () => ({ vi.mock('@fastgpt/service/core/ai/sandbox/runtime/entrypoint', () => ({
runAgentSandboxEntrypoint: runAgentSandboxEntrypointMock, runAgentSandboxEntrypoint: runAgentSandboxEntrypointMock,
runSandboxEntrypoint:
({ sandboxEntrypoint }: { sandboxEntrypoint?: string }) =>
async (context: { sandbox: unknown; workDirectory: string }) => {
await runAgentSandboxEntrypointMock({
sandbox: context.sandbox,
sandboxEntrypoint,
workDirectory: context.workDirectory
});
return context;
},
withAgentSandboxInitLease: withAgentSandboxInitLeaseMock withAgentSandboxInitLease: withAgentSandboxInitLeaseMock
})); }));
...@@ -52,6 +64,10 @@ vi.mock('@fastgpt/service/core/ai/sandbox/runtime/home', () => ({ ...@@ -52,6 +64,10 @@ vi.mock('@fastgpt/service/core/ai/sandbox/runtime/home', () => ({
resolveSandboxHome: resolveSandboxHomeMock resolveSandboxHome: resolveSandboxHomeMock
})); }));
vi.mock('@fastgpt/service/core/ai/sandbox/runtime/mirrors', () => ({
prepareSandboxRuntimeMirrors: prepareSandboxRuntimeMirrorsMock
}));
vi.mock('@fastgpt/service/core/ai/skill/runtime', () => ({ vi.mock('@fastgpt/service/core/ai/skill/runtime', () => ({
getAgentSkillInfos: getAgentSkillInfosMock, getAgentSkillInfos: getAgentSkillInfosMock,
getBuiltinSkillsRootPath: (homeDirectory: string) => `${homeDirectory}/.fastgpt/skills`, getBuiltinSkillsRootPath: (homeDirectory: string) => `${homeDirectory}/.fastgpt/skills`,
...@@ -98,6 +114,10 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -98,6 +114,10 @@ describe('ensureAgentSandboxRuntime', () => {
it('runs selected skill lifecycle inside sandbox init lease', async () => { it('runs selected skill lifecycle inside sandbox init lease', async () => {
const { ensureAgentSandboxRuntime } = const { ensureAgentSandboxRuntime } =
await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare'); await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare');
const prepareAction = vi.fn(async (context) => ({
...context,
skillScanDirectories: [...context.skillScanDirectories, '/home/sandbox/.fastgpt/skills']
}));
const result = await ensureAgentSandboxRuntime({ const result = await ensureAgentSandboxRuntime({
appId: 'app_1', appId: 'app_1',
...@@ -108,6 +128,7 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -108,6 +128,7 @@ describe('ensureAgentSandboxRuntime', () => {
needSandboxRuntime: true, needSandboxRuntime: true,
sandboxEntrypoint: 'pip install -r requirements.txt', sandboxEntrypoint: 'pip install -r requirements.txt',
skillIds: ['skill_1'], skillIds: ['skill_1'],
prepareActions: [prepareAction],
currentFiles currentFiles
}); });
...@@ -122,6 +143,15 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -122,6 +143,15 @@ describe('ensureAgentSandboxRuntime', () => {
skillIds: ['skill_1'], skillIds: ['skill_1'],
workDirectory: '/workspace' workDirectory: '/workspace'
}); });
expect(prepareSandboxRuntimeMirrorsMock).toHaveBeenCalledWith({
sandbox: sandboxProviderMock
});
expect(prepareSandboxRuntimeMirrorsMock.mock.invocationCallOrder[0]).toBeLessThan(
injectAgentSkillFilesToSandboxMock.mock.invocationCallOrder[0]
);
expect(prepareSandboxRuntimeMirrorsMock.mock.invocationCallOrder[0]).toBeLessThan(
runAgentSandboxEntrypointMock.mock.invocationCallOrder[0]
);
expect(injectInputFilesToSandboxMock).toHaveBeenCalledWith(sandboxProviderMock, currentFiles); expect(injectInputFilesToSandboxMock).toHaveBeenCalledWith(sandboxProviderMock, currentFiles);
expect(readSandboxPwdMock).toHaveBeenCalledWith(sandboxClientMock); expect(readSandboxPwdMock).toHaveBeenCalledWith(sandboxClientMock);
expect(runAgentSandboxEntrypointMock).toHaveBeenCalledWith({ expect(runAgentSandboxEntrypointMock).toHaveBeenCalledWith({
...@@ -138,9 +168,20 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -138,9 +168,20 @@ describe('ensureAgentSandboxRuntime', () => {
} }
] ]
}); });
expect(prepareAction).toHaveBeenCalledWith(
expect.objectContaining({
deployedSkillVersions: [
{
versionId: 'version_1',
targetDir: '/workspace/skills/version_1'
}
],
skillScanDirectories: []
})
);
expect(getAgentSkillInfosMock).toHaveBeenCalledWith({ expect(getAgentSkillInfosMock).toHaveBeenCalledWith({
sandbox: sandboxProviderMock, sandbox: sandboxProviderMock,
skillDirectories: ['/workspace/skills/version_1'] skillDirectories: ['/workspace/skills/version_1', '/home/sandbox/.fastgpt/skills']
}); });
expect(result).toEqual({ expect(result).toEqual({
sandboxClient: sandboxClientMock, sandboxClient: sandboxClientMock,
...@@ -157,7 +198,7 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -157,7 +198,7 @@ describe('ensureAgentSandboxRuntime', () => {
}); });
}); });
it('runs custom prepare actions in selected skill lifecycle', async () => { it('runs edit-debug lifecycle without deploying selected skills or builtin skills by default', async () => {
const { ensureAgentSandboxRuntime } = const { ensureAgentSandboxRuntime } =
await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare'); await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare');
const prepareAction = vi.fn(async (context) => ({ const prepareAction = vi.fn(async (context) => ({
...@@ -170,76 +211,22 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -170,76 +211,22 @@ describe('ensureAgentSandboxRuntime', () => {
userId: 'user_1', userId: 'user_1',
chatId: 'chat_1', chatId: 'chat_1',
teamId: 'team_1', teamId: 'team_1',
needSandboxRuntime: true,
skillIds: ['skill_1'],
prepareActions: [prepareAction],
currentFiles
});
expect(prepareAction).toHaveBeenCalledWith(
expect.objectContaining({
deployedSkillVersions: [
{
versionId: 'version_1',
targetDir: '/workspace/skills/version_1'
}
],
skillScanDirectories: []
})
);
expect(getAgentSkillInfosMock).toHaveBeenCalledWith({
sandbox: sandboxProviderMock,
skillDirectories: ['/workspace/skills/version_1', '/home/sandbox/.fastgpt/skills']
});
});
it('runs edit-debug lifecycle without deploying selected skills or builtin skills by default', async () => {
const { ensureAgentSandboxRuntime } =
await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare');
await ensureAgentSandboxRuntime({
appId: 'app_1',
userId: 'user_1',
chatId: 'chat_1',
teamId: 'team_1',
tmbId: 'tmb_1', tmbId: 'tmb_1',
needSandboxRuntime: true, needSandboxRuntime: true,
skillIds: [], skillIds: [],
editSkillId: 'edit_skill_1', editSkillId: 'edit_skill_1',
prepareActions: [prepareAction],
currentFiles currentFiles
}); });
expect(prepareSandboxRuntimeMirrorsMock).toHaveBeenCalledWith({
sandbox: sandboxProviderMock
});
expect(prepareSandboxRuntimeMirrorsMock.mock.invocationCallOrder[0]).toBeLessThan(
injectInputFilesToSandboxMock.mock.invocationCallOrder[0]
);
expect(injectInputFilesToSandboxMock).toHaveBeenCalledWith(sandboxProviderMock, currentFiles); expect(injectInputFilesToSandboxMock).toHaveBeenCalledWith(sandboxProviderMock, currentFiles);
expect(syncBuiltinSkillsToSandboxMock).not.toHaveBeenCalled(); expect(syncBuiltinSkillsToSandboxMock).not.toHaveBeenCalled();
expect(getAgentSkillInfosMock).toHaveBeenCalledWith({
sandbox: sandboxProviderMock,
skillDirectories: ['/workspace']
});
expect(injectAgentSkillFilesToSandboxMock).not.toHaveBeenCalled();
expect(runAgentSandboxEntrypointMock).not.toHaveBeenCalled();
expect(runAgentSkillVersionEntrypointsMock).not.toHaveBeenCalled();
});
it('runs custom prepare actions in edit-debug lifecycle', async () => {
const { ensureAgentSandboxRuntime } =
await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare');
const prepareAction = vi.fn(async (context) => ({
...context,
skillScanDirectories: [...context.skillScanDirectories, '/home/sandbox/.fastgpt/skills']
}));
await ensureAgentSandboxRuntime({
appId: 'app_1',
userId: 'user_1',
chatId: 'chat_1',
teamId: 'team_1',
needSandboxRuntime: true,
skillIds: [],
editSkillId: 'edit_skill_1',
prepareActions: [prepareAction],
currentFiles
});
expect(prepareAction).toHaveBeenCalledWith( expect(prepareAction).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
sandboxClient: sandboxClientMock, sandboxClient: sandboxClientMock,
...@@ -251,6 +238,9 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -251,6 +238,9 @@ describe('ensureAgentSandboxRuntime', () => {
sandbox: sandboxProviderMock, sandbox: sandboxProviderMock,
skillDirectories: ['/workspace', '/home/sandbox/.fastgpt/skills'] skillDirectories: ['/workspace', '/home/sandbox/.fastgpt/skills']
}); });
expect(injectAgentSkillFilesToSandboxMock).not.toHaveBeenCalled();
expect(runAgentSandboxEntrypointMock).not.toHaveBeenCalled();
expect(runAgentSkillVersionEntrypointsMock).not.toHaveBeenCalled();
}); });
it('creates builtin skill prepare action with lazy source loading', async () => { it('creates builtin skill prepare action with lazy source loading', async () => {
...@@ -270,6 +260,7 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -270,6 +260,7 @@ describe('ensureAgentSandboxRuntime', () => {
const getSources = vi.fn(async () => builtinSkillSources); const getSources = vi.fn(async () => builtinSkillSources);
const result = await createBuiltinSkillPrepareAction({ getSources })({ const result = await createBuiltinSkillPrepareAction({ getSources })({
sandbox: sandboxProviderMock,
sandboxClient: sandboxClientMock, sandboxClient: sandboxClientMock,
workDirectory: '/workspace', workDirectory: '/workspace',
deployedSkillVersions: [], deployedSkillVersions: [],
...@@ -290,7 +281,6 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -290,7 +281,6 @@ describe('ensureAgentSandboxRuntime', () => {
it('returns empty skill infos when sandbox runtime is not needed', async () => { it('returns empty skill infos when sandbox runtime is not needed', async () => {
const { ensureAgentSandboxRuntime } = const { ensureAgentSandboxRuntime } =
await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare'); await import('@fastgpt/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare');
prepareAgentSandboxRuntimeMock.mockResolvedValueOnce(undefined);
await expect( await expect(
ensureAgentSandboxRuntime({ ensureAgentSandboxRuntime({
...@@ -306,6 +296,7 @@ describe('ensureAgentSandboxRuntime', () => { ...@@ -306,6 +296,7 @@ describe('ensureAgentSandboxRuntime', () => {
).resolves.toEqual({ ).resolves.toEqual({
skillInfos: [] skillInfos: []
}); });
expect(prepareAgentSandboxRuntimeMock).not.toHaveBeenCalled();
expect(withAgentSandboxInitLeaseMock).not.toHaveBeenCalled(); expect(withAgentSandboxInitLeaseMock).not.toHaveBeenCalled();
}); });
}); });
projects/app/.env.template
...@@ -68,6 +68,10 @@ AGENT_SANDBOX_MAX_FILE_SIZE=10 ...@@ -68,6 +68,10 @@ AGENT_SANDBOX_MAX_FILE_SIZE=10
AGENT_SANDBOX_ARCHIVE_MAX_SIZE=50 AGENT_SANDBOX_ARCHIVE_MAX_SIZE=50
# Skill sandbox 包大小上限（MB），用于 Skill 包上传、下载和打包发布校验 # Skill sandbox 包大小上限（MB），用于 Skill 包上传、下载和打包发布校验
AGENT_SANDBOX_SKILL_MAX_SIZE=10 AGENT_SANDBOX_SKILL_MAX_SIZE=10
# Agent 沙箱内 npm/yarn/pnpm/bun 使用的 npm registry（可选）
AGENT_SANDBOX_NPM_REGISTRY=
# Agent 沙箱内 pip/python -m pip/uv 使用的 PyPI index URL（可选）
AGENT_SANDBOX_PYPI_INDEX_URL=
# ==================== 沙盒代理 (agent-sandbox-proxy) 与网络配置 ==================== # ==================== 沙盒代理 (agent-sandbox-proxy) 与网络配置 ====================
# JWT 验签与内网安全物理阻断密钥 (必须与 Rust Proxy 的 AGENT_SANDBOX_PROXY_SECRET 环境变量保持完全一致) # JWT 验签与内网安全物理阻断密钥 (必须与 Rust Proxy 的 AGENT_SANDBOX_PROXY_SECRET 环境变量保持完全一致)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment