Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit 7b1c53fa by DigHuang Committed by GitHub
Options

refactor(sandbox): stream sandbox file upload directly (#7412) 

* refactor(skill): stream skill package import directly to S3

* refactor(sandbox): stream sandbox file upload directly via octet-stream

* refactor(sandbox): derive limits from storage size

* refactor(sandbox): rename storage size env

* refactor(sandbox): use Gi storage env name

* chore: keep v4.15 deployment templates unchanged
parent d37f8ddc
Showing with 971 additions and 770 deletions
document/content/self-host/config/env.en.mdx
......@@ -72,7 +72,7 @@ These variables are mainly validated by `packages/service/env.ts` and apply to `
| `AGENT_SANDBOX_PROXY_URL` | Empty | Browser-accessible WebSocket URL for agent-sandbox-proxy. Required by `fastgpt-app` when Agent Sandbox is enabled; must start with `ws://` or `wss://`. |
| `AGENT_SANDBOX_PREVIEW_PROXY_URL` | Empty | Browser-accessible HTTP(S) URL for Sandbox file previews. You must add it to both `fastgpt-app` and `fastgpt-pro` when Agent Sandbox is enabled. Use an origin separate from the FastGPT application. |
| `AGENT_SANDBOX_FREE_TIP` | `false` | Whether the frontend shows the Agent Sandbox free-use hint. |
| `AGENT_SANDBOX_DISK_MB` | `1024` | Agent Sandbox disk-size baseline, in MB. Cold archive packages use the full value; Skill packages and IDE single-file operations use half the value, rounded to the nearest MB. |
| `AGENT_SANDBOX_STORAGE_SIZE_GI` | `1` | Agent Sandbox storage size in Gi. FastGPT derives the archive, Skill, and single-file limits from this value. |
| `AGENT_SANDBOX_SUSPEND_MINUTES` | `60` | Number of inactive minutes before a running Agent Sandbox is automatically suspended. |
| `AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS` | `7` | Number of inactive days before a suspended Agent Sandbox is automatically archived. |
| `AGENT_SANDBOX_MAX_EDIT_DEBUG` | `100` | Limit for Agent edit/debug sandboxes. |
......
document/content/self-host/config/env.mdx
......@@ -72,7 +72,7 @@ description: projects/app、projects/code-sandbox 与 pro/admin 环境变量说
| `AGENT_SANDBOX_PROXY_URL` | 空 | 浏览器访问 agent-sandbox-proxy 的 WebSocket 地址；`fastgpt-app` 启用 Agent Sandbox 时必填，必须以 `ws://` 或 `wss://` 开头。 |
| `AGENT_SANDBOX_PREVIEW_PROXY_URL` | 空 | 浏览器访问 Sandbox 文件预览的 HTTP(S) 地址；`fastgpt-app` 和 `fastgpt-pro` 启用 Agent Sandbox 时都必须增加。建议使用与 FastGPT 主站不同的 origin。 |
| `AGENT_SANDBOX_FREE_TIP` | `false` | 前端是否展示 Agent Sandbox 免费提示。 |
| `AGENT_SANDBOX_DISK_MB` | `1024` | Agent 沙箱磁盘大小基准，单位 MB；冷归档包上限等于该值，Skill 包和 IDE 单文件上限为该值的一半并四舍五入。 |
| `AGENT_SANDBOX_STORAGE_SIZE_GI` | `1` | Agent Sandbox 存储容量，单位 Gi；FastGPT 根据该值计算归档、Skill 和单文件限制。 |
| `AGENT_SANDBOX_SUSPEND_MINUTES` | `60` | 运行中的 Agent 沙箱持续未活跃多少分钟后自动暂停。 |
| `AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS` | `7` | 已暂停的 Agent 沙箱持续未活跃多少天后自动归档。 |
| `AGENT_SANDBOX_MAX_EDIT_DEBUG` | `100` | Agent 编辑/调试沙箱数量限制。 |
......
document/content/self-host/config/sandbox/common.en.mdx
......@@ -71,14 +71,13 @@ AGENT_SANDBOX_PYPI_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
Configure the following variables in `fastgpt-app` and `fastgpt-pro` when you need to adjust resource limits:
| Variable | Default | Description |
| ------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | Maximum CPU count for each Agent Sandbox instance. |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Maximum memory for each Agent Sandbox instance, in MiB. |
| `AGENT_SANDBOX_STORAGE_SIZE` | `1Gi` | Agent Sandbox storage size, preserving the Kubernetes quantity unit. Used as the Sealos Devbox storage limit and for new OpenSandbox PVCs in Kubernetes mode. |
| `AGENT_SANDBOX_DISK_MB` | `1024` | Baseline Agent Sandbox disk size in MB. It affects cold archive package limits, Skill package limits, and IDE single-file limits. |
| `AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES` | `67108864` | Maximum IDE Agent WebSocket message size in bytes. |
| `AGENT_SANDBOX_WS_MAX_FRAME_BYTES` | `16777216` | Maximum IDE Agent WebSocket frame size in bytes. |
| Variable | Default | Description |
| ------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------------ |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | Maximum CPU count for each Agent Sandbox instance. |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Maximum memory for each Agent Sandbox instance, in MiB. |
| `AGENT_SANDBOX_STORAGE_SIZE_GI` | `1` | Agent Sandbox storage size in Gi. FastGPT derives its archive, Skill, and single-file limits as storage in MB / 2 - 150. |
| `AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES` | `67108864` | Maximum IDE Agent WebSocket message size in bytes. |
| `AGENT_SANDBOX_WS_MAX_FRAME_BYTES` | `16777216` | Maximum IDE Agent WebSocket frame size in bytes. |
### Lifecycle variables
......
document/content/self-host/config/sandbox/common.mdx
......@@ -71,14 +71,13 @@ AGENT_SANDBOX_PYPI_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
`fastpgt-app` 和 `fastgpt-pro` 中你可以通过以下变量控制资源限制：
| 变量 | 默认值 | 说明 |
| ------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | Agent Sandbox 单实例 CPU 核数上限。 |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Agent Sandbox 单实例内存上限，单位 MiB。 |
| `AGENT_SANDBOX_STORAGE_SIZE` | `1Gi` | Agent Sandbox 存储容量，保持 Kubernetes 资源量单位；用于 Sealos Devbox 存储上限，以及 OpenSandbox Kubernetes 模式下创建新 PVC。 |
| `AGENT_SANDBOX_DISK_MB` | `1024` | Agent 沙盒磁盘大小基准，单位 MB；会影响冷归档包、Skill 包和 IDE 单文件大小限制。 |
| `AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES` | `67108864` | IDE Agent WebSocket 单消息大小上限，单位字节。 |
| `AGENT_SANDBOX_WS_MAX_FRAME_BYTES` | `16777216` | IDE Agent WebSocket 单帧大小上限，单位字节。 |
| 变量 | 默认值 | 说明 |
| ------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------- |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | Agent Sandbox 单实例 CPU 核数上限。 |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Agent Sandbox 单实例内存上限，单位 MiB。 |
| `AGENT_SANDBOX_STORAGE_SIZE_GI` | `1` | Agent Sandbox 存储容量，单位 Gi；同时按“存储容量换算为 MB / 2 - 150”计算 FastGPT 的归档、Skill 和单文件限制。 |
| `AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES` | `67108864` | IDE Agent WebSocket 单消息大小上限，单位字节。 |
| `AGENT_SANDBOX_WS_MAX_FRAME_BYTES` | `16777216` | IDE Agent WebSocket 单帧大小上限，单位字节。 |
### 生命周期变量
......
document/content/self-host/config/sandbox/opensandbox.en.mdx
......@@ -64,7 +64,7 @@ AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN=replace_with_volume_manager_token
AGENT_SANDBOX_CPU_COUNT=1
AGENT_SANDBOX_MEMORY_MIB=2048
# OpenSandbox persistent volume size. Effective only when creating new PVCs in Kubernetes mode.
AGENT_SANDBOX_STORAGE_SIZE=1Gi
AGENT_SANDBOX_STORAGE_SIZE_GI=1
```
If your `docker-compose.yml` already uses `x-agent-sandbox-config` to inject Agent Sandbox variables, fill these values in that anchor so both `fastgpt-app` and `fastgpt-pro` inherit the same configuration.
......
document/content/self-host/config/sandbox/opensandbox.mdx
......@@ -63,7 +63,7 @@ AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN=replace_with_volume_manager_token
AGENT_SANDBOX_CPU_COUNT=1
AGENT_SANDBOX_MEMORY_MIB=2048
# OpenSandbox 持久卷容量，仅 K8s 模式下创建新 PVC 时有效
AGENT_SANDBOX_STORAGE_SIZE=1Gi
AGENT_SANDBOX_STORAGE_SIZE_GI=1
```
如果你的 `docker-compose.yml` 已经使用 `x-agent-sandbox-config` 统一注入 Agent Sandbox 变量，可直接在该 anchor 中填入上述值，确保 `fastgpt-app` 和 `fastgpt-pro` 都继承该配置。
......
document/content/self-host/config/sandbox/sealosdevbox.en.mdx
......@@ -31,10 +31,10 @@ AGENT_SANDBOX_SEALOS_TOKEN=replace_with_sealos_devbox_token
# Sandbox image version
AGENT_SANDBOX_SEALOS_IMAGE=hub.hzh.sealos.run/labring/devbox-sandbox:v0.2.0
# Per-instance Devbox resource limits. Keep the Kubernetes quantity unit for storage.
# Per-instance Devbox resource limits. Storage size is in Gi.
AGENT_SANDBOX_CPU_COUNT=1
AGENT_SANDBOX_MEMORY_MIB=2048
AGENT_SANDBOX_STORAGE_SIZE=1Gi
AGENT_SANDBOX_STORAGE_SIZE_GI=1
```
## FAQ
......
document/content/self-host/config/sandbox/sealosdevbox.mdx
......@@ -30,10 +30,10 @@ AGENT_SANDBOX_SEALOS_TOKEN=replace_with_sealos_devbox_token
# 沙盒镜像版本
AGENT_SANDBOX_SEALOS_IMAGE=hub.hzh.sealos.run/labring/devbox-sandbox:v0.2.0
# Devbox 单实例资源上限；存储容量保持 Kubernetes 资源量单位
# Devbox 单实例资源上限；存储容量单位为 GB
AGENT_SANDBOX_CPU_COUNT=1
AGENT_SANDBOX_MEMORY_MIB=2048
AGENT_SANDBOX_STORAGE_SIZE=1Gi
AGENT_SANDBOX_STORAGE_SIZE_GI=1
```
## 常见问题
......
document/content/self-host/upgrading/4-16/41601.en.mdx
......@@ -50,13 +50,13 @@ A preview URL is a temporary, read-only bearer capability and is not limited to
This release also adds the following optional settings:
| Variable | Default | Description |
| ------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------- |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | CPU core limit for each Agent Sandbox instance. |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Memory limit for each Agent Sandbox instance, in MiB. |
| `AGENT_SANDBOX_STORAGE_SIZE` | `1Gi` | Agent Sandbox storage size. Used as the Sealos Devbox storage limit and for new OpenSandbox PVCs in Kubernetes mode. |
| `AGENT_SANDBOX_SUSPEND_MINUTES` | `60` | Minutes of inactivity before a running Sandbox is stopped. |
| `AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS` | `7` | Days of inactivity before a stopped Sandbox is archived. |
| Variable | Default | Description |
| ------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------- |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | CPU core limit for each Agent Sandbox instance. |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Memory limit for each Agent Sandbox instance, in MiB. |
| `AGENT_SANDBOX_STORAGE_SIZE_GI` | `1` | Agent Sandbox storage size in Gi. Used as the Sealos Devbox storage limit and for new OpenSandbox PVCs in Kubernetes mode. |
| `AGENT_SANDBOX_SUSPEND_MINUTES` | `60` | Minutes of inactivity before a running Sandbox is stopped. |
| `AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS` | `7` | Days of inactivity before a stopped Sandbox is archived. |
The E2B Sandbox Provider has been removed. Deployments previously using E2B must switch to `opensandbox` or `sealosdevbox` and remove the `AGENT_SANDBOX_E2B_API_KEY` environment variable.
......
document/content/self-host/upgrading/4-16/41601.mdx
......@@ -22,13 +22,13 @@ AGENT_SANDBOX_PREVIEW_PROXY_URL=https://sandbox-proxy.example.com
本版本还新增以下可选配置：
| 变量 | 默认值 | 说明 |
| ------------------------------------- | ------ | --------------------------------------------------------------------------------------------------- |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | Agent Sandbox 单实例 CPU 核数上限。 |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Agent Sandbox 单实例内存上限，单位 MiB。 |
| `AGENT_SANDBOX_STORAGE_SIZE` | `1Gi` | Agent Sandbox 存储容量；用于 Sealos Devbox 存储上限，以及 OpenSandbox Kubernetes 模式下创建新 PVC。 |
| `AGENT_SANDBOX_SUSPEND_MINUTES` | `60` | 运行中的 Sandbox 未活跃多久后自动暂停，单位分钟。 |
| `AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS` | `7` | 已暂停 Sandbox 未活跃多久后自动归档，单位天。 |
| 变量 | 默认值 | 说明 |
| ------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------ |
| `AGENT_SANDBOX_CPU_COUNT` | `1` | Agent Sandbox 单实例 CPU 核数上限。 |
| `AGENT_SANDBOX_MEMORY_MIB` | `2048` | Agent Sandbox 单实例内存上限，单位 MiB。 |
| `AGENT_SANDBOX_STORAGE_SIZE_GI` | `1` | Agent Sandbox 存储容量，单位 Gi；用于 Sealos Devbox 存储上限，以及 OpenSandbox Kubernetes 模式下创建新 PVC。 |
| `AGENT_SANDBOX_SUSPEND_MINUTES` | `60` | 运行中的 Sandbox 未活跃多久后自动暂停，单位分钟。 |
| `AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS` | `7` | 已暂停 Sandbox 未活跃多久后自动归档，单位天。 |
E2B Sandbox Provider 已移除。此前配置过 E2B 的环境需要切换为 `opensandbox` 或 `sealosdevbox`，并删除 `AGENT_SANDBOX_E2B_API_KEY`。
......
document/data/doc-last-modified.json
......@@ -167,8 +167,8 @@
"content/plugin/model-presets.mdx": "2026-06-04T16:10:15+08:00",
"content/plugin/system-tool-development.en.mdx": "2026-07-02T11:54:55+08:00",
"content/plugin/system-tool-development.mdx": "2026-07-02T11:54:55+08:00",
"content/self-host/config/env.en.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/env.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/env.en.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/env.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/model/intro.en.mdx": "2026-06-04T16:10:15+08:00",
"content/self-host/config/model/intro.mdx": "2026-06-04T16:10:15+08:00",
"content/self-host/config/model/minimax.en.mdx": "2026-06-03T10:40:17+08:00",
......@@ -179,12 +179,12 @@
"content/self-host/config/object-storage.mdx": "2026-07-26T21:56:28+08:00",
"content/self-host/config/remote-debug-suite.en.mdx": "2026-06-27T22:05:51+08:00",
"content/self-host/config/remote-debug-suite.mdx": "2026-06-27T22:05:51+08:00",
"content/self-host/config/sandbox/common.en.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/sandbox/common.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/sandbox/opensandbox.en.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/sandbox/opensandbox.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/sandbox/sealosdevbox.en.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/sandbox/sealosdevbox.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/config/sandbox/common.en.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/sandbox/common.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/sandbox/opensandbox.en.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/sandbox/opensandbox.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/sandbox/sealosdevbox.en.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/sandbox/sealosdevbox.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/config/signoz.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/config/signoz.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/custom-models/bge-rerank.en.mdx": "2026-04-26T21:08:47+08:00",
......@@ -325,12 +325,12 @@
"content/self-host/upgrading/4-15/4152.mdx": "2026-07-25T00:27:20+08:00",
"content/self-host/upgrading/4-15/4153.en.mdx": "2026-07-18T12:51:59+08:00",
"content/self-host/upgrading/4-15/4153.mdx": "2026-07-18T12:51:59+08:00",
"content/self-host/upgrading/4-15/4154.en.mdx": "2026-07-25T00:27:20+08:00",
"content/self-host/upgrading/4-15/4154.mdx": "2026-07-25T00:27:20+08:00",
"content/self-host/upgrading/4-15/4155.en.mdx": "2026-07-29T14:48:32+08:00",
"content/self-host/upgrading/4-15/4155.mdx": "2026-07-29T14:48:32+08:00",
"content/self-host/upgrading/4-16/41601.en.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/upgrading/4-16/41601.mdx": "2026-07-29T19:55:49+08:00",
"content/self-host/upgrading/4-15/4154.en.mdx": "2026-07-30T11:22:58+08:00",
"content/self-host/upgrading/4-15/4154.mdx": "2026-07-30T11:22:58+08:00",
"content/self-host/upgrading/4-15/4155.en.mdx": "2026-07-30T11:22:58+08:00",
"content/self-host/upgrading/4-15/4155.mdx": "2026-07-30T11:22:58+08:00",
"content/self-host/upgrading/4-16/41601.en.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/upgrading/4-16/41601.mdx": "2026-07-30T14:44:03+08:00",
"content/self-host/upgrading/outdated/40.en.mdx": "2026-07-25T00:27:20+08:00",
"content/self-host/upgrading/outdated/40.mdx": "2026-07-25T00:27:20+08:00",
"content/self-host/upgrading/outdated/41.en.mdx": "2026-07-25T00:27:20+08:00",
......
packages/global/openapi/core/ai/sandbox/api.ts
......@@ -34,30 +34,30 @@ export const SandboxDownloadResponseSchema = z
.string()
.meta({ format: 'binary', description: '文件流或 ZIP 包' });
/**
* 上传文件到沙盒工作区 - multipart/form-data 文档结构。
*/
export const SandboxUploadMultipartSchema = z.object({
file: z.any().meta({
format: 'binary',
description: '上传文件，multipart/form-data 的 file 字段'
}),
data: createOutLinkChatTargetInputSchema({
...SandboxBaseShape,
path: z.string().meta({
example: 'src/main.py',
description: '目标文件路径，相对于当前 Chat Session 目录'
})
}).meta({
description: '上传参数，JSON 序列化后传入 multipart/form-data 的 data 字段'
/* ============================================================================
* API: 上传文件到沙盒工作区
* Route: POST /api/core/ai/sandbox/upload
* Method: POST
* Description: 通过原始二进制请求流上传文件到当前 Chat Session
* Tags: ['Sandbox', 'Write']
* ============================================================================ */
export const SandboxUploadQueryRawSchema = createOutLinkChatTargetInputSchema({
...SandboxBaseShape,
path: z.string().meta({
example: 'src/main.py',
description: '目标文件路径，相对于当前 Chat Session 目录'
})
});
export const SandboxUploadBodySchema = withSandboxTarget({
export const SandboxUploadQuerySchema = withSandboxTarget({
path: z.string().meta({
example: 'src/main.py',
description: '目标文件路径，相对于当前 Chat Session 目录'
})
});
export const SandboxUploadFileSchema = z.string().meta({
format: 'binary',
description: '文件原始二进制内容'
});
export const SandboxUploadResponseSchema = z.object({
path: z.string().meta({
example: 'src/main.py',
......@@ -68,8 +68,9 @@ export const SandboxUploadResponseSchema = z.object({
description: '写入字节数'
})
});
export type SandboxUploadBody = z.input<typeof SandboxUploadBodySchema>;
export type SandboxUploadRuntimeBody = z.output<typeof SandboxUploadBodySchema>;
export type SandboxUploadQuery = z.input<typeof SandboxUploadQuerySchema>;
export type SandboxUploadRuntimeQuery = z.output<typeof SandboxUploadQuerySchema>;
export type SandboxUploadFile = z.infer<typeof SandboxUploadFileSchema>;
export type SandboxUploadResponse = z.infer<typeof SandboxUploadResponseSchema>;
/**
......
packages/global/openapi/core/ai/sandbox/index.ts
......@@ -3,7 +3,8 @@ import { DevApiTagsMap } from '../../../tag';
import {
SandboxDownloadBodyRawSchema,
SandboxDownloadResponseSchema,
SandboxUploadMultipartSchema,
SandboxUploadFileSchema,
SandboxUploadQueryRawSchema,
SandboxUploadResponseSchema,
SandboxCheckExistBodyRawSchema,
SandboxCheckExistResponseSchema,
......@@ -42,15 +43,15 @@ export const SandboxPath: OpenAPIPath = {
post: {
summary: '上传文件到沙盒',
description:
'通过 multipart/form-data 上传文件，并写入当前 Chat Session 路径。`file` 字段为二进制文件，`data` 字段为 JSON 序列化后的上传参数对象',
'将原始二进制请求流直接写入当前 Chat Session 路径，不在 FastGPT 节点生成临时文件',
tags: [DevApiTagsMap.sandbox],
requestParams: {
query: SandboxUploadQueryRawSchema
},
requestBody: {
content: {
'multipart/form-data': {
schema: SandboxUploadMultipartSchema,
encoding: {
data: { contentType: 'application/json' }
}
'application/octet-stream': {
schema: SandboxUploadFileSchema
}
}
},
......
packages/global/openapi/core/ai/skill/api.ts
......@@ -138,13 +138,14 @@ export const GetSkillDetailResponseSchema = z.object({
});
export type GetSkillDetailResponse = z.infer<typeof GetSkillDetailResponseSchema>;
export const ImportSkillBodySchema = z.object({
export const ImportSkillQuerySchema = z.object({
filename: z.string().min(1).describe('上传的 .zip 文件原始文件名'),
parentId: z.string().nullable().optional().describe('导入的目标目录 ID'),
name: z.string().optional().describe('导入后的技能名称'),
description: z.string().optional().describe('导入后的技能描述'),
avatar: z.string().optional().describe('导入后的技能头像')
});
export type ImportSkillBody = z.infer<typeof ImportSkillBodySchema>;
export type ImportSkillQuery = z.infer<typeof ImportSkillQuerySchema>;
export const ImportSkillResponseSchema = IdSchema;
export type ImportSkillResponse = z.infer<typeof ImportSkillResponseSchema>;
......@@ -321,28 +322,3 @@ export type SwitchSkillVersionBody = z.infer<typeof SwitchSkillVersionBodySchema
export const SwitchSkillVersionResponseSchema = z.void();
export type SwitchSkillVersionResponse = z.infer<typeof SwitchSkillVersionResponseSchema>;
export const ImportSkillMultipartRequestSchema = {
type: 'object' as const,
properties: {
file: {
type: 'string' as const,
format: 'binary' as const,
description:
'技能压缩包文件，支持 workspace 包（根目录包含非空 skills/）或单 skill 包（根目录或一级目录包含 SKILL.md），支持 ZIP / TAR / TAR.GZ'
},
name: {
type: 'string' as const,
description: '导入后的技能名称，可选'
},
description: {
type: 'string' as const,
description: '导入后的技能描述，可选'
},
avatar: {
type: 'string' as const,
description: '导入后的技能头像，可选'
}
},
required: ['file'] as string[]
};
packages/global/openapi/core/ai/skill/index.ts
......@@ -12,7 +12,7 @@ import {
GetSkillDetailResponseSchema,
GetSkillFolderPathQuerySchema,
GetSkillFolderPathResponseSchema,
ImportSkillMultipartRequestSchema,
ImportSkillQuerySchema,
ImportSkillResponseSchema,
ListAppsBySkillIdQuerySchema,
ListSkillVersionsBodySchema,
......@@ -136,12 +136,19 @@ export const SkillPath: OpenAPIPath = {
'/core/ai/skill/import': {
post: {
summary: '导入技能',
description: '上传 ZIP / TAR / TAR.GZ 技能压缩包并导入为技能',
description: '以原始请求体上传 .zip 技能包并导入为技能',
tags: [DevApiTagsMap.aiSkill],
requestParams: {
query: ImportSkillQuerySchema
},
requestBody: {
content: {
'multipart/form-data': {
schema: ImportSkillMultipartRequestSchema
'application/octet-stream': {
schema: {
type: 'string',
format: 'binary',
description: '技能包原始二进制内容'
}
}
}
},
......
packages/service/common/file/stream.ts 0 → 100644
import { Readable } from 'node:stream';
type CreateSizeLimitedStreamParams = {
stream: Readable;
maxBytes: number;
createExceededError: (bytesRead: number) => Error;
};
/**
* 为 Node Readable 增加实际读取字节上限，避免缺失或伪造 Content-Length 绕过限制。
*/
export function createSizeLimitedStream({
stream,
maxBytes,
createExceededError
}: CreateSizeLimitedStreamParams): Readable {
return Readable.from(
(async function* () {
let bytesRead = 0;
for await (const chunk of stream) {
const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
bytesRead += buffer.length;
if (bytesRead > maxBytes) {
throw createExceededError(bytesRead);
}
yield buffer;
}
})()
);
}
packages/service/common/s3/buckets/base.ts
......@@ -17,7 +17,7 @@ import { MongoS3TTL } from '../models/ttl';
import { addHours, addMinutes, differenceInHours, differenceInSeconds } from 'date-fns';
import { getLogger, LogCategories } from '../../logger';
import { addS3DelJob } from '../queue/delete';
import { type UploadFileByBufferParams, UploadFileByBodySchema } from '../contracts/type';
import { type UploadFileByBodyParams, UploadFileByBodySchema } from '../contracts/type';
import type { createStorage } from '@fastgpt-sdk/storage';
import { parseFileExtensionFromUrl } from '@fastgpt/global/common/string/tools';
import { getContentDisposition } from '@fastgpt/global/common/file/tools';
......@@ -314,12 +314,13 @@ export class S3BaseBucket {
});
}
async uploadFileByBody(params: UploadFileByBufferParams) {
async uploadFileByBody(params: UploadFileByBodyParams) {
const {
key,
body,
filename,
contentType,
contentLength,
expiredTime = addHours(new Date(), 1)
} = UploadFileByBodySchema.parse(params);
......@@ -332,7 +333,8 @@ export class S3BaseBucket {
await this.client.uploadObject({
key,
body,
contentType: contentType || 'application/octet-stream',
contentType: contentType ?? 'application/octet-stream',
contentLength,
metadata: {
contentDisposition: `attachment; filename="${encodeURIComponent(filename)}"`,
originFilename: encodeURIComponent(filename),
......
packages/service/common/s3/contracts/type.ts
......@@ -100,9 +100,9 @@ export type UploadImage2S3BucketParams = z.infer<typeof UploadImage2S3BucketPara
export const UploadFileByBodySchema = z.object({
body: z.union([z.instanceof(Buffer), z.string(), z.instanceof(Readable)]),
contentType: z.string().optional(),
contentLength: z.number().int().nonnegative().optional(),
key: StorageObjectKeySchema,
filename: z.string().nonempty(),
expiredTime: z.coerce.date().optional()
});
export type UploadFileByBodyParams = z.infer<typeof UploadFileByBodySchema>;
export type UploadFileByBufferParams = UploadFileByBodyParams;
packages/service/common/s3/sources/skill/index.ts
import type { ClientSession } from 'mongoose';
import type { Readable } from 'node:stream';
import { S3PrivateBucket } from '../../buckets/private';
import { removeS3TTL } from '../../utils';
......@@ -42,18 +43,20 @@ export class S3SkillSource extends S3PrivateBucket {
teamId: string;
skillId: string;
packageObjectId: string;
body: Buffer;
body: Buffer | Readable;
contentLength?: number;
expiredTime?: Date;
}) {
const { teamId, skillId, packageObjectId, body, expiredTime } = params;
const { teamId, skillId, packageObjectId, body, contentLength, expiredTime } = params;
const key = getSkillPackageKey({ teamId, skillId, packageObjectId });
return this.uploadFileByBody({
key,
body,
contentLength,
contentType: 'application/zip',
filename: 'package.zip',
...(expiredTime && { expiredTime })
expiredTime
});
}
......
packages/service/core/ai/sandbox/application/runtime/skill/core.ts
......@@ -7,7 +7,7 @@ import type { ISandbox } from '@fastgpt-sdk/sandbox-adapter';
import { shellQuote } from '@fastgpt/global/common/string/utils';
import { MongoAgentSkills } from '../../../../skill/model/schema';
import { MongoAgentSkillsVersion } from '../../../../skill/version/schema';
import { downloadSkillPackage } from '../../../../skill/package';
import { downloadSkillPackageStream } from '../../../../skill/package';
import { parseSkillMarkdown } from '../../../../skill/utils';
import { getLogger, LogCategories } from '../../../../../../common/logger';
import type { DeployedSkillInfo, DeployedSkillVersion } from './types';
......@@ -16,6 +16,7 @@ import { joinSandboxPath } from '../../../utils';
import { authSkillByTmbId } from '../../../../../../support/permission/skill/auth';
import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant';
import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import { Readable } from 'node:stream';
export type { DeployedSkillInfo, DeployedSkillVersion } from './types';
......@@ -269,54 +270,42 @@ export const injectAgentSkillFilesToSandbox = async ({
);
const maxPackageBytes = getAgentSandboxSkillMaxBytes();
const results = await Promise.all(
missingSkills.map(async ({ skill, version, versionId, targetDir }) => {
try {
const rawPackageBuffer = await downloadSkillPackage({ storageKey: version.storageKey });
const tempDir = joinSandboxPath(
skillsRootPath,
`.tmp-${getSafeRuntimePathSegment(versionId)}-${Date.now()}-${Math.random()
.toString(36)
.slice(2)}`
);
const zipPath = joinSandboxPath(tempDir, 'package.zip');
const quotedTempDir = shellQuote(tempDir);
const quotedTargetDir = shellQuote(targetDir);
const unzipCommand = `(${[
`cd ${quotedTempDir}`,
`unzip -Z -t package.zip | awk -v max=${maxPackageBytes} 'BEGIN { ok=0 } /uncompressed,/ { ok=(($3 + 0) <= max) } END { exit ok ? 0 : 1 }'`,
`unzip -Z1 package.zip | awk 'BEGIN { ok=1 } /^\\// || /(^|\\/)\\.\\.($|\\/)/ { ok=0 } END { exit ok ? 0 : 1 }'`,
`unzip -o -q package.zip`,
`rm -f package.zip`,
`rm -rf ${quotedTargetDir}`,
`mv ${quotedTempDir} ${quotedTargetDir}`
].join(' && ')})`;
return {
targetDir,
tempDir,
writeEntry: {
path: zipPath,
data: rawPackageBuffer
},
unzipCommand
};
} catch (error) {
logger.error('[Agent Skills] Failed to prepare skill package', {
skillName: skill.name,
error
});
throw error;
}
})
);
const writeEntries = results.map((r) => r.writeEntry);
const unzipCommands = results.map((r) => r.unzipCommand);
const results = missingSkills.map(({ version, versionId, targetDir }) => {
const tempDir = joinSandboxPath(
skillsRootPath,
`.tmp-${getSafeRuntimePathSegment(versionId)}-${Date.now()}-${Math.random()
.toString(36)
.slice(2)}`
);
const zipPath = joinSandboxPath(tempDir, 'package.zip');
const quotedTempDir = shellQuote(tempDir);
const quotedTargetDir = shellQuote(targetDir);
const unzipCommand = `(${[
`cd ${quotedTempDir}`,
`unzip -Z -t package.zip | awk -v max=${maxPackageBytes} 'BEGIN { ok=0 } /uncompressed,/ { ok=(($3 + 0) <= max) } END { exit ok ? 0 : 1 }'`,
`unzip -Z1 package.zip | awk 'BEGIN { ok=1 } /^\\// || /(^|\\/)\\.\\.($|\\/)/ { ok=0 } END { exit ok ? 0 : 1 }'`,
`unzip -o -q package.zip`,
`rm -f package.zip`,
`rm -rf ${quotedTargetDir}`,
`mv ${quotedTempDir} ${quotedTargetDir}`
].join(' && ')})`;
return {
storageKey: version.storageKey,
targetDir,
tempDir,
zipPath,
unzipCommand
};
});
// 1. Batch write all ZIP packages directly to their respective folders in a single call
if (writeEntries.length > 0) {
// 1. Stream each ZIP package directly to its temporary directory.
if (results.length > 0) {
const tempDirs = results.map(({ tempDir }) => tempDir);
const cleanupTempDirs = () =>
Promise.all(
tempDirs.map((tempDir) => sandbox.execute(`rm -rf ${shellQuote(tempDir)}`).catch(() => {}))
);
const mkdirTempResult = await sandbox.execute(
`mkdir -p ${tempDirs.map((dir) => shellQuote(dir)).join(' ')}`
);
......@@ -326,26 +315,33 @@ export const injectAgentSkillFilesToSandbox = async ({
);
}
const writeResults = await sandbox.writeFiles(writeEntries);
const failedWrite = writeResults.find((result) => result.error);
if (failedWrite) {
await Promise.all(
results.map(({ tempDir }) =>
sandbox.execute(`rm -rf ${shellQuote(tempDir)}`).catch(() => {})
)
);
throw new Error(`Failed to write skill ZIP packages: ${failedWrite.error?.message}`);
try {
for (const { storageKey, zipPath } of results) {
const packageStream = await downloadSkillPackageStream({ storageKey });
try {
const [writeResult] = await sandbox.writeFiles([
{
path: zipPath,
data: Readable.toWeb(packageStream) as ReadableStream<Uint8Array>
}
]);
if (writeResult?.error) {
throw new Error(`Failed to write skill ZIP package: ${writeResult.error.message}`);
}
} finally {
if (!packageStream.destroyed) packageStream.destroy();
}
}
} catch (error) {
await cleanupTempDirs();
throw error;
}
// 2. Execute a single unified decompression command inside the sandbox container
const finalUnzipCmd = unzipCommands.join(' && ');
const finalUnzipCmd = results.map(({ unzipCommand }) => unzipCommand).join(' && ');
const extractResult = await sandbox.execute(finalUnzipCmd);
if (extractResult.exitCode !== 0) {
await Promise.all(
results.map(({ tempDir }) =>
sandbox.execute(`rm -rf ${shellQuote(tempDir)}`).catch(() => {})
)
);
await cleanupTempDirs();
throw new Error(
`Failed to decompress skill packages inside sandbox: ${extractResult.stderr}`
);
......
packages/service/core/ai/sandbox/application/runtime/skill/prepare.ts
/**
* 沙盒业务层：定义 Skill 版本包下载和部署的 prepare step。
* 沙盒业务层：定义 Skill 版本包的 prepare step。
*
* 只服务 sandbox runtime 初始化链路，不负责 Skill 版本创建或权限校验。
*/
......@@ -8,47 +8,25 @@ import { shellQuote } from '@fastgpt/global/common/string/utils';
import type { SandboxPrepareContext, SandboxPrepareStep } from '../prepare';
import { joinSandboxPath } from '../../../utils';
import { getAgentSandboxSkillMaxBytes } from '../../../config';
import { DEFAULT_GITIGNORE_CONTENT, downloadSkillPackage } from '../../../../skill/package';
import { DEFAULT_GITIGNORE_CONTENT, downloadSkillPackageStream } from '../../../../skill/package';
import { Readable } from 'node:stream';
export type SkillPackagePrepareContext = SandboxPrepareContext & {
packageBuffer?: Buffer;
workspaceHasContent?: boolean;
};
export type SkillPackagePrepareStep = SandboxPrepareStep<SkillPackagePrepareContext>;
/** 下载指定 skill 版本包，并挂到 prepare context，供后续部署 step 使用。 */
export const downloadSkillPackageToContext =
/** 将对象存储中的 skill 包流式写入 sandbox，并解压到当前工作目录。 */
export const deploySkillPackage =
({
storageKey,
onProgress
}: {
storageKey: string;
onProgress?: (phase: SandboxStatusPhase) => void;
}): SkillPackagePrepareStep =>
async (context) => {
onProgress?.('downloadingPackage');
return {
...context,
packageBuffer: await downloadSkillPackage({ storageKey })
};
};
/** 将已下载的 skill ZIP 写入 sandbox 并解压到当前工作目录。 */
export const deployDownloadedSkillPackage =
({
skillsRootPath,
onProgress
}: {
storageKey: string;
skillsRootPath: string;
onProgress?: (phase: SandboxStatusPhase) => void;
}): SkillPackagePrepareStep =>
}): SandboxPrepareStep<SkillPackagePrepareContext> =>
async (context) => {
if (!context.packageBuffer) {
throw new Error('Skill package buffer is required before deployment');
}
onProgress?.('uploadingPackage');
const prepareSkillsRootResult = await context.sandbox.execute(
`mkdir -p ${shellQuote(skillsRootPath)}`
);
......@@ -59,12 +37,21 @@ export const deployDownloadedSkillPackage =
const zipPath = joinSandboxPath(skillsRootPath, 'package.zip');
const maxPackageBytes = getAgentSandboxSkillMaxBytes();
const writeResults = await context.sandbox.writeFiles([
{
path: zipPath,
data: context.packageBuffer
}
]);
onProgress?.('downloadingPackage');
const packageStream = await downloadSkillPackageStream({ storageKey });
onProgress?.('uploadingPackage');
const writeResults = await context.sandbox
.writeFiles([
{
path: zipPath,
data: Readable.toWeb(packageStream) as ReadableStream<Uint8Array>
}
])
.finally(() => {
if (!packageStream.destroyed) {
packageStream.destroy();
}
});
const failedWrite = writeResults.find((result) => result.error);
if (failedWrite) {
throw new Error(`Failed to write skill package ZIP: ${failedWrite.error?.message}`);
......
packages/service/core/ai/sandbox/application/skillEdit/runtime.ts
......@@ -57,11 +57,7 @@ import {
prepareWorkDirectory,
prepareSandbox
} from '../runtime/prepare';
import {
deployDownloadedSkillPackage,
downloadSkillPackageToContext,
type SkillPackagePrepareContext
} from '../runtime/skill/prepare';
import { deploySkillPackage, type SkillPackagePrepareContext } from '../runtime/skill/prepare';
import { normalizeSandboxImage } from '../runtime/image';
import {
getSandboxRuntimeUpgradeStatus,
......@@ -328,12 +324,9 @@ export async function initSkillEditRuntimeSandbox({
const prepareSteps = [
preparePackageMirrors(),
prepareWorkDirectory(),
downloadSkillPackageToContext({
storageKey: currentVersion.storageKey,
onProgress: reportProgress(sessionId)
}),
...(shouldCleanWorkspaceBeforeDeploy ? [emptyWorkDirectory()] : []),
deployDownloadedSkillPackage({
deploySkillPackage({
storageKey: currentVersion.storageKey,
skillsRootPath: runtimeProfile.skillsRootPath,
onProgress: reportProgress(sessionId)
})
......
packages/service/core/ai/sandbox/config.ts
......@@ -7,10 +7,19 @@
import { serviceEnv } from '../../../env';
const MB_BYTES = 1024 * 1024;
const toRoundedMBBytes = (mb: number) => Math.round(mb) * MB_BYTES;
const RESERVED_DISK_MB = 150;
/** 获取 Agent sandbox 磁盘基准字节数，按 MB 四舍五入。 */
export const getAgentSandboxDiskBytes = () => toRoundedMBBytes(serviceEnv.AGENT_SANDBOX_DISK_MB);
/** 获取 Agent sandbox 磁盘基准字节数，按存储容量换算并预留 150MB。 */
export const getAgentSandboxDiskBytes = () => {
const storageMB = serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI * 1024;
const diskMB = Math.round(storageMB / 2 - RESERVED_DISK_MB);
if (diskMB <= 0) {
throw new Error(
`AGENT_SANDBOX_STORAGE_SIZE_GI must be greater than ${RESERVED_DISK_MB * 2}MiB`
);
}
return diskMB * MB_BYTES;
};
/** 获取 sandbox 冷归档包大小上限，等于磁盘基准。 */
export const getAgentSandboxArchiveMaxBytes = getAgentSandboxDiskBytes;
......@@ -22,9 +31,8 @@ export const getAgentSandboxSuspendMinutes = () => serviceEnv.AGENT_SANDBOX_SUSP
export const getAgentSandboxArchiveInactiveDays = () =>
serviceEnv.AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS;
/** 获取 Skill 包大小上限，按磁盘基准的一半四舍五入。 */
export const getAgentSandboxSkillMaxBytes = () =>
toRoundedMBBytes(serviceEnv.AGENT_SANDBOX_DISK_MB * 0.5);
/** 获取 Skill 包大小上限，等于磁盘基准。 */
export const getAgentSandboxSkillMaxBytes = getAgentSandboxDiskBytes;
/** 获取 IDE 单文件大小上限，复用 Skill 包大小上限。 */
export const getAgentSandboxMaxFileBytes = getAgentSandboxSkillMaxBytes;
/** 获取 IDE 单文件大小上限，等于磁盘基准。 */
export const getAgentSandboxMaxFileBytes = getAgentSandboxDiskBytes;
packages/service/core/ai/sandbox/infrastructure/provider/runtimeProfile/sealosdevbox.ts
......@@ -7,6 +7,7 @@ import { serviceEnv } from '../../../../../../env';
import type { SandboxRuntimeProfile } from './types';
import { getSandboxSkillsRootPath, mergeStringRecord } from './utils';
import { parseImageSpec } from '@fastgpt-sdk/sandbox-adapter';
import { getAgentSandboxDiskBytes } from '../../../config';
/**
* 构建 Sealos Devbox 的 FastGPT 运行态 profile。
......@@ -31,7 +32,9 @@ export function buildSealosRuntimeProfile(): SandboxRuntimeProfile {
throw new Error('AGENT_SANDBOX_SEALOS_IMAGE is required for sealosdevbox provider');
}
const env = mergeStringRecord(createConfig.env, input.env);
const env = mergeStringRecord(mergeStringRecord(createConfig.env, input.env), {
DEVBOX_SDK_MAX_FILE_SIZE: String(getAgentSandboxDiskBytes())
});
const metadata = mergeStringRecord(createConfig.metadata, input.metadata);
const storageLimit = (() => {
if (input.resourceLimits?.storageSize !== undefined) {
......@@ -40,7 +43,7 @@ export function buildSealosRuntimeProfile(): SandboxRuntimeProfile {
if (createConfig.resourceLimits?.storageSize !== undefined) {
return { storageSize: createConfig.resourceLimits.storageSize };
}
return { storageSize: serviceEnv.AGENT_SANDBOX_STORAGE_SIZE };
return { storageSize: `${serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI}Gi` };
})();
const resourceLimits = {
cpuCount:
......
packages/service/core/ai/sandbox/infrastructure/volume/config.ts
......@@ -22,6 +22,6 @@ export function getVolumeManagerEnvConfig(): VolumeManagerConfig {
enable: true,
url: serviceEnv.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL!,
token: serviceEnv.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN,
storageSize: serviceEnv.AGENT_SANDBOX_STORAGE_SIZE
storageSize: `${serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI}Gi`
};
}
packages/service/core/ai/sandbox/interface/config/index.ts
/** 沙盒接口层：暴露 Agent sandbox 对外可用的配置派生值。 */
export {
getAgentSandboxArchiveMaxBytes,
getAgentSandboxDiskBytes,
getAgentSandboxMaxFileBytes,
getAgentSandboxSkillMaxBytes
} from '../../config';
packages/service/core/ai/skill/manage/import.ts
import { AgentSkillSourceEnum, AgentSkillTypeEnum } from '@fastgpt/global/core/ai/skill/constants';
import type { SkillPackageType } from '@fastgpt/global/core/ai/skill/type';
import type {
RuntimeSkillMetadataType,
SkillPackageType
} from '@fastgpt/global/core/ai/skill/type';
import { Types } from '../../../../common/mongo';
import { mongoSessionRun } from '../../../../common/mongo/sessionRun';
import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import {
extractRuntimeSkillsFromPackage,
removeSkillPackageTTL,
uploadSkillPackage
} from '../package';
import { removeSkillPackageTTL, uploadSkillPackageStream } from '../package';
import { MongoAgentSkills } from '../model/schema';
import { createVersion } from '../version';
import { updateCurrentVersion } from './update';
import type { Readable } from 'node:stream';
type ImportSkillParams = {
skill: SkillPackageType['skill'];
teamId: string;
tmbId: string;
packageStream: Readable;
contentLength?: number;
parentId?: string | null;
};
/**
* Import skill from a validated package.
* Import an opaque package stream as the initial Skill version.
*
* Caller owns archive parsing. This function uploads the package before opening the Mongo
* transaction, then binds currentVersionId and removes the temporary S3 TTL inside the
* transaction. If Mongo fails, the uploaded package keeps its TTL and is cleaned by the shared S3
* cleanup flow.
* The package is stored as the initial version and linked as currentVersionId.
* Import intentionally does not parse or validate package contents. This function uploads the
* package before opening the Mongo transaction, then binds currentVersionId and removes the
* temporary S3 TTL inside the transaction. If Mongo fails, the uploaded package keeps its TTL and
* is cleaned by the shared S3 cleanup flow.
* Runtime metadata remains empty until the workspace is saved and deployed from the edit sandbox.
*/
export async function importSkill(
packageData: SkillPackageType,
teamId: string,
tmbId: string,
zipBuffer: Buffer,
parentId?: string | null
): Promise<string> {
const { skill } = packageData;
const runtimeSkills = await extractRuntimeSkillsFromPackage(zipBuffer);
export async function importSkill({
skill,
teamId,
tmbId,
packageStream,
contentLength,
parentId
}: ImportSkillParams): Promise<string> {
const runtimeSkills: RuntimeSkillMetadataType[] = [];
const newSkill = new MongoAgentSkills({
parentId: parentId || null,
parentId: parentId ?? null,
type: AgentSkillTypeEnum.skill,
source: AgentSkillSourceEnum.personal,
name: skill.name,
......@@ -48,11 +56,12 @@ export async function importSkill(
const newSkillId = newSkill._id.toString();
const versionId = new Types.ObjectId().toString();
const storageInfo = await uploadSkillPackage({
const storageInfo = await uploadSkillPackageStream({
teamId,
skillId: newSkillId,
packageObjectId: versionId,
zipBuffer
packageStream,
contentLength
});
return mongoSessionRun(async (session) => {
......
packages/service/core/ai/skill/package/storage.ts
......@@ -10,6 +10,8 @@ import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import type { ClientSession } from '../../../../common/mongo';
import { getAgentSandboxSkillMaxBytes } from '../../sandbox/interface/config';
import { readStreamToBuffer } from '../../../../common/s3/utils';
import type { Readable } from 'node:stream';
import { createSizeLimitedStream } from '../../../../common/file/stream';
export type SkillStorageInfo = {
key: string;
......@@ -26,6 +28,11 @@ export type DownloadSkillPackageParams = {
storageKey: string;
};
type UploadSkillPackageStreamParams = Omit<UploadSkillPackageParams, 'zipBuffer'> & {
packageStream: Readable;
contentLength?: number;
};
/**
* 上传 Skill ZIP 包到私有对象存储。
*
......@@ -54,6 +61,35 @@ export async function uploadSkillPackage(
}
/**
* 将 Skill ZIP 流直接上传到私有对象存储。
*
* contentLength 仅透传对象存储 SDK；实际传输仍逐块计数，防止伪造长度绕过限制。
*/
export async function uploadSkillPackageStream(
params: UploadSkillPackageStreamParams
): Promise<SkillStorageInfo> {
const { teamId, skillId, packageObjectId, packageStream, contentLength } = params;
const maxBytes = getAgentSandboxSkillMaxBytes();
const boundedStream = createSizeLimitedStream({
stream: packageStream,
maxBytes,
createExceededError: () => new Error(SkillErrEnum.archiveTooLarge)
});
const bucket = getS3SkillSource();
const { key } = await bucket.uploadPackage({
teamId,
skillId,
packageObjectId,
body: boundedStream,
contentLength
});
return { key };
}
/**
* 移除 Skill 包的临时 S3 TTL。
*
* uploadSkillPackage 通过 S3 bucket 封装上传，默认会写入一条 TTL 记录。只有当版本记录和
......@@ -94,6 +130,32 @@ export async function downloadSkillPackage(params: DownloadSkillPackageParams):
}
/**
* 从私有对象存储返回带实际字节上限的 Skill ZIP 流。
*
* 上限在消费流时逐块执行，避免对象元数据缺失或被错误填写时把超大包继续传入 Sandbox。
*/
export async function downloadSkillPackageStream(
params: DownloadSkillPackageParams
): Promise<Readable> {
const { storageKey } = params;
const maxBytes = getAgentSandboxSkillMaxBytes();
const response = await getS3SkillSource().client.downloadObject({
key: storageKey
});
if (!response.body) {
throw new Error(`Failed to download skill package: ${storageKey}`);
}
return createSizeLimitedStream({
stream: response.body,
maxBytes,
createExceededError: () =>
new Error(`Skill package exceeds maximum allowed size (${maxBytes / 1024 / 1024}MB)`)
});
}
/**
* 删除单个版本的 Skill ZIP 包。
*/
export async function deleteSkillPackage(storageKey: string): Promise<void> {
......
packages/service/env.ts
......@@ -103,8 +103,8 @@ export const serviceEnv = createEnv({
AGENT_SANDBOX_MEMORY_MIB: IntSchema.min(1).default(2048).meta({
description: 'Agent Sandbox 实例的内存上限（MiB）'
}),
AGENT_SANDBOX_STORAGE_SIZE: z.string().trim().min(1).default('1Gi').meta({
description: 'Agent Sandbox 存储容量（Kubernetes 资源量格式）'
AGENT_SANDBOX_STORAGE_SIZE_GI: NumSchema.min(1).default(1).meta({
description: 'Agent Sandbox 存储容量，单位 Gi'
}),
// OpenSandbox配置
AGENT_SANDBOX_OPENSANDBOX_BASEURL: UrlSchema.optional(),
......@@ -121,10 +121,6 @@ export const serviceEnv = createEnv({
'Set to true when the OpenSandbox server uses a user-defined Docker network (not bridge), ' +
'as networkPolicy is only supported with network_mode=bridge.'
}),
AGENT_SANDBOX_DISK_MB: NumSchema.min(1).default(1024).meta({
description:
'Agent sandbox 磁盘大小基准（MB）。冷归档包上限等于该值，Skill 包和 IDE 单文件上限按该值的一半四舍五入计算。'
}),
AGENT_SANDBOX_SUSPEND_MINUTES: IntSchema.min(1).default(60).meta({
description: 'Agent sandbox 持续未活跃多少分钟后自动暂停'
}),
......
packages/service/test/core/ai/sandbox/application/runtime/client.integration.test.ts
......@@ -43,8 +43,7 @@ const runFullIntegration = process.env.SANDBOX_INTEGRATION_FULL === 'true';
vi.mock('@fastgpt/service/env', () => ({
...(() => {
const envBool = (value: string | undefined) => value === 'true';
const getAgentSandboxDiskMB = () => Number(process.env.AGENT_SANDBOX_DISK_MB ?? 1024);
const agentSandboxDiskMB = getAgentSandboxDiskMB();
const agentSandboxStorageSize = Number(process.env.AGENT_SANDBOX_STORAGE_SIZE_GI ?? 1);
return {
serviceEnv: {
......@@ -65,7 +64,7 @@ vi.mock('@fastgpt/service/env', () => ({
process.env.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL,
AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN:
process.env.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN,
AGENT_SANDBOX_DISK_MB: agentSandboxDiskMB
AGENT_SANDBOX_STORAGE_SIZE_GI: agentSandboxStorageSize
}
};
})()
......
packages/service/test/core/ai/sandbox/application/runtime/client.test.ts
......@@ -3,7 +3,7 @@ import { SandboxNotFoundError } from '@fastgpt-sdk/sandbox-adapter';
import { beforeEach, describe, expect, it, vi } from 'vitest';
vi.mock('@fastgpt/service/env', () => ({
serviceEnv: { AGENT_SANDBOX_PROVIDER: 'sealosdevbox', AGENT_SANDBOX_DISK_MB: 20 }
serviceEnv: { AGENT_SANDBOX_PROVIDER: 'sealosdevbox', AGENT_SANDBOX_STORAGE_SIZE_GI: 1 }
}));
const mocks = vi.hoisted(() => ({
......
packages/service/test/core/ai/sandbox/application/runtime/skill/core.test.ts
......@@ -232,11 +232,13 @@ description: Zeta skill
skillDirectories: deployedSkillVersions.map(({ targetDir }) => targetDir)
});
expect(sandbox.writeFiles).toHaveBeenCalledTimes(1);
expect(sandbox.writeFiles).toHaveBeenCalledTimes(2);
expect(skill1TargetDir).not.toBe(skill2TargetDir);
const writtenFilePaths = sandbox.writeFiles.mock.calls[0][0].map(
(entry: { path: string }) => entry.path
);
const writeEntries = sandbox.writeFiles.mock.calls.flatMap(([entries]) => entries);
const writtenFilePaths = writeEntries.map((entry: { path: string }) => entry.path);
expect(
writeEntries.every((entry: { data: unknown }) => entry.data instanceof ReadableStream)
).toBe(true);
expect(writtenFilePaths).toEqual(
expect.arrayContaining([
expect.stringContaining(`/workspace/projects/.tmp-${String(skill1VersionId)}`),
......@@ -431,10 +433,10 @@ description: Missing skill
skillDirectories: deployedSkillVersions.map(({ targetDir }) => targetDir)
});
expect(sandbox.writeFiles).toHaveBeenCalledTimes(1);
const writtenFilePaths = sandbox.writeFiles.mock.calls[0][0].map(
(entry: { path: string }) => entry.path
);
expect(sandbox.writeFiles).toHaveBeenCalledTimes(2);
const writtenFilePaths = sandbox.writeFiles.mock.calls
.flatMap(([entries]) => entries)
.map((entry: { path: string }) => entry.path);
expect(writtenFilePaths).toEqual(
expect.arrayContaining([
expect.stringContaining(`/workspace/projects/.tmp-${String(existingSkillVersionId)}`),
......@@ -889,7 +891,7 @@ description: Latest current skill
tmbId,
workDirectory: '/workspace'
})
).rejects.toThrow('Failed to write skill ZIP packages: write failed');
).rejects.toThrow('Failed to write skill ZIP package: write failed');
expect(sandbox.execute).not.toHaveBeenCalledWith(`rm -rf '${skillTargetDir}'`);
expect(sandbox.readFiles).not.toHaveBeenCalled();
});
......
packages/service/test/core/ai/sandbox/application/runtime/skill/prepare.test.ts 0 → 100644
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { Readable } from 'node:stream';
const mocks = vi.hoisted(() => ({
downloadSkillPackageStream: vi.fn()
}));
vi.mock('@fastgpt/service/core/ai/skill/package', async (importOriginal) => ({
...(await importOriginal<typeof import('@fastgpt/service/core/ai/skill/package')>()),
downloadSkillPackageStream: mocks.downloadSkillPackageStream
}));
import { deploySkillPackage } from '@fastgpt/service/core/ai/sandbox/application/runtime/skill/prepare';
const successResult = {
exitCode: 0,
stdout: '',
stderr: ''
};
describe('skill package streaming prepare', () => {
beforeEach(() => {
vi.clearAllMocks();
});
it('streams the S3 package into sandbox.writeFiles', async () => {
const source = Readable.from([Buffer.from('chunk-1'), Buffer.from('chunk-2')]);
mocks.downloadSkillPackageStream.mockResolvedValue(source);
let uploaded = Buffer.alloc(0);
const sandbox = {
execute: vi.fn().mockResolvedValue(successResult),
writeFiles: vi.fn(async ([entry]: Array<{ path: string; data: ReadableStream }>) => {
expect(entry.path).toBe('/workspace/skills/package.zip');
expect(entry.data).toBeInstanceOf(ReadableStream);
const chunks: Buffer[] = [];
for await (const chunk of entry.data) {
chunks.push(Buffer.from(chunk));
}
uploaded = Buffer.concat(chunks);
return [{ path: entry.path, bytesWritten: uploaded.length, error: null }];
})
};
const baseContext = {
sandbox: sandbox as any,
workDirectory: '/workspace'
};
await deploySkillPackage({
storageKey: 'version.zip',
skillsRootPath: '/workspace/skills'
})(baseContext);
expect(mocks.downloadSkillPackageStream).toHaveBeenCalledWith({ storageKey: 'version.zip' });
expect(uploaded.toString()).toBe('chunk-1chunk-2');
expect(source.destroyed).toBe(true);
expect(sandbox.execute.mock.calls.at(-1)?.[0]).toContain('unzip -Z1');
});
it('destroys an unconsumed S3 stream when sandbox upload fails', async () => {
const source = Readable.from([Buffer.from('content')]);
mocks.downloadSkillPackageStream.mockResolvedValue(source);
const sandbox = {
execute: vi.fn().mockResolvedValue(successResult),
writeFiles: vi.fn().mockResolvedValue([
{
path: '/workspace/skills/package.zip',
bytesWritten: 0,
error: new Error('upload failed')
}
])
};
await expect(
deploySkillPackage({ storageKey: 'version.zip', skillsRootPath: '/workspace/skills' })({
sandbox: sandbox as any,
workDirectory: '/workspace'
})
).rejects.toThrow('Failed to write skill package ZIP');
expect(source.destroyed).toBe(true);
});
});
packages/service/test/core/ai/sandbox/application/skillEdit/runtime.test.ts
......@@ -21,8 +21,7 @@ const mocks = vi.hoisted(() => ({
preparePackageMirrors: vi.fn(),
prepareWorkDirectory: vi.fn(),
emptyWorkDirectory: vi.fn(),
downloadSkillPackageToContext: vi.fn(),
deployDownloadedSkillPackage: vi.fn()
deploySkillPackage: vi.fn()
}));
vi.mock('@fastgpt/service/core/ai/skill/model/schema', () => ({
......@@ -112,13 +111,12 @@ vi.mock('@fastgpt/service/core/ai/sandbox/application/runtime/prepare', () => ({
}));
vi.mock('@fastgpt/service/core/ai/sandbox/application/runtime/skill/prepare', () => ({
downloadSkillPackageToContext: mocks.downloadSkillPackageToContext,
deployDownloadedSkillPackage: mocks.deployDownloadedSkillPackage
deploySkillPackage: mocks.deploySkillPackage
}));
vi.mock('@fastgpt/service/env', () => ({
serviceEnv: {
AGENT_SANDBOX_DISK_MB: 2,
AGENT_SANDBOX_STORAGE_SIZE_GI: 1,
AGENT_SANDBOX_MAX_EDIT_DEBUG: undefined
}
}));
......@@ -298,8 +296,7 @@ describe('skill edit runtime initialization', () => {
mocks.preparePackageMirrors.mockReturnValue({ step: 'mirrors' });
mocks.prepareWorkDirectory.mockReturnValue({ step: 'workdir' });
mocks.emptyWorkDirectory.mockReturnValue({ step: 'empty' });
mocks.downloadSkillPackageToContext.mockReturnValue({ step: 'download' });
mocks.deployDownloadedSkillPackage.mockReturnValue({ step: 'deploy' });
mocks.deploySkillPackage.mockReturnValue({ step: 'deploy' });
mocks.updateSandboxInstanceRecordBySandboxId.mockResolvedValue(createResource());
mocks.countRunningSandboxInstancesBySourceType.mockResolvedValue(0);
mocks.disconnectSandbox.mockResolvedValue(undefined);
......
packages/service/test/core/ai/sandbox/config.test.ts
......@@ -3,32 +3,37 @@ import { serviceEnv } from '@fastgpt/service/env';
import {
getAgentSandboxArchiveInactiveDays,
getAgentSandboxArchiveMaxBytes,
getAgentSandboxDiskBytes,
getAgentSandboxMaxFileBytes,
getAgentSandboxSkillMaxBytes,
getAgentSandboxSuspendMinutes
} from '@fastgpt/service/core/ai/sandbox/config';
describe('agent sandbox config', () => {
const originalAgentSandboxDiskMB = serviceEnv.AGENT_SANDBOX_DISK_MB;
const originalAgentSandboxStorageSize = serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI;
const originalAgentSandboxSuspendMinutes = serviceEnv.AGENT_SANDBOX_SUSPEND_MINUTES;
const originalAgentSandboxArchiveInactiveDays = serviceEnv.AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS;
afterEach(() => {
serviceEnv.AGENT_SANDBOX_DISK_MB = originalAgentSandboxDiskMB;
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = originalAgentSandboxStorageSize;
serviceEnv.AGENT_SANDBOX_SUSPEND_MINUTES = originalAgentSandboxSuspendMinutes;
serviceEnv.AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS = originalAgentSandboxArchiveInactiveDays;
});
it('derives size limits from AGENT_SANDBOX_DISK_MB', () => {
serviceEnv.AGENT_SANDBOX_DISK_MB = 1024;
expect(getAgentSandboxArchiveMaxBytes()).toBe(1024 * 1024 * 1024);
expect(getAgentSandboxSkillMaxBytes()).toBe(512 * 1024 * 1024);
expect(getAgentSandboxMaxFileBytes()).toBe(512 * 1024 * 1024);
it('derives all size limits from AGENT_SANDBOX_STORAGE_SIZE_GI', () => {
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = 1;
expect(getAgentSandboxArchiveMaxBytes()).toBe(362 * 1024 * 1024);
expect(getAgentSandboxDiskBytes()).toBe(362 * 1024 * 1024);
expect(getAgentSandboxSkillMaxBytes()).toBe(362 * 1024 * 1024);
expect(getAgentSandboxMaxFileBytes()).toBe(362 * 1024 * 1024);
serviceEnv.AGENT_SANDBOX_DISK_MB = 333;
expect(getAgentSandboxArchiveMaxBytes()).toBe(333 * 1024 * 1024);
expect(getAgentSandboxSkillMaxBytes()).toBe(167 * 1024 * 1024);
expect(getAgentSandboxMaxFileBytes()).toBe(167 * 1024 * 1024);
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = 2;
expect(getAgentSandboxDiskBytes()).toBe(874 * 1024 * 1024);
});
it('rejects storage sizes without enough space for the reserved system capacity', () => {
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = 0.29;
expect(() => getAgentSandboxDiskBytes()).toThrow('AGENT_SANDBOX_STORAGE_SIZE_GI');
});
it('reads lifecycle thresholds from service env', () => {
......
packages/service/test/core/ai/sandbox/infrastructure/provider/adapter.test.ts
......@@ -17,7 +17,7 @@ vi.mock('@fastgpt/service/env', () => ({
AGENT_SANDBOX_OPENSANDBOX_IMAGE_TAG: 'test',
AGENT_SANDBOX_SEALOS_BASEURL: 'http://mock-sealos.local',
AGENT_SANDBOX_SEALOS_TOKEN: 'mock-sealos-token',
AGENT_SANDBOX_DISK_MB: 20
AGENT_SANDBOX_STORAGE_SIZE_GI: 1
}
}));
......
packages/service/test/core/ai/sandbox/infrastructure/provider/config.test.ts
......@@ -13,12 +13,11 @@ const originalEnv = {
AGENT_SANDBOX_OPENSANDBOX_IMAGE_TAG: process.env.AGENT_SANDBOX_OPENSANDBOX_IMAGE_TAG,
AGENT_SANDBOX_CPU_COUNT: process.env.AGENT_SANDBOX_CPU_COUNT,
AGENT_SANDBOX_MEMORY_MIB: process.env.AGENT_SANDBOX_MEMORY_MIB,
AGENT_SANDBOX_STORAGE_SIZE: process.env.AGENT_SANDBOX_STORAGE_SIZE,
AGENT_SANDBOX_STORAGE_SIZE_GI: process.env.AGENT_SANDBOX_STORAGE_SIZE_GI,
AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL:
process.env.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL,
AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN:
process.env.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN,
AGENT_SANDBOX_DISK_MB: process.env.AGENT_SANDBOX_DISK_MB,
AGENT_SANDBOX_PROXY_SECRET: process.env.AGENT_SANDBOX_PROXY_SECRET,
AGENT_SANDBOX_PROXY_URL: process.env.AGENT_SANDBOX_PROXY_URL,
AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES: process.env.AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES,
......@@ -73,7 +72,7 @@ describe('sandbox provider config', () => {
);
vi.stubEnv('AGENT_SANDBOX_CPU_COUNT', originalEnv.AGENT_SANDBOX_CPU_COUNT);
vi.stubEnv('AGENT_SANDBOX_MEMORY_MIB', originalEnv.AGENT_SANDBOX_MEMORY_MIB);
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE', originalEnv.AGENT_SANDBOX_STORAGE_SIZE);
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE_GI', originalEnv.AGENT_SANDBOX_STORAGE_SIZE_GI);
vi.stubEnv(
'AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL',
originalEnv.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL
......@@ -82,7 +81,6 @@ describe('sandbox provider config', () => {
'AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN',
originalEnv.AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN
);
vi.stubEnv('AGENT_SANDBOX_DISK_MB', originalEnv.AGENT_SANDBOX_DISK_MB);
vi.stubEnv(
'AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES',
originalEnv.AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES
......@@ -129,7 +127,7 @@ describe('sandbox provider config', () => {
vi.stubEnv('AGENT_SANDBOX_SEALOS_IMAGE', 'default-sealos-image:latest');
vi.stubEnv('AGENT_SANDBOX_CPU_COUNT', '2');
vi.stubEnv('AGENT_SANDBOX_MEMORY_MIB', '4096');
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE', '5G');
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE_GI', '5');
vi.stubEnv('AGENT_SANDBOX_WS_MAX_MESSAGE_BYTES', '67108864');
vi.stubEnv('AGENT_SANDBOX_WS_MAX_FRAME_BYTES', '16777216');
......@@ -152,14 +150,15 @@ describe('sandbox provider config', () => {
repository: 'default-sealos-image',
tag: 'latest'
},
resourceLimits: { cpuCount: 2, memoryMiB: 4096, storageSize: '5G' },
resourceLimits: { cpuCount: 2, memoryMiB: 4096, storageSize: '5Gi' },
workingDir: '/home/devbox/workspace',
upstreamID: 'session-1',
env: {
FASTGPT_SESSION_ID: 'session-1',
FASTGPT_WORKDIR: '/home/devbox/workspace',
IDE_AGENT_ENABLED: 'true',
FASTGPT_IDE_MAX_FILE_BYTES: '536870912',
DEVBOX_SDK_MAX_FILE_SIZE: '2527068160',
FASTGPT_IDE_MAX_FILE_BYTES: '2527068160',
FASTGPT_IDE_WS_MAX_MESSAGE_BYTES: '67108864',
FASTGPT_IDE_WS_MAX_FRAME_BYTES: '16777216'
}
......@@ -186,7 +185,7 @@ describe('sandbox provider config', () => {
AGENT_SANDBOX_PROVIDER: 'sealosdevbox',
AGENT_SANDBOX_SEALOS_BASEURL: undefined,
AGENT_SANDBOX_SEALOS_TOKEN: undefined,
AGENT_SANDBOX_DISK_MB: 20
AGENT_SANDBOX_STORAGE_SIZE_GI: 1
}
}));
......@@ -386,7 +385,7 @@ describe('sandbox provider config', () => {
AGENT_SANDBOX_OPENSANDBOX_USE_SERVER_PROXY: true,
AGENT_SANDBOX_CPU_COUNT: 1,
AGENT_SANDBOX_MEMORY_MIB: 2048,
AGENT_SANDBOX_DISK_MB: 20
AGENT_SANDBOX_STORAGE_SIZE_GI: 1
}
}));
......
packages/service/test/core/ai/sandbox/infrastructure/provider/runtimeProfile.test.ts
......@@ -5,7 +5,8 @@ const originalEnv = {
AGENT_SANDBOX_OPENSANDBOX_IMAGE_REPO: process.env.AGENT_SANDBOX_OPENSANDBOX_IMAGE_REPO,
AGENT_SANDBOX_OPENSANDBOX_IMAGE_TAG: process.env.AGENT_SANDBOX_OPENSANDBOX_IMAGE_TAG,
AGENT_SANDBOX_SEALOS_WORK_DIRECTORY: process.env.AGENT_SANDBOX_SEALOS_WORK_DIRECTORY,
AGENT_SANDBOX_SEALOS_IMAGE: process.env.AGENT_SANDBOX_SEALOS_IMAGE
AGENT_SANDBOX_SEALOS_IMAGE: process.env.AGENT_SANDBOX_SEALOS_IMAGE,
AGENT_SANDBOX_STORAGE_SIZE_GI: process.env.AGENT_SANDBOX_STORAGE_SIZE_GI
};
const loadSandboxRuntimeProfileModule = async () => {
......@@ -29,6 +30,7 @@ describe('sandbox runtime profile', () => {
originalEnv.AGENT_SANDBOX_SEALOS_WORK_DIRECTORY
);
vi.stubEnv('AGENT_SANDBOX_SEALOS_IMAGE', originalEnv.AGENT_SANDBOX_SEALOS_IMAGE);
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE_GI', originalEnv.AGENT_SANDBOX_STORAGE_SIZE_GI);
});
it('uses fixed /workspace as opensandbox work directory', async () => {
......@@ -74,6 +76,7 @@ describe('sandbox runtime profile', () => {
vi.stubEnv('AGENT_SANDBOX_PROVIDER', '');
vi.stubEnv('AGENT_SANDBOX_SEALOS_WORK_DIRECTORY', '/custom/devbox/workspace');
vi.stubEnv('AGENT_SANDBOX_SEALOS_IMAGE', 'runtime/fastgpt:stable');
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE_GI', '1');
const { getSandboxRuntimeProfile } = await loadSandboxRuntimeProfileModule();
......@@ -117,6 +120,7 @@ describe('sandbox runtime profile', () => {
FASTGPT_SESSION_ID: 'session-1',
FASTGPT_WORKDIR: '/custom/devbox/workspace',
IDE_AGENT_ENABLED: 'true',
DEVBOX_SDK_MAX_FILE_SIZE: '379584512',
FASTGPT_IDE_MAX_FILE_BYTES: '10485760',
FASTGPT_IDE_WS_MAX_MESSAGE_BYTES: '67108864',
FASTGPT_IDE_WS_MAX_FRAME_BYTES: '16777216'
......
packages/service/test/core/ai/sandbox/infrastructure/volume/config.test.ts
......@@ -16,7 +16,7 @@ describe('sandbox volume config', () => {
serviceEnv: {
AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL: 'http://volume-manager.local',
AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN: 'volume-token',
AGENT_SANDBOX_STORAGE_SIZE: '5Gi'
AGENT_SANDBOX_STORAGE_SIZE_GI: 5
}
}));
......
packages/service/test/core/ai/skill/controller.test.ts
import { describe, expect, it, beforeAll, afterAll, beforeEach } from 'vitest';
import JSZip from 'jszip';
import { Types } from '@fastgpt/service/common/mongo';
import { MongoAgentSkills } from '@fastgpt/service/core/ai/skill/model/schema';
import {
......@@ -17,6 +16,7 @@ import {
AgentSkillSourceEnum,
AgentSkillCategoryEnum
} from '@fastgpt/global/core/ai/skill/constants';
import { Readable } from 'node:stream';
describe('AgentSkill Controller', () => {
let testTeamId: string;
......@@ -341,56 +341,56 @@ describe('AgentSkill Controller', () => {
// ==================== Import Skill ====================
describe('importSkill', () => {
const createImportZipBuffer = async (skillName: string) => {
const zip = new JSZip();
zip.file('skills/imported/SKILL.md', `---\nname: ${skillName}\n---\n`);
return zip.generateAsync({ type: 'nodebuffer' });
};
const packageContent = Buffer.from('opaque package content');
it('should import skill from package', async () => {
const packageData = {
skill: {
name: 'Imported Skill',
description: 'An imported skill',
category: [AgentSkillCategoryEnum.tool]
}
const skillData = {
name: 'Imported Skill',
description: 'An imported skill',
category: [AgentSkillCategoryEnum.tool]
};
const mockZipBuffer = await createImportZipBuffer('imported-skill');
const skillId = await importSkill(packageData, testTeamId, testTmbId, mockZipBuffer);
const skillId = await importSkill({
skill: skillData,
teamId: testTeamId,
tmbId: testTmbId,
packageStream: Readable.from(packageContent),
contentLength: packageContent.length
});
expect(skillId).toBeDefined();
const skill = await MongoAgentSkills.findById(skillId);
expect(skill?.name).toBe(packageData.skill.name);
expect(skill?.description).toBe(packageData.skill.description);
expect(skill?.name).toBe(skillData.name);
expect(skill?.description).toBe(skillData.description);
expect(skill?.source).toBe(AgentSkillSourceEnum.personal);
expect(skill?.currentRuntimeSkills.map((item) => item.toObject())).toEqual([
{
name: 'imported-skill',
description: '',
path: 'skills/imported/SKILL.md'
}
]);
expect(skill?.currentRuntimeSkills).toHaveLength(0);
});
it('should allow importing duplicate name without error', async () => {
const packageData = {
skill: {
name: 'Duplicate Import',
description: 'A skill',
category: []
}
const skillData = {
name: 'Duplicate Import',
description: 'A skill',
category: []
};
const mockZipBuffer = await createImportZipBuffer('duplicate-import');
// First import
const firstSkillId = await importSkill(packageData, testTeamId, testTmbId, mockZipBuffer);
const firstSkillId = await importSkill({
skill: skillData,
teamId: testTeamId,
tmbId: testTmbId,
packageStream: Readable.from(packageContent),
contentLength: packageContent.length
});
// Second import should succeed with a different ID
const secondSkillId = await importSkill(packageData, testTeamId, testTmbId, mockZipBuffer);
const secondSkillId = await importSkill({
skill: skillData,
teamId: testTeamId,
tmbId: testTmbId,
packageStream: Readable.from(packageContent),
contentLength: packageContent.length
});
expect(firstSkillId).toBeDefined();
expect(secondSkillId).toBeDefined();
......
packages/service/test/core/ai/skill/storage.test.ts
import { describe, expect, it, vi, beforeEach } from 'vitest';
import {
uploadSkillPackage,
uploadSkillPackageStream,
downloadSkillPackage,
downloadSkillPackageStream,
deleteSkillPackage
} from '@fastgpt/service/core/ai/skill/package';
import { getS3SkillSource } from '@fastgpt/service/common/s3/sources/skill';
import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import { serviceEnv } from '@fastgpt/service/env';
import { Readable } from 'node:stream';
const s3SkillSourceMocks = vi.hoisted(() => {
const getSkillPackageKey = ({
......@@ -65,6 +68,14 @@ describe('storage', () => {
vi.clearAllMocks();
});
const consumeStream = async (stream: Readable) => {
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
}
return Buffer.concat(chunks);
};
// ==================== uploadSkillPackage ====================
describe('uploadSkillPackage', () => {
it('should upload skill package successfully', async () => {
......@@ -87,11 +98,11 @@ describe('storage', () => {
});
it('should reject zip buffers larger than the upload limit before uploading to S3', async () => {
const originalAgentSandboxDiskMB = serviceEnv.AGENT_SANDBOX_DISK_MB;
serviceEnv.AGENT_SANDBOX_DISK_MB = 2;
const originalAgentSandboxStorageSize = serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI;
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = 0.296875;
try {
const tooLargeBuffer = Buffer.alloc(1024 * 1024 + 1);
const tooLargeBuffer = Buffer.alloc(2 * 1024 * 1024 + 1);
await expect(
uploadSkillPackage({
......@@ -105,7 +116,60 @@ describe('storage', () => {
expect(getS3SkillSource).not.toHaveBeenCalled();
expect(s3SkillSourceMocks.uploadPackageMock).not.toHaveBeenCalled();
} finally {
serviceEnv.AGENT_SANDBOX_DISK_MB = originalAgentSandboxDiskMB;
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = originalAgentSandboxStorageSize;
}
});
});
describe('uploadSkillPackageStream', () => {
it('streams chunks to S3 without joining them into a package buffer', async () => {
let uploadedBody: Buffer | undefined;
s3SkillSourceMocks.uploadPackageMock.mockImplementationOnce(async (params) => {
expect(params.body).toBeInstanceOf(Readable);
uploadedBody = await consumeStream(params.body);
return {
key: `agent-skills/${params.teamId}/${params.skillId}/${params.packageObjectId}.zip`
};
});
const packageStream = Readable.from([Buffer.from('chunk-1'), Buffer.from('chunk-2')]);
const result = await uploadSkillPackageStream({
teamId: mockTeamId,
skillId: mockSkillId,
packageObjectId: mockVersionId,
packageStream,
contentLength: 14
});
expect(result.key).toBe(`agent-skills/${mockTeamId}/${mockSkillId}/${mockVersionId}.zip`);
expect(uploadedBody?.toString()).toBe('chunk-1chunk-2');
expect(s3SkillSourceMocks.uploadPackageMock).toHaveBeenCalledWith(
expect.objectContaining({
contentLength: 14,
body: expect.any(Readable)
})
);
});
it('aborts a stream whose actual bytes exceed the package limit', async () => {
const originalAgentSandboxStorageSize = serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI;
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = 0.296875;
s3SkillSourceMocks.uploadPackageMock.mockImplementationOnce(async (params) => {
await consumeStream(params.body);
return { key: 'should-not-complete' };
});
try {
await expect(
uploadSkillPackageStream({
teamId: mockTeamId,
skillId: mockSkillId,
packageObjectId: mockVersionId,
packageStream: Readable.from([Buffer.alloc(2 * 1024 * 1024), Buffer.from('x')])
})
).rejects.toThrow(SkillErrEnum.archiveTooLarge);
} finally {
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = originalAgentSandboxStorageSize;
}
});
});
......@@ -134,6 +198,34 @@ describe('storage', () => {
});
});
describe('downloadSkillPackageStream', () => {
it('returns the S3 object as a bounded Readable', async () => {
const storageKey = `agent-skills/${mockTeamId}/${mockSkillId}/${mockVersionId}.zip`;
const stream = await downloadSkillPackageStream({ storageKey });
expect(stream).toBeInstanceOf(Readable);
await expect(consumeStream(stream)).resolves.toEqual(Buffer.from('mock zip content'));
});
it('fails while consuming an object larger than the package limit', async () => {
const originalAgentSandboxStorageSize = serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI;
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = 0.296875;
s3SkillSourceMocks.downloadObjectMock.mockResolvedValueOnce({
body: Readable.from([Buffer.alloc(2 * 1024 * 1024), Buffer.from('x')])
});
try {
const stream = await downloadSkillPackageStream({ storageKey: 'too-large.zip' });
await expect(consumeStream(stream)).rejects.toThrow(
'Skill package exceeds maximum allowed size'
);
} finally {
serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI = originalAgentSandboxStorageSize;
}
});
});
// ==================== deleteSkillPackage ====================
describe('deleteSkillPackage', () => {
it('should delete skill package successfully', async () => {
......
packages/service/test/env.test.ts
......@@ -4,10 +4,9 @@ const validInvokeTokenSecret = 'fastgpt_test_invoke_token_secret_32';
const originalEnv = {
SYSTEM_MAX_STRING_LENGTH_M: process.env.SYSTEM_MAX_STRING_LENGTH_M,
AGENT_SANDBOX_DISK_MB: process.env.AGENT_SANDBOX_DISK_MB,
AGENT_SANDBOX_CPU_COUNT: process.env.AGENT_SANDBOX_CPU_COUNT,
AGENT_SANDBOX_MEMORY_MIB: process.env.AGENT_SANDBOX_MEMORY_MIB,
AGENT_SANDBOX_STORAGE_SIZE: process.env.AGENT_SANDBOX_STORAGE_SIZE,
AGENT_SANDBOX_STORAGE_SIZE_GI: process.env.AGENT_SANDBOX_STORAGE_SIZE_GI,
FE_DOMAIN: process.env.FE_DOMAIN,
AGENT_SANDBOX_SUSPEND_MINUTES: process.env.AGENT_SANDBOX_SUSPEND_MINUTES,
AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS: process.env.AGENT_SANDBOX_ARCHIVE_INACTIVE_DAYS,
......@@ -39,10 +38,9 @@ const importServiceEnv = async () => {
describe('serviceEnv', () => {
afterEach(() => {
vi.stubEnv('SYSTEM_MAX_STRING_LENGTH_M', originalEnv.SYSTEM_MAX_STRING_LENGTH_M);
vi.stubEnv('AGENT_SANDBOX_DISK_MB', originalEnv.AGENT_SANDBOX_DISK_MB);
vi.stubEnv('AGENT_SANDBOX_CPU_COUNT', originalEnv.AGENT_SANDBOX_CPU_COUNT);
vi.stubEnv('AGENT_SANDBOX_MEMORY_MIB', originalEnv.AGENT_SANDBOX_MEMORY_MIB);
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE', originalEnv.AGENT_SANDBOX_STORAGE_SIZE);
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE_GI', originalEnv.AGENT_SANDBOX_STORAGE_SIZE_GI);
vi.stubEnv('FE_DOMAIN', originalEnv.FE_DOMAIN);
vi.stubEnv('AGENT_SANDBOX_SUSPEND_MINUTES', originalEnv.AGENT_SANDBOX_SUSPEND_MINUTES);
vi.stubEnv(
......@@ -257,20 +255,6 @@ describe('serviceEnv', () => {
});
});
it('validates AGENT_SANDBOX_DISK_MB during service env init', async () => {
vi.stubEnv('FILE_TOKEN_KEY', 'filetokenkey');
vi.stubEnv('AES256_SECRET_KEY', 'fastgptsecret');
vi.stubEnv('INVOKE_TOKEN_SECRET', validInvokeTokenSecret);
vi.stubEnv('AGENT_SANDBOX_DISK_MB', undefined);
const defaultEnv = await importServiceEnv();
expect(defaultEnv.serviceEnv.AGENT_SANDBOX_DISK_MB).toBe(1024);
vi.stubEnv('AGENT_SANDBOX_DISK_MB', '333');
const customEnv = await importServiceEnv();
expect(customEnv.serviceEnv.AGENT_SANDBOX_DISK_MB).toBe(333);
});
it('validates shared Agent Sandbox resource limits during service env init', async () => {
vi.stubEnv('FILE_TOKEN_KEY', 'filetokenkey');
vi.stubEnv('AES256_SECRET_KEY', 'fastgptsecret');
......@@ -278,19 +262,19 @@ describe('serviceEnv', () => {
vi.stubEnv('AGENT_SANDBOX_CPU_COUNT', undefined);
vi.stubEnv('AGENT_SANDBOX_MEMORY_MIB', undefined);
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE', undefined);
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE_GI', undefined);
const defaultEnv = await importServiceEnv();
expect(defaultEnv.serviceEnv.AGENT_SANDBOX_CPU_COUNT).toBe(1);
expect(defaultEnv.serviceEnv.AGENT_SANDBOX_MEMORY_MIB).toBe(2048);
expect(defaultEnv.serviceEnv.AGENT_SANDBOX_STORAGE_SIZE).toBe('1Gi');
expect(defaultEnv.serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI).toBe(1);
vi.stubEnv('AGENT_SANDBOX_CPU_COUNT', '2.5');
vi.stubEnv('AGENT_SANDBOX_MEMORY_MIB', '4096');
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE', '5G');
vi.stubEnv('AGENT_SANDBOX_STORAGE_SIZE_GI', '5');
const customEnv = await importServiceEnv();
expect(customEnv.serviceEnv.AGENT_SANDBOX_CPU_COUNT).toBe(2.5);
expect(customEnv.serviceEnv.AGENT_SANDBOX_MEMORY_MIB).toBe(4096);
expect(customEnv.serviceEnv.AGENT_SANDBOX_STORAGE_SIZE).toBe('5G');
expect(customEnv.serviceEnv.AGENT_SANDBOX_STORAGE_SIZE_GI).toBe(5);
});
it('validates Agent Sandbox lifecycle thresholds during service env init', async () => {
......
pro @ 80528276
Subproject commit a704153cf360c5b4ca60ac16c5788b97aecd8198
Subproject commit 8052827606acad0588c2b497ce205e562657ee6e
projects/app/.env.template
......@@ -54,8 +54,8 @@ AGENT_SANDBOX_SEALOS_WORK_DIRECTORY=/home/devbox/workspace
# Agent Sandbox 单实例 CPU 核数和内存上限（MiB）
AGENT_SANDBOX_CPU_COUNT=1
AGENT_SANDBOX_MEMORY_MIB=2048
# Agent Sandbox 存储容量，保持 Kubernetes 资源量单位
AGENT_SANDBOX_STORAGE_SIZE=1Gi
# Agent Sandbox 存储容量，单位 Gi
AGENT_SANDBOX_STORAGE_SIZE_GI=1
# OpenSandbox 配置（PROVIDER=opensandbox 时生效）
AGENT_SANDBOX_OPENSANDBOX_BASEURL=http://localhost:8090
AGENT_SANDBOX_OPENSANDBOX_API_KEY=my_secure_sandbox_key_123
......@@ -68,8 +68,6 @@ AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN=vmtoken
# 活跃编辑/调试沙箱数量上限
AGENT_SANDBOX_MAX_EDIT_DEBUG=100
# Agent 沙箱磁盘大小基准（MB）。冷归档包等于该值，Skill 包和 IDE 单文件上限为该值的一半并四舍五入。
AGENT_SANDBOX_DISK_MB=1024
# 运行中 Agent 沙箱持续未活跃多少分钟后自动暂停
AGENT_SANDBOX_SUSPEND_MINUTES=60
# 已暂停 Agent 沙箱持续未活跃多少天后自动归档
......
projects/app/src/pageComponents/chat/SandboxEditor/api.ts
......@@ -6,11 +6,11 @@ import type {
SandboxGetTicketResponse,
SandboxGetHtmlPreviewLinkBody,
SandboxGetHtmlPreviewLinkResponse,
SandboxUploadBody,
SandboxUploadQuery,
SandboxUploadResponse
} from '@fastgpt/global/openapi/core/ai/sandbox/api';
import { parseContentDispositionFilename } from '@fastgpt/global/common/file/tools';
import { POST } from '@/web/common/api/request';
import { POST, POSTRawFile } from '@/web/common/api/request';
import { useSystemStore } from '@/web/common/system/useSystemStore';
import type { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
......@@ -27,7 +27,7 @@ type SandboxDownloadClientBody = SandboxClientBody<SandboxDownloadBody>;
type SandboxCheckExistClientBody = SandboxClientBody<SandboxCheckExistBody>;
type SandboxGetTicketClientBody = SandboxClientBody<SandboxGetTicketBody>;
type SandboxGetHtmlPreviewLinkClientBody = SandboxClientBody<SandboxGetHtmlPreviewLinkBody>;
type SandboxUploadClientBody = SandboxClientBody<SandboxUploadBody>;
type SandboxUploadClientQuery = SandboxClientBody<SandboxUploadQuery>;
/**
* share 模式下后端 schema 要求只传 outLinkAuthData，真实 appId 由鉴权解析。
......@@ -124,17 +124,18 @@ export const getSandboxTicket = async (data: SandboxGetTicketClientBody) =>
POST<SandboxGetTicketResponse>('/core/ai/sandbox/getTicket', normalizeSandboxRequest(data));
/**
* 通过主站 API 复用 sandbox provider 文件上传能力，避免走 ide-agent WebSocket base64。
* 将文件作为原始请求体流式写入 sandbox，避免 WebSocket base64 和主站临时文件。
*/
export const uploadSandboxFile = async ({
file,
...data
}: SandboxUploadClientBody & { file: File }) => {
const formData = new FormData();
formData.append('file', file);
formData.append('data', JSON.stringify(normalizeSandboxRequest(data)));
return POST<SandboxUploadResponse>('/core/ai/sandbox/upload', formData, {
timeout: 10 * 60 * 1000
}: SandboxUploadClientQuery & { file: File }) => {
return POSTRawFile<SandboxUploadResponse>({
url: '/core/ai/sandbox/upload',
file,
query: normalizeSandboxRequest(data),
config: {
timeout: 10 * 60 * 1000
}
});
};
projects/app/src/pageComponents/chat/SandboxEditor/components/FileTree.tsx
......@@ -342,6 +342,7 @@ const FileTree = ({
const [isUploading, setIsUploading] = useState(false);
const fileInputRef = useRef<HTMLInputElement>(null);
const uploadLockRef = useRef(false);
const treeRootRef = useRef<HTMLDivElement>(null);
const menuRef = useRef<HTMLDivElement>(null);
......@@ -516,16 +517,18 @@ const FileTree = ({
};
const handleUploadClick = () => {
if (uploadLockRef.current) return;
fileInputRef.current?.click();
};
const handleFileChange = async (e: React.ChangeEvent<HTMLInputElement>) => {
const files = e.target.files;
if (!files || files.length === 0) return;
if (!files || files.length === 0 || uploadLockRef.current) return;
const targetDirPath = getTargetDirPathFromSelected();
try {
uploadLockRef.current = true;
setIsUploading(true);
await onUploadFiles(files, targetDirPath);
} catch (error) {
......@@ -535,6 +538,7 @@ const FileTree = ({
status: 'error'
});
} finally {
uploadLockRef.current = false;
setIsUploading(false);
if (fileInputRef.current) fileInputRef.current.value = '';
}
......@@ -833,6 +837,7 @@ const FileTree = ({
multiple
ref={fileInputRef}
onChange={handleFileChange}
disabled={isUploading}
style={{ display: 'none' }}
/>
......@@ -901,9 +906,10 @@ const FileTree = ({
w="24px"
h="24px"
borderRadius="6px"
_hover={{ bg: 'rgba(15, 23, 42, 0.05)' }}
cursor="pointer"
onClick={handleUploadClick}
_hover={isUploading ? undefined : { bg: 'rgba(15, 23, 42, 0.05)' }}
cursor={isUploading ? 'not-allowed' : 'pointer'}
aria-disabled={isUploading}
onClick={isUploading ? undefined : handleUploadClick}
transition="background 0.2s"
>
{isUploading ? (
......
projects/app/src/pageComponents/dashboard/skill/ImportSkillModal.tsx
......@@ -16,7 +16,7 @@ import { getUploadAvatarPresignedUrl } from '@/web/common/file/api';
import { type FieldErrors, useForm, useWatch } from 'react-hook-form';
import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel';
const ACCEPT_TYPES = '.zip';
const ACCEPT_TYPE = '.zip';
const DEFAULT_SKILL_AVATAR = 'core/skill/default';
type ImportSkillFormType = {
......@@ -35,11 +35,6 @@ type Props = {
onSuccess?: () => void;
};
const isValidFile = (file: File) => {
const name = file.name.toLowerCase();
return name.endsWith('.zip');
};
const ImportSkillModal = ({ parentId, onClose, onSuccess }: Props) => {
const { t } = useTranslation();
const { toast } = useToast();
......@@ -67,21 +62,19 @@ const ImportSkillModal = ({ parentId, onClose, onSuccess }: Props) => {
});
const { File: FileInput, onOpen } = useSelectFile({
fileType: ACCEPT_TYPES,
fileType: ACCEPT_TYPE,
multiple: false,
maxCount: 1
});
const { runAsync: onImport, loading: isImporting } = useRequest(
({ name, avatar, file }: ValidImportSkillFormType) => {
const formData = new FormData();
formData.append('file', file);
if (name.trim()) {
formData.append('name', name.trim());
}
formData.append('avatar', avatar);
if (parentId) formData.append('parentId', parentId);
return importSkill(formData);
return importSkill({
file,
name: name.trim() || undefined,
avatar,
parentId: parentId ?? undefined
});
},
{
onSuccess() {
......@@ -115,11 +108,12 @@ const ImportSkillModal = ({ parentId, onClose, onSuccess }: Props) => {
const handleFile = useCallback(
(file: File) => {
if (!isValidFile(file)) {
const ext = file.name.slice(file.name.lastIndexOf('.'));
if (!file.name.toLowerCase().endsWith(ACCEPT_TYPE)) {
toast({
status: 'warning',
title: t('skill:unsupported_file_format', { ext })
title: t('skill:unsupported_file_format', {
ext: file.name.slice(file.name.lastIndexOf('.'))
})
});
return;
}
......@@ -249,9 +243,7 @@ const ImportSkillModal = ({ parentId, onClose, onSuccess }: Props) => {
: t('file:select_and_drag_file_tip')}
</Box>
<Box color={'myGray.500'} fontSize={'xs'} mt={1}>
{t('skill:import_skill_file_type_tip', {
ext: ACCEPT_TYPES.split(',').join(' ')
})}
{t('skill:import_skill_file_type_tip', { ext: ACCEPT_TYPE })}
</Box>
{typeof maxUploadBytes === 'number' && (
<Box color={'myGray.500'} fontSize={'xs'}>
......
projects/app/src/pages/api/core/ai/sandbox/upload.ts
import { NextAPI } from '@/service/middleware/entry';
import { type ApiRequestProps } from '@fastgpt/next/type';
import { authSandboxRuntimeSession } from '@/service/core/sandbox/access';
import { multer } from '@fastgpt/service/common/file/multer';
import { WritePermissionVal } from '@fastgpt/global/support/permission/constant';
import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError';
import {
SandboxUploadBodySchema,
SandboxUploadQuerySchema,
SandboxUploadResponseSchema,
type SandboxUploadQuery,
type SandboxUploadResponse
} from '@fastgpt/global/openapi/core/ai/sandbox/api';
import { getAgentSandboxMaxFileBytes } from '@fastgpt/service/core/ai/sandbox/interface/config';
......@@ -16,72 +16,70 @@ import {
} from '@fastgpt/service/core/ai/sandbox/interface/runtime';
import { prepareSandboxFileParentDirectories } from '@fastgpt/service/core/ai/sandbox/interface/file';
import { Readable } from 'node:stream';
import { createSizeLimitedStream } from '@fastgpt/service/common/file/stream';
async function handler(req: ApiRequestProps): Promise<SandboxUploadResponse> {
const contentType = req.headers['content-type'] ?? '';
if (!contentType.includes('multipart/form-data')) {
return Promise.reject('Content-Type must be multipart/form-data');
}
const filepaths: string[] = [];
async function handler(
req: ApiRequestProps<unknown, SandboxUploadQuery>
): Promise<SandboxUploadResponse> {
const { sourceType, sourceId, chatId, path, outLinkAuthData } = parseApiInput({
req,
querySchema: SandboxUploadQuerySchema
}).query;
try {
const maxFileBytes = getAgentSandboxMaxFileBytes();
const form = await multer.resolveFormData({
request: req,
maxFileSize: Math.ceil(maxFileBytes / 1024 / 1024)
});
if (form.fileMetadata.path) {
filepaths.push(form.fileMetadata.path);
}
const maxFileBytes = getAgentSandboxMaxFileBytes();
const contentLengthHeader = req.headers['content-length'];
const contentLength = contentLengthHeader ? Number(contentLengthHeader) : undefined;
const createFileTooLargeError = (size: number) =>
new Error(`File is too large (${size} bytes > ${maxFileBytes} bytes)`);
const { sourceType, sourceId, chatId, path, outLinkAuthData } = parseApiInput({
req: { body: form.data },
bodySchema: SandboxUploadBodySchema
}).body;
if (contentLength !== undefined && contentLength > maxFileBytes) {
throw createFileTooLargeError(contentLength);
}
if (form.fileMetadata.size > maxFileBytes) {
return Promise.reject(
`File is too large (${form.fileMetadata.size} bytes > ${maxFileBytes} bytes)`
);
}
const {
uid,
sourceType: resolvedSourceType,
sourceId: resolvedSourceId
} = await authSandboxRuntimeSession({
req,
sourceType,
sourceId,
chatId,
outLinkAuthData,
per: WritePermissionVal
});
const {
uid,
const sandbox = await getSandboxClient(
buildSandboxClientQueryFromChatSource({
sourceType: resolvedSourceType,
sourceId: resolvedSourceId
} = await authSandboxRuntimeSession({
req,
sourceType,
sourceId,
chatId,
outLinkAuthData,
per: WritePermissionVal
});
const sandbox = await getSandboxClient(
buildSandboxClientQueryFromChatSource({
sourceType: resolvedSourceType,
sourceId: resolvedSourceId,
userId: uid,
chatId
})
);
sourceId: resolvedSourceId,
userId: uid,
chatId
})
);
const providerPath = sandbox.resolveRuntimePath(path, { allowAbsolutePath: true });
await prepareSandboxFileParentDirectories(sandbox.provider, [providerPath]);
await sandbox.provider.writeFileStream(
providerPath,
Readable.toWeb(form.getReadStream()) as ReadableStream<Uint8Array>
);
const providerPath = sandbox.resolveRuntimePath(path, { allowAbsolutePath: true });
await prepareSandboxFileParentDirectories(sandbox.provider, [providerPath]);
return SandboxUploadResponseSchema.parse({
path,
bytesWritten: form.fileMetadata.size
});
} finally {
multer.clearDiskTempFiles(filepaths);
const boundedStream = createSizeLimitedStream({
stream: req,
maxBytes: maxFileBytes,
createExceededError: createFileTooLargeError
});
const [writeResult] = await sandbox.provider.writeFiles([
{
path: providerPath,
data: Readable.toWeb(boundedStream) as ReadableStream<Uint8Array>
}
]);
if (!writeResult || writeResult.error) {
throw writeResult?.error ?? new Error('Sandbox did not return a file write result');
}
return SandboxUploadResponseSchema.parse({
path,
bytesWritten: writeResult.bytesWritten
});
}
export default NextAPI(handler);
......
projects/app/src/pages/api/core/ai/skill/import.ts
......@@ -4,182 +4,90 @@ import { authSkill } from '@fastgpt/service/support/permission/skill/auth';
import { WritePermissionVal } from '@fastgpt/global/support/permission/constant';
import { TeamSkillCreatePermissionVal } from '@fastgpt/global/support/permission/user/constant';
import { importSkill } from '@fastgpt/service/core/ai/skill/manage';
import { validateZipStructure } from '@fastgpt/service/core/ai/skill/package';
import {
ImportSkillBodySchema,
type ImportSkillBody,
ImportSkillQuerySchema,
ImportSkillResponseSchema,
type ImportSkillQuery,
type ImportSkillResponse
} from '@fastgpt/global/core/ai/skill/api';
import type { SkillPackageType } from '@fastgpt/global/core/ai/skill/type';
import {
AgentSkillCategoryEnum,
AgentSkillTypeEnum
} from '@fastgpt/global/core/ai/skill/constants';
import { multer } from '@fastgpt/service/common/file/multer';
import fs from 'fs/promises';
import { addAuditLog, getI18nSkillType } from '@fastgpt/service/support/user/audit/util';
import { AuditEventEnum } from '@fastgpt/global/support/user/audit/constants';
import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import type { ApiRequestProps } from '@fastgpt/next/type';
import { getLogger, LogCategories } from '@fastgpt/service/common/logger';
import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError';
import { getAgentSandboxSkillMaxBytes } from '@fastgpt/service/core/ai/sandbox/interface/config';
const logger = getLogger(LogCategories.MODULE.AGENT_SKILLS.IMPORT);
/**
* 归一化上传文件名。
*
* 浏览器或网关可能把 multipart filename 中的 UTF-8 字节按 latin1 传给 multer，
* 这里先兼容百分号编码，再在全量字符都属于 latin1 范围时尝试还原 UTF-8 中文名。
*/
const normalizeUploadedFilename = (filename: string) => {
const decoded = (() => {
try {
return decodeURIComponent(filename);
} catch {
return filename;
}
})();
const chars = Array.from(decoded);
if (chars.some((char) => char.charCodeAt(0) > 0xff)) {
return decoded;
}
const repaired = Buffer.from(decoded, 'latin1').toString('utf8');
return repaired.includes('\uFFFD') ? decoded : repaired;
};
const getSkillNameFromArchiveFilename = (filename: string) => {
const basename = filename.split(/[\\/]/).pop() || filename;
return basename.replace(/\.(zip|tar\.gz|tgz|tar)$/i, '').trim();
};
export const config = {
api: {
bodyParser: false
}
};
async function handler(req: ApiRequestProps<ImportSkillBody>): Promise<ImportSkillResponse> {
const filepaths: string[] = [];
try {
// Read env limit before multer so both use the same value
const maxSkillPackageSize = getAgentSandboxSkillMaxBytes();
// Convert bytes to MB for multer (multer expects MB)
const maxSkillPackageSizeMB = Math.ceil(maxSkillPackageSize / 1024 / 1024);
const result = await multer.resolveFormData<ImportSkillBody>({
request: req,
maxFileSize: maxSkillPackageSizeMB
});
async function handler(
req: ApiRequestProps<unknown, ImportSkillQuery>
): Promise<ImportSkillResponse> {
const query = parseApiInput({ req, querySchema: ImportSkillQuerySchema }).query;
const filename = query.filename.split(/[\\/]/).pop() ?? query.filename;
filepaths.push(result.fileMetadata.path);
const file = result.fileMetadata;
const normalizedOriginalName = normalizeUploadedFilename(file.originalname || '');
const body = parseApiInput({
req: { body: result.data },
bodySchema: ImportSkillBodySchema
}).body;
if (!filename.toLowerCase().endsWith('.zip')) {
return Promise.reject(SkillErrEnum.invalidArchiveFormat);
}
if (!normalizedOriginalName.toLowerCase().endsWith('.zip')) {
return Promise.reject(SkillErrEnum.invalidArchiveFormat);
}
const maxSkillPackageSize = getAgentSandboxSkillMaxBytes();
const contentLengthHeader = req.headers['content-length'];
const contentLength = contentLengthHeader ? Number(contentLengthHeader) : undefined;
// Authenticate user and check permission
let teamId: string;
let tmbId: string;
if (contentLength !== undefined && contentLength > maxSkillPackageSize) {
return Promise.reject(SkillErrEnum.archiveTooLarge);
}
if (body.parentId) {
// If importing into a folder, check write permission on the parent folder
const authResult = await authSkill({
// 在消费文件流前完成权限校验，避免未授权请求占用对象存储上传带宽。
const { teamId, tmbId } = query.parentId
? await authSkill({
req,
authToken: true,
authApiKey: true,
skillId: body.parentId,
skillId: query.parentId,
per: WritePermissionVal
});
teamId = authResult.teamId;
tmbId = authResult.tmbId;
} else {
// If importing to root, check team-level skill create permission
const authResult = await authUserPer({
})
: await authUserPer({
req,
authToken: true,
authApiKey: true,
per: TeamSkillCreatePermissionVal
});
teamId = authResult.teamId;
tmbId = authResult.tmbId;
}
// Check archive size (multer already enforces the limit, this is a secondary guard)
const stats = await fs.stat(file.path);
if (stats.size > maxSkillPackageSize) {
logger.warn('Archive file size exceeds maximum', {
sizeMB: (stats.size / 1024 / 1024).toFixed(2),
maxMB: (maxSkillPackageSize / 1024 / 1024).toFixed(2)
});
return Promise.reject(SkillErrEnum.archiveTooLarge);
}
// Directly read the ZIP archive buffer from disk without any in-memory decompression
const zipBuffer = await fs.readFile(file.path);
// Light-weight structure validation: the package must contain an exact SKILL.md entry.
const validation = await validateZipStructure(zipBuffer, {
maxUncompressedBytes: maxSkillPackageSize
});
if (!validation.valid) {
return Promise.reject(SkillErrEnum.invalidSkillPackage);
const skillName = query.name?.trim() || filename.replace(/\.[^.]+$/, '').trim() || 'package';
const skillId = await importSkill({
skill: {
name: skillName,
description: query.description?.trim() ?? '',
category: [AgentSkillCategoryEnum.other],
avatar: query.avatar
},
teamId,
tmbId,
packageStream: req,
contentLength,
parentId: query.parentId ?? null
});
void addAuditLog({
tmbId,
teamId,
event: AuditEventEnum.IMPORT_SKILL,
params: {
skillName,
skillType: getI18nSkillType(AgentSkillTypeEnum.skill)
}
});
// 用户未手动命名时，展示名严格来自上传 ZIP 文件名，不读取 SKILL.md 作为兜底。
const pkgName =
body.name?.trim() ||
getSkillNameFromArchiveFilename(normalizedOriginalName || 'package') ||
'package';
const pkgDescription = body.description?.trim() ?? '';
// Build skill package using package-level metadata only
const skillPackage: SkillPackageType = {
skill: {
name: pkgName,
description: pkgDescription,
category: [AgentSkillCategoryEnum.other],
avatar: body.avatar
}
};
// Create ONE DB record and upload the raw ZIP buffer straight to S3
const skillId = await importSkill(
skillPackage,
teamId,
tmbId,
zipBuffer,
body.parentId || null
);
// Add audit log
(async () => {
addAuditLog({
tmbId,
teamId,
event: AuditEventEnum.IMPORT_SKILL,
params: {
skillName: pkgName,
skillType: getI18nSkillType(AgentSkillTypeEnum.skill)
}
});
})();
return skillId;
} finally {
multer.clearDiskTempFiles(filepaths);
}
return ImportSkillResponseSchema.parse(skillId);
}
export default NextAPI(handler);
projects/app/src/web/common/api/request.ts
......@@ -241,13 +241,16 @@ function request(
{ cancelToken, maxQuantity, withCredentials, dataAsBody, deduplicate, ...config }: ConfigType,
method: Method
): any {
/* 去空 */
for (const key in data) {
const val = data[key];
if (data[key] === undefined) {
delete data[key];
} else if (val instanceof Date) {
data[key] = dayjs(val).format();
// 只归一化普通参数对象和数组，避免改写 File、Blob 等原始请求体的只读属性。
const dataPrototype = data && typeof data === 'object' ? Object.getPrototypeOf(data) : undefined;
if (Array.isArray(data) || dataPrototype === Object.prototype || dataPrototype === null) {
for (const key in data) {
const val = data[key];
if (data[key] === undefined) {
delete data[key];
} else if (val instanceof Date) {
data[key] = dayjs(val).format();
}
}
}
......@@ -316,6 +319,34 @@ export function POST<T = undefined>(url: string, data = {}, config: ConfigType =
return request(url, data, config, 'POST');
}
/** 将 File 作为原始请求体上传，对象类型 query 参数按 JSON 序列化。 */
export function POSTRawFile<T = undefined>({
url,
file,
query,
config = {}
}: {
url: string;
file: File;
query: Record<string, unknown>;
config?: ConfigType;
}): Promise<T> {
const searchParams = new URLSearchParams();
Object.entries(query).forEach(([key, value]) => {
if (value !== undefined && value !== null) {
searchParams.set(key, typeof value === 'string' ? value : JSON.stringify(value));
}
});
return POST<T>(`${url}?${searchParams.toString()}`, file, {
...config,
headers: {
'Content-Type': 'application/octet-stream',
...config.headers
}
});
}
export function PUT<T = undefined>(url: string, data = {}, config: ConfigType = {}): Promise<T> {
return request(url, data, config, 'PUT');
}
......
projects/app/src/web/core/skill/api.ts
import { GET, DELETE, POST } from '@/web/common/api/request';
import { GET, DELETE, POST, POSTRawFile } from '@/web/common/api/request';
import { streamFetch, type StreamResponseType } from '@/web/common/api/fetch';
import { downloadFetch } from '@/web/common/system/utils';
import { useSystemStore } from '@/web/common/system/useSystemStore';
......@@ -26,7 +26,8 @@ import type {
ListSkillVersionsResponse,
SkillRuntimeBody,
SwitchSkillVersionBody,
UpdateSkillVersionBody
UpdateSkillVersionBody,
ImportSkillQuery
} from '@fastgpt/global/core/ai/skill/api';
import type { SandboxRuntimeStatusResponse } from '@fastgpt/global/core/ai/sandbox/type';
import type { GetResourceFolderListProps } from '@fastgpt/global/common/parentFolder/type';
......@@ -67,8 +68,19 @@ export const postCopySkill = (data: CopySkillBody) =>
/** 删除 Skill */
export const deleteSkill = (skillId: string) => DELETE('/core/ai/skill/delete', { skillId });
/** 导入 Skill 压缩包 */
export const importSkill = (formData: FormData) => POST<string>('/core/ai/skill/import', formData);
/** 以原始请求体导入 Skill，文件流不落 FastGPT 本地临时目录。 */
export const importSkill = ({
file,
...query
}: Omit<ImportSkillQuery, 'filename'> & { file: File }) =>
POSTRawFile<string>({
url: '/core/ai/skill/import',
file,
query: {
filename: file.name,
...query
}
});
/** 从 Sandbox 打包并发布新版本 */
export const postSaveDeploySkill = (data: SaveDeploySkillBody) =>
......
projects/app/test/api/core/ai/sandbox/upload.test.ts
......@@ -6,13 +6,10 @@ import { Readable } from 'node:stream';
const mocks = vi.hoisted(() => ({
authSandboxRuntimeSession: vi.fn(),
buildSandboxClientQueryFromChatSource: vi.fn(),
clearDiskTempFiles: vi.fn(),
createDirectories: vi.fn(),
getAgentSandboxMaxFileBytes: vi.fn(),
getReadStream: vi.fn(),
getSandboxClient: vi.fn(),
resolveFormData: vi.fn(),
writeFileStream: vi.fn()
writeFiles: vi.fn()
}));
vi.mock('@/service/middleware/entry', () => ({
......@@ -23,13 +20,6 @@ vi.mock('@/service/core/sandbox/access', () => ({
authSandboxRuntimeSession: mocks.authSandboxRuntimeSession
}));
vi.mock('@fastgpt/service/common/file/multer', () => ({
multer: {
resolveFormData: mocks.resolveFormData,
clearDiskTempFiles: mocks.clearDiskTempFiles
}
}));
vi.mock('@fastgpt/service/core/ai/sandbox/interface/config', () => ({
getAgentSandboxMaxFileBytes: mocks.getAgentSandboxMaxFileBytes
}));
......@@ -41,58 +31,67 @@ vi.mock('@fastgpt/service/core/ai/sandbox/interface/runtime', () => ({
import handler from '@/pages/api/core/ai/sandbox/upload';
const createReq = () =>
({
const createReq = (headers: Record<string, string> = {}) =>
Object.assign(Readable.from([new Uint8Array([1, 2]), new Uint8Array([3, 4])]), {
body: undefined,
query: {
appId: '507f1f77bcf86cd799439011',
chatId: 'chat-1',
path: 'uploads/a.txt'
},
headers: {
'content-type': 'multipart/form-data; boundary=test'
'content-type': 'application/octet-stream',
'content-length': '4',
...headers
}
}) as any;
/** 消费 provider 收到的 Web Stream，验证路由传递的是实际文件字节。 */
const readWebStream = async (stream: ReadableStream<Uint8Array>) => {
const reader = stream.getReader();
const chunks: Uint8Array[] = [];
while (true) {
const { done, value } = await reader.read();
if (done) break;
chunks.push(value);
}
return Buffer.concat(chunks.map((chunk) => Buffer.from(chunk)));
};
describe('sandbox upload API', () => {
let uploadedContent: Buffer;
beforeEach(() => {
vi.clearAllMocks();
uploadedContent = Buffer.alloc(0);
mocks.getAgentSandboxMaxFileBytes.mockReturnValue(10 * 1024 * 1024);
mocks.getReadStream.mockReturnValue(Readable.from([new Uint8Array([1, 2, 3])]));
mocks.resolveFormData.mockResolvedValue({
data: {
appId: '507f1f77bcf86cd799439011',
chatId: 'chat-1',
path: 'uploads/a.txt'
},
fileMetadata: {
path: '/tmp/upload-a.txt',
size: 3
},
getReadStream: mocks.getReadStream
});
mocks.authSandboxRuntimeSession.mockResolvedValue({
uid: 'user-1',
sourceType: ChatSourceTypeEnum.app,
sourceId: '507f1f77bcf86cd799439011'
});
mocks.buildSandboxClientQueryFromChatSource.mockReturnValue({ sandboxId: 'sandbox-1' });
mocks.writeFileStream.mockResolvedValue(undefined);
mocks.writeFiles.mockImplementation(async ([entry]) => {
uploadedContent = await readWebStream(entry.data);
return [{ path: entry.path, bytesWritten: uploadedContent.length, error: null }];
});
mocks.getSandboxClient.mockResolvedValue({
provider: {
createDirectories: mocks.createDirectories,
writeFileStream: mocks.writeFileStream
writeFiles: mocks.writeFiles
},
resolveRuntimePath: (path: string) => `/workspace/sessions/chat-1/${path}`
});
});
it('uploads multipart file through sandbox provider after write auth', async () => {
it('streams the raw request body to the sandbox after write auth', async () => {
const req = createReq();
await expect(handler(req)).resolves.toEqual({
path: 'uploads/a.txt',
bytesWritten: 3
bytesWritten: 4
});
expect(mocks.resolveFormData).toHaveBeenCalledWith({
request: req,
maxFileSize: 10
});
expect(mocks.authSandboxRuntimeSession).toHaveBeenCalledWith({
req,
sourceType: ChatSourceTypeEnum.app,
......@@ -108,13 +107,28 @@ describe('sandbox upload API', () => {
chatId: 'chat-1'
});
expect(mocks.createDirectories).toHaveBeenCalledWith(['/workspace/sessions/chat-1/uploads']);
expect(mocks.writeFileStream).toHaveBeenCalledTimes(1);
const [[path, stream]] = mocks.writeFileStream.mock.calls;
expect(path).toBe('/workspace/sessions/chat-1/uploads/a.txt');
expect(stream).toBeInstanceOf(ReadableStream);
expect(mocks.writeFiles).toHaveBeenCalledWith([
{
path: '/workspace/sessions/chat-1/uploads/a.txt',
data: expect.any(ReadableStream)
}
]);
expect(uploadedContent).toEqual(Buffer.from([1, 2, 3, 4]));
expect(mocks.authSandboxRuntimeSession.mock.invocationCallOrder[0]).toBeLessThan(
mocks.writeFiles.mock.invocationCallOrder[0]
);
expect(mocks.createDirectories.mock.invocationCallOrder[0]).toBeLessThan(
mocks.writeFileStream.mock.invocationCallOrder[0]
mocks.writeFiles.mock.invocationCallOrder[0]
);
expect(mocks.clearDiskTempFiles).toHaveBeenCalledWith(['/tmp/upload-a.txt']);
});
it('rejects an oversized Content-Length before auth', async () => {
mocks.getAgentSandboxMaxFileBytes.mockReturnValue(3);
const req = createReq();
await expect(handler(req)).rejects.toThrow('File is too large (4 bytes > 3 bytes)');
expect(mocks.authSandboxRuntimeSession).not.toHaveBeenCalled();
expect(mocks.writeFiles).not.toHaveBeenCalled();
});
});
projects/app/test/api/core/ai/skill/import.invalidPackage.test.ts deleted 100644 → 0
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import JSZip from 'jszip';
import fs from 'fs/promises';
import os from 'os';
import path from 'path';
import handler from '@/pages/api/core/ai/skill/import';
import { MongoAgentSkills } from '@fastgpt/service/core/ai/skill/model/schema';
import { MongoAgentSkillsVersion } from '@fastgpt/service/core/ai/skill/version/schema';
import { downloadSkillPackage } from '@fastgpt/service/core/ai/skill/package';
import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { getUser } from '@test/datas/users';
import { Call } from '@test/utils/request';
import { multer } from '@fastgpt/service/common/file/multer';
vi.mock('@fastgpt/service/common/file/multer', () => ({
multer: {
resolveFormData: vi.fn(),
clearDiskTempFiles: vi.fn()
}
}));
describe('skill/import invalid package', () => {
let tmpDir: string;
beforeEach(async () => {
vi.clearAllMocks();
tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'fastgpt-skill-import-'));
});
afterEach(async () => {
await fs.rm(tmpDir, { recursive: true, force: true });
});
it('缺少 skills 目录的压缩包不能创建 skill', async () => {
const user = await getUser(`skill-import-invalid-${getNanoid(6)}`);
const zip = new JSZip();
zip.file('README.md', '# no skill entry');
const archivePath = path.join(tmpDir, 'no-skill-md.zip');
await fs.writeFile(archivePath, await zip.generateAsync({ type: 'nodebuffer' }));
vi.mocked(multer.resolveFormData).mockResolvedValue({
data: {},
fileMetadata: {
path: archivePath,
originalname: 'no-skill-md.zip'
}
} as Awaited<ReturnType<typeof multer.resolveFormData>>);
const res = await Call(handler, {
auth: user
});
expect(res.code).not.toBe(200);
expect(res.error).toBe(SkillErrEnum.invalidSkillPackage);
await expect(MongoAgentSkills.findOne({ teamId: user.teamId }).lean()).resolves.toBeNull();
await expect(MongoAgentSkillsVersion.countDocuments({ tmbId: user.tmbId })).resolves.toBe(0);
});
it('单 skill 压缩包会按原始上传结构保存', async () => {
const user = await getUser(`skill-import-single-${getNanoid(6)}`);
const zip = new JSZip();
zip.file('single-skill/SKILL.md', '---\nname: single\n---\n');
zip.file('single-skill/src/main.ts', 'export default 1;');
const archivePath = path.join(tmpDir, 'single-skill.zip');
await fs.writeFile(archivePath, await zip.generateAsync({ type: 'nodebuffer' }));
vi.mocked(multer.resolveFormData).mockResolvedValue({
data: {},
fileMetadata: {
path: archivePath,
originalname: 'single-skill.zip'
}
} as Awaited<ReturnType<typeof multer.resolveFormData>>);
const res = await Call(handler, {
auth: user
});
expect(res.code).toBe(200);
const skill = await MongoAgentSkills.findOne({ teamId: user.teamId }).lean();
expect(skill?.name).toBe('single-skill');
expect(skill?.currentRuntimeSkills).toEqual([
{
name: 'single',
description: '',
path: 'skills/single/SKILL.md'
}
]);
const version = await MongoAgentSkillsVersion.findOne({ skillId: skill?._id }).lean();
expect(version?.storageKey).toBeTruthy();
expect(version?.runtimeSkills).toEqual([
{
name: 'single',
description: '',
path: 'skills/single/SKILL.md'
}
]);
const storedZip = await downloadSkillPackage({ storageKey: version!.storageKey });
const stored = await JSZip.loadAsync(storedZip);
const files = Object.keys(stored.files).filter((key) => !stored.files[key].dir);
expect(files).toEqual(
expect.arrayContaining(['single-skill/SKILL.md', 'single-skill/src/main.ts'])
);
expect(files).not.toContain('skills/single-skill/SKILL.md');
});
});
projects/app/test/api/core/ai/skill/import.stream.test.ts 0 → 100644
import { describe, expect, it } from 'vitest';
import { Readable } from 'node:stream';
import handler from '@/pages/api/core/ai/skill/import';
import { MongoAgentSkills } from '@fastgpt/service/core/ai/skill/model/schema';
import { MongoAgentSkillsVersion } from '@fastgpt/service/core/ai/skill/version/schema';
import { downloadSkillPackage } from '@fastgpt/service/core/ai/skill/package';
import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import { getAgentSandboxSkillMaxBytes } from '@fastgpt/service/core/ai/sandbox/interface/config';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { getUser } from '@test/datas/users';
import type { MockReqType } from '@test/mocks/request';
type ImportRequestParams = {
content: Buffer;
filename: string;
user: NonNullable<MockReqType['auth']>;
query?: Record<string, string>;
headers?: Record<string, string>;
};
/** 使用原始 body 调用导入接口，保留 Node Readable 的背压语义。 */
const callImport = async ({
content,
filename,
user,
query = {},
headers = {}
}: ImportRequestParams) => {
const req = Object.assign(Readable.from([content]), {
body: undefined,
query: {
filename,
...query
},
headers: {
'content-type': 'application/octet-stream',
'content-length': String(content.length),
...headers
},
auth: user
});
return handler(req as any, { writableFinished: false } as any) as Promise<{
code: number;
data?: string;
error?: unknown;
}>;
};
describe('skill/import stream', () => {
it('不校验 ZIP 内容和 MIME，并将原始请求流保存为初始版本', async () => {
const user = await getUser(`skill-import-opaque-${getNanoid(6)}`);
const content = Buffer.from('opaque-package-content');
const res = await callImport({
content,
filename: 'opaque.zip',
user,
headers: { 'content-type': 'text/plain' }
});
expect(res).toEqual(expect.objectContaining({ code: 200 }));
const skill = await MongoAgentSkills.findById(res.data).lean();
expect(skill?.name).toBe('opaque');
const version = await MongoAgentSkillsVersion.findOne({ skillId: skill?._id }).lean();
await expect(downloadSkillPackage({ storageKey: version!.storageKey })).resolves.toEqual(
content
);
});
it('仅按 Content-Length 提前拒绝超出大小限制的文件', async () => {
const user = await getUser(`skill-import-size-${getNanoid(6)}`);
const content = Buffer.from('content');
const res = await callImport({
content,
filename: 'package.zip',
user,
headers: {
'content-length': String(getAgentSandboxSkillMaxBytes() + 1),
'content-type': 'text/plain'
}
});
expect(res.error).toBe(SkillErrEnum.archiveTooLarge);
await expect(
MongoAgentSkills.findOne({ name: 'package', teamId: user.teamId })
).resolves.toBeNull();
});
});
projects/app/test/web/common/api/request.test.ts
......@@ -8,6 +8,7 @@ import {
responseError,
AUTH_ERROR_EVENT_NAME,
GET,
POST,
instance
} from '../../../../src/web/common/api/request';
import { TeamErrEnum } from '@fastgpt/global/common/error/code/team';
......@@ -126,6 +127,28 @@ describe('request utils', () => {
});
});
describe('raw request body', () => {
it('should send File without mutating its readonly properties', async () => {
const file = new File(['skill package'], 'skill.zip', { type: 'application/octet-stream' });
const requestSpy = vi.spyOn(instance, 'request').mockResolvedValue({
data: { code: 200, data: 'skill-id', message: 'success' }
});
await expect(
POST('/core/ai/skill/import?filename=skill.zip', file, {
headers: { 'Content-Type': 'application/octet-stream' }
})
).resolves.toBe('skill-id');
expect(requestSpy).toHaveBeenCalledWith(
expect.objectContaining({
data: file,
method: 'POST'
})
);
});
});
describe('checkMaxQuantity', () => {
it('should return empty object when maxQuantity is not set', () => {
const result = checkMaxQuantity({ url: 'test', maxQuantity: undefined });
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment