Commit 3920b6b7 by YeYuheng Committed by GitHub

feat: persist runtime skill metadata (#7225)

* feat: persist runtime skill metadata

* fix: address runtime skill metadata review

* refactor: gate runtime skill metadata in skill list
parent ca48ec84
...@@ -419,33 +419,32 @@ runtimeSkills: [ ...@@ -419,33 +419,32 @@ runtimeSkills: [
2. `MongoAgentSkills.currentRuntimeSkills` 2. `MongoAgentSkills.currentRuntimeSkills`
- 建议存。 - 建议存。
- 缓存当前版本的子 Skill 列表。 - 缓存 `currentVersionId` 指向版本的子 Skill 列表。
- 辅助生成和列表查询可以直接读主表,避免每次 join 当前版本表。 - 辅助生成和列表查询可以直接读主表,避免每次 join 当前版本表。
### 平台 Skill name/description 这里的“最新版本信息”应以当前生效版本为准,而不是按 `createdAt` 最大的版本为准。
平台 Skill `name` 不被子 Skill 覆盖,仍然是 Skill 应用名。 现有版本模型中:
平台 Skill `description` 可以在发布时确定性更新为子 Skill 描述拼接后的短文本,但必须控制在 500 字符内。 - `MongoAgentSkills.currentVersionId` 是当前生效版本指针。
- `getCurrentVersion(skillId)` 先读取主表 `currentVersionId`,再查询对应的 `MongoAgentSkillsVersion`
- 保存发布新版本时,`saveDeploySkillFromSandbox()` 会通过 `updateCurrentVersion(skillId, versionId)` 把新版本切为当前版本。
- 版本列表可以按 `createdAt` 倒序展示历史版本,但用户也可以通过版本切换把历史版本重新设为当前版本。
建议规则 因此第二阶段实现必须保证
```ts - 新建版本时,把解析出的子 Skill 元数据写入 `MongoAgentSkillsVersion.runtimeSkills`
const parts = runtimeSkills.map((item) => `${item.name}: ${item.description}`); - 当前版本发生变化时,把目标版本的 `runtimeSkills` 同步写入 `MongoAgentSkills.currentRuntimeSkills`
const description = joinAndTruncate(parts, 500); - 辅助生成读取 `MongoAgentSkills.currentRuntimeSkills`,拿到的是当前生效版本的子 Skill 信息。
``` - `version/switch.ts` 切换历史版本时,必须同步刷新 `currentRuntimeSkills`,否则会出现 `currentVersionId` 已切换但辅助生成仍展示旧子 Skill 的不一致问题。
超长时使用确定性截断,不调用 LLM,例如: ### 平台 Skill name/description
```text 平台 Skill `name` 不被子 Skill 覆盖,仍然是 Skill 应用名。
data-cleaning: 清洗表格数据;chart-reporting: 生成图表报告;...等 6 个子 Skill
```
如果用户手动维护了平台描述,是否发布时自动覆盖需要产品确认。建议第一版采用: 平台 Skill `description` 不由子 Skill 描述自动填充或覆盖。导入时用户填写什么就写入什么;用户未填写时保持空字符串。
- 创建/导入时平台描述为空:自动写入拼接描述。 原因是平台描述属于 Skill 应用级元信息,子 Skill 描述属于运行时能力元信息。第二阶段只把子 Skill 信息写入 `runtimeSkills` / `currentRuntimeSkills`,供辅助生成展示和匹配使用,不反向改写平台 Skill 主表字段。
- 保存发布已有 Skill 时:只更新 `currentRuntimeSkills`,不自动覆盖用户手写 `description`
- 如需覆盖,后续在发布弹窗中增加开关。
### 解析时机 ### 解析时机
...@@ -490,17 +489,13 @@ data-cleaning: 清洗表格数据;chart-reporting: 生成图表报告;...等 ...@@ -490,17 +489,13 @@ data-cleaning: 清洗表格数据;chart-reporting: 生成图表报告;...等
### Prompt 长度控制 ### Prompt 长度控制
如果某个 Skill 应用包含很多子 Skill,资源列表应限制展示长度: 第二阶段资源列表完整展示当前版本里的所有子 Skill:
- 单个 Skill 最多展示前 10 个子 Skill。
- 每个子 Skill 描述按固定字符数截断。
- 超出时追加:
```md - 不限制单个 Skill 展示的子 Skill 数量。
- ...还有 N 个子 Skill - 不截断子 Skill 描述。
``` - 不追加“还有 N 个子 Skill”这类摘要提示。
这样避免辅助生成 prompt 被少数大型 Skill 包撑爆 如果后续出现 prompt 过长问题,应先基于真实包规模和模型上下文窗口做数据评估,再单独设计压缩策略;第二阶段不提前加入展示限制
## 数据迁移与兼容 ## 数据迁移与兼容
...@@ -534,7 +529,7 @@ data-cleaning: 清洗表格数据;chart-reporting: 生成图表报告;...等 ...@@ -534,7 +529,7 @@ data-cleaning: 清洗表格数据;chart-reporting: 生成图表报告;...等
- 重复 `name` 发布失败。 - 重复 `name` 发布失败。
- 缺少 `name` 发布失败。 - 缺少 `name` 发布失败。
-`SKILL.md` 发布失败。 -`SKILL.md` 发布失败。
- `description` 拼接不超过 500 字符 - 导入 Skill 时不使用子 Skill 描述自动填充平台 `description`
- 保存发布新版本后: - 保存发布新版本后:
- `MongoAgentSkillsVersion.runtimeSkills` 写入。 - `MongoAgentSkillsVersion.runtimeSkills` 写入。
- `MongoAgentSkills.currentRuntimeSkills` 更新。 - `MongoAgentSkills.currentRuntimeSkills` 更新。
...@@ -568,6 +563,6 @@ data-cleaning: 清洗表格数据;chart-reporting: 生成图表报告;...等 ...@@ -568,6 +563,6 @@ data-cleaning: 清洗表格数据;chart-reporting: 生成图表报告;...等
- [ ] 导入包时写入 runtime skill metadata。 - [ ] 导入包时写入 runtime skill metadata。
- [ ] 复制 Skill 时复制 runtime skill metadata。 - [ ] 复制 Skill 时复制 runtime skill metadata。
- [ ] 保存发布 sandbox 包时写入 runtime skill metadata。 - [ ] 保存发布 sandbox 包时写入 runtime skill metadata。
- [ ] 实现平台 description 的确定性拼接与 500 字符限制 - [ ] 版本切换时同步刷新主表 `currentRuntimeSkills`
- [ ] 辅助生成资源列表展示子 Skill 详情,并做数量和长度截断 - [ ] 辅助生成资源列表完整展示当前版本的全部子 Skill 详情
- [ ] 补充发布、导入、复制、辅助生成资源列表测试。 - [ ] 补充发布、导入、复制、辅助生成资源列表测试。
...@@ -27,6 +27,13 @@ export const SandboxStatusSchema = z.enum([ ...@@ -27,6 +27,13 @@ export const SandboxStatusSchema = z.enum([
SandboxStatusEnum.stopped SandboxStatusEnum.stopped
] as const); ] as const);
export const RuntimeSkillMetadataSchema = z.object({
name: z.string(),
description: z.string(),
path: z.string()
});
export type RuntimeSkillMetadataType = z.infer<typeof RuntimeSkillMetadataSchema>;
export const AgentSkillSchema = z.object({ export const AgentSkillSchema = z.object({
_id: z.string(), _id: z.string(),
parentId: z.string().nullable().optional(), parentId: z.string().nullable().optional(),
...@@ -43,6 +50,7 @@ export const AgentSkillSchema = z.object({ ...@@ -43,6 +50,7 @@ export const AgentSkillSchema = z.object({
updateTime: z.coerce.date(), updateTime: z.coerce.date(),
deleteTime: z.coerce.date().nullable().optional(), deleteTime: z.coerce.date().nullable().optional(),
currentVersionId: z.string().optional(), currentVersionId: z.string().optional(),
currentRuntimeSkills: z.array(RuntimeSkillMetadataSchema).optional(),
creationStatus: AgentSkillCreationStatusSchema.optional(), creationStatus: AgentSkillCreationStatusSchema.optional(),
creationError: z.string().optional() creationError: z.string().optional()
}); });
...@@ -90,6 +98,7 @@ export const AgentSkillsVersionSchema = z.object({ ...@@ -90,6 +98,7 @@ export const AgentSkillsVersionSchema = z.object({
tmbId: z.string(), tmbId: z.string(),
versionName: z.string().optional(), versionName: z.string().optional(),
storageKey: z.string(), storageKey: z.string(),
runtimeSkills: z.array(RuntimeSkillMetadataSchema),
importSource: AgentSkillsVersionImportSourceSchema.optional(), importSource: AgentSkillsVersionImportSourceSchema.optional(),
createdAt: z.coerce.date() createdAt: z.coerce.date()
}); });
......
...@@ -8,7 +8,11 @@ import { mongoSessionRun } from '../../../../../common/mongo/sessionRun'; ...@@ -8,7 +8,11 @@ import { mongoSessionRun } from '../../../../../common/mongo/sessionRun';
import { Types } from '../../../../../common/mongo'; import { Types } from '../../../../../common/mongo';
import { getLogger, LogCategories } from '../../../../../common/logger'; import { getLogger, LogCategories } from '../../../../../common/logger';
import { updateCurrentVersion } from '../../../skill/manage'; import { updateCurrentVersion } from '../../../skill/manage';
import { removeSkillPackageTTL, uploadSkillPackage } from '../../../skill/package'; import {
extractRuntimeSkillsFromPackage,
removeSkillPackageTTL,
uploadSkillPackage
} from '../../../skill/package';
import { packageSkillInSandbox } from './runtime'; import { packageSkillInSandbox } from './runtime';
import { getEditDebugSandboxId } from '../../../skill/edit/config'; import { getEditDebugSandboxId } from '../../../skill/edit/config';
import { createVersion } from '../../../skill/version'; import { createVersion } from '../../../skill/version';
...@@ -80,6 +84,7 @@ export async function saveDeploySkillFromSandbox({ ...@@ -80,6 +84,7 @@ export async function saveDeploySkillFromSandbox({
const versionId = new Types.ObjectId().toString(); const versionId = new Types.ObjectId().toString();
const createdAt = new Date(); const createdAt = new Date();
const resolvedVersionName = versionName || formatTime2YMDHMS(createdAt); const resolvedVersionName = versionName || formatTime2YMDHMS(createdAt);
const runtimeSkills = await extractRuntimeSkillsFromPackage(packageBuffer);
let storageInfo; let storageInfo;
try { try {
...@@ -97,7 +102,12 @@ export async function saveDeploySkillFromSandbox({ ...@@ -97,7 +102,12 @@ export async function saveDeploySkillFromSandbox({
} }
const deployResult = await mongoSessionRun(async (session) => { const deployResult = await mongoSessionRun(async (session) => {
const isVersionLinked = await updateCurrentVersion(skillId, versionId, session); const isVersionLinked = await updateCurrentVersion({
skillId,
currentVersionId: versionId,
runtimeSkills,
session
});
if (!isVersionLinked) { if (!isVersionLinked) {
// skill 可能在打包上传期间被删除。此时不能移除 S3 TTL,让孤儿包继续走 TTL 清理。 // skill 可能在打包上传期间被删除。此时不能移除 S3 TTL,让孤儿包继续走 TTL 清理。
throw new UserError('Skill not found'); throw new UserError('Skill not found');
...@@ -109,7 +119,8 @@ export async function saveDeploySkillFromSandbox({ ...@@ -109,7 +119,8 @@ export async function saveDeploySkillFromSandbox({
skillId, skillId,
tmbId, tmbId,
versionName: resolvedVersionName, versionName: resolvedVersionName,
storageKey: storageInfo.key storageKey: storageInfo.key,
runtimeSkills
}, },
session session
); );
......
...@@ -12,6 +12,7 @@ import { updateCurrentVersion, updateSkillCreationFailed } from '../update'; ...@@ -12,6 +12,7 @@ import { updateCurrentVersion, updateSkillCreationFailed } from '../update';
import { import {
createBlankSkillWorkspacePackage, createBlankSkillWorkspacePackage,
deleteSkillPackage, deleteSkillPackage,
extractRuntimeSkillsFromPackage,
removeSkillPackageTTL, removeSkillPackageTTL,
type SkillStorageInfo, type SkillStorageInfo,
uploadSkillPackage uploadSkillPackage
...@@ -142,6 +143,7 @@ export async function completePendingSkillCreation(data: AgentSkillCreateJobData ...@@ -142,6 +143,7 @@ export async function completePendingSkillCreation(data: AgentSkillCreateJobData
try { try {
const zipBuffer = await createBlankSkillWorkspacePackage(); const zipBuffer = await createBlankSkillWorkspacePackage();
const runtimeSkills = await extractRuntimeSkillsFromPackage(zipBuffer, { allowEmpty: true });
const versionId = new Types.ObjectId().toString(); const versionId = new Types.ObjectId().toString();
const storageInfo = await uploadSkillPackage({ const storageInfo = await uploadSkillPackage({
...@@ -154,7 +156,12 @@ export async function completePendingSkillCreation(data: AgentSkillCreateJobData ...@@ -154,7 +156,12 @@ export async function completePendingSkillCreation(data: AgentSkillCreateJobData
// versionId 必须在同一个事务里绑定;否则可能出现版本列表有记录但当前指针缺失。 // versionId 必须在同一个事务里绑定;否则可能出现版本列表有记录但当前指针缺失。
const isStorageLinked = await mongoSessionRun(async (session) => { const isStorageLinked = await mongoSessionRun(async (session) => {
const isUpdated = await updateCurrentVersion(skillId, versionId, session); const isUpdated = await updateCurrentVersion({
skillId,
currentVersionId: versionId,
runtimeSkills,
session
});
if (!isUpdated) { if (!isUpdated) {
return false; return false;
} }
...@@ -164,7 +171,8 @@ export async function completePendingSkillCreation(data: AgentSkillCreateJobData ...@@ -164,7 +171,8 @@ export async function completePendingSkillCreation(data: AgentSkillCreateJobData
skillId, skillId,
tmbId, tmbId,
versionName: 'Initial blank workspace', versionName: 'Initial blank workspace',
storageKey: storageInfo.key storageKey: storageInfo.key,
runtimeSkills
}, },
session session
); );
......
...@@ -3,7 +3,11 @@ import type { SkillPackageType } from '@fastgpt/global/core/ai/skill/type'; ...@@ -3,7 +3,11 @@ import type { SkillPackageType } from '@fastgpt/global/core/ai/skill/type';
import { Types } from '../../../../common/mongo'; import { Types } from '../../../../common/mongo';
import { mongoSessionRun } from '../../../../common/mongo/sessionRun'; import { mongoSessionRun } from '../../../../common/mongo/sessionRun';
import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill'; import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill';
import { removeSkillPackageTTL, uploadSkillPackage } from '../package'; import {
extractRuntimeSkillsFromPackage,
removeSkillPackageTTL,
uploadSkillPackage
} from '../package';
import { MongoAgentSkills } from '../model/schema'; import { MongoAgentSkills } from '../model/schema';
import { createVersion } from '../version'; import { createVersion } from '../version';
import { updateCurrentVersion } from './update'; import { updateCurrentVersion } from './update';
...@@ -25,6 +29,7 @@ export async function importSkill( ...@@ -25,6 +29,7 @@ export async function importSkill(
parentId?: string | null parentId?: string | null
): Promise<string> { ): Promise<string> {
const { skill } = packageData; const { skill } = packageData;
const runtimeSkills = await extractRuntimeSkillsFromPackage(zipBuffer);
const newSkill = new MongoAgentSkills({ const newSkill = new MongoAgentSkills({
parentId: parentId || null, parentId: parentId || null,
...@@ -53,7 +58,12 @@ export async function importSkill( ...@@ -53,7 +58,12 @@ export async function importSkill(
return mongoSessionRun(async (session) => { return mongoSessionRun(async (session) => {
await newSkill.save({ session }); await newSkill.save({ session });
await updateCurrentVersion(newSkillId, versionId, session); await updateCurrentVersion({
skillId: newSkillId,
currentVersionId: versionId,
runtimeSkills,
session
});
await createVersion( await createVersion(
{ {
...@@ -61,7 +71,8 @@ export async function importSkill( ...@@ -61,7 +71,8 @@ export async function importSkill(
skillId: newSkillId, skillId: newSkillId,
tmbId, tmbId,
versionName: 'Initial import', versionName: 'Initial import',
storageKey: storageInfo.key storageKey: storageInfo.key,
runtimeSkills
}, },
session session
); );
......
...@@ -25,6 +25,7 @@ type ListReadableAgentSkillsParams = ListSkillsQuery & { ...@@ -25,6 +25,7 @@ type ListReadableAgentSkillsParams = ListSkillsQuery & {
teamPer: TeamPermission; teamPer: TeamPermission;
creationStatus?: AgentSkillCreationStatusEnum; creationStatus?: AgentSkillCreationStatusEnum;
withSourceMember?: boolean; withSourceMember?: boolean;
withCurrentRuntimeSkills?: boolean;
}; };
const mergeMongoAndQuery = (...queries: Record<string, unknown>[]) => { const mergeMongoAndQuery = (...queries: Record<string, unknown>[]) => {
...@@ -59,7 +60,8 @@ export const listReadableAgentSkills = async ({ ...@@ -59,7 +60,8 @@ export const listReadableAgentSkills = async ({
pageSize, pageSize,
withAppCount, withAppCount,
creationStatus, creationStatus,
withSourceMember = true withSourceMember = true,
withCurrentRuntimeSkills = false
}: ListReadableAgentSkillsParams) => { }: ListReadableAgentSkillsParams) => {
const selectedSkillIds = skillIds?.filter(Boolean) ?? []; const selectedSkillIds = skillIds?.filter(Boolean) ?? [];
const isSkillIdsQuery = selectedSkillIds.length > 0; const isSkillIdsQuery = selectedSkillIds.length > 0;
...@@ -244,6 +246,9 @@ export const listReadableAgentSkills = async ({ ...@@ -244,6 +246,9 @@ export const listReadableAgentSkills = async ({
category: skill.category, category: skill.category,
inheritPermission: skill.inheritPermission, inheritPermission: skill.inheritPermission,
currentVersionId: skill.currentVersionId ? String(skill.currentVersionId) : undefined, currentVersionId: skill.currentVersionId ? String(skill.currentVersionId) : undefined,
...(withCurrentRuntimeSkills
? { currentRuntimeSkills: skill.currentRuntimeSkills ?? [] }
: {}),
creationStatus: skill.creationStatus, creationStatus: skill.creationStatus,
tmbId: skill.tmbId, tmbId: skill.tmbId,
parentId: skill.parentId, parentId: skill.parentId,
......
import { AgentSkillCreationStatusEnum } from '@fastgpt/global/core/ai/skill/constants'; import { AgentSkillCreationStatusEnum } from '@fastgpt/global/core/ai/skill/constants';
import type { RuntimeSkillMetadataType } from '@fastgpt/global/core/ai/skill/type';
import type { ClientSession } from '../../../../common/mongo'; import type { ClientSession } from '../../../../common/mongo';
import { MongoAgentSkills } from '../model/schema'; import { MongoAgentSkills } from '../model/schema';
import type { UpdateSkillData } from './types'; import type { UpdateSkillData } from './types';
type UpdateCurrentVersionParams = {
skillId: string;
currentVersionId: string;
runtimeSkills: RuntimeSkillMetadataType[];
session?: ClientSession;
};
/** /**
* Update skill metadata. * Update skill metadata.
* *
...@@ -32,16 +40,18 @@ export async function updateSkill( ...@@ -32,16 +40,18 @@ export async function updateSkill(
* 这里返回 matchedCount,而不是在 skill 行消失时抛错。异步创建可能在用户删除 * 这里返回 matchedCount,而不是在 skill 行消失时抛错。异步创建可能在用户删除
* pending skill 后才完成,调用方会依赖这个 boolean 判断刚上传的包是否需要清理。 * pending skill 后才完成,调用方会依赖这个 boolean 判断刚上传的包是否需要清理。
*/ */
export async function updateCurrentVersion( export async function updateCurrentVersion({
skillId: string, skillId,
currentVersionId: string, currentVersionId,
session?: ClientSession runtimeSkills,
): Promise<boolean> { session
}: UpdateCurrentVersionParams): Promise<boolean> {
const result = await MongoAgentSkills.updateOne( const result = await MongoAgentSkills.updateOne(
{ _id: skillId, deleteTime: null }, { _id: skillId, deleteTime: null },
{ {
$set: { $set: {
currentVersionId, currentVersionId,
currentRuntimeSkills: runtimeSkills,
creationStatus: AgentSkillCreationStatusEnum.ready, creationStatus: AgentSkillCreationStatusEnum.ready,
updateTime: new Date() updateTime: new Date()
}, },
......
...@@ -17,6 +17,24 @@ export type MongoAgentSkillSchemaType = AgentSkillSchemaType; ...@@ -17,6 +17,24 @@ export type MongoAgentSkillSchemaType = AgentSkillSchemaType;
const { Schema } = connectionMongo; const { Schema } = connectionMongo;
export const RuntimeSkillMetadataSchema = new Schema(
{
name: {
type: String,
required: true
},
description: {
type: String,
default: ''
},
path: {
type: String,
required: true
}
},
{ _id: false }
);
/** /**
* Agent Skill 主表结构。 * Agent Skill 主表结构。
* *
...@@ -87,6 +105,11 @@ const AgentSkillsSchema = new Schema({ ...@@ -87,6 +105,11 @@ const AgentSkillsSchema = new Schema({
type: Schema.Types.ObjectId, type: Schema.Types.ObjectId,
ref: agentSkillsVersionCollectionName ref: agentSkillsVersionCollectionName
}, },
// 当前生效版本中的子 Skill 元数据缓存,供列表和辅助生成直接读取。
currentRuntimeSkills: {
type: [RuntimeSkillMetadataSchema],
default: []
},
creationStatus: { creationStatus: {
type: String, type: String,
enum: Object.values(AgentSkillCreationStatusEnum), enum: Object.values(AgentSkillCreationStatusEnum),
......
...@@ -7,3 +7,4 @@ ...@@ -7,3 +7,4 @@
export * from './zipBuilder'; export * from './zipBuilder';
export * from './storage'; export * from './storage';
export * from './constants'; export * from './constants';
export * from './runtimeMetadata';
import JSZip from 'jszip';
import type { RuntimeSkillMetadataType } from '@fastgpt/global/core/ai/skill/type';
import { parseSkillMarkdown } from '../utils';
import { validateZipSafety } from './zipBuilder';
export type ExtractRuntimeSkillsFromPackageOptions = {
allowEmpty?: boolean;
};
/**
* 从 Skill 包中解析运行态子 Skill 元数据。
*
* 标准 workspace 读取 `skills/<skillDir>/SKILL.md`;历史单 skill 包允许从根目录或
* 一级目录 `SKILL.md` 兜底解析,并把 path 规范化为运行态可识别的
* `skills/<name>/SKILL.md`。初始空白 workspace 可通过 `allowEmpty` 跳过空包校验。
*/
export async function extractRuntimeSkillsFromPackage(
zipBuffer: Buffer,
options: ExtractRuntimeSkillsFromPackageOptions = {}
): Promise<RuntimeSkillMetadataType[]> {
const zip = await JSZip.loadAsync(zipBuffer);
const safety = validateZipSafety(zip);
if (!safety.valid) {
throw new Error(safety.error || 'Invalid skill package');
}
const allSkillMdEntries = Object.entries(zip.files)
.filter(([, file]) => !file.dir)
.map(([path, file]) => ({
path: normalizeRuntimeSkillPath(path),
file
}))
.sort((a, b) => a.path.localeCompare(b.path));
const workspaceEntries = allSkillMdEntries.filter((entry) =>
isWorkspaceRuntimeSkillMdPath(entry.path)
);
const skillMdEntries =
workspaceEntries.length > 0
? workspaceEntries
: allSkillMdEntries.filter((entry) => isLegacySkillMdPath(entry.path));
if (skillMdEntries.length === 0) {
if (options.allowEmpty) return [];
throw new Error('Skill package must contain at least one skills/<name>/SKILL.md');
}
const nameSet = new Set<string>();
const runtimeSkills: RuntimeSkillMetadataType[] = [];
for (const entry of skillMdEntries) {
const content = await entry.file.async('string');
const { frontmatter, error } = parseSkillMarkdown(content);
if (error) {
throw new Error(`${entry.path}: ${error}`);
}
const name = typeof frontmatter.name === 'string' ? frontmatter.name.trim() : '';
if (!name) {
throw new Error(`${entry.path}: frontmatter name is required`);
}
if (nameSet.has(name)) {
throw new Error(`Duplicate runtime skill name: ${name}`);
}
nameSet.add(name);
runtimeSkills.push({
name,
description:
typeof frontmatter.description === 'string' ? frontmatter.description.trim() : '',
path: getRuntimeSkillMetadataPath(entry.path, name)
});
}
return runtimeSkills;
}
function normalizeRuntimeSkillPath(path: string): string {
return path.replace(/\\/g, '/').replace(/^\.\/+/, '');
}
function isWorkspaceRuntimeSkillMdPath(path: string): boolean {
return /^skills\/[^/]+\/SKILL\.md$/i.test(path);
}
function isLegacySkillMdPath(path: string): boolean {
return /^SKILL\.md$/i.test(path) || /^[^/]+\/SKILL\.md$/i.test(path);
}
function getRuntimeSkillMetadataPath(path: string, name: string): string {
return isWorkspaceRuntimeSkillMdPath(path) ? path : `skills/${name}/SKILL.md`;
}
...@@ -270,7 +270,7 @@ export async function validateDeployableSkillWorkspacePackage( ...@@ -270,7 +270,7 @@ export async function validateDeployableSkillWorkspacePackage(
firstLevelSkillDirs.add(firstLevelDirMatch[1]); firstLevelSkillDirs.add(firstLevelDirMatch[1]);
} }
const skillMdMatch = normalized.match(/^skills\/([^/]+)\/SKILL\.md$/); const skillMdMatch = normalized.match(/^skills\/([^/]+)\/SKILL\.md$/i);
if (skillMdMatch?.[1]) { if (skillMdMatch?.[1]) {
executableSkillDirs.add(skillMdMatch[1]); executableSkillDirs.add(skillMdMatch[1]);
} }
...@@ -321,7 +321,7 @@ function normalizeZipEntryPathForSafety(path: string): string { ...@@ -321,7 +321,7 @@ function normalizeZipEntryPathForSafety(path: string): string {
return path.replace(/\\/g, '/').replace(/^\.\/+/, ''); return path.replace(/\\/g, '/').replace(/^\.\/+/, '');
} }
function validateZipSafety( export function validateZipSafety(
zip: JSZip, zip: JSZip,
options: { maxUncompressedBytes?: number } = {} options: { maxUncompressedBytes?: number } = {}
): ZipSafetyValidationResult { ): ZipSafetyValidationResult {
......
...@@ -5,6 +5,7 @@ import { ...@@ -5,6 +5,7 @@ import {
} from '@fastgpt/global/core/ai/skill/constants'; } from '@fastgpt/global/core/ai/skill/constants';
import { TeamMemberCollectionName } from '@fastgpt/global/support/user/team/constant'; import { TeamMemberCollectionName } from '@fastgpt/global/support/user/team/constant';
import type { AgentSkillsVersionSchemaType } from '@fastgpt/global/core/ai/skill/type'; import type { AgentSkillsVersionSchemaType } from '@fastgpt/global/core/ai/skill/type';
import { RuntimeSkillMetadataSchema } from '../model/schema';
const { Schema } = connectionMongo; const { Schema } = connectionMongo;
...@@ -34,6 +35,11 @@ const AgentSkillsVersionSchema = new Schema({ ...@@ -34,6 +35,11 @@ const AgentSkillsVersionSchema = new Schema({
type: String, type: String,
required: true required: true
}, },
// 该版本包内实际包含的子 Skill 元数据。
runtimeSkills: {
type: [RuntimeSkillMetadataSchema],
default: []
},
// 导入来源信息,仅导入场景存在。 // 导入来源信息,仅导入场景存在。
importSource: { importSource: {
originalFilename: String, originalFilename: String,
......
import type { RuntimeSkillMetadataType } from '@fastgpt/global/core/ai/skill/type';
export type CreateVersionData = { export type CreateVersionData = {
versionId?: string; versionId?: string;
skillId: string; skillId: string;
tmbId: string; tmbId: string;
versionName?: string; versionName?: string;
storageKey: string; storageKey: string;
runtimeSkills: RuntimeSkillMetadataType[];
importSource?: { importSource?: {
originalFilename: string; originalFilename: string;
importedAt: Date; importedAt: Date;
......
...@@ -12,7 +12,8 @@ const mocks = vi.hoisted(() => ({ ...@@ -12,7 +12,8 @@ const mocks = vi.hoisted(() => ({
updateSandboxInstanceRecordBySandboxId: vi.fn(), updateSandboxInstanceRecordBySandboxId: vi.fn(),
updateCurrentVersion: vi.fn(), updateCurrentVersion: vi.fn(),
uploadSkillPackage: vi.fn(), uploadSkillPackage: vi.fn(),
validateZipStructure: vi.fn() validateZipStructure: vi.fn(),
extractRuntimeSkillsFromPackage: vi.fn()
})); }));
vi.mock('@fastgpt/service/common/mongo/sessionRun', () => ({ vi.mock('@fastgpt/service/common/mongo/sessionRun', () => ({
...@@ -31,6 +32,7 @@ vi.mock('@fastgpt/service/core/ai/skill/package', async (importOriginal) => { ...@@ -31,6 +32,7 @@ vi.mock('@fastgpt/service/core/ai/skill/package', async (importOriginal) => {
const actual = await importOriginal<typeof import('@fastgpt/service/core/ai/skill/package')>(); const actual = await importOriginal<typeof import('@fastgpt/service/core/ai/skill/package')>();
return { return {
...actual, ...actual,
extractRuntimeSkillsFromPackage: mocks.extractRuntimeSkillsFromPackage,
removeSkillPackageTTL: mocks.removeSkillPackageTTL, removeSkillPackageTTL: mocks.removeSkillPackageTTL,
uploadSkillPackage: mocks.uploadSkillPackage, uploadSkillPackage: mocks.uploadSkillPackage,
validateZipStructure: mocks.validateZipStructure validateZipStructure: mocks.validateZipStructure
...@@ -67,11 +69,16 @@ vi.mock('@fastgpt/service/core/ai/sandbox/infrastructure/instance/repository', ( ...@@ -67,11 +69,16 @@ vi.mock('@fastgpt/service/core/ai/sandbox/infrastructure/instance/repository', (
updateSandboxInstanceRecordBySandboxId: mocks.updateSandboxInstanceRecordBySandboxId updateSandboxInstanceRecordBySandboxId: mocks.updateSandboxInstanceRecordBySandboxId
})); }));
vi.mock('@fastgpt/service/core/ai/skill/model/schema', () => ({ vi.mock('@fastgpt/service/core/ai/skill/model/schema', async (importOriginal) => {
const actual =
await importOriginal<typeof import('@fastgpt/service/core/ai/skill/model/schema')>();
return {
...actual,
MongoAgentSkills: { MongoAgentSkills: {
updateOne: mocks.mongoAgentSkillsUpdateOne updateOne: mocks.mongoAgentSkillsUpdateOne
} }
})); };
});
import { SandboxStatusEnum } from '@fastgpt/global/core/ai/sandbox/constants'; import { SandboxStatusEnum } from '@fastgpt/global/core/ai/sandbox/constants';
import { saveDeploySkillFromSandbox } from '@fastgpt/service/core/ai/sandbox/application/skillEdit/deploy'; import { saveDeploySkillFromSandbox } from '@fastgpt/service/core/ai/sandbox/application/skillEdit/deploy';
...@@ -90,6 +97,13 @@ describe('saveDeploySkillFromSandbox', () => { ...@@ -90,6 +97,13 @@ describe('saveDeploySkillFromSandbox', () => {
}); });
mocks.packageSkillInSandbox.mockResolvedValue(Buffer.from('mock zip')); mocks.packageSkillInSandbox.mockResolvedValue(Buffer.from('mock zip'));
mocks.validateZipStructure.mockResolvedValue({ valid: true }); mocks.validateZipStructure.mockResolvedValue({ valid: true });
mocks.extractRuntimeSkillsFromPackage.mockResolvedValue([
{
name: 'runtime-skill',
description: 'Runtime skill',
path: 'skills/runtime-skill/SKILL.md'
}
]);
mocks.uploadSkillPackage.mockResolvedValue({ mocks.uploadSkillPackage.mockResolvedValue({
key: 'agent-skills/team-1/skill-1/version-1.zip' key: 'agent-skills/team-1/skill-1/version-1.zip'
}); });
...@@ -150,6 +164,30 @@ describe('saveDeploySkillFromSandbox', () => { ...@@ -150,6 +164,30 @@ describe('saveDeploySkillFromSandbox', () => {
}) })
}) })
); );
expect(mocks.updateCurrentVersion).toHaveBeenCalledWith({
skillId: 'skill-1',
currentVersionId: expect.any(String),
runtimeSkills: [
{
name: 'runtime-skill',
description: 'Runtime skill',
path: 'skills/runtime-skill/SKILL.md'
}
],
session: { id: 'mock-session' }
});
expect(mocks.createVersion).toHaveBeenCalledWith(
expect.objectContaining({
runtimeSkills: [
{
name: 'runtime-skill',
description: 'Runtime skill',
path: 'skills/runtime-skill/SKILL.md'
}
]
}),
{ id: 'mock-session' }
);
}); });
it('rejects sandbox records that do not belong to the deploying team', async () => { it('rejects sandbox records that do not belong to the deploying team', async () => {
......
import { describe, expect, it, beforeAll, afterAll, beforeEach } from 'vitest'; import { describe, expect, it, beforeAll, afterAll, beforeEach } from 'vitest';
import JSZip from 'jszip';
import { Types } from '@fastgpt/service/common/mongo'; import { Types } from '@fastgpt/service/common/mongo';
import { MongoAgentSkills } from '@fastgpt/service/core/ai/skill/model/schema'; import { MongoAgentSkills } from '@fastgpt/service/core/ai/skill/model/schema';
import { import {
...@@ -340,6 +341,12 @@ describe('AgentSkill Controller', () => { ...@@ -340,6 +341,12 @@ describe('AgentSkill Controller', () => {
// ==================== Import Skill ==================== // ==================== Import Skill ====================
describe('importSkill', () => { describe('importSkill', () => {
const createImportZipBuffer = async (skillName: string) => {
const zip = new JSZip();
zip.file('skills/imported/SKILL.md', `---\nname: ${skillName}\n---\n`);
return zip.generateAsync({ type: 'nodebuffer' });
};
it('should import skill from package', async () => { it('should import skill from package', async () => {
const packageData = { const packageData = {
skill: { skill: {
...@@ -349,8 +356,7 @@ describe('AgentSkill Controller', () => { ...@@ -349,8 +356,7 @@ describe('AgentSkill Controller', () => {
} }
}; };
// Create a mock ZIP buffer const mockZipBuffer = await createImportZipBuffer('imported-skill');
const mockZipBuffer = Buffer.from('mock zip content');
const skillId = await importSkill(packageData, testTeamId, testTmbId, mockZipBuffer); const skillId = await importSkill(packageData, testTeamId, testTmbId, mockZipBuffer);
...@@ -360,6 +366,13 @@ describe('AgentSkill Controller', () => { ...@@ -360,6 +366,13 @@ describe('AgentSkill Controller', () => {
expect(skill?.name).toBe(packageData.skill.name); expect(skill?.name).toBe(packageData.skill.name);
expect(skill?.description).toBe(packageData.skill.description); expect(skill?.description).toBe(packageData.skill.description);
expect(skill?.source).toBe(AgentSkillSourceEnum.personal); expect(skill?.source).toBe(AgentSkillSourceEnum.personal);
expect(skill?.currentRuntimeSkills.map((item) => item.toObject())).toEqual([
{
name: 'imported-skill',
description: '',
path: 'skills/imported/SKILL.md'
}
]);
}); });
it('should allow importing duplicate name without error', async () => { it('should allow importing duplicate name without error', async () => {
...@@ -371,7 +384,7 @@ describe('AgentSkill Controller', () => { ...@@ -371,7 +384,7 @@ describe('AgentSkill Controller', () => {
} }
}; };
const mockZipBuffer = Buffer.from('mock zip content'); const mockZipBuffer = await createImportZipBuffer('duplicate-import');
// First import // First import
const firstSkillId = await importSkill(packageData, testTeamId, testTmbId, mockZipBuffer); const firstSkillId = await importSkill(packageData, testTeamId, testTmbId, mockZipBuffer);
......
...@@ -52,7 +52,8 @@ describe('versionController', () => { ...@@ -52,7 +52,8 @@ describe('versionController', () => {
skillId: testSkillId, skillId: testSkillId,
tmbId: testTmbId, tmbId: testTmbId,
versionName: 'Initial creation', versionName: 'Initial creation',
storageKey: `agent-skills/${testTeamId}/${testSkillId}/version-v0.zip` storageKey: `agent-skills/${testTeamId}/${testSkillId}/version-v0.zip`,
runtimeSkills: []
}; };
const versionId = await createVersion(versionData); const versionId = await createVersion(versionData);
...@@ -70,12 +71,14 @@ describe('versionController', () => { ...@@ -70,12 +71,14 @@ describe('versionController', () => {
const v0Id = await createVersion({ const v0Id = await createVersion({
skillId: testSkillId, skillId: testSkillId,
tmbId: testTmbId, tmbId: testTmbId,
storageKey: 'v0' storageKey: 'v0',
runtimeSkills: []
}); });
const v1Id = await createVersion({ const v1Id = await createVersion({
skillId: testSkillId, skillId: testSkillId,
tmbId: testTmbId, tmbId: testTmbId,
storageKey: 'v1' storageKey: 'v1',
runtimeSkills: []
}); });
const [v0, v1] = await Promise.all([ const [v0, v1] = await Promise.all([
......
...@@ -6,7 +6,8 @@ import { ...@@ -6,7 +6,8 @@ import {
validateDeployableSkillWorkspacePackage, validateDeployableSkillWorkspacePackage,
validateZipStructure, validateZipStructure,
extractSkillPackage, extractSkillPackage,
standardizeSkillPackageBySkillMdName standardizeSkillPackageBySkillMdName,
extractRuntimeSkillsFromPackage
} from '@fastgpt/service/core/ai/skill/package'; } from '@fastgpt/service/core/ai/skill/package';
describe('zipBuilder', () => { describe('zipBuilder', () => {
...@@ -132,6 +133,95 @@ ${largeMarkdown}`; ...@@ -132,6 +133,95 @@ ${largeMarkdown}`;
}); });
}); });
// ==================== extractRuntimeSkillsFromPackage ====================
describe('extractRuntimeSkillsFromPackage', () => {
it('should extract runtime skills from workspace package', async () => {
const zip = new JSZip();
zip.file(
'skills/data-cleaning/SKILL.md',
'---\nname: data-cleaning\ndescription: Clean table data\n---\n'
);
zip.file(
'skills/chart-reporting/SKILL.md',
'---\nname: chart-reporting\ndescription: Build chart reports\n---\n'
);
const buffer = await zip.generateAsync({ type: 'nodebuffer' });
const result = await extractRuntimeSkillsFromPackage(buffer);
expect(result).toEqual([
{
name: 'chart-reporting',
description: 'Build chart reports',
path: 'skills/chart-reporting/SKILL.md'
},
{
name: 'data-cleaning',
description: 'Clean table data',
path: 'skills/data-cleaning/SKILL.md'
}
]);
});
it('should normalize legacy single skill package path', async () => {
const zip = new JSZip();
zip.file('legacy-skill/SKILL.MD', '---\nname: legacy-skill\n---\n');
const buffer = await zip.generateAsync({ type: 'nodebuffer' });
const result = await extractRuntimeSkillsFromPackage(buffer);
expect(result).toEqual([
{
name: 'legacy-skill',
description: '',
path: 'skills/legacy-skill/SKILL.md'
}
]);
});
it('should reject duplicate runtime skill names', async () => {
const zip = new JSZip();
zip.file('skills/a/SKILL.md', '---\nname: same\n---\n');
zip.file('skills/b/SKILL.md', '---\nname: same\n---\n');
const buffer = await zip.generateAsync({ type: 'nodebuffer' });
await expect(extractRuntimeSkillsFromPackage(buffer)).rejects.toThrow(
'Duplicate runtime skill name: same'
);
});
it('should reject SKILL.md without frontmatter name', async () => {
const zip = new JSZip();
zip.file('skills/a/SKILL.md', '---\ndescription: Missing name\n---\n');
const buffer = await zip.generateAsync({ type: 'nodebuffer' });
await expect(extractRuntimeSkillsFromPackage(buffer)).rejects.toThrow(
'frontmatter name is required'
);
});
it('should reuse ZIP safety validation for unsafe entry paths', async () => {
const zip = new JSZip();
zip.file('skills/../SKILL.md', '---\nname: unsafe\n---\n');
const buffer = await zip.generateAsync({ type: 'nodebuffer' });
await expect(extractRuntimeSkillsFromPackage(buffer)).rejects.toThrow(
'Unsafe ZIP entry path'
);
});
it('should allow empty initial workspace when requested', async () => {
const buffer = await createBlankSkillWorkspacePackage();
await expect(extractRuntimeSkillsFromPackage(buffer, { allowEmpty: true })).resolves.toEqual(
[]
);
});
});
// ==================== validateZipStructure ==================== // ==================== validateZipStructure ====================
describe('validateZipStructure', () => { describe('validateZipStructure', () => {
it('should validate zip with SKILL.md at root', async () => { it('should validate zip with SKILL.md at root', async () => {
...@@ -247,15 +337,31 @@ ${largeMarkdown}`; ...@@ -247,15 +337,31 @@ ${largeMarkdown}`;
expect(result.error).toContain('missing-entry'); expect(result.error).toContain('missing-entry');
}); });
it('should require uppercase SKILL.md for deployable skill folders', async () => { it('should accept case-insensitive SKILL.md for deployable skill folders', async () => {
const zip = new JSZip(); const zip = new JSZip();
zip.file('skills/lowercase-entry/skill.md', '---\nname: lowercase-entry\n---\n'); zip.file('skills/lowercase-entry/skill.md', '---\nname: lowercase-entry\n---\n');
const buffer = await zip.generateAsync({ type: 'nodebuffer' }); const buffer = await zip.generateAsync({ type: 'nodebuffer' });
const result = await validateDeployableSkillWorkspacePackage(buffer); const result = await validateDeployableSkillWorkspacePackage(buffer);
expect(result.valid).toBe(false); expect(result.valid).toBe(true);
expect(result.error).toContain('lowercase-entry'); });
it('should extract runtime skills from case-insensitive SKILL.md entries', async () => {
const zip = new JSZip();
zip.file('skills/lowercase-entry/skill.md', '---\nname: lowercase-entry\n---\n');
zip.file('UPPERCASE/SKILL.MD', '---\nname: uppercase-entry\n---\n');
const buffer = await zip.generateAsync({ type: 'nodebuffer' });
const result = await extractRuntimeSkillsFromPackage(buffer);
expect(result).toEqual([
{
name: 'lowercase-entry',
description: '',
path: 'skills/lowercase-entry/skill.md'
}
]);
}); });
it('should reject unsafe zip entry paths before workspace structure validation', async () => { it('should reject unsafe zip entry paths before workspace structure validation', async () => {
......
Subproject commit b3c06fa7b196f6a7917065117c27432f1a526fb2 Subproject commit bed1e59a243e8d3062450d6eab799b37d5c718e0
...@@ -106,8 +106,15 @@ async function handler(req: ApiRequestProps<CopySkillBody>): Promise<CopySkillRe ...@@ -106,8 +106,15 @@ async function handler(req: ApiRequestProps<CopySkillBody>): Promise<CopySkillRe
session session
); );
const runtimeSkills = sourceVersion.runtimeSkills ?? skill.currentRuntimeSkills ?? [];
// Point the copied skill to the copied package version. // Point the copied skill to the copied package version.
await updateCurrentVersion(newId, versionId, session); await updateCurrentVersion({
skillId: newId,
currentVersionId: versionId,
runtimeSkills,
session
});
// Create the initial v0 version record // Create the initial v0 version record
await createVersion( await createVersion(
...@@ -116,7 +123,8 @@ async function handler(req: ApiRequestProps<CopySkillBody>): Promise<CopySkillRe ...@@ -116,7 +123,8 @@ async function handler(req: ApiRequestProps<CopySkillBody>): Promise<CopySkillRe
skillId: newId, skillId: newId,
tmbId, tmbId,
versionName: 'Copied from ' + skill.name, versionName: 'Copied from ' + skill.name,
storageKey: storageInfo.key storageKey: storageInfo.key,
runtimeSkills
}, },
session session
); );
......
...@@ -40,6 +40,7 @@ async function handler( ...@@ -40,6 +40,7 @@ async function handler(
{ {
$set: { $set: {
currentVersionId: targetVersion._id, currentVersionId: targetVersion._id,
currentRuntimeSkills: targetVersion.runtimeSkills ?? [],
updateTime: new Date() updateTime: new Date()
} }
}, },
......
...@@ -81,8 +81,22 @@ describe('skill/import invalid package', () => { ...@@ -81,8 +81,22 @@ describe('skill/import invalid package', () => {
const skill = await MongoAgentSkills.findOne({ teamId: user.teamId }).lean(); const skill = await MongoAgentSkills.findOne({ teamId: user.teamId }).lean();
expect(skill?.name).toBe('single-skill'); expect(skill?.name).toBe('single-skill');
expect(skill?.currentRuntimeSkills).toEqual([
{
name: 'single',
description: '',
path: 'skills/single/SKILL.md'
}
]);
const version = await MongoAgentSkillsVersion.findOne({ skillId: skill?._id }).lean(); const version = await MongoAgentSkillsVersion.findOne({ skillId: skill?._id }).lean();
expect(version?.storageKey).toBeTruthy(); expect(version?.storageKey).toBeTruthy();
expect(version?.runtimeSkills).toEqual([
{
name: 'single',
description: '',
path: 'skills/single/SKILL.md'
}
]);
const storedZip = await downloadSkillPackage({ storageKey: version!.storageKey }); const storedZip = await downloadSkillPackage({ storageKey: version!.storageKey });
const stored = await JSZip.loadAsync(storedZip); const stored = await JSZip.loadAsync(storedZip);
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment