Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit e927b94c by Archer Committed by GitHub
Options

perf: quote code (#6977) 

* perf: quote code

* fix: type

* doc

* doc

* add zod schema
parent 22ebfacb
Showing with 339 additions and 201 deletions
document/content/self-host/upgrading/4-15/41502.mdx
...@@ -24,6 +24,7 @@ description: 'FastGPT V4.15.0-beta2 更新说明' ...@@ -24,6 +24,7 @@ description: 'FastGPT V4.15.0-beta2 更新说明'
1. 支持 Skill 编辑，Agent 支持 Skill 使用，目前仅支持静态 Skill，无法反向调用系统工具。 1. 支持 Skill 编辑，Agent 支持 Skill 使用，目前仅支持静态 Skill，无法反向调用系统工具。
2. 重写 agentV2 loop 逻辑。 2. 重写 agentV2 loop 逻辑。
3. 知识库搜索支持原生多模态 embedding 模型以及图搜图。 3. 知识库搜索支持原生多模态 embedding 模型以及图搜图。
4. Chat API dataId 校验：`/v1/chat/completions`、`/v2/chat/completions` 与 `chatTest` 在工作流执行前校验本轮 `dataId` 是否与请求内或当前会话已有记录重复；重复时直接返回业务错误，避免脏数据进入工作流与流恢复合并逻辑。
## ⚙️ 优化 ## ⚙️ 优化
...@@ -44,13 +45,14 @@ description: 'FastGPT V4.15.0-beta2 更新说明' ...@@ -44,13 +45,14 @@ description: 'FastGPT V4.15.0-beta2 更新说明'
1. 工作流，单节点调试，存在异常默认值。 1. 工作流，单节点调试，存在异常默认值。
2. 模型配置，defaultConfig 覆盖异常。 2. 模型配置，defaultConfig 覆盖异常。
3. 切换团队时，清除本地 chat 缓存。 3. 切换团队时，清除本地 chat 缓存。
4. 流恢复表单输入：刷新或断线续传后，已提交的表单输入值（含 `fileSelect` 文件列表）能正确回填到交互节点内，不再出现空表单或文件消失。 4. 对话流恢复：
5. 流恢复内容保留：自动续传开始时保留已加载的 AI 输出与节点响应；completed 记录覆盖时不再丢失已恢复的交互表单值与 flow 节点响应。 - 刷新或断线续传后，已提交的表单输入值（含 `fileSelect` 文件列表）能正确回填到交互节点内，不再出现空表单或文件消失。
6. 流恢复交互状态：已提交表单后不再重复追加过期未提交交互；恢复过程中表单默认值能随 `formInputResult` 同步更新。 - 自动续传开始时保留已加载的 AI 输出与节点响应；completed 记录覆盖时不再丢失已恢复的交互表单值与 flow 节点响应。
7. 流恢复历史标题：新对话发起后，侧栏临时历史项优先展示用户输入生成的标题，服务端标题落库后再覆盖，避免长时间显示「新对话」。 - 已提交表单后不再重复追加过期未提交交互；恢复过程中表单默认值能随 `formInputResult` 同步更新。
8. 切换应用历史串线：修复切换不同应用时，侧栏或会话内容短暂展示其他应用聊天记录的问题。 - 新对话发起后，侧栏临时历史项优先展示用户输入生成的标题，服务端标题落库后再覆盖，避免长时间显示「新对话」。
9. 停止对话提示：移除停止时的 warning toast，改为与后端生成态同步的状态提示。 - 切换不同应用时，侧栏或会话内容短暂展示其他应用聊天记录的问题。
10. Chat API dataId 校验：`/v1/chat/completions`、`/v2/chat/completions` 与 `chatTest` 在工作流执行前校验本轮 `dataId` 是否与请求内或当前会话已有记录重复；重复时直接返回业务错误，避免脏数据进入工作流与流恢复合并逻辑。 5. 停止对话提示：移除停止时的 warning toast，改为与后端生成态同步的状态提示。
6. v1/completions 接口，nodeResponse 中，quoteList 未返回 `q` , `a`。
## 代码优化 ## 代码优化
......
document/data/doc-last-modified.json
...@@ -187,8 +187,8 @@ ...@@ -187,8 +187,8 @@
"content/self-host/custom-models/ollama.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/custom-models/ollama.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/custom-models/xinference.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/custom-models/xinference.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/custom-models/xinference.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/custom-models/xinference.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/deploy/docker.en.mdx": "2026-05-07T15:08:21+08:00", "content/self-host/deploy/docker.en.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/deploy/docker.mdx": "2026-05-07T15:08:21+08:00", "content/self-host/deploy/docker.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/deploy/sealos.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/deploy/sealos.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/deploy/sealos.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/deploy/sealos.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/design/dataset.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/design/dataset.en.mdx": "2026-04-26T21:08:47+08:00",
...@@ -233,8 +233,8 @@ ...@@ -233,8 +233,8 @@
"content/self-host/upgrading/4-14/4140.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4140.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/4141.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4141.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/4141.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4141.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/41410.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/41410.en.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/4-14/41410.mdx": "2026-05-23T22:47:02+08:00", "content/self-host/upgrading/4-14/41410.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/4-14/41411.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/41411.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/41411.mdx": "2026-04-26T21:28:27+08:00", "content/self-host/upgrading/4-14/41411.mdx": "2026-04-26T21:28:27+08:00",
"content/self-host/upgrading/4-14/41412.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/41412.en.mdx": "2026-04-26T21:08:47+08:00",
...@@ -278,13 +278,13 @@ ...@@ -278,13 +278,13 @@
"content/self-host/upgrading/4-14/4149.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4149.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-14/4149.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/4-14/4149.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/4-15/4150.mdx": "2026-05-20T17:52:26+08:00", "content/self-host/upgrading/4-15/4150.mdx": "2026-05-20T17:52:26+08:00",
"content/self-host/upgrading/4-15/41502.mdx": "2026-05-23T23:33:02+08:00", "content/self-host/upgrading/4-15/41502.mdx": "2026-05-24T17:01:11+08:00",
"content/self-host/upgrading/outdated/40.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/40.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/40.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/40.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/41.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/41.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/41.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/41.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4100.en.mdx": "2026-05-07T15:06:40+08:00", "content/self-host/upgrading/outdated/4100.en.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/4100.mdx": "2026-05-07T15:06:40+08:00", "content/self-host/upgrading/outdated/4100.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/4101.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4101.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4101.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4101.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4110.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4110.en.mdx": "2026-04-26T21:08:47+08:00",
...@@ -363,8 +363,8 @@ ...@@ -363,8 +363,8 @@
"content/self-host/upgrading/outdated/4818.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4818.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4819.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4819.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4819.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4819.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/482.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/482.en.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/482.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/482.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/4820.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4820.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4820.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4820.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4821.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4821.en.mdx": "2026-04-26T21:08:47+08:00",
...@@ -373,8 +373,8 @@ ...@@ -373,8 +373,8 @@
"content/self-host/upgrading/outdated/4822.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4822.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4823.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4823.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4823.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4823.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/483.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/483.en.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/483.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/483.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/484.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/484.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/484.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/484.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/485.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/485.en.mdx": "2026-04-26T21:08:47+08:00",
...@@ -387,8 +387,8 @@ ...@@ -387,8 +387,8 @@
"content/self-host/upgrading/outdated/488.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/488.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/489.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/489.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/489.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/489.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/490.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/490.en.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/490.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/490.mdx": "2026-05-24T00:53:50+08:00",
"content/self-host/upgrading/outdated/491.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/491.en.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/491.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/491.mdx": "2026-04-26T21:08:47+08:00",
"content/self-host/upgrading/outdated/4910.en.mdx": "2026-04-26T21:08:47+08:00", "content/self-host/upgrading/outdated/4910.en.mdx": "2026-04-26T21:08:47+08:00",
......
packages/global/common/error/code/common.ts
...@@ -22,19 +22,19 @@ const datasetErr = [ ...@@ -22,19 +22,19 @@ const datasetErr = [
}, },
{ {
statusText: CommonErrEnum.fileNotFound, statusText: CommonErrEnum.fileNotFound,
message: 'error.fileNotFound' message: i18nT('common:error.fileNotFound')
}, },
{ {
statusText: CommonErrEnum.unAuthFile, statusText: CommonErrEnum.unAuthFile,
message: 'error.unAuthFile' message: i18nT('common:error.unAuthFile')
}, },
{ {
statusText: CommonErrEnum.missingParams, statusText: CommonErrEnum.missingParams,
message: 'error.missingParams' message: i18nT('common:error.missingParams')
}, },
{ {
statusText: CommonErrEnum.inheritPermissionError, statusText: CommonErrEnum.inheritPermissionError,
message: 'error.inheritPermissionError' message: i18nT('common:error.inheritPermissionError')
} }
]; ];
export default datasetErr.reduce((acc, cur, index) => { export default datasetErr.reduce((acc, cur, index) => {
......
packages/global/common/error/code/dataset.ts
...@@ -20,11 +20,11 @@ export enum DatasetErrEnum { ...@@ -20,11 +20,11 @@ export enum DatasetErrEnum {
const datasetErr = [ const datasetErr = [
{ {
statusText: DatasetErrEnum.sameApiCollection, statusText: DatasetErrEnum.sameApiCollection,
message: i18nT('dataset:same_api_collection') message: i18nT('common:core.dataset.error.sameApiCollection')
}, },
{ {
statusText: DatasetErrEnum.notSupportSync, statusText: DatasetErrEnum.notSupportSync,
message: i18nT('dataset:collection_not_support_sync') message: i18nT('common:core.dataset.error.notSupportSync')
}, },
{ {
statusText: DatasetErrEnum.unExist, statusText: DatasetErrEnum.unExist,
...@@ -36,35 +36,39 @@ const datasetErr = [ ...@@ -36,35 +36,39 @@ const datasetErr = [
}, },
{ {
statusText: DatasetErrEnum.unAuthDataset, statusText: DatasetErrEnum.unAuthDataset,
message: 'core.dataset.error.unAuthDataset' message: i18nT('common:core.dataset.error.unAuthDataset')
}, },
{ {
statusText: DatasetErrEnum.unAuthDatasetCollection, statusText: DatasetErrEnum.unAuthDatasetCollection,
message: 'core.dataset.error.unAuthDatasetCollection' message: i18nT('common:core.dataset.error.unAuthDatasetCollection')
}, },
{ {
statusText: DatasetErrEnum.unAuthDatasetData, statusText: DatasetErrEnum.unAuthDatasetData,
message: 'core.dataset.error.unAuthDatasetData' message: i18nT('common:core.dataset.error.unAuthDatasetData')
}, },
{ {
statusText: DatasetErrEnum.unAuthDatasetFile, statusText: DatasetErrEnum.unAuthDatasetFile,
message: 'core.dataset.error.unAuthDatasetFile' message: i18nT('common:core.dataset.error.unAuthDatasetFile')
}, },
{ {
statusText: DatasetErrEnum.unCreateCollection, statusText: DatasetErrEnum.unCreateCollection,
message: 'core.dataset.error.unCreateCollection' message: i18nT('common:core.dataset.error.unCreateCollection')
}, },
{ {
statusText: DatasetErrEnum.unLinkCollection, statusText: DatasetErrEnum.unLinkCollection,
message: 'core.dataset.error.unLinkCollection' message: i18nT('common:core.dataset.error.unLinkCollection')
}, },
{ {
statusText: DatasetErrEnum.invalidVectorModelOrQAModel, statusText: DatasetErrEnum.invalidVectorModelOrQAModel,
message: 'core.dataset.error.invalidVectorModelOrQAModel' message: i18nT('common:core.dataset.error.invalidVectorModelOrQAModel')
}, },
{ {
statusText: DatasetErrEnum.canNotEditAdminPermission, statusText: DatasetErrEnum.canNotEditAdminPermission,
message: 'core.dataset.error.canNotEditAdminPermission' message: i18nT('common:core.dataset.error.canNotEditAdminPermission')
},
{
statusText: DatasetErrEnum.noApiServer,
message: i18nT('common:core.dataset.error.noApiServer')
} }
]; ];
export default datasetErr.reduce((acc, cur, index) => { export default datasetErr.reduce((acc, cur, index) => {
......
packages/global/core/chat/type.ts
import { SearchDataResponseItemSchema } from '../dataset/type'; import { SearchDataResponseQuoteListItemSchema } from '../dataset/type';
import { import {
ChatFileTypeEnum, ChatFileTypeEnum,
ChatGenerateStatusEnum, ChatGenerateStatusEnum,
ChatRoleEnum, ChatRoleEnum,
type ChatSourceEnum ChatSourceEnum
} from './constants'; } from './constants';
import { FlowNodeTypeEnum } from '../workflow/node/constant'; import { FlowNodeTypeEnum } from '../workflow/node/constant';
import { DispatchNodeResponseKeyEnum } from '../workflow/runtime/constants'; import { DispatchNodeResponseKeyEnum } from '../workflow/runtime/constants';
import { AppSchemaTypeSchema, type AppSchemaType, type VariableItemType } from '../app/type'; import { AppSchemaTypeSchema, VariableItemTypeSchema } from '../app/type';
import { DispatchNodeResponseSchema } from '../workflow/runtime/type'; import { DispatchNodeResponseSchema } from '../workflow/runtime/type';
import { WorkflowInteractiveResponseTypeSchema } from '../workflow/template/system/interactive/type'; import { WorkflowInteractiveResponseTypeSchema } from '../workflow/template/system/interactive/type';
import type { FlowNodeInputItemType } from '../workflow/type/io'; import { FlowNodeInputItemTypeSchema } from '../workflow/type/io';
import z from 'zod'; import z from 'zod';
import { import {
AgentLoopAskSchema, AgentLoopAskSchema,
...@@ -19,6 +19,7 @@ import { ...@@ -19,6 +19,7 @@ import {
AgentPlanSchema, AgentPlanSchema,
AgentPlanStatusSchema AgentPlanStatusSchema
} from '../ai/agent/type'; } from '../ai/agent/type';
import { ObjectIdSchema } from '../../common/type/mongo';
export const ChatHistoryItemResSchema = DispatchNodeResponseSchema.extend({ export const ChatHistoryItemResSchema = DispatchNodeResponseSchema.extend({
nodeId: z.string(), nodeId: z.string(),
...@@ -80,49 +81,56 @@ export const SkillModuleResponseItemSchema = z.object({ ...@@ -80,49 +81,56 @@ export const SkillModuleResponseItemSchema = z.object({
export type SkillModuleResponseItemType = z.infer<typeof SkillModuleResponseItemSchema>; export type SkillModuleResponseItemType = z.infer<typeof SkillModuleResponseItemSchema>;
/* --------- chat ---------- */ /* --------- chat ---------- */
export type ChatSchemaType = { export const ChatSchema = z.object({
_id: string; _id: ObjectIdSchema,
chatId: string; chatId: z.string(),
userId: string; userId: ObjectIdSchema,
teamId: string; teamId: ObjectIdSchema,
tmbId: string; tmbId: ObjectIdSchema,
appId: string; appId: ObjectIdSchema.meta({
appVersionId?: string; description: '目前已经变成 sourceId，可能是 app 的，也可能是 skill 的'
createTime: Date; }),
updateTime: Date; appVersionId: ObjectIdSchema.optional().meta({ description: 'appId 为 app 时候才有' }),
title: string; createTime: z.coerce.date(),
customTitle: string; updateTime: z.coerce.date(),
top: boolean; title: z.string(),
source: `${ChatSourceEnum}`; customTitle: z.string().optional(),
sourceName?: string; top: z.boolean().default(false),
source: z.enum(ChatSourceEnum),
shareId?: string; sourceName: z.string().optional(),
outLinkUid?: string;
shareId: z.string().optional(),
variableList?: VariableItemType[]; outLinkUid: z.string().optional(),
welcomeText?: string;
variables: Record<string, any>; variableList: z.array(VariableItemTypeSchema).optional(),
pluginInputs?: FlowNodeInputItemType[]; welcomeText: z.string().optional(),
metadata?: Record<string, any>; variables: z.record(z.string(), z.any()),
pluginInputs: z.array(FlowNodeInputItemTypeSchema).optional(),
metadata: z.record(z.string(), z.any()).optional(),
// Boolean flags for efficient filtering // Boolean flags for efficient filtering
hasGoodFeedback?: boolean; hasGoodFeedback: z.boolean().optional(),
hasBadFeedback?: boolean; hasBadFeedback: z.boolean().optional(),
hasUnreadGoodFeedback?: boolean; hasUnreadGoodFeedback: z.boolean().optional(),
hasUnreadBadFeedback?: boolean; hasUnreadBadFeedback: z.boolean().optional(),
// Error count (redundant field for performance) // Error count (redundant field for performance)
errorCount?: number; errorCount: z.number().optional(),
/** 旧数据可能无此字段；业务上按 done 处理 */ /** 旧数据可能无此字段；业务上按 done 处理 */
chatGenerateStatus?: ChatGenerateStatusEnum; chatGenerateStatus: z
hasBeenRead: boolean; .enum(ChatGenerateStatusEnum)
.default(ChatGenerateStatusEnum.done)
.meta({ description: '生成状态' }),
hasBeenRead: z.boolean().default(true),
deleteTime?: Date | null; deleteTime: z.coerce.date().nullish()
}; });
export type ChatSchemaType = z.infer<typeof ChatSchema>;
export type ChatWithAppSchema = Omit<ChatSchemaType, 'appId'> & { export const ChatWithAppSchema = ChatSchema.omit({ appId: true }).extend({
appId: AppSchemaType; appId: AppSchemaTypeSchema
}; });
export type ChatWithAppSchema = z.infer<typeof ChatWithAppSchema>;
/* --------- chat item ---------- */ /* --------- chat item ---------- */
// User // User
...@@ -320,7 +328,7 @@ export type ToolCiteLinksType = z.infer<typeof ToolCiteLinksSchema>; ...@@ -320,7 +328,7 @@ export type ToolCiteLinksType = z.infer<typeof ToolCiteLinksSchema>;
export const ResponseTagItemSchema = z.object({ export const ResponseTagItemSchema = z.object({
useAgentSandbox: z.boolean().optional(), useAgentSandbox: z.boolean().optional(),
totalQuoteList: z.array(SearchDataResponseItemSchema).optional(), totalQuoteList: z.array(SearchDataResponseQuoteListItemSchema).optional(),
toolCiteLinks: z.array(ToolCiteLinksSchema).optional(), toolCiteLinks: z.array(ToolCiteLinksSchema).optional(),
errorText: ErrorTextItemSchema.optional(), errorText: ErrorTextItemSchema.optional(),
llmModuleAccount: z.number().optional().meta({ deprecated: true }), llmModuleAccount: z.number().optional().meta({ deprecated: true }),
......
packages/global/core/dataset/type.ts
...@@ -428,6 +428,25 @@ export const SearchDataResponseItemSchema = DatasetDataItemSchema.omit({ ...@@ -428,6 +428,25 @@ export const SearchDataResponseItemSchema = DatasetDataItemSchema.omit({
.meta({ description: '搜索数据响应项' }); .meta({ description: '搜索数据响应项' });
export type SearchDataResponseItemType = z.infer<typeof SearchDataResponseItemSchema>; export type SearchDataResponseItemType = z.infer<typeof SearchDataResponseItemSchema>;
export const SearchDataResponseQuoteItemSchema = SearchDataResponseItemSchema.pick({
id: true,
chunkIndex: true,
datasetId: true,
collectionId: true,
sourceId: true,
sourceName: true,
score: true
}).meta({ description: '搜索数据引用响应项（精简）' });
export type SearchDataResponseQuoteItemType = z.infer<typeof SearchDataResponseQuoteItemSchema>;
export const SearchDataResponseQuoteListItemSchema = z.union([
SearchDataResponseItemSchema,
SearchDataResponseQuoteItemSchema
]);
export type SearchDataResponseQuoteListItemType = z.infer<
typeof SearchDataResponseQuoteListItemSchema
>;
export const DatasetCiteItemSchema = z export const DatasetCiteItemSchema = z
.object({ .object({
_id: ObjectIdSchema.meta({ description: '数据 ID' }), _id: ObjectIdSchema.meta({ description: '数据 ID' }),
......
packages/global/core/workflow/runtime/type.ts
...@@ -24,7 +24,7 @@ import type { ...@@ -24,7 +24,7 @@ import type {
InteractiveNodeResponseType, InteractiveNodeResponseType,
WorkflowInteractiveResponseType WorkflowInteractiveResponseType
} from '../template/system/interactive/type'; } from '../template/system/interactive/type';
import { SearchDataResponseItemSchema } from '../../dataset/type'; import { SearchDataResponseQuoteListItemSchema } from '../../dataset/type';
import type { localeType } from '../../../common/i18n/type'; import type { localeType } from '../../../common/i18n/type';
import { type ChatFileStoreValue, type UserChatItemValueItemType } from '../../chat/type'; import { type ChatFileStoreValue, type UserChatItemValueItemType } from '../../chat/type';
import { DatasetSearchModeEnum } from '../../dataset/constants'; import { DatasetSearchModeEnum } from '../../dataset/constants';
...@@ -198,7 +198,7 @@ export const DispatchNodeResponseSchema = z ...@@ -198,7 +198,7 @@ export const DispatchNodeResponseSchema = z
temperature: z.number().optional().meta({ description: '温度' }), temperature: z.number().optional().meta({ description: '温度' }),
maxToken: z.number().optional().meta({ description: '最大 token' }), maxToken: z.number().optional().meta({ description: '最大 token' }),
quoteList: z quoteList: z
.array(SearchDataResponseItemSchema) .array(SearchDataResponseQuoteListItemSchema)
.optional() .optional()
.meta({ description: '知识库引用列表' }), .meta({ description: '知识库引用列表' }),
reasoningText: z.string().optional().meta({ description: '思考文本' }), reasoningText: z.string().optional().meta({ description: '思考文本' }),
......
packages/global/test/core/chat/type.test.ts
import { describe, expect, it } from 'vitest'; import { describe, expect, it } from 'vitest';
import { import {
ChatSchema,
ToolModuleResponseItemSchema, ToolModuleResponseItemSchema,
UserChatItemFileItemSchema, UserChatItemFileItemSchema,
UserChatItemValueItemSchema, UserChatItemValueItemSchema,
...@@ -12,7 +13,66 @@ import { ...@@ -12,7 +13,66 @@ import {
ToolCiteLinksSchema, ToolCiteLinksSchema,
RuntimeUserPromptSchema RuntimeUserPromptSchema
} from '@fastgpt/global/core/chat/type'; } from '@fastgpt/global/core/chat/type';
import { ChatRoleEnum, ChatFileTypeEnum } from '@fastgpt/global/core/chat/constants'; import {
ChatRoleEnum,
ChatFileTypeEnum,
ChatGenerateStatusEnum,
ChatSourceEnum
} from '@fastgpt/global/core/chat/constants';
describe('ChatSchema', () => {
const chatId = 'chat-1';
const objectId = '68ee0bd23d17260b7829b137';
it('should validate chat document and coerce date fields', () => {
const result = ChatSchema.safeParse({
_id: objectId,
chatId,
userId: objectId,
teamId: objectId,
tmbId: objectId,
appId: objectId,
createTime: '2026-05-24T00:00:00.000Z',
updateTime: new Date('2026-05-24T00:00:01.000Z'),
title: '历史记录',
customTitle: '',
top: false,
source: ChatSourceEnum.online,
variables: {},
errorCount: 0,
chatGenerateStatus: ChatGenerateStatusEnum.done,
hasBeenRead: false,
deleteTime: null
});
expect(result.success).toBe(true);
if (result.success) {
expect(result.data.createTime).toBeInstanceOf(Date);
expect(result.data.deleteTime).toBeNull();
}
});
it('should reject invalid chat source', () => {
const result = ChatSchema.safeParse({
_id: objectId,
chatId,
userId: objectId,
teamId: objectId,
tmbId: objectId,
appId: objectId,
createTime: new Date(),
updateTime: new Date(),
title: '历史记录',
customTitle: '',
top: false,
source: 'invalid',
variables: {},
hasBeenRead: false
});
expect(result.success).toBe(false);
});
});
describe('ToolModuleResponseItemSchema', () => { describe('ToolModuleResponseItemSchema', () => {
it('should validate valid tool response', () => { it('should validate valid tool response', () => {
......
packages/service/core/chat/saveChat.ts
...@@ -38,6 +38,7 @@ import { getFlatAppResponses } from '@fastgpt/global/core/chat/utils'; ...@@ -38,6 +38,7 @@ import { getFlatAppResponses } from '@fastgpt/global/core/chat/utils';
import { getErrText } from '@fastgpt/global/common/error/utils'; import { getErrText } from '@fastgpt/global/common/error/utils';
import { getNanoid } from '@fastgpt/global/common/string/tools'; import { getNanoid } from '@fastgpt/global/common/string/tools';
import { normalizeChatFileStoreValues } from './fileStoreValue'; import { normalizeChatFileStoreValues } from './fileStoreValue';
import { cloneDeep } from 'lodash';
export type Props = { export type Props = {
chatId: string; chatId: string;
...@@ -178,10 +179,10 @@ const formatAiContent = ({ ...@@ -178,10 +179,10 @@ const formatAiContent = ({
const { responseData, ...aiResponse } = aiContent; const { responseData, ...aiResponse } = aiContent;
const citeCollectionIds = new Set<string>(); const citeCollectionIds = new Set<string>();
const cloneResponseData = cloneDeep(responseData || []);
const dealResponseData = (responseItem: ChatHistoryItemResType) => { getFlatAppResponses(cloneResponseData).forEach((responseItem: ChatHistoryItemResType) => {
if (responseItem.moduleType === FlowNodeTypeEnum.datasetSearchNode && responseItem.quoteList) { if (responseItem.moduleType === FlowNodeTypeEnum.datasetSearchNode && responseItem.quoteList) {
// @ts-ignore
responseItem.quoteList = responseItem.quoteList.map((quote) => { responseItem.quoteList = responseItem.quoteList.map((quote) => {
citeCollectionIds.add(quote.collectionId); citeCollectionIds.add(quote.collectionId);
return { return {
...@@ -195,10 +196,9 @@ const formatAiContent = ({ ...@@ -195,10 +196,9 @@ const formatAiContent = ({
}; };
}); });
} }
}; });
getFlatAppResponses(responseData || []).forEach(dealResponseData);
const errorCount = responseData?.filter((item) => item.errorText).length ?? 0; const errorCount = cloneResponseData?.filter((item) => item.errorText).length ?? 0;
return { return {
aiResponse: { aiResponse: {
...@@ -207,7 +207,7 @@ const formatAiContent = ({ ...@@ -207,7 +207,7 @@ const formatAiContent = ({
errorMsg, errorMsg,
citeCollectionIds: Array.from(citeCollectionIds) citeCollectionIds: Array.from(citeCollectionIds)
}, },
nodeResponses: responseData, nodeResponses: cloneResponseData,
citeCollectionIds, citeCollectionIds,
errorCount errorCount
}; };
......
packages/service/test/core/chat/saveChat.test.ts
...@@ -349,6 +349,11 @@ describe('pushChatRecords', () => { ...@@ -349,6 +349,11 @@ describe('pushChatRecords', () => {
await pushChatRecords(props); await pushChatRecords(props);
expect(props.aiContent.responseData?.[0]?.quoteList?.[0]).toMatchObject({
q: quote.q,
a: quote.a
});
const responses = await MongoChatItemResponse.find({ const responses = await MongoChatItemResponse.find({
appId: testAppId, appId: testAppId,
chatId: props.chatId chatId: props.chatId
......
packages/web/hooks/useLinkedScroll.tsx
...@@ -2,7 +2,7 @@ import { useCallback, useEffect, useRef, useState, type ReactNode } from 'react' ...@@ -2,7 +2,7 @@ import { useCallback, useEffect, useRef, useState, type ReactNode } from 'react'
import { type LinkedListResponse, type LinkedPaginationProps } from '@fastgpt/global/openapi/api'; import { type LinkedListResponse, type LinkedPaginationProps } from '@fastgpt/global/openapi/api';
import { Box, type BoxProps } from '@chakra-ui/react'; import { Box, type BoxProps } from '@chakra-ui/react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { useScroll, useMemoizedFn, useDebounceEffect, useLatest } from 'ahooks'; import { useScroll, useDebounceEffect } from 'ahooks';
import MyBox from '../components/common/MyBox'; import MyBox from '../components/common/MyBox';
import { useRequest } from './useRequest'; import { useRequest } from './useRequest';
...@@ -42,37 +42,44 @@ export function useLinkedScroll< ...@@ -42,37 +42,44 @@ export function useLinkedScroll<
const isInit = useRef(false); const isInit = useRef(false);
const scrollToItem = useCallback( const scrollToItem = useCallback(
async (id?: string) => { (id?: string) => {
if (!id) { const targetId =
id = defaultScroll === 'top' ? dataList[0]?.id : dataList[dataList.length - 1]?.id; id || (defaultScroll === 'top' ? dataList[0]?.id : dataList[dataList.length - 1]?.id);
}
const itemIndex = dataList.findIndex((item) => item.id === id); if (!targetId) {
if (itemIndex === -1) {
return; return;
} }
const element = itemRefs.current.get(id); const itemIndex = dataList.findIndex((item) => item.id === targetId);
if (!element || !containerRef.current) { if (itemIndex === -1) {
requestAnimationFrame(() => scrollToItem(id));
return; return;
} }
const elementRect = element.getBoundingClientRect(); const tryScroll = () => {
const containerRect = containerRef.current.getBoundingClientRect(); const element = itemRefs.current.get(targetId);
if (!element || !containerRef.current) {
requestAnimationFrame(tryScroll);
return;
}
const elementRect = element.getBoundingClientRect();
const containerRect = containerRef.current.getBoundingClientRect();
const scrollTop = containerRef.current.scrollTop + elementRect.top - containerRect.top;
const scrollTop = containerRef.current.scrollTop + elementRect.top - containerRect.top; containerRef.current.scrollTo({
top: scrollTop
});
};
containerRef.current.scrollTo({ tryScroll();
top: scrollTop
});
}, },
[dataList, defaultScroll] [dataList, defaultScroll]
); );
const { runAsync: callApi, loading: isLoading } = useRequest(api); const { runAsync: callApi, loading: isLoading } = useRequest(api, { errorToast: '' });
let scrollSign = useRef(false); const scrollSign = useRef(false);
const { runAsync: loadInitData } = useRequest( const { runAsync: loadInitData } = useRequest(
async ({ scrollWhenFinish, refresh } = { scrollWhenFinish: true, refresh: false }) => { async ({ scrollWhenFinish, refresh } = { scrollWhenFinish: true, refresh: false }) => {
if (isLoading) return; if (isLoading) return;
......
projects/app/src/components/core/chat/ChatContainer/ChatBox/components/QuoteList.tsx
import React, { useMemo } from 'react'; import React, { useMemo } from 'react';
import { Box, useTheme } from '@chakra-ui/react'; import { Box, useTheme } from '@chakra-ui/react';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import type { SearchDataResponseQuoteListItemType } from '@fastgpt/global/core/dataset/type';
import QuoteItem, { formatScore } from '@/components/core/dataset/QuoteItem'; import QuoteItem, { formatScore } from '@/components/core/dataset/QuoteItem';
import { useContextSelector } from 'use-context-selector'; import { useContextSelector } from 'use-context-selector';
import { WorkflowRuntimeContext } from '../../context/workflowRuntimeContext'; import { WorkflowRuntimeContext } from '../../context/workflowRuntimeContext';
...@@ -15,7 +15,7 @@ const QuoteList = React.memo(function QuoteList({ ...@@ -15,7 +15,7 @@ const QuoteList = React.memo(function QuoteList({
rawSearch = [] rawSearch = []
}: { }: {
chatItemDataId?: string; chatItemDataId?: string;
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseQuoteListItemType[];
}) { }) {
const theme = useTheme(); const theme = useTheme();
const { appId, outLinkAuthData } = useChatStore(); const { appId, outLinkAuthData } = useChatStore();
...@@ -62,7 +62,7 @@ const QuoteList = React.memo(function QuoteList({ ...@@ -62,7 +62,7 @@ const QuoteList = React.memo(function QuoteList({
}; };
} }
return { ...item, q: item.q || '' }; return { ...item, q: 'q' in item ? item.q : '' };
}); });
return processedData.sort((a, b) => { return processedData.sort((a, b) => {
......
projects/app/src/components/core/chat/ChatContainer/ChatBox/components/ResponseTags.tsx
import React, { useMemo, useState } from 'react'; import React, { useMemo, useState } from 'react';
import { Flex, useDisclosure, Box } from '@chakra-ui/react'; import { Flex, useDisclosure, Box } from '@chakra-ui/react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import type { SearchDataResponseQuoteListItemType } from '@fastgpt/global/core/dataset/type';
import dynamic from 'next/dynamic'; import dynamic from 'next/dynamic';
import MyTag from '@fastgpt/web/components/common/Tag/index'; import MyTag from '@fastgpt/web/components/common/Tag/index';
import MyTooltip from '@fastgpt/web/components/common/MyTooltip'; import MyTooltip from '@fastgpt/web/components/common/MyTooltip';
...@@ -95,7 +95,7 @@ const ResponseTags = ({ ...@@ -95,7 +95,7 @@ const ResponseTags = ({
// Dataset citations // Dataset citations
const datasetItems = Object.values( const datasetItems = Object.values(
quoteList.reduce((acc: Record<string, SearchDataResponseItemType[]>, cur) => { quoteList.reduce((acc: Record<string, SearchDataResponseQuoteListItemType[]>, cur) => {
if (!acc[cur.collectionId]) { if (!acc[cur.collectionId]) {
acc[cur.collectionId] = [cur]; acc[cur.collectionId] = [cur];
} }
...@@ -107,9 +107,10 @@ const ResponseTags = ({ ...@@ -107,9 +107,10 @@ const ResponseTags = ({
type: 'dataset' as const, type: 'dataset' as const,
key: item.collectionId, key: item.collectionId,
displayText: item.sourceName, displayText: item.sourceName,
icon: item.imageId icon:
? 'core/dataset/imageFill' 'imageId' in item && item.imageId
: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }), ? 'core/dataset/imageFill'
: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }),
onClick: () => { onClick: () => {
onOpenCiteModal({ onOpenCiteModal({
collectionId: item.collectionId, collectionId: item.collectionId,
......
projects/app/src/components/core/dataset/QuoteItem.tsx
import React, { useMemo, useState } from 'react'; import React, { useMemo, useState } from 'react';
import { Box, Flex, Link, Progress } from '@chakra-ui/react'; import { Box, Flex, Link, Progress } from '@chakra-ui/react';
import RawSourceBox from '@/components/core/dataset/RawSourceBox'; import RawSourceBox from '@/components/core/dataset/RawSourceBox';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import type { SearchDataResponseQuoteItemType } from '@fastgpt/global/core/dataset/type';
import NextLink from 'next/link'; import NextLink from 'next/link';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
...@@ -85,7 +85,11 @@ const QuoteItem = ({ ...@@ -85,7 +85,11 @@ const QuoteItem = ({
canEditDataset, canEditDataset,
...RawSourceBoxProps ...RawSourceBoxProps
}: { }: {
quoteItem: SearchDataResponseItemType; quoteItem: SearchDataResponseQuoteItemType & {
q: string;
a?: string;
imagePreivewUrl?: string;
};
canDownloadSource?: boolean; canDownloadSource?: boolean;
canEditData?: boolean; canEditData?: boolean;
canEditDataset?: boolean; canEditDataset?: boolean;
......
projects/app/src/global/core/chat/utils.ts
...@@ -5,7 +5,7 @@ import type { ...@@ -5,7 +5,7 @@ import type {
ToolCiteLinksType, ToolCiteLinksType,
ErrorTextItemType ErrorTextItemType
} from '@fastgpt/global/core/chat/type'; } from '@fastgpt/global/core/chat/type';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import type { SearchDataResponseQuoteListItemType } from '@fastgpt/global/core/dataset/type';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { getFlatAppResponses } from '@fastgpt/global/core/chat/utils'; import { getFlatAppResponses } from '@fastgpt/global/core/chat/utils';
import { sandboxToolMap } from '@fastgpt/global/core/ai/sandbox/tools'; import { sandboxToolMap } from '@fastgpt/global/core/ai/sandbox/tools';
...@@ -32,8 +32,8 @@ export function transformPreviewHistories( ...@@ -32,8 +32,8 @@ export function transformPreviewHistories(
const extractCitationIdsFromText = (text: string): string[] => { const extractCitationIdsFromText = (text: string): string[] => {
if (!text) return []; if (!text) return [];
// Match [24-bit hexadecimal ID](CITE) format // Match [24-bit hexadecimal ID](CITE|QUOTE) format. Markdown rendering supports both.
const citeRegex = /\[([a-f0-9]{24})\]\(CITE\)/gi; const citeRegex = /\[([a-f0-9]{24})\]\((?:CITE|QUOTE)\)/gi;
const matches = text.match(citeRegex); const matches = text.match(citeRegex);
if (!matches) return []; if (!matches) return [];
...@@ -111,7 +111,7 @@ export function addStatisticalDataToHistoryItem(historyItem: ChatItemMiniType) { ...@@ -111,7 +111,7 @@ export function addStatisticalDataToHistoryItem(historyItem: ChatItemMiniType) {
}, },
{ {
useAgentSandbox: false, useAgentSandbox: false,
totalQuoteList: [] as SearchDataResponseItemType[], totalQuoteList: [] as SearchDataResponseQuoteListItemType[],
toolCiteLinks: [] as ToolCiteLinksType[], toolCiteLinks: [] as ToolCiteLinksType[],
linkDedupe: new Set<string>(), linkDedupe: new Set<string>(),
errorText: undefined as ErrorTextItemType | undefined, errorText: undefined as ErrorTextItemType | undefined,
......
projects/app/src/pageComponents/chat/ChatQuoteList/CollectionQuoteItem.tsx
import Markdown from '@/components/Markdown'; import Markdown from '@/components/Markdown';
import { Box, Flex } from '@chakra-ui/react'; import { Box, Flex } from '@chakra-ui/react';
import MyTooltip from '@fastgpt/web/components/common/MyTooltip'; import MyTooltip from '@fastgpt/web/components/common/MyTooltip';
import { type Dispatch, type MutableRefObject, type SetStateAction, useState } from 'react'; import { type MutableRefObject, useState } from 'react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
import { useCopyData } from '@fastgpt/web/hooks/useCopyData'; import { useCopyData } from '@fastgpt/web/hooks/useCopyData';
...@@ -23,7 +23,7 @@ const CollectionQuoteItem = ({ ...@@ -23,7 +23,7 @@ const CollectionQuoteItem = ({
}: { }: {
quoteRefs: MutableRefObject<Map<string, HTMLDivElement | null>>; quoteRefs: MutableRefObject<Map<string, HTMLDivElement | null>>;
quoteIndex: number; quoteIndex: number;
setQuoteIndex: Dispatch<SetStateAction<number>>; setQuoteIndex: (quoteIndex: number) => void;
refreshList: () => void; refreshList: () => void;
canEdit: boolean; canEdit: boolean;
......
projects/app/src/pageComponents/chat/ChatQuoteList/CollectionQuoteReader.tsx
import { Box, Flex, HStack } from '@chakra-ui/react'; import { Box, Flex, HStack } from '@chakra-ui/react';
import { type SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import { type SearchDataResponseQuoteListItemType } from '@fastgpt/global/core/dataset/type';
import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils'; import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
import { useRouter } from 'next/router'; import { useRouter } from 'next/router';
...@@ -28,7 +28,7 @@ const CollectionReader = ({ ...@@ -28,7 +28,7 @@ const CollectionReader = ({
metadata, metadata,
onClose onClose
}: { }: {
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseQuoteListItemType[];
metadata: GetCollectionQuoteDataProps; metadata: GetCollectionQuoteDataProps;
onClose: () => void; onClose: () => void;
}) => { }) => {
...@@ -39,7 +39,7 @@ const CollectionReader = ({ ...@@ -39,7 +39,7 @@ const CollectionReader = ({
const canDownloadSource = useContextSelector(ChatItemContext, (v) => v.canDownloadSource); const canDownloadSource = useContextSelector(ChatItemContext, (v) => v.canDownloadSource);
const { collectionId, datasetId, chatItemDataId, sourceId, sourceName, quoteId } = metadata; const { collectionId, datasetId, chatItemDataId, sourceId, sourceName, quoteId } = metadata;
const [quoteIndex, setQuoteIndex] = useState(0); const [selectedQuote, setSelectedQuote] = useState<{ sourceQuoteId?: string; id: string }>();
// Get dataset permission // Get dataset permission
const { data: datasetData } = useRequest(async () => await getDatasetPermission(datasetId), { const { data: datasetData } = useRequest(async () => await getDatasetPermission(datasetId), {
...@@ -48,7 +48,7 @@ const CollectionReader = ({ ...@@ -48,7 +48,7 @@ const CollectionReader = ({
}); });
const filterResults = useMemo(() => { const filterResults = useMemo(() => {
const res = rawSearch return rawSearch
.filter((item) => item.collectionId === collectionId) .filter((item) => item.collectionId === collectionId)
.sort((a, b) => { .sort((a, b) => {
const chunkDiff = (a.chunkIndex || 0) - (b.chunkIndex || 0); const chunkDiff = (a.chunkIndex || 0) - (b.chunkIndex || 0);
...@@ -56,15 +56,28 @@ const CollectionReader = ({ ...@@ -56,15 +56,28 @@ const CollectionReader = ({
return a.id.localeCompare(b.id); return a.id.localeCompare(b.id);
}); });
}, [collectionId, rawSearch]);
if (quoteId) { const quoteIndex = useMemo(() => {
setQuoteIndex(res.findIndex((item) => item.id === quoteId)); const selectedQuoteId =
} else { selectedQuote && selectedQuote.sourceQuoteId === quoteId ? selectedQuote.id : quoteId;
setQuoteIndex(0); if (!selectedQuoteId) return 0;
}
return Math.max(
filterResults.findIndex((item) => item.id === selectedQuoteId),
0
);
}, [filterResults, quoteId, selectedQuote]);
return res; const setQuoteIndex = (index: number) => {
}, [collectionId, quoteId, rawSearch]); const nextQuote = filterResults[index];
if (!nextQuote) return;
setSelectedQuote({
sourceQuoteId: quoteId,
id: nextQuote.id
});
};
const currentQuoteItem = useMemo(() => { const currentQuoteItem = useMemo(() => {
const item = filterResults[quoteIndex]; const item = filterResults[quoteIndex];
...@@ -281,7 +294,7 @@ const CollectionReader = ({ ...@@ -281,7 +294,7 @@ const CollectionReader = ({
{isLoading || datasetDataList.length > 0 ? ( {isLoading || datasetDataList.length > 0 ? (
<ScrollData flex={'1 0 0'} mt={2} px={5} py={1}> <ScrollData flex={'1 0 0'} mt={2} px={5} py={1}>
<Flex flexDir={'column'}> <Flex flexDir={'column'}>
{formatedDataList.map((item, index) => ( {formatedDataList.map((item) => (
<CollectionQuoteItem <CollectionQuoteItem
key={item._id} key={item._id}
quoteRefs={itemRefs as React.MutableRefObject<Map<string, HTMLDivElement | null>>} quoteRefs={itemRefs as React.MutableRefObject<Map<string, HTMLDivElement | null>>}
......
projects/app/src/pageComponents/chat/ChatQuoteList/QuoteReader.tsx
import { Box, Flex } from '@chakra-ui/react'; import { Box, Flex } from '@chakra-ui/react';
import { type SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import { type SearchDataResponseQuoteListItemType } from '@fastgpt/global/core/dataset/type';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
import MyBox from '@fastgpt/web/components/common/MyBox'; import MyBox from '@fastgpt/web/components/common/MyBox';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
...@@ -16,7 +16,7 @@ const QuoteReader = ({ ...@@ -16,7 +16,7 @@ const QuoteReader = ({
metadata, metadata,
onClose onClose
}: { }: {
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseQuoteListItemType[];
metadata: GetAllQuoteDataProps; metadata: GetAllQuoteDataProps;
onClose: () => void; onClose: () => void;
}) => { }) => {
......
projects/app/src/pageComponents/chat/ChatQuoteList/index.tsx
import React from 'react'; import React from 'react';
import { type SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import { type SearchDataResponseQuoteListItemType } from '@fastgpt/global/core/dataset/type';
import { type GetQuoteProps } from '@/web/core/chat/context/chatItemContext'; import { type GetQuoteProps } from '@/web/core/chat/context/chatItemContext';
import CollectionQuoteReader from './CollectionQuoteReader'; import CollectionQuoteReader from './CollectionQuoteReader';
import QuoteReader from './QuoteReader'; import QuoteReader from './QuoteReader';
...@@ -9,7 +9,7 @@ const ChatQuoteList = ({ ...@@ -9,7 +9,7 @@ const ChatQuoteList = ({
metadata, metadata,
onClose onClose
}: { }: {
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseQuoteListItemType[];
metadata: GetQuoteProps; metadata: GetQuoteProps;
onClose: () => void; onClose: () => void;
}) => { }) => {
......
projects/app/src/pages/api/core/chat/record/getCollectionQuote.ts
...@@ -50,7 +50,7 @@ async function handler(req: ApiRequestProps): Promise<GetCollectionQuoteResType> ...@@ -50,7 +50,7 @@ async function handler(req: ApiRequestProps): Promise<GetCollectionQuoteResType>
teamToken teamToken
}), }),
MongoChatItem.findOne({ appId, chatId, dataId: chatItemDataId }, 'time').lean(), MongoChatItem.findOne({ appId, chatId, dataId: chatItemDataId }, 'time').lean(),
authCollectionInChat({ appId, chatId, chatItemDataId, collectionIds: [collectionId] }) authCollectionInChat({ appId, chatId, collectionIds: [collectionId] })
]); ]);
if (!showFullText || !chat || !chatItem || initialAnchor === undefined) { if (!showFullText || !chat || !chatItem || initialAnchor === undefined) {
......
projects/app/src/pages/api/core/chat/record/getQuote.ts
...@@ -38,7 +38,7 @@ async function handler(req: ApiRequestProps): Promise<GetQuoteResponseType> { ...@@ -38,7 +38,7 @@ async function handler(req: ApiRequestProps): Promise<GetQuoteResponseType> {
teamToken teamToken
}), }),
MongoChatItem.findOne({ appId, chatId, dataId: chatItemDataId }, 'time').lean(), MongoChatItem.findOne({ appId, chatId, dataId: chatItemDataId }, 'time').lean(),
authCollectionInChat({ appId, chatId, chatItemDataId, collectionIds: collectionIdList }) authCollectionInChat({ appId, chatId, collectionIds: collectionIdList })
]); ]);
if (!chat || !chatItem || !showCite) return Promise.reject(ChatErrEnum.unAuthChat); if (!chat || !chatItem || !showCite) return Promise.reject(ChatErrEnum.unAuthChat);
......
projects/app/src/pages/api/core/dataset/collection/export.ts
...@@ -41,8 +41,7 @@ async function handler(req: ApiRequestProps, res: NextApiResponse) { ...@@ -41,8 +41,7 @@ async function handler(req: ApiRequestProps, res: NextApiResponse) {
}; };
} }
const { appId, chatId, chatItemDataId, shareId, outLinkUid, teamId, teamToken, chatTime } = const { appId, chatId, shareId, outLinkUid, teamId, teamToken, chatTime } = parseBody;
parseBody;
/* /*
1. auth chat read permission 1. auth chat read permission
2. auth collection quote in chat 2. auth collection quote in chat
...@@ -60,7 +59,7 @@ async function handler(req: ApiRequestProps, res: NextApiResponse) { ...@@ -60,7 +59,7 @@ async function handler(req: ApiRequestProps, res: NextApiResponse) {
teamToken teamToken
}), }),
getCollectionWithDataset(collectionId), getCollectionWithDataset(collectionId),
authCollectionInChat({ appId, chatId, chatItemDataId, collectionIds: [collectionId] }) authCollectionInChat({ appId, chatId, collectionIds: [collectionId] })
]); ]);
if (!authRes.canDownloadSource) { if (!authRes.canDownloadSource) {
......
projects/app/src/pages/api/core/dataset/collection/read.ts
...@@ -48,7 +48,7 @@ async function handler(req: ApiRequestProps): Promise<ReadCollectionSourceRespon ...@@ -48,7 +48,7 @@ async function handler(req: ApiRequestProps): Promise<ReadCollectionSourceRespon
teamToken teamToken
}), }),
getCollectionWithDataset(collectionId), getCollectionWithDataset(collectionId),
authCollectionInChat({ appId, chatId, chatItemDataId, collectionIds: [collectionId] }) authCollectionInChat({ appId, chatId, collectionIds: [collectionId] })
]); ]);
if (!authRes.canDownloadSource) { if (!authRes.canDownloadSource) {
......
projects/app/src/pages/api/core/dataset/data/getQuoteData.ts
...@@ -23,7 +23,7 @@ async function handler(req: ApiRequestProps): Promise<GetQuoteDataResponse> { ...@@ -23,7 +23,7 @@ async function handler(req: ApiRequestProps): Promise<GetQuoteDataResponse> {
// Auth // Auth
const { collection, q, a } = await (async () => { const { collection, q, a } = await (async () => {
if (body.chatId && body.appId && body.chatItemDataId) { if (body.chatId && body.appId && body.chatItemDataId) {
const { appId, chatId, shareId, outLinkUid, teamId, teamToken, chatItemDataId } = body; const { appId, chatId, shareId, outLinkUid, teamId, teamToken } = body;
await authChatCrud({ await authChatCrud({
req, req,
authToken: true, authToken: true,
...@@ -55,7 +55,6 @@ async function handler(req: ApiRequestProps): Promise<GetQuoteDataResponse> { ...@@ -55,7 +55,6 @@ async function handler(req: ApiRequestProps): Promise<GetQuoteDataResponse> {
authCollectionInChat({ authCollectionInChat({
appId, appId,
chatId, chatId,
chatItemDataId,
collectionIds: [datasetData.collectionId] collectionIds: [datasetData.collectionId]
}) })
]); ]);
......
projects/app/src/service/support/permission/auth/chat.ts
import { type ChatHistoryItemResType, type ChatSchemaType } from '@fastgpt/global/core/chat/type'; import { type ChatSchemaType } from '@fastgpt/global/core/chat/type';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema'; import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import { type AuthModeType } from '@fastgpt/service/support/permission/type'; import { type AuthModeType } from '@fastgpt/service/support/permission/type';
import { authOutLink } from './outLink'; import { authOutLink } from './outLink';
...@@ -8,13 +8,12 @@ import { AuthUserTypeEnum, ReadPermissionVal } from '@fastgpt/global/support/per ...@@ -8,13 +8,12 @@ import { AuthUserTypeEnum, ReadPermissionVal } from '@fastgpt/global/support/per
import { authApp } from '@fastgpt/service/support/permission/app/auth'; import { authApp } from '@fastgpt/service/support/permission/app/auth';
import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema'; import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { DatasetErrEnum } from '@fastgpt/global/common/error/code/dataset'; import { DatasetErrEnum } from '@fastgpt/global/common/error/code/dataset';
import { getFlatAppResponses } from '@fastgpt/global/core/chat/utils';
import { MongoChatItemResponse } from '@fastgpt/service/core/chat/chatItemResponseSchema';
import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants'; import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
import type { HelperBotTypeEnum } from '@fastgpt/global/core/chat/helperBot/type'; import type { HelperBotTypeEnum } from '@fastgpt/global/core/chat/helperBot/type';
import { MongoHelperBotChat } from '@fastgpt/service/core/chat/HelperBot/chatSchema'; import { MongoHelperBotChat } from '@fastgpt/service/core/chat/HelperBot/chatSchema';
import { authCert } from '@fastgpt/service/support/permission/auth/common'; import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { MongoApp } from '@fastgpt/service/core/app/schema'; import { MongoApp } from '@fastgpt/service/core/app/schema';
import { Types } from 'mongoose';
/* /*
检查chat的权限： 检查chat的权限：
...@@ -237,69 +236,54 @@ export async function authChatCrud({ ...@@ -237,69 +236,54 @@ export async function authChatCrud({
return Promise.reject(ChatErrEnum.unAuthChat); return Promise.reject(ChatErrEnum.unAuthChat);
} }
/**
* 校验文档是否来自当前会话引用。
*
* 只依赖 ChatItem 上的 citeCollectionIds 判断 collection 是否在当前会话中被引用，
* 避免读取和解析完整 responseData。
*/
export const authCollectionInChat = async ({ export const authCollectionInChat = async ({
collectionIds, collectionIds,
appId, appId,
chatId, chatId
chatItemDataId
}: { }: {
collectionIds: string[]; collectionIds: string[];
appId: string; appId: string;
chatId: string; chatId: string;
chatItemDataId: string;
}) => { }) => {
try { const appObjectId = Types.ObjectId.isValid(String(appId))
// 1. 使用 citeCollectionIds 字段来判断 ? new Types.ObjectId(String(appId))
const chatItems = await MongoChatItem.find( : appId;
{ const targetCollectionIds = collectionIds.map(String);
appId,
const [authResult] = await MongoChatItem.aggregate<{ isAuthorized: boolean }>([
{
$match: {
appId: appObjectId,
chatId, chatId,
obj: ChatRoleEnum.AI obj: ChatRoleEnum.AI
}, }
'citeCollectionIds' },
) { $sort: { _id: -1 } },
.sort({ _id: -1 }) { $limit: 50 },
.limit(50) { $unwind: '$citeCollectionIds' },
.lean(); {
const citeCollectionIds = new Set( $group: {
chatItems.map((item) => ('citeCollectionIds' in item ? item.citeCollectionIds : [])).flat() _id: null,
); citeCollectionIds: { $addToSet: { $toString: '$citeCollectionIds' } }
if (collectionIds.every((id) => citeCollectionIds.has(id))) { }
return; },
} {
$project: {
// Adapt <=4.13.0 _id: 0,
const chatItem = (await MongoChatItem.findOne( isAuthorized: { $setIsSubset: [targetCollectionIds, '$citeCollectionIds'] }
{ }
appId,
chatId,
dataId: chatItemDataId
},
'responseData'
).lean()) as { time: Date; responseData?: ChatHistoryItemResType[] };
if (!chatItem) return Promise.reject(DatasetErrEnum.unAuthDatasetFile);
// Concat response data
if (!chatItem.responseData || chatItem.responseData.length === 0) {
const chatItemResponses = await MongoChatItemResponse.find(
{ appId, chatId, chatItemDataId },
{ data: 1 }
).lean();
chatItem.responseData = chatItemResponses.map((item) => item.data);
} }
]);
// 找 responseData 里，是否有该文档 id if (authResult?.isAuthorized) {
const flatResData = getFlatAppResponses(chatItem.responseData || []); return;
}
const quoteListSet = new Set(
flatResData.map((item) => item.quoteList?.map((quote) => quote.collectionId) || []).flat()
);
if (collectionIds.every((id) => quoteListSet.has(id))) {
return;
}
} catch (error) {}
return Promise.reject(DatasetErrEnum.unAuthDatasetFile); return Promise.reject(DatasetErrEnum.unAuthDatasetFile);
}; };
......
projects/app/src/web/core/chat/context/chatItemContext.tsx
import { type ChatBoxInputFormType } from '@/components/core/chat/ChatContainer/ChatBox/type'; import { type ChatBoxInputFormType } from '@/components/core/chat/ChatContainer/ChatBox/type';
import { PluginRunBoxTabEnum } from '@/components/core/chat/ChatContainer/PluginRunBox/constants'; import { PluginRunBoxTabEnum } from '@/components/core/chat/ChatContainer/PluginRunBox/constants';
import React, { type ReactNode, useCallback, useEffect, useMemo, useRef, useState } from 'react'; import React, { type ReactNode, useCallback, useMemo, useRef, useState } from 'react';
import { createContext } from 'use-context-selector'; import { createContext } from 'use-context-selector';
import { type ComponentRef as ChatComponentRef } from '@/components/core/chat/ChatContainer/ChatBox/type'; import { type ComponentRef as ChatComponentRef } from '@/components/core/chat/ChatContainer/ChatBox/type';
import { useForm, type UseFormReturn } from 'react-hook-form'; import { useForm, type UseFormReturn } from 'react-hook-form';
...@@ -8,7 +8,7 @@ import { defaultChatData } from '@/global/core/chat/constants'; ...@@ -8,7 +8,7 @@ import { defaultChatData } from '@/global/core/chat/constants';
import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { AppTypeEnum } from '@fastgpt/global/core/app/constants';
import { type AppChatConfigType, type VariableItemType } from '@fastgpt/global/core/app/type'; import { type AppChatConfigType, type VariableItemType } from '@fastgpt/global/core/app/type';
import { type FlowNodeInputItemType } from '@fastgpt/global/core/workflow/type/io'; import { type FlowNodeInputItemType } from '@fastgpt/global/core/workflow/type/io';
import { type SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import { type SearchDataResponseQuoteListItemType } from '@fastgpt/global/core/dataset/type';
import { type OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat'; import { type OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
import type { ChatGenerateStatusEnum } from '@fastgpt/global/core/chat/constants'; import type { ChatGenerateStatusEnum } from '@fastgpt/global/core/chat/constants';
...@@ -65,7 +65,7 @@ export type GetAllQuoteDataProps = GetQuoteDataBasicProps & { ...@@ -65,7 +65,7 @@ export type GetAllQuoteDataProps = GetQuoteDataBasicProps & {
}; };
export type GetQuoteProps = GetAllQuoteDataProps | GetCollectionQuoteDataProps; export type GetQuoteProps = GetAllQuoteDataProps | GetCollectionQuoteDataProps;
export type QuoteDataType = { export type QuoteDataType = {
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseQuoteListItemType[];
metadata: GetQuoteProps; metadata: GetQuoteProps;
}; };
export type OnOpenCiteModalProps = { export type OnOpenCiteModalProps = {
...@@ -131,7 +131,7 @@ export const ChatItemContext = createContext<ChatItemContextType>({ ...@@ -131,7 +131,7 @@ export const ChatItemContext = createContext<ChatItemContextType>({
} }
}); });
/* /*
Chat 对象的上下文 Chat 对象的上下文
*/ */
const ChatItemContextProvider = ({ const ChatItemContextProvider = ({
......
projects/app/test/api/core/dataset/data/getQuoteData.test.ts
...@@ -201,7 +201,6 @@ describe('getQuoteData handler', () => { ...@@ -201,7 +201,6 @@ describe('getQuoteData handler', () => {
expect(authCollectionInChatMock).toHaveBeenCalledWith({ expect(authCollectionInChatMock).toHaveBeenCalledWith({
appId: VALID_APP_ID, appId: VALID_APP_ID,
chatId: 'chat_123', chatId: 'chat_123',
chatItemDataId: 'item_456',
collectionIds: ['col_1'] collectionIds: ['col_1']
}); });
expect(authDatasetDataMock).not.toHaveBeenCalled(); expect(authDatasetDataMock).not.toHaveBeenCalled();
......
projects/app/test/global/core/chat/utils.test.ts
...@@ -85,4 +85,38 @@ describe('addStatisticalDataToHistoryItem', () => { ...@@ -85,4 +85,38 @@ describe('addStatisticalDataToHistoryItem', () => {
expect(addStatisticalDataToHistoryItem(historyItem).useAgentSandbox).toBe(false); expect(addStatisticalDataToHistoryItem(historyItem).useAgentSandbox).toBe(false);
}); });
it('includes dataset quote tags that use QUOTE markdown links', () => {
const quoteId = '507f1f77bcf86cd799439011';
const historyItem: ChatItemMiniType = {
obj: ChatRoleEnum.AI,
value: [
{
text: {
content: `done [${quoteId}](QUOTE)`
}
}
],
responseData: [
{
id: 'dataset-response',
nodeId: 'dataset-node',
moduleName: 'Dataset Search',
moduleType: FlowNodeTypeEnum.datasetSearchNode,
quoteList: [
{
id: quoteId,
chunkIndex: 0,
datasetId: 'dataset-1',
collectionId: 'collection-1',
sourceName: 'doc.pdf',
score: [{ type: 'embedding', value: 0.9, index: 0 }]
}
]
}
]
};
expect(addStatisticalDataToHistoryItem(historyItem).totalQuoteList).toHaveLength(1);
});
}); });
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment