Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit d2d7eac9 by Archer Committed by GitHub
Options

4.6.4 (#588)

parent 84cf6b56
Showing with 822 additions and 330 deletions
docSite/content/docs/course/webSync.md 0 → 100644
---
title: 'Web 站点同步'
description: 'FastGPT Web 站点同步功能介绍和使用方式'
icon: 'language'
draft: false
toc: true
weight: 105
---
![](/imgs/webSync1.jpg)
## 什么是 Web 站点同步
Web 站点同步利用爬虫的技术，可以通过一个入口网站，自动捕获`同域名`下的所有网站，目前最多支持`200`个子页面。出于合规与安全角度，FastGPT 仅支持`静态站点`的爬取，主要用于各个文档站点快速构建知识库。
Tips: 国内的媒体站点基本不可用，公众号、csdn、知乎等。可以通过终端发送`curl`请求检测是否为静态站点，例如：
```bash
curl ai.fastgpt.in
```
## 如何使用
### 1. 新建知识库，选择 Web 站点同步
![](/imgs/webSync2.jpg)
![](/imgs/webSync3.jpg)
### 2. 点击配置站点信息
![](/imgs/webSync4.jpg)
### 3. 填写网址和选择器
![](/imgs/webSync5.jpg)
好了， 现在点击开始同步，静等系统自动抓取网站信息即可。
## 创建应用，绑定知识库
![](/imgs/webSync6.jpg)
## 选择器如何使用
选择器是 HTML CSS JS 的产物，你可以通过选择器来定位到你需要抓取的具体内容，而不是整个站点。使用方式为：
### 首先打开浏览器调试面板（通常是 F12，或者【右键 - 检查】）
![](/imgs/webSync7.jpg)
![](/imgs/webSync8.jpg)
### 输入对应元素的选择器
[菜鸟教程 css 选择器](https://www.runoob.com/cssref/css-selectors.html)，具体选择器的使用方式可以参考菜鸟教程。
上图中，我们选中了一个区域，对应的是`div`标签，它有 `data-prismjs-copy`, `data-prismjs-copy-success`, `data-prismjs-copy-error` 三个属性，这里我们用到一个就够。所以选择器是：
**`div[data-prismjs-copy]`**
除了属性选择器，常见的还有类和ID选择器。例如：
![](/imgs/webSync9.jpg)
上图 class 里的是类名（可能包含多个类名，都是空格隔开的，选择一个即可），选择器可以为：**`.docs-content`**
### 多选择器使用
在开头的演示中，我们对 FastGPT 文档是使用了多选择器的方式来选择，通过逗号隔开了两个选择器。
![](/imgs/webSync10.jpg)
我们希望选中上图两个标签中的内容，此时就需要两组选择器。一组是：`.docs-content .mb-0.d-flex`，含义是 `docs-content` 类下同时包含 `mb-0`和`d-flex` 两个类的子元素；
另一组是`.docs-content div[data-prismjs-copy]`，含义是`docs-content` 类下包含`data-prismjs-copy`属性的`div`元素。
把两组选择器用逗号隔开即可：`.docs-content .mb-0.d-flex, .docs-content div[data-prismjs-copy]`
\ No newline at end of file
docSite/content/docs/development/one-api.md
--- ---
title: '接入微软、ChatGLM、本地模型等' title: '接入微软、ChatGLM、本地模型等'
description: '通过接入 One API 来实现对各种大模型的支持' description: '部署和接入 OneAPI，实现对各种大模型的支持'
icon: 'Api' icon: 'Api'
draft: false draft: false
toc: true toc: true
weight: 708 weight: 708
--- ---
* 默认情况下，FastGPT 只配置了 GPT 的 3 个模型，如果你需要接入其他模型，需要进行一些额外配置。 * 默认情况下，FastGPT 只配置了 GPT 的模型，如果你需要接入其他模型，需要进行一些额外配置。
* [One API](https://github.com/songquanpeng/one-api) 是一个 OpenAI 接口管理 & 分发系统，可以通过标准的 OpenAI API 格式访问所有的大模型，开箱即用。 * [One API](https://github.com/songquanpeng/one-api) 是一个 OpenAI 接口管理 & 分发系统，可以通过标准的 OpenAI API 格式访问所有的大模型，开箱即用。
* FastGPT 可以通过接入 OneAPI 来实现对不同大模型的支持。OneAPI 的部署方法也很简单。
FastGPT 可以通过接入 One API 来实现对各种大模型的支持。部署方法也很简单。
## MySQL 版本 ## MySQL 版本
......
docSite/content/docs/development/openapi/dataset.md
...@@ -179,7 +179,7 @@ curl --location --request POST 'https://fastgpt.run/api/core/dataset/searchTest' ...@@ -179,7 +179,7 @@ curl --location --request POST 'https://fastgpt.run/api/core/dataset/searchTest'
{{< tab tabName="响应示例" >}} {{< tab tabName="响应示例" >}}
{{< markdownify >}} {{< markdownify >}}
返回 top limit 结果 返回 top k 结果， limit 为预估条数，会按每条数据 800 tokens 的长度进行预估，20条也就是返回 16000 tokens 长度的数据，最多测试 30000 tokens 的数据。
```bash ```bash
{ {
......
docSite/content/docs/development/upgrading/464.md 0 → 100644
---
title: 'V4.6.4(需要初始化)'
description: 'FastGPT V4.6.4'
icon: 'upgrade'
draft: false
toc: true
weight: 832
---
## 1。执行初始化 API
发起 1 个 HTTP 请求 ({{rootkey}} 替换成环境变量里的 `rootkey`，{{host}} 替换成自己域名)
1. https://xxxxx/api/admin/initv464
```bash
curl --location --request POST 'https://{{host}}/api/admin/initv464' \
--header 'rootkey: {{rootkey}}' \
--header 'Content-Type: application/json'
```
初始化说明：
1. 初始化 PG 的createTime字段
2. 初始化 Mongo 中 chat 的 feedback 字段
## V4.6.4 功能介绍
1. 重写 - 分享链接身份逻辑，采用 localID 记录用户的ID。
2. 商业版新增 - 分享链接 SSO 方案，通过`身份鉴权`地址，仅需`3个接口`即可完全接入已有用户系统。具体参考[分享链接身份鉴权](/docs/development/openapi/share/)
3. 调整 - 知识库搜索模块 topk 逻辑，采用 MaxToken 计算，兼容不同长度的文本块
4. 调整鉴权顺序，提高 apikey 的优先级，避免cookie抢占 apikey 的鉴权。
5. 链接读取支持多选择器。参考[Web 站点同步用法](/docs/course/webSync)
6. 修复 - 分享链接图片上传鉴权问题
7. 修复 - Mongo 连接池未释放问题。
8. 修复 - Dataset Intro 无法更新
9. 修复 - md 代码块问题
10. 修复 - root 权限问题
11. 优化 docker file
packages/global/common/system/constants.ts 0 → 100644
export const HUMAN_ICON = `/icon/human.svg`;
export const LOGO_ICON = `/icon/logo.svg`;
packages/global/common/system/types/index.d.ts
...@@ -9,6 +9,7 @@ export type FeConfigsType = { ...@@ -9,6 +9,7 @@ export type FeConfigsType = {
hide_app_flow?: boolean; hide_app_flow?: boolean;
concatMd?: string; concatMd?: string;
docUrl?: string; docUrl?: string;
chatbotUrl?: string;
openAPIDocUrl?: string; openAPIDocUrl?: string;
systemTitle?: string; systemTitle?: string;
googleClientVerKey?: string; googleClientVerKey?: string;
......
packages/global/core/chat/api.d.ts 0 → 100644
export type UpdateChatFeedbackProps = {
chatId: string;
chatItemId: string;
shareId?: string;
outLinkUid?: string;
userBadFeedback?: string;
userGoodFeedback?: string;
};
packages/global/core/chat/constants.ts
...@@ -50,9 +50,6 @@ export enum ChatStatusEnum { ...@@ -50,9 +50,6 @@ export enum ChatStatusEnum {
finish = 'finish' finish = 'finish'
} }
export const HUMAN_ICON = `/icon/human.svg`;
export const LOGO_ICON = `/icon/logo.svg`;
export const IMG_BLOCK_KEY = 'img-block'; export const IMG_BLOCK_KEY = 'img-block';
export const FILE_BLOCK_KEY = 'file-block'; export const FILE_BLOCK_KEY = 'file-block';
......
packages/global/core/chat/type.d.ts
...@@ -38,7 +38,9 @@ export type ChatItemSchema = { ...@@ -38,7 +38,9 @@ export type ChatItemSchema = {
time: Date; time: Date;
obj: `${ChatRoleEnum}`; obj: `${ChatRoleEnum}`;
value: string; value: string;
userFeedback?: string; userGoodFeedback?: string;
userBadFeedback?: string;
robotBadFeedback?: string;
adminFeedback?: AdminFbkType; adminFeedback?: AdminFbkType;
[ModuleOutputKeyEnum.responseData]?: ChatHistoryItemResType[]; [ModuleOutputKeyEnum.responseData]?: ChatHistoryItemResType[];
}; };
...@@ -56,7 +58,8 @@ export type ChatItemType = { ...@@ -56,7 +58,8 @@ export type ChatItemType = {
dataId?: string; dataId?: string;
obj: ChatItemSchema['obj']; obj: ChatItemSchema['obj'];
value: any; value: any;
userFeedback?: string; userGoodFeedback?: string;
userBadFeedback?: string;
adminFeedback?: ChatItemSchema['feedback']; adminFeedback?: ChatItemSchema['feedback'];
[ModuleOutputKeyEnum.responseData]?: ChatHistoryItemResType[]; [ModuleOutputKeyEnum.responseData]?: ChatHistoryItemResType[];
}; };
......
packages/global/core/dataset/api.d.ts
...@@ -25,6 +25,8 @@ export type CreateDatasetCollectionParams = { ...@@ -25,6 +25,8 @@ export type CreateDatasetCollectionParams = {
chunkSize?: number; chunkSize?: number;
fileId?: string; fileId?: string;
rawLink?: string; rawLink?: string;
qaPrompt?: string;
hashRawText?: string;
metadata?: Record<string, any>; metadata?: Record<string, any>;
}; };
...@@ -40,6 +42,7 @@ export type PgSearchRawType = { ...@@ -40,6 +42,7 @@ export type PgSearchRawType = {
export type PushDatasetDataChunkProps = { export type PushDatasetDataChunkProps = {
q: string; // embedding content q: string; // embedding content
a?: string; // bonus content a?: string; // bonus content
chunkIndex?: number;
indexes?: Omit<DatasetDataIndexItemType, 'dataId'>[]; indexes?: Omit<DatasetDataIndexItemType, 'dataId'>[];
}; };
......
packages/global/core/dataset/type.d.ts
...@@ -45,6 +45,8 @@ export type DatasetCollectionSchemaType = { ...@@ -45,6 +45,8 @@ export type DatasetCollectionSchemaType = {
chunkSize: number; chunkSize: number;
fileId?: string; fileId?: string;
rawLink?: string; rawLink?: string;
qaPrompt?: string;
hashRawText?: string;
metadata?: Record<string, any>; metadata?: Record<string, any>;
}; };
...@@ -134,6 +136,7 @@ export type DatasetDataItemType = { ...@@ -134,6 +136,7 @@ export type DatasetDataItemType = {
sourceId?: string; sourceId?: string;
q: string; q: string;
a: string; a: string;
chunkIndex: number;
indexes: DatasetDataIndexItemType[]; indexes: DatasetDataIndexItemType[];
isOwner: boolean; isOwner: boolean;
canWrite: boolean; canWrite: boolean;
......
packages/global/support/user/constant.ts
export enum UserStatusEnum {
active = 'active',
forbidden = 'forbidden'
}
export const userStatusMap = {
[UserStatusEnum.active]: {
label: 'support.user.status.active'
},
[UserStatusEnum.forbidden]: {
label: 'support.user.status.forbidden'
}
};
export enum OAuthEnum { export enum OAuthEnum {
github = 'github', github = 'github',
google = 'google' google = 'google'
} }
export enum UserAuthTypeEnum {
register = 'register',
findPassword = 'findPassword'
}
packages/global/support/user/type.d.ts
import { InformTypeEnum } from './constant'; import { InformTypeEnum, UserStatusEnum } from './constant';
import { TeamItemType } from './team/type'; import { TeamItemType } from './team/type';
export type UserModelSchema = { export type UserModelSchema = {
...@@ -12,6 +12,7 @@ export type UserModelSchema = { ...@@ -12,6 +12,7 @@ export type UserModelSchema = {
openaiKey: string; openaiKey: string;
createTime: number; createTime: number;
timezone: string; timezone: string;
status: `${UserStatusEnum}`;
openaiAccount?: { openaiAccount?: {
key: string; key: string;
baseUrl: string; baseUrl: string;
......
packages/service/common/api/plusRequest.ts
...@@ -70,7 +70,7 @@ instance.interceptors.request.use(requestStart, (err) => Promise.reject(err)); ...@@ -70,7 +70,7 @@ instance.interceptors.request.use(requestStart, (err) => Promise.reject(err));
instance.interceptors.response.use(responseSuccess, (err) => Promise.reject(err)); instance.interceptors.response.use(responseSuccess, (err) => Promise.reject(err));
export function request(url: string, data: any, config: ConfigType, method: Method): any { export function request(url: string, data: any, config: ConfigType, method: Method): any {
if (!global.systemEnv?.pluginBaseUrl) { if (global.systemEnv && !global.systemEnv?.pluginBaseUrl) {
return Promise.reject('该功能为商业版特有...'); return Promise.reject('该功能为商业版特有...');
} }
......
packages/service/common/file/image/schema.ts
...@@ -8,6 +8,10 @@ const ImageSchema = new Schema({ ...@@ -8,6 +8,10 @@ const ImageSchema = new Schema({
ref: TeamCollectionName, ref: TeamCollectionName,
required: true required: true
}, },
createTime: {
type: Date,
default: () => new Date()
},
binary: { binary: {
type: Buffer type: Buffer
}, },
...@@ -25,7 +29,10 @@ try { ...@@ -25,7 +29,10 @@ try {
console.log(error); console.log(error);
} }
export const MongoImage: Model<{ teamId: string; binary: Buffer; metadata?: Record<string, any> }> = export const MongoImage: Model<{
models['image'] || model('image', ImageSchema); teamId: string;
binary: Buffer;
metadata?: { fileId?: string };
}> = models['image'] || model('image', ImageSchema);
MongoImage.syncIndexes(); MongoImage.syncIndexes();
packages/service/common/mongo/controller.ts deleted 100644 → 0
/* add logger */
export const addLog = {
info: (msg: string, obj?: Record<string, any>) => {
global.logger?.info(msg, { meta: obj });
},
error: (msg: string, error?: any) => {
global.logger?.error(msg, {
meta: {
stack: error?.stack,
...(error?.config && {
config: {
headers: error.config.headers,
url: error.config.url,
data: error.config.data
}
}),
...(error?.response && {
response: {
status: error.response.status,
statusText: error.response.statusText
}
})
}
});
}
};
packages/service/common/mongo/init.ts
import mongoose from './index'; import mongoose from './index';
import { createLogger, format, transports } from 'winston';
import 'winston-mongodb';
/** /**
* connect MongoDB and init data * connect MongoDB and init data
...@@ -36,7 +34,6 @@ export async function connectMongo({ ...@@ -36,7 +34,6 @@ export async function connectMongo({
}); });
console.log('mongo connected'); console.log('mongo connected');
initLogger();
afterHook && (await afterHook()); afterHook && (await afterHook());
} catch (error) { } catch (error) {
...@@ -45,35 +42,3 @@ export async function connectMongo({ ...@@ -45,35 +42,3 @@ export async function connectMongo({
global.mongodb = undefined; global.mongodb = undefined;
} }
} }
function initLogger() {
global.logger = createLogger({
transports: [
new transports.MongoDB({
db: process.env.MONGODB_URI as string,
collection: 'server_logs',
options: {
useUnifiedTopology: true
},
cappedSize: 500000000,
tryReconnect: true,
metaKey: 'meta',
format: format.combine(format.timestamp(), format.json())
}),
new transports.Console({
format: format.combine(
format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss' }),
format.printf((info) => {
if (info.level === 'error') {
console.log(info.meta);
return `[${info.level.toLocaleUpperCase()}]: ${[info.timestamp]}: ${info.message}`;
}
return `[${info.level.toLocaleUpperCase()}]: ${[info.timestamp]}: ${info.message}${
info.meta ? `: ${JSON.stringify(info.meta)}` : ''
}`;
})
)
})
]
});
}
packages/service/common/mongo/type.d.ts
...@@ -3,5 +3,4 @@ import type { Logger } from 'winston'; ...@@ -3,5 +3,4 @@ import type { Logger } from 'winston';
declare global { declare global {
var mongodb: Mongoose | undefined; var mongodb: Mongoose | undefined;
var logger: Logger;
} }
packages/service/common/pg/index.ts
...@@ -171,9 +171,10 @@ export async function initPg() { ...@@ -171,9 +171,10 @@ export async function initPg() {
tmb_id VARCHAR(50) NOT NULL, tmb_id VARCHAR(50) NOT NULL,
dataset_id VARCHAR(50) NOT NULL, dataset_id VARCHAR(50) NOT NULL,
collection_id VARCHAR(50) NOT NULL, collection_id VARCHAR(50) NOT NULL,
data_id VARCHAR(50) NOT NULL data_id VARCHAR(50) NOT NULL,
createTime TIMESTAMP DEFAULT CURRENT_TIMESTAMP
); );
CREATE INDEX IF NOT EXISTS vector_index ON ${PgDatasetTableName} USING hnsw (vector vector_ip_ops) WITH (m = 24, ef_construction = 64); CREATE INDEX IF NOT EXISTS vector_index ON ${PgDatasetTableName} USING hnsw (vector vector_ip_ops) WITH (m = 32, ef_construction = 64);
`); `);
console.log('init pg successful'); console.log('init pg successful');
......
packages/service/common/response/index.ts
import type { NextApiResponse } from 'next'; import type { NextApiResponse } from 'next';
import { sseResponseEventEnum } from './constant'; import { sseResponseEventEnum } from './constant';
import { proxyError, ERROR_RESPONSE, ERROR_ENUM } from '@fastgpt/global/common/error/errorCode'; import { proxyError, ERROR_RESPONSE, ERROR_ENUM } from '@fastgpt/global/common/error/errorCode';
import { addLog } from '../mongo/controller'; import { addLog } from '../system/log';
import { clearCookie } from '../../support/permission/controller'; import { clearCookie } from '../../support/permission/controller';
export interface ResponseType<T = any> { export interface ResponseType<T = any> {
......
packages/service/common/system/config/controller.ts 0 → 100644
import { SystemConfigsTypeEnum } from '@fastgpt/global/common/system/config/constants';
import { MongoSystemConfigs } from './schema';
import { FeConfigsType } from '@fastgpt/global/common/system/types';
export const getFastGPTFeConfig = async () => {
const res = await MongoSystemConfigs.findOne({
type: SystemConfigsTypeEnum.fastgpt
}).sort({
createTime: -1
});
const config: FeConfigsType = res?.value?.FeConfig || {};
return config;
};
packages/service/common/system/config/schema.ts
...@@ -19,14 +19,15 @@ const systemConfigSchema = new Schema({ ...@@ -19,14 +19,15 @@ const systemConfigSchema = new Schema({
type: Date, type: Date,
default: () => new Date() default: () => new Date()
} }
}) });
try { try {
systemConfigSchema.index({ createTime: -1 }, { expireAfterSeconds: 90 * 24 * 60 * 60 }); systemConfigSchema.index({ createTime: -1 }, { expireAfterSeconds: 90 * 24 * 60 * 60 });
systemConfigSchema.index({ type: 1 });
} catch (error) { } catch (error) {
console.log(error); console.log(error);
} }
export const MongoSystemConfigs: Model<SystemConfigsType>= export const MongoSystemConfigs: Model<SystemConfigsType> =
models[collectionName] || model(collectionName, systemConfigSchema); models[collectionName] || model(collectionName, systemConfigSchema);
MongoSystemConfigs.syncIndexes(); MongoSystemConfigs.syncIndexes();
packages/service/common/system/log.ts 0 → 100644
import dayjs from 'dayjs';
/* add logger */
export const addLog = {
log(level: 'info' | 'warn' | 'error', msg: string, obj: Record<string, any> = {}) {
console.log(
`[${level.toLocaleUpperCase()}] ${dayjs().format(
'YYYY-MM-DD HH:mm:ss'
)} ${msg}: ${JSON.stringify(obj)}`
);
const lokiUrl = process.env.LOKI_LOG_URL as string;
if (!lokiUrl) return;
try {
fetch(lokiUrl, {
method: 'POST',
headers: {
'Content-type': 'application/json'
},
body: JSON.stringify({
streams: [
{
stream: {
level
},
values: [
[
`${Date.now() * 1000000}`,
JSON.stringify({
message: msg,
...obj
})
]
]
}
]
})
});
} catch (error) {}
},
info(msg: string, obj?: Record<string, any>) {
this.log('info', msg, obj);
},
warn(msg: string, obj?: Record<string, any>) {
this.log('warn', msg, obj);
},
error(msg: string, error?: any) {
this.log('error', msg, {
stack: error?.stack,
...(error?.config && {
config: {
headers: error.config.headers,
url: error.config.url,
data: error.config.data
}
}),
...(error?.response && {
response: {
status: error.response.status,
statusText: error.response.statusText
}
})
});
}
};
packages/service/core/chat/chatItemSchema.ts
...@@ -54,7 +54,14 @@ const ChatItemSchema = new Schema({ ...@@ -54,7 +54,14 @@ const ChatItemSchema = new Schema({
type: String, type: String,
default: '' default: ''
}, },
userFeedback: { userGoodFeedback: {
type: String
},
userFeedback: String,
userBadFeedback: {
type: String
},
robotBadFeedback: {
type: String type: String
}, },
adminFeedback: { adminFeedback: {
...@@ -77,7 +84,10 @@ try { ...@@ -77,7 +84,10 @@ try {
ChatItemSchema.index({ userId: 1 }); ChatItemSchema.index({ userId: 1 });
ChatItemSchema.index({ appId: 1 }); ChatItemSchema.index({ appId: 1 });
ChatItemSchema.index({ chatId: 1 }); ChatItemSchema.index({ chatId: 1 });
ChatItemSchema.index({ userFeedback: 1 }); ChatItemSchema.index({ userGoodFeedback: 1 });
ChatItemSchema.index({ userBadFeedback: 1 });
ChatItemSchema.index({ robotBadFeedback: 1 });
ChatItemSchema.index({ adminFeedback: 1 });
} catch (error) { } catch (error) {
console.log(error); console.log(error);
} }
......
packages/service/core/dataset/collection/controller.ts
...@@ -6,6 +6,8 @@ import type { CreateDatasetCollectionParams } from '@fastgpt/global/core/dataset ...@@ -6,6 +6,8 @@ import type { CreateDatasetCollectionParams } from '@fastgpt/global/core/dataset
import { MongoDatasetCollection } from './schema'; import { MongoDatasetCollection } from './schema';
export async function createOneCollection({ export async function createOneCollection({
teamId,
tmbId,
name, name,
parentId, parentId,
datasetId, datasetId,
...@@ -14,8 +16,8 @@ export async function createOneCollection({ ...@@ -14,8 +16,8 @@ export async function createOneCollection({
chunkSize = 0, chunkSize = 0,
fileId, fileId,
rawLink, rawLink,
teamId, qaPrompt,
tmbId, hashRawText,
metadata = {} metadata = {}
}: CreateDatasetCollectionParams & { teamId: string; tmbId: string }) { }: CreateDatasetCollectionParams & { teamId: string; tmbId: string }) {
const { _id } = await MongoDatasetCollection.create({ const { _id } = await MongoDatasetCollection.create({
...@@ -29,6 +31,8 @@ export async function createOneCollection({ ...@@ -29,6 +31,8 @@ export async function createOneCollection({
chunkSize, chunkSize,
fileId, fileId,
rawLink, rawLink,
qaPrompt,
hashRawText,
metadata metadata
}); });
...@@ -71,3 +75,19 @@ export function createDefaultCollection({ ...@@ -71,3 +75,19 @@ export function createDefaultCollection({
updateTime: new Date('2099') updateTime: new Date('2099')
}); });
} }
// check same collection
export const getSameRawTextCollection = async ({
datasetId,
hashRawText
}: {
datasetId: string;
hashRawText?: string;
}) => {
const collection = await MongoDatasetCollection.findOne({
datasetId,
hashRawText
});
return collection;
};
packages/service/core/dataset/collection/schema.ts
...@@ -72,6 +72,12 @@ const DatasetCollectionSchema = new Schema({ ...@@ -72,6 +72,12 @@ const DatasetCollectionSchema = new Schema({
rawLink: { rawLink: {
type: String type: String
}, },
qaPrompt: {
type: String
},
hashRawText: {
type: String
},
metadata: { metadata: {
type: Object, type: Object,
default: {} default: {}
...@@ -82,6 +88,7 @@ try { ...@@ -82,6 +88,7 @@ try {
DatasetCollectionSchema.index({ datasetId: 1 }); DatasetCollectionSchema.index({ datasetId: 1 });
DatasetCollectionSchema.index({ datasetId: 1, parentId: 1 }); DatasetCollectionSchema.index({ datasetId: 1, parentId: 1 });
DatasetCollectionSchema.index({ updateTime: -1 }); DatasetCollectionSchema.index({ updateTime: -1 });
DatasetCollectionSchema.index({ hashRawText: -1 });
} catch (error) { } catch (error) {
console.log(error); console.log(error);
} }
......
packages/service/core/dataset/data/controller.ts
...@@ -16,22 +16,22 @@ export async function delDatasetRelevantData({ datasetIds }: { datasetIds: strin ...@@ -16,22 +16,22 @@ export async function delDatasetRelevantData({ datasetIds }: { datasetIds: strin
datasetId: { $in: datasetIds } datasetId: { $in: datasetIds }
}); });
// delete related files await delay(2000);
await Promise.all(
datasetIds.map((id) => delFileByMetadata({ bucketName: BucketNameEnum.dataset, datasetId: id }))
);
await delay(500);
// delete pg data
await deletePgDataById(`dataset_id IN ('${datasetIds.join("','")}')`);
// delete dataset.datas // delete dataset.datas
await MongoDatasetData.deleteMany({ datasetId: { $in: datasetIds } }); await MongoDatasetData.deleteMany({ datasetId: { $in: datasetIds } });
// delete pg data
await deletePgDataById(`dataset_id IN ('${datasetIds.join("','")}')`);
// delete collections // delete collections
await MongoDatasetCollection.deleteMany({ await MongoDatasetCollection.deleteMany({
datasetId: { $in: datasetIds } datasetId: { $in: datasetIds }
}); });
// delete related files
await Promise.all(
datasetIds.map((id) => delFileByMetadata({ bucketName: BucketNameEnum.dataset, datasetId: id }))
);
} }
/** /**
* delete all data by collectionIds * delete all data by collectionIds
...@@ -51,6 +51,18 @@ export async function delCollectionRelevantData({ ...@@ -51,6 +51,18 @@ export async function delCollectionRelevantData({
collectionId: { $in: collectionIds } collectionId: { $in: collectionIds }
}); });
await delay(2000);
// delete dataset.datas
await MongoDatasetData.deleteMany({ collectionId: { $in: collectionIds } });
// delete pg data
await deletePgDataById(`collection_id IN ('${collectionIds.join("','")}')`);
// delete collections
await MongoDatasetCollection.deleteMany({
_id: { $in: collectionIds }
});
// delete file and imgs // delete file and imgs
await Promise.all([ await Promise.all([
delImgByFileIdList(filterFileIds), delImgByFileIdList(filterFileIds),
...@@ -59,13 +71,6 @@ export async function delCollectionRelevantData({ ...@@ -59,13 +71,6 @@ export async function delCollectionRelevantData({
fileIdList: filterFileIds fileIdList: filterFileIds
}) })
]); ]);
await delay(500);
// delete pg data
await deletePgDataById(`collection_id IN ('${collectionIds.join("','")}')`);
// delete dataset.datas
await MongoDatasetData.deleteMany({ collectionId: { $in: collectionIds } });
} }
/** /**
* delete one data by mongoDataId * delete one data by mongoDataId
......
packages/service/core/dataset/data/schema.ts
...@@ -71,7 +71,6 @@ const DatasetDataSchema = new Schema({ ...@@ -71,7 +71,6 @@ const DatasetDataSchema = new Schema({
], ],
default: [] default: []
}, },
// metadata
updateTime: { updateTime: {
type: Date, type: Date,
default: () => new Date() default: () => new Date()
...@@ -89,6 +88,7 @@ try { ...@@ -89,6 +88,7 @@ try {
DatasetDataSchema.index({ teamId: 1 }); DatasetDataSchema.index({ teamId: 1 });
DatasetDataSchema.index({ datasetId: 1 }); DatasetDataSchema.index({ datasetId: 1 });
DatasetDataSchema.index({ collectionId: 1 }); DatasetDataSchema.index({ collectionId: 1 });
DatasetDataSchema.index({ updateTime: -1 });
// full text index // full text index
DatasetDataSchema.index({ datasetId: 1, fullTextToken: 'text' }); DatasetDataSchema.index({ datasetId: 1, fullTextToken: 'text' });
DatasetDataSchema.index({ inited: 1 }); DatasetDataSchema.index({ inited: 1 });
......
packages/service/package.json
...@@ -13,8 +13,7 @@ ...@@ -13,8 +13,7 @@
"nextjs-cors": "^2.1.2", "nextjs-cors": "^2.1.2",
"pg": "^8.10.0", "pg": "^8.10.0",
"tunnel": "^0.0.6", "tunnel": "^0.0.6",
"winston": "^3.10.0", "dayjs": "^1.11.7"
"winston-mongodb": "^5.1.1"
}, },
"devDependencies": { "devDependencies": {
"@types/cookie": "^0.5.2", "@types/cookie": "^0.5.2",
......
packages/service/support/user/schema.ts
...@@ -3,10 +3,16 @@ const { Schema, model, models } = connectionMongo; ...@@ -3,10 +3,16 @@ const { Schema, model, models } = connectionMongo;
import { hashStr } from '@fastgpt/global/common/string/tools'; import { hashStr } from '@fastgpt/global/common/string/tools';
import { PRICE_SCALE } from '@fastgpt/global/support/wallet/bill/constants'; import { PRICE_SCALE } from '@fastgpt/global/support/wallet/bill/constants';
import type { UserModelSchema } from '@fastgpt/global/support/user/type'; import type { UserModelSchema } from '@fastgpt/global/support/user/type';
import { UserStatusEnum, userStatusMap } from '@fastgpt/global/support/user/constant';
export const userCollectionName = 'users'; export const userCollectionName = 'users';
const UserSchema = new Schema({ const UserSchema = new Schema({
status: {
type: String,
enum: Object.keys(userStatusMap),
default: UserStatusEnum.active
},
username: { username: {
// 可以是手机/邮箱，新的验证都只用手机 // 可以是手机/邮箱，新的验证都只用手机
type: String, type: String,
......
projects/app/.env.template
...@@ -21,3 +21,5 @@ MONGODB_URI=mongodb://username:password@0.0.0.0:27017/fastgpt?authSource=admin ...@@ -21,3 +21,5 @@ MONGODB_URI=mongodb://username:password@0.0.0.0:27017/fastgpt?authSource=admin
PG_URL=postgresql://username:password@host:port/postgres PG_URL=postgresql://username:password@host:port/postgres
# 首页路径 # 首页路径
HOME_URL=/ HOME_URL=/
# Loki Log Path
# LOKI_LOG_URL=
\ No newline at end of file
projects/app/public/docs/versionIntro.md
### Fast GPT V4.6.3 ### Fast GPT V4.6.4
1. 商业版新增 - web站点同步 1. 重写 - 分享链接身份逻辑，采用 localID 记录用户的ID。
2. 新增 - 集合元数据记录 2. 商业版新增 - 分享链接 SSO 方案，通过`身份鉴权`地址，仅需`3个接口`即可完全接入已有用户系统。具体参考[分享链接身份鉴权](https://doc.fastgpt.in/docs/development/openapi/share/)
3. 优化 - url 读取内容 3. 调整 - 知识库搜索模块 topk 逻辑，采用 MaxToken 计算，兼容不同长度的文本块
4. 优化 - 流读取文件，防止内存溢出 4. 调整鉴权顺序，提高 apikey 的优先级，避免cookie抢占 apikey 的鉴权。
5. [知识库结构详解](https://doc.fastgpt.in/docs/use-cases/datasetengine/) 5. 链接读取支持多选择器。参考[Web 站点同步用法](https://doc.fastgpt.in/docs/course/webSync)
6. [知识库提示词详解](https://doc.fastgpt.in/docs/use-cases/ai_settings/#引用模板--引用提示词) 6. 修复 - 分享链接图片上传鉴权问题
7. [使用文档](https://doc.fastgpt.in/docs/intro/) 7. 修复 - Mongo 连接池未释放问题。
8. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow) 8. 修复 - Dataset Intro 无法更新
9. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/) 9. 修复 - md 代码块问题
10. 修复 - root 权限问题
11. 优化 docker file
12. [知识库结构详解](https://doc.fastgpt.in/docs/use-cases/datasetengine/)
13. [知识库提示词详解](https://doc.fastgpt.in/docs/use-cases/ai_settings/#引用模板--引用提示词)
14. [使用文档](https://doc.fastgpt.in/docs/intro/)
15. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow)
16. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/)
projects/app/public/locales/en/common.json
...@@ -65,7 +65,6 @@ ...@@ -65,7 +65,6 @@
"Confirm to clear share chat history": " Are you sure to delete all chats?", "Confirm to clear share chat history": " Are you sure to delete all chats?",
"Converting to text": "Converting to text...", "Converting to text": "Converting to text...",
"Exit Chat": "Exit", "Exit Chat": "Exit",
"Feedback Close": "Close Feedback",
"Feedback Failed": "Feedback Failed", "Feedback Failed": "Feedback Failed",
"Feedback Mark": "Mark", "Feedback Mark": "Mark",
"Feedback Modal": "Feedback", "Feedback Modal": "Feedback",
...@@ -81,7 +80,6 @@ ...@@ -81,7 +80,6 @@
"Question Guide Tips": "I guess what you're asking is", "Question Guide Tips": "I guess what you're asking is",
"Quote": "Quote", "Quote": "Quote",
"Read Mark Description": "Read mark description", "Read Mark Description": "Read mark description",
"Read User Feedback": "Read user feedback",
"Select Mark Kb": "Select Dataset", "Select Mark Kb": "Select Dataset",
"Select Mark Kb Desc": "Select a dataset to store the expected answers", "Select Mark Kb Desc": "Select a dataset to store the expected answers",
"You need to a chat app": "You need to a chat app", "You need to a chat app": "You need to a chat app",
...@@ -201,6 +199,9 @@ ...@@ -201,6 +199,9 @@
"confirm": { "confirm": {
"Common Tip": "Operational Confirm" "Common Tip": "Operational Confirm"
}, },
"course": {
"Read Course": "Read Course"
},
"empty": { "empty": {
"Common Tip": "No data" "Common Tip": "No data"
}, },
...@@ -234,6 +235,10 @@ ...@@ -234,6 +235,10 @@
}, },
"speech": { "speech": {
"error tip": "Speech Failed" "error tip": "Speech Failed"
},
"system": {
"Help Chatbot": "Chatbot Helper",
"Use Helper": "UsingHelp"
} }
}, },
"core": { "core": {
...@@ -257,6 +262,9 @@ ...@@ -257,6 +262,9 @@
"TTS Tip": "After this function is enabled, the voice playback function can be used after each conversation. Use of this feature may incur additional charges.", "TTS Tip": "After this function is enabled, the voice playback function can be used after each conversation. Use of this feature may incur additional charges.",
"Welcome Text": "Welcome Text", "Welcome Text": "Welcome Text",
"create app": "Create App", "create app": "Create App",
"logs": {
"Source And Time": "Source & Time"
},
"setting": "App Setting", "setting": "App Setting",
"simple": { "simple": {
"mode template select": "Template" "mode template select": "Template"
...@@ -283,6 +291,16 @@ ...@@ -283,6 +291,16 @@
"Speaking": "I'm listening...", "Speaking": "I'm listening...",
"Stop Speak": "Stop Speak", "Stop Speak": "Stop Speak",
"Type a message": "Input problem", "Type a message": "Input problem",
"error": {
"Messages empty": "Interface content is empty, maybe the text is too long ~"
},
"feedback": {
"Close User Good Feedback": "",
"Close User Like": "The user like\nClick to close the tag",
"No Content": "The user did not fill in the specific feedback content",
"Read User dislike": "User dislike\nClick to view content",
"Feedback Close": "Close Feedback"
},
"markdown": { "markdown": {
"Edit Question": "Edit Question", "Edit Question": "Edit Question",
"Quick Question": "Ask the question immediately", "Quick Question": "Ask the question immediately",
...@@ -295,9 +313,6 @@ ...@@ -295,9 +313,6 @@
}, },
"tts": { "tts": {
"Stop Speech": "Stop" "Stop Speech": "Stop"
},
"error": {
"Messages empty": "Interface content is empty, maybe the text is too long ~"
} }
}, },
"dataset": { "dataset": {
...@@ -379,13 +394,13 @@ ...@@ -379,13 +394,13 @@
"id": "Data ID" "id": "Data ID"
}, },
"error": { "error": {
"Start Sync Failed": "Start Sync Failed",
"unAuthDataset": "No access to this knowledge base ", "unAuthDataset": "No access to this knowledge base ",
"unAuthDatasetCollection": "Not authorized to manipulate this data set ", "unAuthDatasetCollection": "Not authorized to manipulate this data set ",
"unAuthDatasetData": "Not authorized to manipulate this data ", "unAuthDatasetData": "Not authorized to manipulate this data ",
"unAuthDatasetFile": "No permission to manipulate this file ", "unAuthDatasetFile": "No permission to manipulate this file ",
"unCreateCollection": "No permission to manipulate this data ", "unCreateCollection": "No permission to manipulate this data ",
"unLinkCollection": "not a network link collection ", "unLinkCollection": "not a network link collection "
"Start Sync Failed": "Start Sync Failed"
}, },
"file": "File", "file": "File",
"folder": "Folder", "folder": "Folder",
...@@ -462,6 +477,7 @@ ...@@ -462,6 +477,7 @@
"Confirm Create Tips": "Confirm to synchronize the site, the synchronization task will start later, please confirm!", "Confirm Create Tips": "Confirm to synchronize the site, the synchronization task will start later, please confirm!",
"Confirm Update Tips": "Are you sure to update the site configuration? The synchronization starts immediately with the new configuration. Please confirm", "Confirm Update Tips": "Are you sure to update the site configuration? The synchronization starts immediately with the new configuration. Please confirm",
"Selector": "Selector", "Selector": "Selector",
"Selector Course": "Selector using tutorial",
"Start Sync": "Start Sync", "Start Sync": "Start Sync",
"UnValid Website Tip": "Your site may not be static and cannot be synchronized" "UnValid Website Tip": "Your site may not be static and cannot be synchronized"
} }
...@@ -669,6 +685,10 @@ ...@@ -669,6 +685,10 @@
"Visual AI orchestration": "Visual AI orchestration", "Visual AI orchestration": "Visual AI orchestration",
"Why FastGPT": "Why {{title}}", "Why FastGPT": "Why {{title}}",
"desc": "AI knowledge base question and answer platform based on LLM large model", "desc": "AI knowledge base question and answer platform based on LLM large model",
"navbar": {
"Use guidance": "Use Guidance",
"chatbot": "Chatbot"
},
"slogan": "Let the AI know more about you" "slogan": "Let the AI know more about you"
}, },
"module": { "module": {
......
projects/app/public/locales/zh/common.json
...@@ -65,7 +65,6 @@ ...@@ -65,7 +65,6 @@
"Confirm to clear share chat history": "确认删除所有聊天记录？", "Confirm to clear share chat history": "确认删除所有聊天记录？",
"Converting to text": "正在转换为文本...", "Converting to text": "正在转换为文本...",
"Exit Chat": "退出聊天", "Exit Chat": "退出聊天",
"Feedback Close": "关闭反馈",
"Feedback Failed": "提交反馈异常", "Feedback Failed": "提交反馈异常",
"Feedback Mark": "标注", "Feedback Mark": "标注",
"Feedback Modal": "结果反馈", "Feedback Modal": "结果反馈",
...@@ -81,7 +80,6 @@ ...@@ -81,7 +80,6 @@
"Question Guide Tips": "猜你想问", "Question Guide Tips": "猜你想问",
"Quote": "引用", "Quote": "引用",
"Read Mark Description": "查看标注功能介绍", "Read Mark Description": "查看标注功能介绍",
"Read User Feedback": "查看用户反馈",
"Select Mark Kb": "选择知识库", "Select Mark Kb": "选择知识库",
"Select Mark Kb Desc": "选择一个知识库存储预期答案", "Select Mark Kb Desc": "选择一个知识库存储预期答案",
"You need to a chat app": "你需要创建一个应用", "You need to a chat app": "你需要创建一个应用",
...@@ -201,6 +199,9 @@ ...@@ -201,6 +199,9 @@
"confirm": { "confirm": {
"Common Tip": "操作确认" "Common Tip": "操作确认"
}, },
"course": {
"Read Course": "查看教程"
},
"empty": { "empty": {
"Common Tip": "没有什么数据噢~" "Common Tip": "没有什么数据噢~"
}, },
...@@ -234,6 +235,10 @@ ...@@ -234,6 +235,10 @@
}, },
"speech": { "speech": {
"error tip": "语音转文字失败" "error tip": "语音转文字失败"
},
"system": {
"Help Chatbot": "机器人助手",
"Use Helper": "使用帮助"
} }
}, },
"core": { "core": {
...@@ -257,6 +262,9 @@ ...@@ -257,6 +262,9 @@
"TTS Tip": "开启后，每次对话后可使用语音播放功能。使用该功能可能产生额外费用。", "TTS Tip": "开启后，每次对话后可使用语音播放功能。使用该功能可能产生额外费用。",
"Welcome Text": "对话开场白", "Welcome Text": "对话开场白",
"create app": "创建属于你的 AI 应用", "create app": "创建属于你的 AI 应用",
"logs": {
"Source And Time": "来源 & 时间"
},
"setting": "应用信息设置", "setting": "应用信息设置",
"simple": { "simple": {
"mode template select": "简易模板" "mode template select": "简易模板"
...@@ -283,6 +291,16 @@ ...@@ -283,6 +291,16 @@
"Speaking": "我在听，请说...", "Speaking": "我在听，请说...",
"Stop Speak": "停止录音", "Stop Speak": "停止录音",
"Type a message": "输入问题", "Type a message": "输入问题",
"error": {
"Messages empty": "接口内容为空，可能文本超长了~"
},
"feedback": {
"Close User Good Feedback": "",
"Close User Like": "用户表示赞同\n点击关闭该标记",
"No Content": "用户没有填写具体反馈内容",
"Read User dislike": "用户表示反对\n点击查看内容",
"Feedback Close": "关闭反馈"
},
"markdown": { "markdown": {
"Edit Question": "编辑问题", "Edit Question": "编辑问题",
"Quick Question": "点我立即提问", "Quick Question": "点我立即提问",
...@@ -295,9 +313,6 @@ ...@@ -295,9 +313,6 @@
}, },
"tts": { "tts": {
"Stop Speech": "停止" "Stop Speech": "停止"
},
"error": {
"Messages empty": "接口内容为空，可能文本超长了~"
} }
}, },
"dataset": { "dataset": {
...@@ -462,6 +477,7 @@ ...@@ -462,6 +477,7 @@
"Confirm Create Tips": "确认同步该站点，同步任务将随后开启，请确认！", "Confirm Create Tips": "确认同步该站点，同步任务将随后开启，请确认！",
"Confirm Update Tips": "确认更新站点配置？会立即按新的配置开始同步，请确认！", "Confirm Update Tips": "确认更新站点配置？会立即按新的配置开始同步，请确认！",
"Selector": "选择器", "Selector": "选择器",
"Selector Course": "选择器使用教程",
"Start Sync": "开始同步", "Start Sync": "开始同步",
"UnValid Website Tip": "您的站点可能非静态站点，无法同步" "UnValid Website Tip": "您的站点可能非静态站点，无法同步"
} }
...@@ -669,6 +685,10 @@ ...@@ -669,6 +685,10 @@
"Visual AI orchestration": "可视化 AI 编排", "Visual AI orchestration": "可视化 AI 编排",
"Why FastGPT": "为什么选择 {{title}}", "Why FastGPT": "为什么选择 {{title}}",
"desc": "基于 LLM 大模型的 AI 知识库问答平台", "desc": "基于 LLM 大模型的 AI 知识库问答平台",
"navbar": {
"Use guidance": "使用引导",
"chatbot": "问答机器人"
},
"slogan": "让 AI 更懂你的知识" "slogan": "让 AI 更懂你的知识"
}, },
"module": { "module": {
......
projects/app/public/simpleTemplates/fastgpt-simple.json
...@@ -14,7 +14,7 @@ ...@@ -14,7 +14,7 @@
"datasets": true, "datasets": true,
"similarity": false, "similarity": false,
"limit": false, "limit": false,
"searchMode": "embedding", "searchMode": false,
"searchEmptyText": false "searchEmptyText": false
}, },
"userGuide": { "userGuide": {
......
projects/app/src/components/Avatar/index.tsx
import React from 'react'; import React from 'react';
import { Image } from '@chakra-ui/react'; import { Image } from '@chakra-ui/react';
import type { ImageProps } from '@chakra-ui/react'; import type { ImageProps } from '@chakra-ui/react';
import { LOGO_ICON } from '@fastgpt/global/core/chat/constants'; import { LOGO_ICON } from '@fastgpt/global/common/system/constants';
const Avatar = ({ w = '30px', src, ...props }: ImageProps) => { const Avatar = ({ w = '30px', src, ...props }: ImageProps) => {
return ( return (
......
projects/app/src/components/ChatBox/FeedbackModal.tsx
...@@ -3,13 +3,15 @@ import { ModalBody, Textarea, ModalFooter, Button } from '@chakra-ui/react'; ...@@ -3,13 +3,15 @@ import { ModalBody, Textarea, ModalFooter, Button } from '@chakra-ui/react';
import MyModal from '../MyModal'; import MyModal from '../MyModal';
import { useRequest } from '@/web/common/hooks/useRequest'; import { useRequest } from '@/web/common/hooks/useRequest';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { userUpdateChatFeedback } from '@/web/core/chat/api'; import { updateChatUserFeedback } from '@/web/core/chat/api';
const FeedbackModal = ({ const FeedbackModal = ({
chatId,
chatItemId, chatItemId,
onSuccess, onSuccess,
onClose onClose
}: { }: {
chatId: string;
chatItemId: string; chatItemId: string;
onSuccess: (e: string) => void; onSuccess: (e: string) => void;
onClose: () => void; onClose: () => void;
...@@ -19,14 +21,15 @@ const FeedbackModal = ({ ...@@ -19,14 +21,15 @@ const FeedbackModal = ({
const { mutate, isLoading } = useRequest({ const { mutate, isLoading } = useRequest({
mutationFn: async () => { mutationFn: async () => {
const val = ref.current?.value || 'N/A'; const val = ref.current?.value || t('core.chat.feedback.No Content');
return userUpdateChatFeedback({ return updateChatUserFeedback({
chatId,
chatItemId, chatItemId,
userFeedback: val userBadFeedback: val
}); });
}, },
onSuccess() { onSuccess() {
onSuccess(ref.current?.value || 'N/A'); onSuccess(ref.current?.value || t('core.chat.feedback.No Content'));
}, },
successToast: t('chat.Feedback Success'), successToast: t('chat.Feedback Success'),
errorToast: t('chat.Feedback Failed') errorToast: t('chat.Feedback Failed')
...@@ -40,11 +43,7 @@ const FeedbackModal = ({ ...@@ -40,11 +43,7 @@ const FeedbackModal = ({
title={t('chat.Feedback Modal')} title={t('chat.Feedback Modal')}
> >
<ModalBody> <ModalBody>
<Textarea <Textarea ref={ref} rows={10} placeholder={t('chat.Feedback Modal Tip')} />
ref={ref}
rows={10}
placeholder={t('chat.Feedback Modal Tip') || 'chat.Feedback Modal Tip'}
/>
</ModalBody> </ModalBody>
<ModalFooter> <ModalFooter>
<Button variant={'base'} mr={2} onClick={onClose}> <Button variant={'base'} mr={2} onClick={onClose}>
......
projects/app/src/components/ChatBox/QuoteModal.tsx
import React, { useCallback, useMemo, useState } from 'react'; import React, { useCallback, useState } from 'react';
import { ModalBody, Box, useTheme, Flex, Progress, Link, Image } from '@chakra-ui/react'; import { ModalBody, Box, useTheme, Flex, Progress, Link } from '@chakra-ui/react';
import { getDatasetDataItemById } from '@/web/core/dataset/api'; import { getDatasetDataItemById } from '@/web/core/dataset/api';
import { useLoading } from '@/web/common/hooks/useLoading'; import { useLoading } from '@/web/common/hooks/useLoading';
import { useToast } from '@/web/common/hooks/useToast'; import { useToast } from '@/web/common/hooks/useToast';
...@@ -11,7 +11,6 @@ import InputDataModal, { ...@@ -11,7 +11,6 @@ import InputDataModal, {
} from '@/pages/dataset/detail/components/InputDataModal'; } from '@/pages/dataset/detail/components/InputDataModal';
import MyModal from '../MyModal'; import MyModal from '../MyModal';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { useRouter } from 'next/router';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type';
import MyTooltip from '../MyTooltip'; import MyTooltip from '../MyTooltip';
import NextLink from 'next/link'; import NextLink from 'next/link';
...@@ -19,21 +18,20 @@ import { useSystemStore } from '@/web/common/system/useSystemStore'; ...@@ -19,21 +18,20 @@ import { useSystemStore } from '@/web/common/system/useSystemStore';
const QuoteModal = ({ const QuoteModal = ({
rawSearch = [], rawSearch = [],
onClose onClose,
isShare
}: { }: {
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseItemType[];
onClose: () => void; onClose: () => void;
isShare: boolean;
}) => { }) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { isPc } = useSystemStore(); const { isPc } = useSystemStore();
const theme = useTheme(); const theme = useTheme();
const router = useRouter();
const { toast } = useToast(); const { toast } = useToast();
const { setIsLoading, Loading } = useLoading(); const { setIsLoading, Loading } = useLoading();
const [editInputData, setEditInputData] = useState<InputDataType & { collectionId: string }>(); const [editInputData, setEditInputData] = useState<InputDataType & { collectionId: string }>();
const isShare = useMemo(() => router.pathname === '/chat/share', [router.pathname]);
/** /**
* click edit, get new DataItem * click edit, get new DataItem
*/ */
......
projects/app/src/components/ChatBox/ReadFeedbackModal.tsx
import React from 'react'; import React from 'react';
import { ModalBody, ModalFooter, Button } from '@chakra-ui/react'; import { ModalBody, ModalFooter, Button } from '@chakra-ui/react';
import MyModal from '../MyModal'; import MyModal from '../MyModal';
import { useRequest } from '@/web/common/hooks/useRequest';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { userUpdateChatFeedback } from '@/web/core/chat/api';
const ReadFeedbackModal = ({ const ReadFeedbackModal = ({
chatItemId,
content, content,
isMarked, onCloseFeedback,
onMark,
onSuccess,
onClose onClose
}: { }: {
chatItemId: string;
content: string; content: string;
isMarked: boolean; onCloseFeedback: () => void;
onMark: () => void;
onSuccess: () => void;
onClose: () => void; onClose: () => void;
}) => { }) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { mutate, isLoading } = useRequest({
mutationFn: async () => {
return userUpdateChatFeedback({
chatItemId,
userFeedback: undefined
});
},
onSuccess() {
onSuccess();
},
errorToast: t('chat.Feedback Update Failed')
});
return ( return (
<MyModal <MyModal
isOpen={true} isOpen={true}
...@@ -44,14 +23,9 @@ const ReadFeedbackModal = ({ ...@@ -44,14 +23,9 @@ const ReadFeedbackModal = ({
> >
<ModalBody>{content}</ModalBody> <ModalBody>{content}</ModalBody>
<ModalFooter> <ModalFooter>
<Button mr={2} isLoading={isLoading} variant={'base'} onClick={mutate}> <Button mr={2} onClick={onCloseFeedback}>
{t('chat.Feedback Close')} {t('core.chat.feedback.Feedback Close')}
</Button>
{!isMarked && (
<Button mr={2} onClick={onMark}>
{t('chat.Feedback Mark')}
</Button> </Button>
)}
</ModalFooter> </ModalFooter>
</MyModal> </MyModal>
); );
......
projects/app/src/components/ChatBox/ResponseTags.tsx
...@@ -13,12 +13,19 @@ import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils'; ...@@ -13,12 +13,19 @@ import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils';
import ChatBoxDivider from '@/components/core/chat/Divider'; import ChatBoxDivider from '@/components/core/chat/Divider';
import MyIcon from '../Icon'; import MyIcon from '../Icon';
import { getFileAndOpen } from '@/web/core/dataset/utils'; import { getFileAndOpen } from '@/web/core/dataset/utils';
import { strIsLink } from '@fastgpt/global/common/string/tools';
const QuoteModal = dynamic(() => import('./QuoteModal'), { ssr: false }); const QuoteModal = dynamic(() => import('./QuoteModal'), { ssr: false });
const ContextModal = dynamic(() => import('./ContextModal'), { ssr: false }); const ContextModal = dynamic(() => import('./ContextModal'), { ssr: false });
const WholeResponseModal = dynamic(() => import('./WholeResponseModal'), { ssr: false }); const WholeResponseModal = dynamic(() => import('./WholeResponseModal'), { ssr: false });
const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemResType[] }) => { const ResponseTags = ({
responseData = [],
isShare
}: {
responseData?: ChatHistoryItemResType[];
isShare: boolean;
}) => {
const theme = useTheme(); const theme = useTheme();
const { isPc } = useSystemStore(); const { isPc } = useSystemStore();
const { t } = useTranslation(); const { t } = useTranslation();
...@@ -62,12 +69,13 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes ...@@ -62,12 +69,13 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes
.map((item) => ({ .map((item) => ({
sourceName: item.sourceName, sourceName: item.sourceName,
sourceId: item.sourceId, sourceId: item.sourceId,
icon: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }) icon: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }),
canReadQuote: !isShare || strIsLink(item.sourceId)
})), })),
historyPreview: chatData?.historyPreview, historyPreview: chatData?.historyPreview,
runningTime: +responseData.reduce((sum, item) => sum + (item.runningTime || 0), 0).toFixed(2) runningTime: +responseData.reduce((sum, item) => sum + (item.runningTime || 0), 0).toFixed(2)
}; };
}, [responseData]); }, [isShare, responseData]);
const TagStyles: BoxProps = { const TagStyles: BoxProps = {
mr: 2, mr: 2,
...@@ -97,7 +105,6 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes ...@@ -97,7 +105,6 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes
}} }}
overflow={'hidden'} overflow={'hidden'}
position={'relative'} position={'relative'}
onClick={() => setQuoteModalData(quoteList)}
> >
<Image src={item.icon} alt={''} mr={1} w={'12px'} /> <Image src={item.icon} alt={''} mr={1} w={'12px'} />
<Box className="textEllipsis" flex={'1 0 0'}> <Box className="textEllipsis" flex={'1 0 0'}>
...@@ -106,7 +113,7 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes ...@@ -106,7 +113,7 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes
<Box <Box
className="controller" className="controller"
display={['flex', 'none']} display={'none'}
pr={2} pr={2}
position={'absolute'} position={'absolute'}
right={0} right={0}
...@@ -127,9 +134,13 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes ...@@ -127,9 +134,13 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes
_hover={{ _hover={{
color: 'green.600' color: 'green.600'
}} }}
onClick={(e) => {
e.stopPropagation();
setQuoteModalData(quoteList);
}}
/> />
</MyTooltip> </MyTooltip>
{item.sourceId && ( {item.sourceId && item.canReadQuote && (
<MyTooltip label={t('core.chat.quote.Read Source')}> <MyTooltip label={t('core.chat.quote.Read Source')}>
<MyIcon <MyIcon
ml={4} ml={4}
...@@ -201,7 +212,11 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes ...@@ -201,7 +212,11 @@ const ResponseTags = ({ responseData = [] }: { responseData?: ChatHistoryItemRes
</MyTooltip> </MyTooltip>
{!!quoteModalData && ( {!!quoteModalData && (
<QuoteModal rawSearch={quoteModalData} onClose={() => setQuoteModalData(undefined)} /> <QuoteModal
rawSearch={quoteModalData}
isShare={isShare}
onClose={() => setQuoteModalData(undefined)}
/>
)} )}
{!!contextModalData && ( {!!contextModalData && (
<ContextModal context={contextModalData} onClose={() => setContextModalData(undefined)} /> <ContextModal context={contextModalData} onClose={() => setContextModalData(undefined)} />
......
projects/app/src/components/Icon/icons/core/chat/feedback/badLight.svg
<?xml version="1.0" standalone="no"?> <?xml version="1.0" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1693117978437" <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1702264166621"
class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="5051" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="3126"
xmlns:xlink="http://www.w3.org/1999/xlink" width="64" height="64"> xmlns:xlink="http://www.w3.org/1999/xlink" width="128" height="128">
<path <path
d="M967.974201 449.978501c11.329434 12.669367 20.158992 29.00055 26.498675 48.987551 6.329684 19.986001 9.499525 40.654967 9.499525 62.013899 0 13.329334-1.499925 25.828709-4.509775 37.498125s-7.179641 22.163892-12.509375 31.483426c-7.339633 11.329434-16.674166 20.158992-27.9986 26.493675-11.329434 6.339683-23.658817 9.504525-36.988151 9.504525L787.959202 665.959702c72.656367 132.658367 90.660467 231.652417 54.012299 296.969152-3.329834 5.989701-7.329634 11.829409-11.9994 17.499125-4.669767 5.674716-9.339533 10.509475-14.014299 14.499275-4.674766 3.984801-9.834508 7.654617-15.484226 11.00445-5.654717 3.349833-11.159442 6.184691-16.514174 8.509575-5.354732 2.324884-10.684466 4.159792-15.9992 5.504725-5.309735 1.344933-10.474476 2.349883-15.484226 3.00985-5.014749 0.659967-9.849508 0.989951-14.499275 0.989951-6.674666 0-12.339383-2.164892-16.989151-6.494675-4.649768-4.329784-7.644618-9.494525-8.989551-15.484226-2.669867-22.678866-10.164492-46.347683-22.493875-71.00645-12.329384-24.658767-28.498575-49.817509-48.507575-75.481226s-43.347833-50.492475-70.011499-74.491275c-79.993-71.9914-170.321484-124.99375-270.985451-158.99205-5.9917 12.669367-15.326234 23.163842-27.9956 31.483426-12.671366 8.319584-26.333683 12.489376-40.989951 12.514374L77.011749 665.9937c-21.330933 0-39.495025-7.499625-54.492275-22.498875C7.522224 628.501575 0.025599 610.337483 0.025599 589.00955L0.025599 128.053597c0-21.330933 7.496625-39.495025 22.493875-54.492275C37.515724 58.567072 55.679816 51.067447 77.011749 51.067447l154.0023 0c15.9992 0 30.333483 4.501775 43.00485 13.504325 12.671366 8.99955 22.0039 20.828959 27.9986 35.483226 31.335433-4.0098 59.00205-9.171541 83.00085-15.486226 23.9988-6.311684 41.157942-11.816409 51.482426-16.509175s23.828809-11.359432 40.509975-19.999c20.009-11.326434 36.508175-19.496025 49.497525-24.510774 12.989351-5.011749 35.328234-10.174491 67.00665-15.484226 31.678416-5.311734 69.506525-7.979601 113.494325-7.9996 18.664067 0 36.328184 2.334883 52.987351 7.00665 16.659167 4.669767 31.653417 11.336433 44.987751 19.996 25.9987 17.344133 41.9979 40.337983 47.9976 68.986551 20.668967 8.661567 37.9981 27.325634 51.9924 55.9972 14.674266 29.9905 22.013899 59.994 22.013899 90.0075l0 7.9996c0 2.0049-0.329984 4.0098-0.989951 6.014699 4.669767 3.327834 8.99955 7.157642 12.989351 11.486426s7.819609 9.161542 11.489426 14.494275c3.669817 5.334733 6.834658 11.339433 9.499525 18.016099 15.339233 29.330533 23.00885 60.323984 23.00885 92.982351 0 10.666467-0.99995 20.498975-3.00985 29.503525-2.0049 8.99955-4.99975 17.171141-8.989551 24.508775-1.344933 1.344933-2.014899 2.346883-2.014899 3.00685l-0.99495 0L967.976201 449.978501zM230.053097 102.0069 77.073746 102.0069c-7.336633 0-13.501325 2.496875-18.494075 7.486626-4.989751 4.99175-7.486626 11.156442-7.486626 18.494075l0 460.950952c0 6.679666 2.496875 12.514374 7.486626 17.509125 4.99175 4.989751 11.156442 7.484626 18.494075 7.484626l152.979351 0c7.339633 0 13.504325-2.494875 18.496075-7.484626 4.989751-4.99475 7.486626-10.829459 7.486626-17.509125L256.035798 127.988601c0-2.664867-0.502975-5.331733-1.504925-7.9996-1.00195-2.664867-2.164892-4.99975-3.486826-7.00665-1.321934-2.0049-3.156842-3.839808-5.504725-5.501725-2.344883-1.664917-4.681766-2.99685-7.00665-3.9998s-5.161742-1.504925-8.509575-1.504925L230.053097 102.0069zM938.986651 497.974101c-7.339633-13.9943-13.669317-20.988951-19.00905-20.988951-3.324834 0-6.659667-0.672966-10.0095-2.017899-3.349833-1.341933-6.189691-3.176841-8.514574-5.501725-2.324884-2.324884-4.159792-5.161742-5.504725-8.511574-1.339933-3.349833-2.014899-6.686666-2.014899-10.014499 0-6.676666 2.494875-12.511374 7.489626-17.504125 4.989751-4.989751 11.154442-7.486626 18.494075-7.486626 2.664867 0 4.669767-1.331933 6.014699-3.9998 3.324834-6.674666 4.99475-16.339183 4.99475-28.988551 0-24.658767-5.334733-48.327584-15.9992-71.00145-9.349533-18.666067-17.679116-27.9986-24.99375-27.9986-7.314634 0-13.479326-2.494875-18.494075-7.486626-5.00975-4.99175-7.509625-10.826459-7.489626-17.501125 0-4.671766 1.00495-8.844558 3.00985-12.511374 2.0099-3.669817 4.99975-6.836658 8.989551-9.504525 0.664967-5.9917 0.159992-13.821309-1.499925-23.485826-1.664917-9.661517-4.659767-20.658967-8.989551-32.988351-4.329784-12.329384-9.494525-22.823859-15.489226-31.485426-7.9996-11.326434-14.99925-16.989151-20.988951-16.989151-7.339633 0-13.504325-2.496875-18.499075-7.486626-4.989751-4.99175-7.484626-10.826459-7.484626-17.504125 0-21.330933-9.504525-37.660117-28.508575-48.987551s-41.502925-16.99115-67.481626-16.99115c-30.678466 0-57.167142 1.00195-79.486026 3.00685-22.308885 2.0069-41.477926 5.344733-57.497125 10.016499s-28.188591 8.841558-36.508175 12.511374c-8.319584 3.666817-19.156042 9.171541-32.510374 16.509175-13.331333 6.676666-24.168792 12.0094-32.508375 15.9992-8.339583 3.989801-20.510974 8.989551-36.508175 15.00625-15.9992 6.014699-34.163292 11.176441-54.492275 15.486226-20.328984 4.309785-43.824809 8.136593-70.491475 11.486426l0 418.970051c111.329434 36.668167 210.661467 94.330283 297.9921 172.981351 83.320834 75.321234 134.313284 150.652467 152.977351 225.9977 18.0091-5.334733 31.338433-15.669217 39.998-31.00845 4.669767-9.339533 7.839608-20.508975 9.504525-33.498325 1.664917-12.99435 1.164942-30.658467-1.504925-52.987351-2.664867-22.338883-11.169442-50.507475-25.498725-84.505775-14.339283-34.0033-34.168292-72.0064-59.487026-114.0093-4.669767-8.659567-4.669767-17.334133 0-26.013699 2.0099-4.0098 5.00475-7.179641 8.989551-9.499525 3.989801-2.329884 8.319584-3.489826 12.99435-3.489826l178.99605 0c9.339533 0 16.679166-3.9998 22.0089-11.9994l2.019899-4.989751c1.339933-2.0099 2.344883-4.0098 3.00985-6.019699 0.659967-2.0049 1.159942-4.169792 1.499925-6.494675 0.339983-2.324884 0.839958-4.819759 1.504925-7.484626l0-7.9996c0.659967-2.669867 0.99495-5.334733 0.99495-7.9996 0-23.338833-4.674766-44.337783-14.019299-63.00385L938.986651 497.974101z" d="M743.328 31.04c71.232 0 127.008 32.96 157.568 93.632 2.784 5.504 3.872 10.944 4.928 16.256 2.24 11.04 5.6 27.776 30.432 57.536 36.32 43.52 29.984 90.08 24.896 127.488-1.312 9.76-2.656 19.456-3.232 29.024-1.856 31.648 3.968 41.184 19.488 66.656 3.136 5.216 6.592 10.848 10.272 17.184 23.68 40.896 22.752 90.912-2.464 133.792-28.96 49.248-82.624 78.624-143.488 78.624-51.552 0-108.256-1.12-141.408-1.952 8.64 50.048 25.44 154.528 25.44 210.656 0 92.64-67.04 125.504-124.448 125.504-63.68 0-113.536-58.688-113.536-133.6 0-57.92 0-65.344-22.976-97.088-60.608-83.776-122.464-142.272-150.304-142.272L117.76 612.48c-49.056 0-88.992-39.392-88.992-87.776L28.768 118.848c0-48.416 39.904-87.808 88.992-87.808l133.888 0c12.896 0 23.392 10.368 23.392 23.104l0 512.224 39.424 0c62.176 0 144.704 101.28 188.416 161.632 31.68 43.744 31.68 63.968 31.68 123.904 0 48.192 29.92 87.424 66.784 87.424 23.296 0 77.632-7.712 77.632-79.328 0-69.824-29.344-228.704-29.664-230.304-1.28-6.848 0.64-13.92 5.28-19.2 4.608-5.28 10.688-8.288 18.464-8 0.864 0 90.72 2.656 168.672 2.656 44.032 0 82.528-20.832 103.008-55.648 16.768-28.48 17.632-61.28 2.304-87.712-3.456-6.016-6.72-11.392-9.792-16.32-16.704-27.456-28.736-47.264-26.048-93.12 0.608-10.752 2.08-21.6 3.552-32.512 4.672-34.368 8.704-64.064-14.624-92.032-32.352-38.784-37.28-63.296-40.192-77.92-0.416-1.984-0.672-3.712-1.408-5.568-15.392-30.624-47.872-67.136-115.168-67.136l-316.704 0c-12.896 0-23.392-10.336-23.392-23.072 0-12.768 10.496-23.104 23.392-23.104L743.328 31.008 743.328 31.04zM117.76 566.368l110.496 0L228.256 77.248 117.76 77.248c-23.232 0-42.176 18.656-42.176 41.632L75.584 524.8C75.552 547.712 94.496 566.368 117.76 566.368L117.76 566.368zM117.76 566.368"
p-id="5052"></path> p-id="3127"></path>
</svg> </svg>
\ No newline at end of file
projects/app/src/components/Icon/icons/core/chat/feedback/goodLight.svg 0 → 100644
<?xml version="1.0" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1702264142648"
class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="2973"
data-spm-anchor-id="a313x.manage_type_myprojects.0.i0.65e73a81QqWlcB" xmlns:xlink="http://www.w3.org/1999/xlink"
width="128" height="128">
<path
d="M743.328 985.536l-316.704 0c-12.896 0-23.392-10.368-23.392-23.104s10.496-23.072 23.392-23.072l316.704 0c67.296 0 99.808-36.512 115.168-67.136 0.768-1.856 1.024-3.552 1.408-5.568 2.912-14.624 7.84-39.168 40.192-77.92 23.328-27.968 19.328-57.664 14.624-92.032-1.472-10.912-2.944-21.76-3.552-32.512-2.688-45.856 9.344-65.664 26.048-93.12 3.04-4.928 6.304-10.304 9.792-16.32 15.328-26.432 14.464-59.232-2.304-87.712-20.48-34.816-59.008-55.648-103.008-55.648-77.952 0-167.808 2.656-168.672 2.656-7.776 0.288-13.888-2.72-18.464-8-4.64-5.28-6.528-12.352-5.28-19.2 0.32-1.6 29.664-160.48 29.664-230.304 0-71.616-54.336-79.328-77.632-79.328-36.832 0-66.784 39.2-66.784 87.424 0 59.936 0 80.16-31.68 123.904-43.712 60.352-126.24 161.632-188.416 161.632L275.04 450.176l0 512.224c0 12.768-10.496 23.104-23.392 23.104L117.76 985.504c-49.056 0-88.992-39.392-88.992-87.808L28.768 491.808c0-48.384 39.904-87.776 88.992-87.776l196.704 0c27.84 0 89.696-58.496 150.304-142.272 22.976-31.712 22.976-39.168 22.976-97.088 0-74.944 49.856-133.6 113.536-133.6 57.408 0 124.448 32.896 124.448 125.504 0 56.128-16.8 160.576-25.44 210.656 33.184-0.8 89.856-1.952 141.408-1.952 60.864 0 114.56 29.376 143.488 78.624 25.216 42.88 26.144 92.928 2.464 133.792-3.68 6.336-7.104 11.968-10.272 17.184-15.52 25.472-21.344 35.008-19.488 66.656 0.576 9.568 1.952 19.296 3.232 29.024 5.088 37.408 11.424 83.968-24.896 127.488-24.832 29.792-28.192 46.496-30.432 57.536-1.056 5.28-2.176 10.752-4.928 16.256C870.336 952.544 814.56 985.536 743.328 985.536L743.328 985.536zM117.76 450.208c-23.232 0-42.176 18.656-42.176 41.6l0 405.952c0 22.976 18.944 41.632 42.176 41.632l110.496 0L228.256 450.208 117.76 450.208 117.76 450.208zM117.76 450.208"
p-id="2974"></path>
</svg>
\ No newline at end of file
projects/app/src/components/Icon/index.tsx
...@@ -78,6 +78,7 @@ const iconPaths = { ...@@ -78,6 +78,7 @@ const iconPaths = {
'support/account/promotionLight': () => import('./icons/support/account/promotionLight.svg'), 'support/account/promotionLight': () => import('./icons/support/account/promotionLight.svg'),
'core/app/logsLight': () => import('./icons/core/app/logsLight.svg'), 'core/app/logsLight': () => import('./icons/core/app/logsLight.svg'),
'core/chat/feedback/badLight': () => import('./icons/core/chat/feedback/badLight.svg'), 'core/chat/feedback/badLight': () => import('./icons/core/chat/feedback/badLight.svg'),
'core/chat/feedback/goodLight': () => import('./icons/core/chat/feedback/goodLight.svg'),
'core/app/markLight': () => import('./icons/core/app/markLight.svg'), 'core/app/markLight': () => import('./icons/core/app/markLight.svg'),
'common/retryLight': () => import('./icons/common/retryLight.svg'), 'common/retryLight': () => import('./icons/common/retryLight.svg'),
'common/rightArrowLight': () => import('./icons/common/rightArrowLight.svg'), 'common/rightArrowLight': () => import('./icons/common/rightArrowLight.svg'),
......
projects/app/src/components/Layout/navbar.tsx
...@@ -3,7 +3,7 @@ import { Box, Flex, Link } from '@chakra-ui/react'; ...@@ -3,7 +3,7 @@ import { Box, Flex, Link } from '@chakra-ui/react';
import { useRouter } from 'next/router'; import { useRouter } from 'next/router';
import { useUserStore } from '@/web/support/user/useUserStore'; import { useUserStore } from '@/web/support/user/useUserStore';
import { useChatStore } from '@/web/core/chat/storeChat'; import { useChatStore } from '@/web/core/chat/storeChat';
import { HUMAN_ICON } from '@fastgpt/global/core/chat/constants'; import { HUMAN_ICON } from '@fastgpt/global/common/system/constants';
import { feConfigs } from '@/web/common/system/staticData'; import { feConfigs } from '@/web/common/system/staticData';
import NextLink from 'next/link'; import NextLink from 'next/link';
import Badge from '../Badge'; import Badge from '../Badge';
...@@ -175,18 +175,17 @@ const Navbar = ({ unread }: { unread: number }) => { ...@@ -175,18 +175,17 @@ const Navbar = ({ unread }: { unread: number }) => {
</Link> </Link>
</Box> </Box>
)} )}
{feConfigs?.docUrl && ( {(feConfigs?.docUrl || feConfigs?.chatbotUrl) && (
<MyTooltip label={t('home.Docs')} placement={'right-end'}> <MyTooltip label={t('common.system.Use Helper')} placement={'right-end'}>
<Box <Link
{...itemStyles} {...itemStyles}
href={feConfigs?.chatbotUrl || getDocPath('/docs/intro')}
target="_blank"
mb={0} mb={0}
color={'#9096a5'} color={'#9096a5'}
onClick={() => {
window.open(getDocPath('/docs/intro'));
}}
> >
<MyIcon name={'common/courseLight'} width={'26px'} height={'26px'} /> <MyIcon name={'common/courseLight'} width={'26px'} height={'26px'} />
</Box> </Link>
</MyTooltip> </MyTooltip>
)} )}
{feConfigs?.show_git && ( {feConfigs?.show_git && (
......
projects/app/src/components/Markdown/chat/QuestionGuide.tsx
...@@ -49,7 +49,7 @@ const QuestionGuide = ({ text }: { text: string }) => { ...@@ -49,7 +49,7 @@ const QuestionGuide = ({ text }: { text: string }) => {
</Box> </Box>
<Box <Box
className="controller" className="controller"
display={['flex', 'none']} display={'none'}
pr={2} pr={2}
position={'absolute'} position={'absolute'}
right={0} right={0}
......
projects/app/src/components/Markdown/index.module.scss
...@@ -173,7 +173,6 @@ ...@@ -173,7 +173,6 @@
} }
.markdown ul, .markdown ul,
.markdown ol { .markdown ol {
margin-bottom: 0;
padding-left: 14px; padding-left: 14px;
} }
.markdown dl { .markdown dl {
......
projects/app/src/components/core/module/Flow/components/modules/NodeCard.tsx
...@@ -15,7 +15,7 @@ import { useSystemStore } from '@/web/common/system/useSystemStore'; ...@@ -15,7 +15,7 @@ import { useSystemStore } from '@/web/common/system/useSystemStore';
import { getPreviewPluginModule } from '@/web/core/plugin/api'; import { getPreviewPluginModule } from '@/web/core/plugin/api';
import { getErrText } from '@fastgpt/global/common/error/utils'; import { getErrText } from '@fastgpt/global/common/error/utils';
import { useConfirm } from '@/web/common/hooks/useConfirm'; import { useConfirm } from '@/web/common/hooks/useConfirm';
import { LOGO_ICON } from '@fastgpt/global/core/chat/constants'; import { LOGO_ICON } from '@fastgpt/global/common/system/constants';
type Props = FlowModuleItemType & { type Props = FlowModuleItemType & {
children?: React.ReactNode | React.ReactNode[] | string; children?: React.ReactNode | React.ReactNode[] | string;
......
projects/app/src/components/support/user/team/TeamManageModal/InviteModal.tsx
...@@ -82,7 +82,7 @@ const InviteModal = ({ ...@@ -82,7 +82,7 @@ const InviteModal = ({
> >
<ModalCloseButton onClick={onClose} /> <ModalCloseButton onClick={onClose} />
<ModalBody> <ModalBody>
<Box mb={2}>{t('common.Username')}</Box> <Box mb={2}>{t('user.Account')}</Box>
<TagTextarea defaultValues={inviteUsernames} onUpdate={setInviteUsernames} /> <TagTextarea defaultValues={inviteUsernames} onUpdate={setInviteUsernames} />
<Box mt={4}> <Box mt={4}>
<MySelect list={inviteTypes} value={selectedInviteType} onchange={setSelectInviteType} /> <MySelect list={inviteTypes} value={selectedInviteType} onchange={setSelectInviteType} />
......
projects/app/src/constants/common.ts
export enum UserAuthTypeEnum {
register = 'register',
findPassword = 'findPassword'
}
export enum TrackEventName { export enum TrackEventName {
windowError = 'windowError', windowError = 'windowError',
pageError = 'pageError', pageError = 'pageError',
......
projects/app/src/global/core/prompt/agent.ts
export const Prompt_AgentQA = { export const Prompt_AgentQA = {
description: `<context></context> 标记中是一段文本，学习它们，并整理学习成果。 description: `<context></context> 标记中是一段文本，学习和分析它，并整理学习成果：
学习要求：
- 提出问题并给出每个问题的答案。 - 提出问题并给出每个问题的答案。
- 答案需详细完整，给出相关原文描述。答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 markdown 元素。 - 答案需详细完整，给出相关原文描述。
- 答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 markdown 元素。
- 最多提出 30 个问题。 - 最多提出 30 个问题。
`, `,
fixedText: `最后，你需要按下面的格式返回多个问题和答案: fixedText: `最后，你需要按下面的格式返回多个问题和答案:
......
projects/app/src/pages/account/components/Info.tsx
...@@ -7,7 +7,8 @@ import { ...@@ -7,7 +7,8 @@ import {
useTheme, useTheme,
Divider, Divider,
Select, Select,
Input Input,
Link
} from '@chakra-ui/react'; } from '@chakra-ui/react';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import { UserUpdateParams } from '@/types/user'; import { UserUpdateParams } from '@/types/user';
...@@ -249,8 +250,10 @@ const UserInfo = () => { ...@@ -249,8 +250,10 @@ const UserInfo = () => {
</Flex> </Flex>
</Box> </Box>
{feConfigs?.docUrl && ( {feConfigs?.docUrl && (
<> <Link
<Flex href={getDocPath('/docs/intro')}
target="_blank"
display={'flex'}
mt={4} mt={4}
w={['85%', '300px']} w={['85%', '300px']}
py={3} py={3}
...@@ -259,11 +262,8 @@ const UserInfo = () => { ...@@ -259,11 +262,8 @@ const UserInfo = () => {
borderWidth={'1.5px'} borderWidth={'1.5px'}
borderRadius={'md'} borderRadius={'md'}
alignItems={'center'} alignItems={'center'}
cursor={'pointer'}
userSelect={'none'} userSelect={'none'}
onClick={() => { textDecoration={'none !important'}
window.open(getDocPath('/docs/intro'));
}}
> >
<MyIcon name={'common/courseLight'} w={'18px'} /> <MyIcon name={'common/courseLight'} w={'18px'} />
<Box ml={2} flex={1}> <Box ml={2} flex={1}>
...@@ -273,8 +273,29 @@ const UserInfo = () => { ...@@ -273,8 +273,29 @@ const UserInfo = () => {
<Box fontSize={'md'} ml={2}> <Box fontSize={'md'} ml={2}>
V{systemVersion} V{systemVersion}
</Box> </Box>
</Flex> </Link>
</> )}
{feConfigs?.chatbotUrl && (
<Link
href={feConfigs.chatbotUrl}
target="_blank"
display={'flex'}
mt={4}
w={['85%', '300px']}
py={3}
px={6}
border={theme.borders.sm}
borderWidth={'1.5px'}
borderRadius={'md'}
alignItems={'center'}
userSelect={'none'}
textDecoration={'none !important'}
>
<MyIcon name={'core/app/aiLight'} w={'18px'} />
<Box ml={2} flex={1}>
{t('common.system.Help Chatbot')}
</Box>
</Link>
)} )}
{feConfigs?.show_openai_account && ( {feConfigs?.show_openai_account && (
<> <>
...@@ -283,7 +304,7 @@ const UserInfo = () => { ...@@ -283,7 +304,7 @@ const UserInfo = () => {
<MyTooltip label={'点击配置账号'}> <MyTooltip label={'点击配置账号'}>
<Flex <Flex
w={['85%', '300px']} w={['85%', '300px']}
py={3} py={4}
px={6} px={6}
border={theme.borders.sm} border={theme.borders.sm}
borderWidth={'1.5px'} borderWidth={'1.5px'}
......
projects/app/src/pages/api/admin/init464.ts 0 → 100644
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { PgClient } from '@fastgpt/service/common/pg';
import { PgDatasetTableName } from '@fastgpt/global/core/dataset/constant';
import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { connectToDatabase } from '@/service/mongo';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const { limit = 50, maxSize = 3 } = req.body as { limit: number; maxSize: number };
await authCert({ req, authRoot: true });
await connectToDatabase();
try {
await PgClient.query(
`ALTER TABLE ${PgDatasetTableName} ADD COLUMN createTime TIMESTAMP DEFAULT CURRENT_TIMESTAMP;`
);
} catch (error) {
console.log(error);
}
try {
const result = await MongoChatItem.updateMany(
{ userFeedback: { $exists: true } },
{ $rename: { userFeedback: 'userBadFeedback' } }
);
console.log(result);
} catch (error) {
console.log(error);
}
jsonRes(res, {
data: {}
});
} catch (error) {
console.log(error);
jsonRes(res, {
code: 500,
error
});
}
}
projects/app/src/pages/api/admin/timeTasks/checkUnValidDatasetFiles.ts
...@@ -6,7 +6,7 @@ import { ...@@ -6,7 +6,7 @@ import {
delFileByFileIdList, delFileByFileIdList,
getGFSCollection getGFSCollection
} from '@fastgpt/service/common/file/gridfs/controller'; } from '@fastgpt/service/common/file/gridfs/controller';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema'; import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { delay } from '@fastgpt/global/common/system/utils'; import { delay } from '@fastgpt/global/common/system/utils';
......
projects/app/src/pages/api/core/app/form2Modules/fastgpt-simple.ts
...@@ -379,14 +379,14 @@ function datasetTemplate({ ...@@ -379,14 +379,14 @@ function datasetTemplate({
}, },
{ {
key: 'similarity', key: 'similarity',
value: 0.4, value: 0.1,
type: FlowNodeInputTypeEnum.slider, type: FlowNodeInputTypeEnum.slider,
label: '相关度', label: '相关度',
connected: true connected: true
}, },
{ {
key: 'limit', key: 'limit',
value: 8, value: 2000,
type: FlowNodeInputTypeEnum.slider, type: FlowNodeInputTypeEnum.slider,
label: '单次搜索上限', label: '单次搜索上限',
connected: true connected: true
......
projects/app/src/pages/api/core/app/getChatLogs.ts
...@@ -60,12 +60,30 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -60,12 +60,30 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
}, },
{ {
$addFields: { $addFields: {
feedbackCount: { userGoodFeedbackCount: {
$size: { $size: {
$filter: { $filter: {
input: '$chatitems', input: '$chatitems',
as: 'item', as: 'item',
cond: { $ifNull: ['$$item.userFeedback', false] } cond: { $ifNull: ['$$item.userGoodFeedback', false] }
}
}
},
userBadFeedbackCount: {
$size: {
$filter: {
input: '$chatitems',
as: 'item',
cond: { $ifNull: ['$$item.userBadFeedback', false] }
}
}
},
robotBadFeedbackCount: {
$size: {
$filter: {
input: '$chatitems',
as: 'item',
cond: { $ifNull: ['$$item.robotBadFeedback', false] }
} }
} }
}, },
...@@ -80,7 +98,14 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -80,7 +98,14 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
} }
} }
}, },
{ $sort: { feedbackCount: -1, updateTime: -1 } }, {
$sort: {
userBadFeedbackCount: -1,
userGoodFeedbackCount: -1,
robotBadFeedbackCount: -1,
updateTime: -1
}
},
{ $skip: (pageNum - 1) * pageSize }, { $skip: (pageNum - 1) * pageSize },
{ $limit: pageSize }, { $limit: pageSize },
{ {
...@@ -91,7 +116,9 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -91,7 +116,9 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
source: 1, source: 1,
time: '$updateTime', time: '$updateTime',
messageCount: { $size: '$chatitems' }, messageCount: { $size: '$chatitems' },
feedbackCount: 1, userGoodFeedbackCount: 1,
userBadFeedbackCount: 1,
robotBadFeedbackCount: 1,
markCount: 1 markCount: 1
} }
} }
......
projects/app/src/pages/api/core/chat/feedback/userUpdate.ts → projects/app/src/pages/api/core/chat/feedback/updateUserFeedback.ts
...@@ -2,15 +2,25 @@ import type { NextApiRequest, NextApiResponse } from 'next'; ...@@ -2,15 +2,25 @@ import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response'; import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo'; import { connectToDatabase } from '@/service/mongo';
import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema'; import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { UpdateChatFeedbackProps } from '@fastgpt/global/core/chat/api';
import { autChatCrud } from '@/service/support/permission/auth/chat';
/* 初始化我的聊天框，需要身份验证 */ /* 初始化我的聊天框，需要身份验证 */
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
const { chatId, chatItemId, shareId, outLinkUid, userBadFeedback, userGoodFeedback } =
req.body as UpdateChatFeedbackProps;
try { try {
await connectToDatabase(); await connectToDatabase();
const { chatItemId, userFeedback = undefined } = req.body as {
chatItemId: string; await autChatCrud({
userFeedback?: string; req,
}; authToken: true,
chatId,
shareId,
outLinkUid,
per: 'r'
});
if (!chatItemId) { if (!chatItemId) {
throw new Error('chatItemId is required'); throw new Error('chatItemId is required');
...@@ -21,7 +31,14 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -21,7 +31,14 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
dataId: chatItemId dataId: chatItemId
}, },
{ {
...(userFeedback ? { userFeedback } : { $unset: { userFeedback: '' } }) $unset: {
...(userBadFeedback === undefined && { userBadFeedback: '' }),
...(userGoodFeedback === undefined && { userGoodFeedback: '' })
},
$set: {
...(userBadFeedback !== undefined && { userBadFeedback }),
...(userGoodFeedback !== undefined && { userGoodFeedback })
}
} }
); );
......
projects/app/src/pages/api/core/chat/init.ts
...@@ -43,7 +43,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -43,7 +43,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
const { history } = await getChatItems({ const { history } = await getChatItems({
chatId, chatId,
limit: 30, limit: 30,
field: `dataId obj value adminFeedback userFeedback ${ModuleOutputKeyEnum.responseData}` field: `dataId obj value adminFeedback userBadFeedback userGoodFeedback robotBadFeedback ${ModuleOutputKeyEnum.responseData}`
}); });
jsonRes<InitChatResponse>(res, { jsonRes<InitChatResponse>(res, {
......
projects/app/src/pages/api/core/chat/outLink/init.ts
...@@ -42,7 +42,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -42,7 +42,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
const { history } = await getChatItems({ const { history } = await getChatItems({
chatId, chatId,
limit: 30, limit: 30,
field: `dataId obj value userFeedback ${ field: `dataId obj value userGoodFeedback userBadFeedback ${
shareChat.responseDetail ? `adminFeedback ${ModuleOutputKeyEnum.responseData}` : '' shareChat.responseDetail ? `adminFeedback ${ModuleOutputKeyEnum.responseData}` : ''
} ` } `
}); });
......
projects/app/src/pages/api/core/dataset/checkExportLimit.ts
...@@ -2,7 +2,7 @@ import type { NextApiRequest, NextApiResponse } from 'next'; ...@@ -2,7 +2,7 @@ import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response'; import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo'; import { connectToDatabase } from '@/service/mongo';
import { MongoUser } from '@fastgpt/service/support/user/schema'; import { MongoUser } from '@fastgpt/service/support/user/schema';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset'; import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema'; import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema';
import { findDatasetIdTreeByTopDatasetId } from '@fastgpt/service/core/dataset/controller'; import { findDatasetIdTreeByTopDatasetId } from '@fastgpt/service/core/dataset/controller';
......
projects/app/src/pages/api/core/dataset/collection/delete.ts
...@@ -33,11 +33,6 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse< ...@@ -33,11 +33,6 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
fileIds: collections.map((item) => item?.fileId || '').filter(Boolean) fileIds: collections.map((item) => item?.fileId || '').filter(Boolean)
}); });
// delete collection
await MongoDatasetCollection.deleteMany({
_id: { $in: delIdList }
});
jsonRes(res); jsonRes(res);
} catch (err) { } catch (err) {
jsonRes(res, { jsonRes(res, {
......
projects/app/src/pages/api/core/dataset/collection/sync/link.ts
...@@ -68,13 +68,10 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse< ...@@ -68,13 +68,10 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
}); });
// delete old collection // delete old collection
await Promise.all([ await delCollectionRelevantData({
delCollectionRelevantData({
collectionIds: [collection._id], collectionIds: [collection._id],
fileIds: collection.fileId ? [collection.fileId] : [] fileIds: collection.fileId ? [collection.fileId] : []
}), });
MongoDatasetCollection.findByIdAndRemove(collection._id)
]);
jsonRes(res); jsonRes(res);
} catch (err) { } catch (err) {
......
projects/app/src/pages/api/core/dataset/data/pushData.ts
...@@ -136,7 +136,7 @@ export async function pushDataToDatasetCollection({ ...@@ -136,7 +136,7 @@ export async function pushDataToDatasetCollection({
model, model,
q: item.q, q: item.q,
a: item.a, a: item.a,
chunkIndex: i, chunkIndex: item.chunkIndex ?? i,
indexes: item.indexes indexes: item.indexes
})) }))
); );
......
projects/app/src/pages/api/core/dataset/exportAll.ts
...@@ -2,7 +2,7 @@ import type { NextApiRequest, NextApiResponse } from 'next'; ...@@ -2,7 +2,7 @@ import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes, responseWriteController } from '@fastgpt/service/common/response'; import { jsonRes, responseWriteController } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo'; import { connectToDatabase } from '@/service/mongo';
import { MongoUser } from '@fastgpt/service/support/user/schema'; import { MongoUser } from '@fastgpt/service/support/user/schema';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset'; import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema'; import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema';
import { findDatasetIdTreeByTopDatasetId } from '@fastgpt/service/core/dataset/controller'; import { findDatasetIdTreeByTopDatasetId } from '@fastgpt/service/core/dataset/controller';
......
projects/app/src/pages/api/core/dataset/searchTest.ts
...@@ -6,10 +6,8 @@ import { connectToDatabase } from '@/service/mongo'; ...@@ -6,10 +6,8 @@ import { connectToDatabase } from '@/service/mongo';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset'; import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { authTeamBalance } from '@/service/support/permission/auth/bill'; import { authTeamBalance } from '@/service/support/permission/auth/bill';
import { pushGenerateVectorBill } from '@/service/support/wallet/bill/push'; import { pushGenerateVectorBill } from '@/service/support/wallet/bill/push';
import { countModelPrice } from '@/service/support/wallet/bill/utils';
import { searchDatasetData } from '@/service/core/dataset/data/pg'; import { searchDatasetData } from '@/service/core/dataset/data/pg';
import { updateApiKeyUsage } from '@fastgpt/service/support/openapi/tools'; import { updateApiKeyUsage } from '@fastgpt/service/support/openapi/tools';
import { ModelTypeEnum } from '@/service/core/ai/model';
import { BillSourceEnum } from '@fastgpt/global/support/wallet/bill/constants'; import { BillSourceEnum } from '@fastgpt/global/support/wallet/bill/constants';
export default withNextCors(async function handler(req: NextApiRequest, res: NextApiResponse<any>) { export default withNextCors(async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
...@@ -38,7 +36,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -38,7 +36,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
const { searchRes, tokenLen } = await searchDatasetData({ const { searchRes, tokenLen } = await searchDatasetData({
text, text,
model: dataset.vectorModel, model: dataset.vectorModel,
limit: Math.min(limit, 50), limit: Math.min(limit * 800, 30000),
datasetIds: [datasetId], datasetIds: [datasetId],
searchMode searchMode
}); });
......
projects/app/src/pages/api/system/getInitData.ts
...@@ -20,6 +20,7 @@ import { ...@@ -20,6 +20,7 @@ import {
import { SimpleModeTemplate_FastGPT_Universal } from '@/global/core/app/constants'; import { SimpleModeTemplate_FastGPT_Universal } from '@/global/core/app/constants';
import { getSimpleTemplatesFromPlus } from '@/service/core/app/utils'; import { getSimpleTemplatesFromPlus } from '@/service/core/app/utils';
import { PluginTypeEnum } from '@fastgpt/global/core/plugin/constants'; import { PluginTypeEnum } from '@fastgpt/global/core/plugin/constants';
import { getFastGPTFeConfig } from '@fastgpt/service/common/system/config/controller';
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
await getInitConfig(); await getInitConfig();
...@@ -73,7 +74,17 @@ export async function getInitConfig() { ...@@ -73,7 +74,17 @@ export async function getInitConfig() {
process.env.NODE_ENV === 'development' ? 'data/config.local.json' : '/app/data/config.json'; process.env.NODE_ENV === 'development' ? 'data/config.local.json' : '/app/data/config.json';
const res = JSON.parse(readFileSync(filename, 'utf-8')) as ConfigFileType; const res = JSON.parse(readFileSync(filename, 'utf-8')) as ConfigFileType;
setDefaultData(res); // get config from database
const dbFeConfig = await getFastGPTFeConfig();
const concatConfig: ConfigFileType = {
...res,
FeConfig: {
...res.FeConfig,
...dbFeConfig
}
};
setDefaultData(concatConfig);
} catch (error) { } catch (error) {
setDefaultData(); setDefaultData();
console.log('get init config error, set default', error); console.log('get init config error, set default', error);
...@@ -83,6 +94,23 @@ export async function getInitConfig() { ...@@ -83,6 +94,23 @@ export async function getInitConfig() {
getSystemVersion(); getSystemVersion();
getModelPrice(); getModelPrice();
getSystemPlugin(); getSystemPlugin();
console.log({
FeConfig: global.feConfigs,
SystemParams: global.systemEnv,
ChatModels: global.chatModels,
QAModels: global.qaModels,
CQModels: global.cqModels,
ExtractModels: global.extractModels,
QGModels: global.qgModels,
VectorModels: global.vectorModels,
ReRankModels: global.reRankModels,
AudioSpeechModels: global.reRankModels,
WhisperModel: global.whisperModel,
price: global.priceMd,
simpleModeTemplates: global.simpleModeTemplates,
communityPlugins: global.communityPlugins
});
} }
export function initGlobal() { export function initGlobal() {
...@@ -125,8 +153,6 @@ export function setDefaultData(res?: ConfigFileType) { ...@@ -125,8 +153,6 @@ export function setDefaultData(res?: ConfigFileType) {
global.whisperModel = res?.WhisperModel || defaultWhisperModel; global.whisperModel = res?.WhisperModel || defaultWhisperModel;
global.priceMd = ''; global.priceMd = '';
console.log(res);
} }
export function getSystemVersion() { export function getSystemVersion() {
...@@ -173,7 +199,6 @@ ${global.audioSpeechModels ...@@ -173,7 +199,6 @@ ${global.audioSpeechModels
.join('\n')} .join('\n')}
${`| 语音输入-${global.whisperModel.name} | ${global.whisperModel.price}/分钟 |`} ${`| 语音输入-${global.whisperModel.name} | ${global.whisperModel.price}/分钟 |`}
`; `;
console.log(global.priceMd);
} }
async function getSimpleModeTemplates() { async function getSimpleModeTemplates() {
...@@ -209,8 +234,6 @@ async function getSimpleModeTemplates() { ...@@ -209,8 +234,6 @@ async function getSimpleModeTemplates() {
} catch (error) { } catch (error) {
global.simpleModeTemplates = [SimpleModeTemplate_FastGPT_Universal]; global.simpleModeTemplates = [SimpleModeTemplate_FastGPT_Universal];
} }
console.log('simple mode templates: ');
console.log(global.simpleModeTemplates);
} }
function getSystemPlugin() { function getSystemPlugin() {
...@@ -236,6 +259,4 @@ function getSystemPlugin() { ...@@ -236,6 +259,4 @@ function getSystemPlugin() {
}); });
global.communityPlugins = fileTemplates; global.communityPlugins = fileTemplates;
console.log('community plugins: ');
console.log(fileTemplates);
} }
projects/app/src/pages/api/user/account/loginByPassword.ts
...@@ -5,6 +5,7 @@ import { createJWT, setCookie } from '@fastgpt/service/support/permission/contro ...@@ -5,6 +5,7 @@ import { createJWT, setCookie } from '@fastgpt/service/support/permission/contro
import { connectToDatabase } from '@/service/mongo'; import { connectToDatabase } from '@/service/mongo';
import { getUserDetail } from '@fastgpt/service/support/user/controller'; import { getUserDetail } from '@fastgpt/service/support/user/controller';
import type { PostLoginProps } from '@fastgpt/global/support/user/api.d'; import type { PostLoginProps } from '@fastgpt/global/support/user/api.d';
import { UserStatusEnum } from '@fastgpt/global/support/user/constant';
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try { try {
...@@ -16,13 +17,20 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -16,13 +17,20 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
} }
// 检测用户是否存在 // 检测用户是否存在
const authCert = await MongoUser.findOne({ const authCert = await MongoUser.findOne(
{
username username
}); },
'status'
);
if (!authCert) { if (!authCert) {
throw new Error('用户未注册'); throw new Error('用户未注册');
} }
if (authCert.status === UserStatusEnum.forbidden) {
throw new Error('账号已停用，无法登录');
}
const user = await MongoUser.findOne({ const user = await MongoUser.findOne({
username, username,
password password
......
projects/app/src/pages/api/v1/chat/completions.ts
...@@ -2,7 +2,7 @@ import type { NextApiRequest, NextApiResponse } from 'next'; ...@@ -2,7 +2,7 @@ import type { NextApiRequest, NextApiResponse } from 'next';
import { authApp } from '@fastgpt/service/support/permission/auth/app'; import { authApp } from '@fastgpt/service/support/permission/auth/app';
import { authCert } from '@fastgpt/service/support/permission/auth/common'; import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { sseErrRes, jsonRes } from '@fastgpt/service/common/response'; import { sseErrRes, jsonRes } from '@fastgpt/service/common/response';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import { withNextCors } from '@fastgpt/service/common/middle/cors'; import { withNextCors } from '@fastgpt/service/common/middle/cors';
import { ChatRoleEnum, ChatSourceEnum } from '@fastgpt/global/core/chat/constants'; import { ChatRoleEnum, ChatSourceEnum } from '@fastgpt/global/core/chat/constants';
import { sseResponseEventEnum } from '@fastgpt/service/common/response/constant'; import { sseResponseEventEnum } from '@fastgpt/service/common/response/constant';
......
projects/app/src/pages/app/detail/components/Logs.tsx
...@@ -18,7 +18,8 @@ import { useTranslation } from 'next-i18next'; ...@@ -18,7 +18,8 @@ import { useTranslation } from 'next-i18next';
import { usePagination } from '@/web/common/hooks/usePagination'; import { usePagination } from '@/web/common/hooks/usePagination';
import { getAppChatLogs } from '@/web/core/app/api'; import { getAppChatLogs } from '@/web/core/app/api';
import dayjs from 'dayjs'; import dayjs from 'dayjs';
import { ChatSourceMap, HUMAN_ICON } from '@fastgpt/global/core/chat/constants'; import { ChatSourceMap } from '@fastgpt/global/core/chat/constants';
import { HUMAN_ICON } from '@fastgpt/global/common/system/constants';
import { AppLogsListItemType } from '@/types/app'; import { AppLogsListItemType } from '@/types/app';
import { useSystemStore } from '@/web/common/system/useSystemStore'; import { useSystemStore } from '@/web/common/system/useSystemStore';
import ChatBox, { type ComponentRef } from '@/components/ChatBox'; import ChatBox, { type ComponentRef } from '@/components/ChatBox';
...@@ -48,7 +49,8 @@ const Logs = ({ appId }: { appId: string }) => { ...@@ -48,7 +49,8 @@ const Logs = ({ appId }: { appId: string }) => {
data: logs, data: logs,
isLoading, isLoading,
Pagination, Pagination,
getData getData,
pageNum
} = usePagination<AppLogsListItemType>({ } = usePagination<AppLogsListItemType>({
api: getAppChatLogs, api: getAppChatLogs,
pageSize: 20, pageSize: 20,
...@@ -90,8 +92,7 @@ const Logs = ({ appId }: { appId: string }) => { ...@@ -90,8 +92,7 @@ const Logs = ({ appId }: { appId: string }) => {
<Table variant={'simple'} fontSize={'sm'}> <Table variant={'simple'} fontSize={'sm'}>
<Thead> <Thead>
<Tr> <Tr>
<Th>{t('app.Logs Source')}</Th> <Th>{t('core.app.logs.Source And Time')}</Th>
<Th>{t('app.Logs Time')}</Th>
<Th>{t('app.Logs Title')}</Th> <Th>{t('app.Logs Title')}</Th>
<Th>{t('app.Logs Message Total')}</Th> <Th>{t('app.Logs Message Total')}</Th>
<Th>{t('app.Feedback Count')}</Th> <Th>{t('app.Feedback Count')}</Th>
...@@ -107,21 +108,44 @@ const Logs = ({ appId }: { appId: string }) => { ...@@ -107,21 +108,44 @@ const Logs = ({ appId }: { appId: string }) => {
title={'点击查看对话详情'} title={'点击查看对话详情'}
onClick={() => setDetailLogsId(item.id)} onClick={() => setDetailLogsId(item.id)}
> >
<Td>{t(ChatSourceMap[item.source]?.name || 'UnKnow')}</Td> <Td>
<Td>{dayjs(item.time).format('YYYY/MM/DD HH:mm')}</Td> <Box>{t(ChatSourceMap[item.source]?.name || 'UnKnow')}</Box>
<Box color={'myGray.500'}>{dayjs(item.time).format('YYYY/MM/DD HH:mm')}</Box>
</Td>
<Td className="textEllipsis" maxW={'250px'}> <Td className="textEllipsis" maxW={'250px'}>
{item.title} {item.title}
</Td> </Td>
<Td>{item.messageCount}</Td> <Td>{item.messageCount}</Td>
<Td w={'100px'}> <Td w={'100px'}>
{!!item?.feedbackCount ? ( {!!item?.userGoodFeedbackCount && (
<Box display={'inline-block'}> <Flex
mb={item?.userGoodFeedbackCount ? 1 : 0}
bg={'green.100'}
color={'green.600'}
px={3}
py={1}
alignItems={'center'}
justifyContent={'center'}
borderRadius={'lg'}
fontWeight={'bold'}
>
<MyIcon
mr={1}
name={'core/chat/feedback/goodLight'}
color={'green.600'}
w={'14px'}
/>
{item.userGoodFeedbackCount}
</Flex>
)}
{!!item?.userBadFeedbackCount && (
<Flex <Flex
bg={'#FFF2EC'} bg={'#FFF2EC'}
color={'#C96330'} color={'#C96330'}
px={3} px={3}
py={1} py={1}
alignItems={'center'} alignItems={'center'}
justifyContent={'center'}
borderRadius={'lg'} borderRadius={'lg'}
fontWeight={'bold'} fontWeight={'bold'}
> >
...@@ -131,11 +155,8 @@ const Logs = ({ appId }: { appId: string }) => { ...@@ -131,11 +155,8 @@ const Logs = ({ appId }: { appId: string }) => {
color={'#C96330'} color={'#C96330'}
w={'14px'} w={'14px'}
/> />
{item.feedbackCount} {item.userBadFeedbackCount}
</Flex> </Flex>
</Box>
) : (
<>-</>
)} )}
</Td> </Td>
<Td>{item.markCount}</Td> <Td>{item.markCount}</Td>
...@@ -168,7 +189,10 @@ const Logs = ({ appId }: { appId: string }) => { ...@@ -168,7 +189,10 @@ const Logs = ({ appId }: { appId: string }) => {
<DetailLogsModal <DetailLogsModal
appId={appId} appId={appId}
chatId={detailLogsId} chatId={detailLogsId}
onClose={() => setDetailLogsId(undefined)} onClose={() => {
setDetailLogsId(undefined);
getData(pageNum);
}}
/> />
)} )}
<MyModal <MyModal
...@@ -297,6 +321,7 @@ function DetailLogsModal({ ...@@ -297,6 +321,7 @@ function DetailLogsModal({
showMarkIcon showMarkIcon
showVoiceIcon={false} showVoiceIcon={false}
userGuideModule={chat?.app?.userGuideModule} userGuideModule={chat?.app?.userGuideModule}
chatId={chatId}
/> />
</Box> </Box>
</Flex> </Flex>
......
projects/app/src/pages/app/detail/components/OutLink/EmbModal.tsx 0 → 100644
import { OutLinkSchema } from '@fastgpt/global/support/outLink/type';
import React from 'react';
import MyModal from '@/components/MyModal';
const EmbModal = ({ share }: { share: OutLinkSchema }) => {
return <MyModal isOpen>EmbModal</MyModal>;
};
export default EmbModal;
projects/app/src/pages/app/detail/components/OutLink/Share.tsx
...@@ -91,11 +91,10 @@ const Share = ({ appId }: { appId: string }) => { ...@@ -91,11 +91,10 @@ const Share = ({ appId }: { appId: string }) => {
<Thead> <Thead>
<Tr> <Tr>
<Th>名称</Th> <Th>名称</Th>
<Th>金额消耗(￥)</Th> <Th>金额消耗</Th>
<Th>返回详情</Th> <Th>返回详情</Th>
{feConfigs?.isPlus && ( {feConfigs?.isPlus && (
<> <>
<Th>金额限制(￥)</Th>
<Th>IP限流（人/分钟）</Th> <Th>IP限流（人/分钟）</Th>
<Th>过期时间</Th> <Th>过期时间</Th>
<Th>身份校验</Th> <Th>身份校验</Th>
...@@ -109,13 +108,19 @@ const Share = ({ appId }: { appId: string }) => { ...@@ -109,13 +108,19 @@ const Share = ({ appId }: { appId: string }) => {
{shareChatList.map((item) => ( {shareChatList.map((item) => (
<Tr key={item._id}> <Tr key={item._id}>
<Td>{item.name}</Td> <Td>{item.name}</Td>
<Td>{formatPrice(item.total)}</Td> <Td>
{formatPrice(item.total)}
{feConfigs?.isPlus
? `${
item.limit && item.limit.credit > -1
? ` / ${item.limit.credit}元`
: ' / 无限制'
}`
: ''}
</Td>
<Td>{item.responseDetail ? '✔' : '✖'}</Td> <Td>{item.responseDetail ? '✔' : '✖'}</Td>
{feConfigs?.isPlus && ( {feConfigs?.isPlus && (
<> <>
<Td>
{item.limit && item.limit.credit > -1 ? `${item.limit.credit}元` : '无限制'}
</Td>
<Td>{item?.limit?.QPM || '-'}</Td> <Td>{item?.limit?.QPM || '-'}</Td>
<Td> <Td>
{item?.limit?.expiredTime {item?.limit?.expiredTime
......
projects/app/src/pages/chat/index.tsx
...@@ -349,6 +349,7 @@ const Chat = ({ appId, chatId }: { appId: string; chatId: string }) => { ...@@ -349,6 +349,7 @@ const Chat = ({ appId, chatId }: { appId: string; chatId: string }) => {
onUpdateVariable={(e) => {}} onUpdateVariable={(e) => {}}
onStartChat={startChat} onStartChat={startChat}
onDelMessage={(e) => delOneHistoryItem({ ...e, chatId })} onDelMessage={(e) => delOneHistoryItem({ ...e, chatId })}
chatId={chatId}
/> />
</Box> </Box>
</Flex> </Flex>
......
projects/app/src/pages/chat/share.tsx
...@@ -358,6 +358,9 @@ const OutLink = ({ ...@@ -358,6 +358,9 @@ const OutLink = ({
onUpdateVariable={(e) => {}} onUpdateVariable={(e) => {}}
onStartChat={startChat} onStartChat={startChat}
onDelMessage={(e) => delOneHistoryItem({ ...e, chatId, shareId, outLinkUid })} onDelMessage={(e) => delOneHistoryItem({ ...e, chatId, shareId, outLinkUid })}
chatId={chatId}
shareId={shareId}
outLinkUid={outLinkUid}
/> />
</Box> </Box>
</Flex> </Flex>
......
projects/app/src/pages/dataset/detail/components/Import/FileSelect.tsx
...@@ -35,7 +35,7 @@ export type FileItemType = { ...@@ -35,7 +35,7 @@ export type FileItemType = {
id: string; // fileId / raw Link id: string; // fileId / raw Link
filename: string; filename: string;
chunks: PushDatasetDataChunkProps[]; chunks: PushDatasetDataChunkProps[];
text: string; // raw text rawText: string; // raw text
icon: string; icon: string;
tokens: number; // total tokens tokens: number; // total tokens
type: DatasetCollectionTypeEnum.file | DatasetCollectionTypeEnum.link; type: DatasetCollectionTypeEnum.file | DatasetCollectionTypeEnum.link;
...@@ -152,7 +152,7 @@ const FileSelect = ({ ...@@ -152,7 +152,7 @@ const FileSelect = ({
filename: file.name, filename: file.name,
icon, icon,
tokens: filterData.reduce((sum, item) => sum + countPromptTokens(item.q), 0), tokens: filterData.reduce((sum, item) => sum + countPromptTokens(item.q), 0),
text: `${header.join(',')}\n${data rawText: `${header.join(',')}\n${data
.map((item) => `"${item[0]}","${item[1]}"`) .map((item) => `"${item[0]}","${item[1]}"`)
.join('\n')}`, .join('\n')}`,
chunks: filterData, chunks: filterData,
...@@ -192,7 +192,7 @@ const FileSelect = ({ ...@@ -192,7 +192,7 @@ const FileSelect = ({
id: nanoid(), id: nanoid(),
filename: file.name, filename: file.name,
icon, icon,
text, rawText: text,
tokens: splitRes.tokens, tokens: splitRes.tokens,
type: DatasetCollectionTypeEnum.file, type: DatasetCollectionTypeEnum.file,
fileId, fileId,
...@@ -228,7 +228,7 @@ const FileSelect = ({ ...@@ -228,7 +228,7 @@ const FileSelect = ({
id: nanoid(), id: nanoid(),
filename: url, filename: url,
icon: '/imgs/files/link.svg', icon: '/imgs/files/link.svg',
text: content, rawText: content,
tokens: splitRes.tokens, tokens: splitRes.tokens,
type: DatasetCollectionTypeEnum.link, type: DatasetCollectionTypeEnum.link,
rawLink: url, rawLink: url,
...@@ -270,7 +270,7 @@ const FileSelect = ({ ...@@ -270,7 +270,7 @@ const FileSelect = ({
id: nanoid(), id: nanoid(),
filename, filename,
icon: '/imgs/files/txt.svg', icon: '/imgs/files/txt.svg',
text: content, rawText: content,
tokens: splitRes.tokens, tokens: splitRes.tokens,
type: DatasetCollectionTypeEnum.file, type: DatasetCollectionTypeEnum.file,
fileId: fileIds[0], fileId: fileIds[0],
......
projects/app/src/pages/dataset/detail/components/Import/ImportModal.tsx
...@@ -49,7 +49,7 @@ const ImportData = ({ ...@@ -49,7 +49,7 @@ const ImportData = ({
collectionTrainingType: DatasetCollectionTrainingModeEnum.chunk collectionTrainingType: DatasetCollectionTrainingModeEnum.chunk
}, },
[ImportTypeEnum.qa]: { [ImportTypeEnum.qa]: {
defaultChunkLen: agentModel?.maxContext * 0.6 || 8000, defaultChunkLen: agentModel?.maxContext * 0.55 || 8000,
chunkOverlapRatio: 0, chunkOverlapRatio: 0,
unitPrice: agentModel?.price || 3, unitPrice: agentModel?.price || 3,
mode: TrainingModeEnum.qa, mode: TrainingModeEnum.qa,
......
projects/app/src/pages/dataset/detail/components/Import/Provider.tsx
...@@ -13,6 +13,7 @@ import { useRequest } from '@/web/common/hooks/useRequest'; ...@@ -13,6 +13,7 @@ import { useRequest } from '@/web/common/hooks/useRequest';
import { postDatasetCollection } from '@/web/core/dataset/api'; import { postDatasetCollection } from '@/web/core/dataset/api';
import { formatPrice } from '@fastgpt/global/support/wallet/bill/tools'; import { formatPrice } from '@fastgpt/global/support/wallet/bill/tools';
import { splitText2Chunks } from '@fastgpt/global/common/string/textSplitter'; import { splitText2Chunks } from '@fastgpt/global/common/string/textSplitter';
import { hashStr } from '@fastgpt/global/common/string/tools';
import { useToast } from '@/web/common/hooks/useToast'; import { useToast } from '@/web/common/hooks/useToast';
import { getErrText } from '@fastgpt/global/common/error/utils'; import { getErrText } from '@fastgpt/global/common/error/utils';
import { import {
...@@ -158,7 +159,9 @@ const Provider = ({ ...@@ -158,7 +159,9 @@ const Provider = ({
fileId: file.fileId, fileId: file.fileId,
rawLink: file.rawLink, rawLink: file.rawLink,
chunkSize: chunkLen, chunkSize: chunkLen,
trainingType: collectionTrainingType trainingType: collectionTrainingType,
qaPrompt: mode === TrainingModeEnum.qa ? prompt : '',
hashRawText: hashStr(file.rawText)
}); });
// upload data // upload data
...@@ -193,7 +196,7 @@ const Provider = ({ ...@@ -193,7 +196,7 @@ const Provider = ({
setFiles((state) => setFiles((state) =>
state.map((file) => { state.map((file) => {
const splitRes = splitText2Chunks({ const splitRes = splitText2Chunks({
text: file.text, text: file.rawText,
chunkLen, chunkLen,
overlapRatio: chunkOverlapRatio overlapRatio: chunkOverlapRatio
}); });
...@@ -287,7 +290,7 @@ export const PreviewFileOrChunk = () => { ...@@ -287,7 +290,7 @@ export const PreviewFileOrChunk = () => {
px={[4, 8]} px={[4, 8]}
my={4} my={4}
contentEditable contentEditable
dangerouslySetInnerHTML={{ __html: previewFile.text }} dangerouslySetInnerHTML={{ __html: previewFile.rawText }}
fontSize={'sm'} fontSize={'sm'}
whiteSpace={'pre-wrap'} whiteSpace={'pre-wrap'}
wordBreak={'break-all'} wordBreak={'break-all'}
......
projects/app/src/pages/dataset/detail/components/Import/UrlFetchModal.tsx
import React from 'react'; import React from 'react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import MyModal from '@/components/MyModal'; import MyModal from '@/components/MyModal';
import { Box, Button, Input, ModalBody, ModalFooter, Textarea } from '@chakra-ui/react'; import { Box, Button, Input, Link, ModalBody, ModalFooter, Textarea } from '@chakra-ui/react';
import { useRequest } from '@/web/common/hooks/useRequest'; import { useRequest } from '@/web/common/hooks/useRequest';
import { postFetchUrls } from '@/web/common/tools/api'; import { postFetchUrls } from '@/web/common/tools/api';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import { UrlFetchResponse } from '@fastgpt/global/common/file/api.d'; import { UrlFetchResponse } from '@fastgpt/global/common/file/api.d';
import { getDocPath } from '@/web/common/system/doc';
import { feConfigs } from '@/web/common/system/staticData';
const UrlFetchModal = ({ const UrlFetchModal = ({
onClose, onClose,
...@@ -68,7 +70,12 @@ const UrlFetchModal = ({ ...@@ -68,7 +70,12 @@ const UrlFetchModal = ({
<Box mt={4}> <Box mt={4}>
<Box fontWeight={'bold'}> <Box fontWeight={'bold'}>
{t('core.dataset.website.Selector')}({t('common.choosable')}) {t('core.dataset.website.Selector')}({t('common.choosable')})
</Box>{' '} </Box>
{feConfigs?.docUrl && (
<Link href={getDocPath('/docs/course/websync/#选择器如何使用')} target="_blank">
{t('core.dataset.website.Selector Course')}
</Link>
)}
<Input {...register('selector')} placeholder="body .content #document" /> <Input {...register('selector')} placeholder="body .content #document" />
</Box> </Box>
</ModalBody> </ModalBody>
......
projects/app/src/pages/dataset/detail/components/Import/WebsiteConfig.tsx
import React from 'react'; import React from 'react';
import MyModal from '@/components/MyModal'; import MyModal from '@/components/MyModal';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { Box, Button, Input, ModalBody, ModalFooter } from '@chakra-ui/react'; import { Box, Button, Input, Link, ModalBody, ModalFooter } from '@chakra-ui/react';
import { strIsLink } from '@fastgpt/global/common/string/tools'; import { strIsLink } from '@fastgpt/global/common/string/tools';
import { useToast } from '@/web/common/hooks/useToast'; import { useToast } from '@/web/common/hooks/useToast';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import { useConfirm } from '@/web/common/hooks/useConfirm'; import { useConfirm } from '@/web/common/hooks/useConfirm';
import { getDocPath } from '@/web/common/system/doc';
import { feConfigs } from '@/web/common/system/staticData';
type FormType = { type FormType = {
url?: string | undefined; url?: string | undefined;
...@@ -49,6 +51,16 @@ const WebsiteConfigModal = ({ ...@@ -49,6 +51,16 @@ const WebsiteConfigModal = ({
<ModalBody> <ModalBody>
<Box fontSize={'sm'} color={'myGray.600'}> <Box fontSize={'sm'} color={'myGray.600'}>
{t('core.dataset.website.Config Description')} {t('core.dataset.website.Config Description')}
{feConfigs?.docUrl && (
<Link
href={getDocPath('/docs/course/websync')}
target="_blank"
textDecoration={'underline'}
fontWeight={'bold'}
>
{t('common.course.Read Course')}
</Link>
)}
</Box> </Box>
<Box mt={2}> <Box mt={2}>
<Box>{t('core.dataset.website.Base Url')}</Box> <Box>{t('core.dataset.website.Base Url')}</Box>
......
projects/app/src/service/core/dataset/data/pg.ts
...@@ -13,15 +13,7 @@ import { jiebaSplit } from '../utils'; ...@@ -13,15 +13,7 @@ import { jiebaSplit } from '../utils';
import { reRankRecall } from '../../ai/rerank'; import { reRankRecall } from '../../ai/rerank';
import { countPromptTokens } from '@fastgpt/global/common/string/tiktoken'; import { countPromptTokens } from '@fastgpt/global/common/string/tiktoken';
export async function insertData2Pg({ export async function insertData2Pg(props: {
mongoDataId,
input,
model,
teamId,
tmbId,
datasetId,
collectionId
}: {
mongoDataId: string; mongoDataId: string;
input: string; input: string;
model: string; model: string;
...@@ -29,9 +21,9 @@ export async function insertData2Pg({ ...@@ -29,9 +21,9 @@ export async function insertData2Pg({
tmbId: string; tmbId: string;
datasetId: string; datasetId: string;
collectionId: string; collectionId: string;
}) { retry?: number;
let retry = 2; }): Promise<{ insertId: string; vectors: number[][]; tokenLen: number }> {
async function insertPg(): Promise<{ insertId: string; vectors: number[][]; tokenLen: number }> { const { mongoDataId, input, model, teamId, tmbId, datasetId, collectionId, retry = 3 } = props;
try { try {
// get vector // get vector
const { vectors, tokenLen } = await getVectorsByText({ const { vectors, tokenLen } = await getVectorsByText({
...@@ -56,15 +48,15 @@ export async function insertData2Pg({ ...@@ -56,15 +48,15 @@ export async function insertData2Pg({
tokenLen tokenLen
}; };
} catch (error) { } catch (error) {
if (--retry < 0) { if (retry <= 0) {
return Promise.reject(error); return Promise.reject(error);
} }
await delay(500); await delay(500);
return insertPg(); return insertData2Pg({
} ...props,
retry: retry - 1
});
} }
return insertPg();
} }
export async function updatePgDataById({ export async function updatePgDataById({
...@@ -128,8 +120,9 @@ export async function searchDatasetData(props: SearchProps) { ...@@ -128,8 +120,9 @@ export async function searchDatasetData(props: SearchProps) {
} }
const rerank = const rerank =
searchMode === DatasetSearchModeEnum.embeddingReRank || global.reRankModels?.[0] &&
searchMode === DatasetSearchModeEnum.embFullTextReRank; (searchMode === DatasetSearchModeEnum.embeddingReRank ||
searchMode === DatasetSearchModeEnum.embFullTextReRank);
const oneChunkToken = 50; const oneChunkToken = 50;
const { embeddingLimit, fullTextLimit } = (() => { const { embeddingLimit, fullTextLimit } = (() => {
...@@ -188,8 +181,6 @@ export async function searchDatasetData(props: SearchProps) { ...@@ -188,8 +181,6 @@ export async function searchDatasetData(props: SearchProps) {
return true; return true;
}); });
// token slice
if (!rerank) { if (!rerank) {
return { return {
searchRes: filterResultsByMaxTokens( searchRes: filterResultsByMaxTokens(
...@@ -264,7 +255,7 @@ export async function embeddingRecall({ ...@@ -264,7 +255,7 @@ export async function embeddingRecall({
{ {
_id: { $in: filterRows.map((item) => item.data_id?.trim()) } _id: { $in: filterRows.map((item) => item.data_id?.trim()) }
}, },
'datasetId collectionId q a indexes' 'datasetId collectionId q a chunkIndex indexes'
).lean() ).lean()
]); ]);
const formatResult = filterRows const formatResult = filterRows
...@@ -281,6 +272,7 @@ export async function embeddingRecall({ ...@@ -281,6 +272,7 @@ export async function embeddingRecall({
id: String(data._id), id: String(data._id),
q: data.q, q: data.q,
a: data.a, a: data.a,
chunkIndex: data.chunkIndex,
indexes: data.indexes, indexes: data.indexes,
datasetId: String(data.datasetId), datasetId: String(data.datasetId),
collectionId: String(data.collectionId), collectionId: String(data.collectionId),
...@@ -322,7 +314,8 @@ export async function fullTextRecall({ text, limit, datasetIds = [] }: SearchPro ...@@ -322,7 +314,8 @@ export async function fullTextRecall({ text, limit, datasetIds = [] }: SearchPro
collectionId: 1, collectionId: 1,
q: 1, q: 1,
a: 1, a: 1,
indexes: 1 indexes: 1,
chunkIndex: 1
} }
) )
.sort({ score: { $meta: 'textScore' } }) .sort({ score: { $meta: 'textScore' } })
...@@ -354,6 +347,7 @@ export async function fullTextRecall({ text, limit, datasetIds = [] }: SearchPro ...@@ -354,6 +347,7 @@ export async function fullTextRecall({ text, limit, datasetIds = [] }: SearchPro
sourceId: collection?.fileId || collection?.rawLink, sourceId: collection?.fileId || collection?.rawLink,
q: item.q, q: item.q,
a: item.a, a: item.a,
chunkIndex: item.chunkIndex,
indexes: item.indexes, indexes: item.indexes,
// @ts-ignore // @ts-ignore
score: item.score score: item.score
...@@ -395,8 +389,6 @@ export async function reRankSearchResult({ ...@@ -395,8 +389,6 @@ export async function reRankSearchResult({
return mergeResult; return mergeResult;
} catch (error) { } catch (error) {
console.log(error);
return data; return data;
} }
} }
......
projects/app/src/service/events/generateQA.ts
...@@ -4,7 +4,7 @@ import { DatasetDataIndexTypeEnum, TrainingModeEnum } from '@fastgpt/global/core ...@@ -4,7 +4,7 @@ import { DatasetDataIndexTypeEnum, TrainingModeEnum } from '@fastgpt/global/core
import { sendOneInform } from '../support/user/inform/api'; import { sendOneInform } from '../support/user/inform/api';
import { getAIApi } from '@fastgpt/service/core/ai/config'; import { getAIApi } from '@fastgpt/service/core/ai/config';
import type { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d'; import type { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import { splitText2Chunks } from '@fastgpt/global/common/string/textSplitter'; import { splitText2Chunks } from '@fastgpt/global/common/string/textSplitter';
import { replaceVariable } from '@fastgpt/global/common/string/tools'; import { replaceVariable } from '@fastgpt/global/common/string/tools';
import { Prompt_AgentQA } from '@/global/core/prompt/agent'; import { Prompt_AgentQA } from '@/global/core/prompt/agent';
...@@ -56,6 +56,7 @@ export async function generateQA(): Promise<any> { ...@@ -56,6 +56,7 @@ export async function generateQA(): Promise<any> {
collectionId: 1, collectionId: 1,
q: 1, q: 1,
model: 1, model: 1,
chunkIndex: 1,
billId: 1, billId: 1,
prompt: 1 prompt: 1
}) })
...@@ -130,7 +131,7 @@ ${replaceVariable(Prompt_AgentQA.fixedText, { text })}`; ...@@ -130,7 +131,7 @@ ${replaceVariable(Prompt_AgentQA.fixedText, { text })}`;
const ai = getAIApi(undefined, 600000); const ai = getAIApi(undefined, 600000);
const chatResponse = await ai.chat.completions.create({ const chatResponse = await ai.chat.completions.create({
model, model,
temperature: 0.01, temperature: 0.3,
messages, messages,
stream: false stream: false
}); });
...@@ -144,7 +145,10 @@ ${replaceVariable(Prompt_AgentQA.fixedText, { text })}`; ...@@ -144,7 +145,10 @@ ${replaceVariable(Prompt_AgentQA.fixedText, { text })}`;
teamId: data.teamId, teamId: data.teamId,
tmbId: data.tmbId, tmbId: data.tmbId,
collectionId: data.collectionId, collectionId: data.collectionId,
data: qaArr, data: qaArr.map((item) => ({
...item,
chunkIndex: data.chunkIndex
})),
mode: TrainingModeEnum.chunk, mode: TrainingModeEnum.chunk,
billId: data.billId billId: data.billId
}); });
......
projects/app/src/service/events/generateVector.ts
...@@ -2,7 +2,7 @@ import { insertData2Dataset } from '@/service/core/dataset/data/controller'; ...@@ -2,7 +2,7 @@ import { insertData2Dataset } from '@/service/core/dataset/data/controller';
import { MongoDatasetTraining } from '@fastgpt/service/core/dataset/training/schema'; import { MongoDatasetTraining } from '@fastgpt/service/core/dataset/training/schema';
import { TrainingModeEnum } from '@fastgpt/global/core/dataset/constant'; import { TrainingModeEnum } from '@fastgpt/global/core/dataset/constant';
import { sendOneInform } from '../support/user/inform/api'; import { sendOneInform } from '../support/user/inform/api';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import { getErrText } from '@fastgpt/global/common/error/utils'; import { getErrText } from '@fastgpt/global/common/error/utils';
import { authTeamBalance } from '@/service/support/permission/auth/bill'; import { authTeamBalance } from '@/service/support/permission/auth/bill';
import { pushGenerateVectorBill } from '@/service/support/wallet/bill/push'; import { pushGenerateVectorBill } from '@/service/support/wallet/bill/push';
......
projects/app/src/service/moduleDispatch/chat/oneapi.ts
...@@ -223,6 +223,7 @@ function filterQuote({ ...@@ -223,6 +223,7 @@ function filterQuote({
score: item.score?.toFixed(4) score: item.score?.toFixed(4)
}); });
} }
const sliceResult = sliceMessagesTB({ const sliceResult = sliceMessagesTB({
maxTokens: model.quoteMaxToken, maxTokens: model.quoteMaxToken,
messages: quoteQA.map((item, index) => ({ messages: quoteQA.map((item, index) => ({
...@@ -234,13 +235,30 @@ function filterQuote({ ...@@ -234,13 +235,30 @@ function filterQuote({
// slice filterSearch // slice filterSearch
const filterQuoteQA = quoteQA.slice(0, sliceResult.length); const filterQuoteQA = quoteQA.slice(0, sliceResult.length);
// filterQuoteQA按collectionId聚合在一起后，再按chunkIndex从小到大排序
const sortQuoteQAMap: Record<string, SearchDataResponseItemType[]> = {};
filterQuoteQA.forEach((item) => {
if (sortQuoteQAMap[item.collectionId]) {
sortQuoteQAMap[item.collectionId].push(item);
} else {
sortQuoteQAMap[item.collectionId] = [item];
}
});
const sortQuoteQAList = Object.values(sortQuoteQAMap).flat();
sortQuoteQAList.sort((a, b) => {
if (a.collectionId === b.collectionId) {
return a.chunkIndex - b.chunkIndex;
}
return 0;
});
const quoteText = const quoteText =
filterQuoteQA.length > 0 filterQuoteQA.length > 0
? `${filterQuoteQA.map((item, index) => getValue(item, index)).join('\n')}` ? `${filterQuoteQA.map((item, index) => getValue(item, index)).join('\n')}`
: ''; : '';
return { return {
filterQuoteQA, filterQuoteQA: sortQuoteQAList,
quoteText quoteText
}; };
} }
......
projects/app/src/service/support/permission/auth/chat.ts
...@@ -45,9 +45,16 @@ export async function autChatCrud({ ...@@ -45,9 +45,16 @@ export async function autChatCrud({
} }
// req auth // req auth
const { tmbId, role } = await authUserRole(props); const { teamId, tmbId, role } = await authUserRole(props);
if (String(teamId) !== String(chat.teamId)) return Promise.reject(ChatErrEnum.unAuthChat);
if (role === TeamMemberRoleEnum.owner) return { uid: outLinkUid }; if (role === TeamMemberRoleEnum.owner) return { uid: outLinkUid };
if (String(tmbId) === String(chat.tmbId)) return { uid: outLinkUid }; if (String(tmbId) === String(chat.tmbId)) return { uid: outLinkUid };
// admin
if (per === 'r' && role === TeamMemberRoleEnum.admin) return { uid: outLinkUid };
return Promise.reject(ChatErrEnum.unAuthChat); return Promise.reject(ChatErrEnum.unAuthChat);
})(); })();
......
projects/app/src/service/support/permission/auth/dataset.ts
import { DatasetDataItemType, DatasetDataSchemaType } from '@fastgpt/global/core/dataset/type'; import { DatasetDataItemType } from '@fastgpt/global/core/dataset/type';
import { AuthResponseType } from '@fastgpt/global/support/permission/type';
import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema'; import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema';
import { authDatasetCollection } from '@fastgpt/service/support/permission/auth/dataset'; import { authDatasetCollection } from '@fastgpt/service/support/permission/auth/dataset';
import { AuthModeType } from '@fastgpt/service/support/permission/type'; import { AuthModeType } from '@fastgpt/service/support/permission/type';
...@@ -27,6 +26,7 @@ export async function authDatasetData({ ...@@ -27,6 +26,7 @@ export async function authDatasetData({
id: String(datasetData._id), id: String(datasetData._id),
q: datasetData.q, q: datasetData.q,
a: datasetData.a, a: datasetData.a,
chunkIndex: datasetData.chunkIndex,
indexes: datasetData.indexes, indexes: datasetData.indexes,
datasetId: String(datasetData.datasetId), datasetId: String(datasetData.datasetId),
collectionId: String(datasetData.collectionId), collectionId: String(datasetData.collectionId),
......
projects/app/src/service/support/wallet/bill/push.ts
...@@ -2,7 +2,7 @@ import { BillSourceEnum, PRICE_SCALE } from '@fastgpt/global/support/wallet/bill ...@@ -2,7 +2,7 @@ import { BillSourceEnum, PRICE_SCALE } from '@fastgpt/global/support/wallet/bill
import { getAudioSpeechModel, getQAModel } from '@/service/core/ai/model'; import { getAudioSpeechModel, getQAModel } from '@/service/core/ai/model';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type.d'; import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type.d';
import { formatPrice } from '@fastgpt/global/support/wallet/bill/tools'; import { formatPrice } from '@fastgpt/global/support/wallet/bill/tools';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import type { ConcatBillProps, CreateBillProps } from '@fastgpt/global/support/wallet/bill/api.d'; import type { ConcatBillProps, CreateBillProps } from '@fastgpt/global/support/wallet/bill/api.d';
import { defaultQGModels } from '@fastgpt/global/core/ai/model'; import { defaultQGModels } from '@fastgpt/global/core/ai/model';
import { POST } from '@fastgpt/service/common/api/plusRequest'; import { POST } from '@fastgpt/service/common/api/plusRequest';
......
projects/app/src/service/utils/chat/saveChat.ts
...@@ -3,7 +3,7 @@ import { MongoApp } from '@fastgpt/service/core/app/schema'; ...@@ -3,7 +3,7 @@ import { MongoApp } from '@fastgpt/service/core/app/schema';
import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants'; import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants';
import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema'; import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema'; import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import { addLog } from '@fastgpt/service/common/mongo/controller'; import { addLog } from '@fastgpt/service/common/system/log';
import { chatContentReplaceBlock } from '@fastgpt/global/core/chat/utils'; import { chatContentReplaceBlock } from '@fastgpt/global/core/chat/utils';
type Props = { type Props = {
......
projects/app/src/types/app.d.ts
...@@ -59,6 +59,8 @@ export type AppLogsListItemType = { ...@@ -59,6 +59,8 @@ export type AppLogsListItemType = {
time: Date; time: Date;
title: string; title: string;
messageCount: number; messageCount: number;
feedbackCount: number; userGoodFeedbackCount: number;
userBadFeedbackCount: number;
robotBadFeedbackCount: number;
markCount: number; markCount: number;
}; };
projects/app/src/web/common/api/request.ts
...@@ -99,10 +99,14 @@ function responseError(err: any) { ...@@ -99,10 +99,14 @@ function responseError(err: any) {
// 有报错响应 // 有报错响应
if (err?.code in TOKEN_ERROR_CODE) { if (err?.code in TOKEN_ERROR_CODE) {
clearToken(); clearToken();
if (window.location.pathname !== '/chat/share') {
window.location.replace( window.location.replace(
`/login?lastRoute=${encodeURIComponent(location.pathname + location.search)}` `/login?lastRoute=${encodeURIComponent(location.pathname + location.search)}`
); );
return Promise.reject({ message: 'token过期，重新登录' }); }
return Promise.reject({ message: '无权操作' });
} }
if (err?.response?.data) { if (err?.response?.data) {
return Promise.reject(err?.response?.data); return Promise.reject(err?.response?.data);
......
projects/app/src/web/core/chat/api.ts
...@@ -13,6 +13,7 @@ import type { ...@@ -13,6 +13,7 @@ import type {
DeleteChatItemProps, DeleteChatItemProps,
UpdateHistoryProps UpdateHistoryProps
} from '@/global/core/chat/api.d'; } from '@/global/core/chat/api.d';
import { UpdateChatFeedbackProps } from '@fastgpt/global/core/chat/api';
/** /**
* 获取初始化聊天内容 * 获取初始化聊天内容
...@@ -49,8 +50,8 @@ export const delChatRecordById = (data: DeleteChatItemProps) => ...@@ -49,8 +50,8 @@ export const delChatRecordById = (data: DeleteChatItemProps) =>
*/ */
export const putChatHistory = (data: UpdateHistoryProps) => PUT('/core/chat/updateHistory', data); export const putChatHistory = (data: UpdateHistoryProps) => PUT('/core/chat/updateHistory', data);
export const userUpdateChatFeedback = (data: { chatItemId: string; userFeedback?: string }) => export const updateChatUserFeedback = (data: UpdateChatFeedbackProps) =>
POST('/core/chat/feedback/userUpdate', data); POST('/core/chat/feedback/updateUserFeedback', data);
export const adminUpdateChatFeedback = (data: AdminUpdateFeedbackParams) => export const updateChatAdminFeedback = (data: AdminUpdateFeedbackParams) =>
POST('/core/chat/feedback/adminUpdate', data); POST('/core/chat/feedback/adminUpdate', data);
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment