Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit aab6ee51 by Archer Committed by GitHub
Options

V4.6.7-production (#759)

parent 91b7d81c
Showing with 1622 additions and 1958 deletions
README.md
...@@ -57,7 +57,7 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b ...@@ -57,7 +57,7 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
- [x] 源文件引用追踪 - [x] 源文件引用追踪
- [x] 模块封装，实现多级复用 - [x] 模块封装，实现多级复用
- [x] 混合检索 & 重排 - [x] 混合检索 & 重排
- [ ] 自查询规划 - [ ] Tool 模块
- [ ] 嵌入 [Laf](https://github.com/labring/laf)，实现在线编写 HTTP 模块 - [ ] 嵌入 [Laf](https://github.com/labring/laf)，实现在线编写 HTTP 模块
- [ ] 插件封装功能 - [ ] 插件封装功能
...@@ -67,10 +67,10 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b ...@@ -67,10 +67,10 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
- [x] 支持知识库单独设置向量模型 - [x] 支持知识库单独设置向量模型
- [x] 源文件存储 - [x] 源文件存储
- [x] 支持手动输入，直接分段，QA 拆分导入 - [x] 支持手动输入，直接分段，QA 拆分导入
- [x] 支持 pdf、word、txt、md 等常用文件，支持 url 读取、CSV 批量导入 - [x] 支持 pdf，docx，txt，html，md，csv
- [ ] 支持 HTML、csv、PPT、Excel 导入 - [x] 支持 url 读取、CSV 批量导入
- [ ] 支持 PPT、Excel 导入
- [ ] 支持文件阅读器 - [ ] 支持文件阅读器
- [ ] 支持差异性文件同步
- [ ] 更多的数据预处理方案 - [ ] 更多的数据预处理方案
`3` 应用调试能力 `3` 应用调试能力
...@@ -81,8 +81,8 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b ...@@ -81,8 +81,8 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
- [ ] 高级编排 DeBug 模式 - [ ] 高级编排 DeBug 模式
`4` OpenAPI 接口 `4` OpenAPI 接口
- [x] completions 接口 (对齐 GPT 接口) - [x] completions 接口 (chat 模式对齐 GPT 接口)
- [ ] 知识库 CRUD - [x] 知识库 CRUD
- [ ] 对话 CRUD - [ ] 对话 CRUD
`5` 运营能力 `5` 运营能力
......
README_en.md
...@@ -49,48 +49,52 @@ Cloud: [fastgpt.in](https://fastgpt.in/) ...@@ -49,48 +49,52 @@ Cloud: [fastgpt.in](https://fastgpt.in/)
## 💡 Features ## 💡 Features
1. Powerful visual workflows: Effortlessly craft AI applications `1` Application Orchestration Features
- [x] Simple mode on deck - no need for manual arrangement - [x] Offers a straightforward mode, eliminating the need for complex orchestration
- [x] User dialogue pre-guidance - [x] Provides clear next-step instructions in dialogues
- [x] Global variables - [x] Facilitates workflow orchestration
- [x] Knowledge base search - [x] Tracks references in source files
- [x] Dialogue via multiple LLM models - [x] Encapsulates modules for enhanced reuse at multiple levels
- [x] Text magic - convert to structured data - [x] Combines search and reordering functions
- [x] Extend with HTTP - [ ] Includes a tool module
- [ ] Embed Laf for on-the-fly HTTP module crafting - [ ] Integrates [Laf](https://github.com/labring/laf) for online HTTP module creation
- [x] Directions for the next dialogue steps - [ ] Plugin encapsulation capabilities
- [x] Tracking source file references
- [ ] Custom file reader `2` Knowledge Base Features
- [ ] Modules are packaged into plug-ins to achieve reuse
- [x] Allows for the mixed use of multiple databases
2. Extensive knowledge base preprocessing - [x] Keeps track of modifications and deletions in data chunks
- [x] Enables specific vector models for each knowledge base
- [x] Reuse and mix multiple knowledge bases - [x] Stores original source files
- [x] Track chunk modifications and deletions - [x] Supports direct input and segment-based QA import
- [x] Supports manual entries, direct segmentation, and QA split imports - [x] Compatible with a variety of file formats: pdf, docx, txt, html, md, csv
- [x] Supports URL fetching and batch CSV imports - [x] Facilitates URL reading and bulk CSV importing
- [x] Supports Set unique vector models for knowledge bases - [ ] Supports PPT and Excel file import
- [x] Store original files - [ ] Features a file reader
- [ ] File learning Agent - [ ] Offers diverse data preprocessing options
3. Multiple effect testing channels `3` Application Debugging Features
- [x] Single-point knowledge base search test - [x] Enables targeted search testing within the knowledge base
- [x] Feedback references and ability to modify and delete during dialogue - [x] Allows feedback, editing, and deletion during conversations
- [x] Complete context presentation - [x] Presents the full context of interactions
- [ ] Complete module intermediate value presentation - [x] Displays all intermediate values within modules
- [ ] Advanced DeBug mode for orchestration
4. OpenAPI
`4` OpenAPI Interface
- [x] completions interface (aligned with GPT interface)
- [ ] Knowledge base CRUD - [x] The completions interface (aligned with GPT's chat mode interface)
- [x] CRUD operations for the knowledge base
5. Operational functions - [ ] CRUD operations for conversations
- [x] Login-free sharing window `5` Operational Features
- [x] One-click embedding with Iframe
- [ ] Unified access to dialogue records - [x] Share without requiring login
- [x] Easy embedding with Iframe
- [x] Customizable chat window embedding with features like default open, drag-and-drop
- [x] Centralizes conversation records for review and annotation
<a href="#readme"> <a href="#readme">
<img src="https://img.shields.io/badge/-Back_to_Top-7d09f1.svg" alt="#" align="right"> <img src="https://img.shields.io/badge/-Back_to_Top-7d09f1.svg" alt="#" align="right">
......
docSite/content/docs/development/openapi/chat.md
...@@ -48,7 +48,7 @@ curl --location --request POST 'https://api.fastgpt.in/api/v1/chat/completions' ...@@ -48,7 +48,7 @@ curl --location --request POST 'https://api.fastgpt.in/api/v1/chat/completions'
{{< /markdownify >}} {{< /markdownify >}}
{{< /tab >}} {{< /tab >}}
{{< tab tabName="detail=true 响应" >}} {{< tab tabName="参数说明" >}}
{{< markdownify >}} {{< markdownify >}}
{{% alert context="info" %}} {{% alert context="info" %}}
...@@ -56,7 +56,7 @@ curl --location --request POST 'https://api.fastgpt.in/api/v1/chat/completions' ...@@ -56,7 +56,7 @@ curl --location --request POST 'https://api.fastgpt.in/api/v1/chat/completions'
- chatId: string | undefined 。 - chatId: string | undefined 。
- 为 `undefined` 时（不传入），不使用 FastGpt 提供的上下文功能，完全通过传入的 messages 构建上下文。 不会将你的记录存储到数据库中，你也无法在记录汇总中查阅到。 - 为 `undefined` 时（不传入），不使用 FastGpt 提供的上下文功能，完全通过传入的 messages 构建上下文。 不会将你的记录存储到数据库中，你也无法在记录汇总中查阅到。
- 为`非空字符串`时，意味着使用 chatId 进行对话，自动从 FastGpt 数据库取历史记录，并使用 messages 数组最后一个内容作为用户问题。请自行确保 chatId 唯一，长度小于250，通常可以是自己系统的对话框ID。 - 为`非空字符串`时，意味着使用 chatId 进行对话，自动从 FastGpt 数据库取历史记录，并使用 messages 数组最后一个内容作为用户问题。请自行确保 chatId 唯一，长度小于250，通常可以是自己系统的对话框ID。
- messages: 结构与 [GPT接口](https://platform.openai.com/docs/api-reference/chat/object) 完全一致。 - messages: 结构与 [GPT接口](https://platform.openai.com/docs/api-reference/chat/object) chat模式一致。
- detail: 是否返回中间值（模块状态，响应的完整结果等），`stream模式`下会通过`event`进行区分，`非stream模式`结果保存在`responseData`中。 - detail: 是否返回中间值（模块状态，响应的完整结果等），`stream模式`下会通过`event`进行区分，`非stream模式`结果保存在`responseData`中。
- variables: 模块变量，一个对象，会替换模块中，输入框内容里的`{{key}}` - variables: 模块变量，一个对象，会替换模块中，输入框内容里的`{{key}}`
{{% /alert %}} {{% /alert %}}
......
docSite/content/docs/development/openapi/dataset.md
...@@ -15,7 +15,9 @@ weight: 853 ...@@ -15,7 +15,9 @@ weight: 853
## 创建训练订单 ## 创建训练订单
**请求示例** {{< tabs tabTotal="2" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```bash ```bash
curl --location --request POST 'https://api.fastgpt.in/api/support/wallet/bill/createTrainingBill' \ curl --location --request POST 'https://api.fastgpt.in/api/support/wallet/bill/createTrainingBill' \
...@@ -26,7 +28,11 @@ curl --location --request POST 'https://api.fastgpt.in/api/support/wallet/bill/c ...@@ -26,7 +28,11 @@ curl --location --request POST 'https://api.fastgpt.in/api/support/wallet/bill/c
}' }'
``` ```
**响应结果** {{< /markdownify >}}
{{< /tab >}}
{{< tab tabName="响应示例" >}}
{{< markdownify >}}
data 为 billId，可用于添加知识库数据时进行账单聚合。 data 为 billId，可用于添加知识库数据时进行账单聚合。
...@@ -39,35 +45,30 @@ data 为 billId，可用于添加知识库数据时进行账单聚合。 ...@@ -39,35 +45,30 @@ data 为 billId，可用于添加知识库数据时进行账单聚合。
} }
``` ```
## 知识库添加数据 {{< /markdownify >}}
{{< /tab >}}
{{< /tabs >}}
## 知识库
{{< tabs tabTotal="4" >}} ### 创建一个知识库
{{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}} {{< tab tabName="请求示例" >}}
{{< markdownify >}} {{< markdownify >}}
```bash ```bash
curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/data/pushData' \ curl --location --request POST 'http://localhost:3000/api/core/dataset/create' \
--header 'Authorization: Bearer apikey' \ --header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \ --header 'Content-Type: application/json' \
--data-raw '{ --data-raw '{
    "collectionId": "64663f451ba1676dbdef0499", "parentId": null,
"trainingMode": "chunk", "type": "dataset",
"prompt": "可选。qa 拆分引导词，chunk 模式下忽略", "name":"测试",
"billId": "可选。如果有这个值，本次的数据会被聚合到一个订单中，这个值可以重复使用。可以参考 [创建训练订单] 获取该值。", "intro":"介绍",
    "data": [ "avatar": "",
{ "vectorModel": "text-embedding-ada-002",
"q": "你是谁？", "agentModel": "gpt-3.5-turbo-16k"
"a": "我是FastGPT助手"
},
{
"q": "你会什么？",
"a": "我什么都会",
"indexes": [{
"type":"custom",
"text":"你好"
}]
}
]
}' }'
``` ```
...@@ -77,101 +78,229 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/data/pus ...@@ -77,101 +78,229 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/data/pus
{{< tab tabName="参数说明" >}} {{< tab tabName="参数说明" >}}
{{< markdownify >}} {{< markdownify >}}
需要先了解 FastGPT 的多路索引概念： {{% alert icon=" " context="success" %}}
- parentId - 父级ID，用于构建目录结构。通常可以为 null 或者直接不传。
- type - `dataset`或者`folder`，代表普通知识库和文件夹。不传则代表创建普通知识库。
- name - 知识库名（必填）
- intro - 介绍（可选）
- avatar - 头像地址（可选）
- vectorModel - 向量模型（建议传空，用系统默认的）
- agentModel - 文本处理模型（建议传空，用系统默认的）
{{% /alert %}}
{{< /markdownify >}}
{{< /tab >}}
{{< tab tabName="响应示例" >}}
{{< markdownify >}}
```json
{
"code": 200,
"statusText": "",
"message": "",
"data": "65abc9bd9d1448617cba5e6c"
}
```
{{< /markdownify >}}
{{< /tab >}}
{{< /tabs >}}
### 获取知识库列表
{{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```bash
curl --location --request GET 'http://localhost:3000/api/core/dataset/list?parentId=' \
--header 'Authorization: Bearer {{authorization}}' \
```
{{< /markdownify >}}
{{< /tab >}}
{{< tab tabName="参数说明" >}}
{{< markdownify >}}
{{% alert icon=" " context="success" %}}
- parentId - 父级ID，不传或为空，代表获取根目录下的知识库
{{% /alert %}}
在 FastGPT 中，你可以为一组数据创建多个索引，如果不指定索引，则系统会自动取对应的 chunk 作为索引。例如前面的请求示例中： {{< /markdownify >}}
{{< /tab >}}
`q：你是谁？a:我是FastGPT助手` 它的`indexes`属性为空，意味着不自定义索引，而是使用默认的索引（你是谁？\n我是FastGPT助手）。 {{< tab tabName="响应示例" >}}
{{< markdownify >}}
在第二组数据中`q:你会什么？a:我什么都会`指定了一个`你好`的索引，因此这组数据的索引为`你好`。
```json ```json
{ {
"collectionId": "文件集合的ID，参考上面的第二张图", "code": 200,
"mode": "chunk | qa ", // chunk 模式: 可自定义索引。qa 模型：无法自定义索引，会自动取 data 中的 q 作为数据，让模型自动生成问答对和索引。 "statusText": "",
"prompt": "QA 拆分提示词，需严格按照模板，建议不要传入。", "message": "",
"data": [ "data": [
{ {
"q": "生成索引的内容，index 模式下最大 tokens 为3000，建议不超过 1000", "_id": "65abc9bd9d1448617cba5e6c",
"a": "预期回答/补充", "parentId": null,
"indexes": "自定义索引", "avatar": "",
}, "name": "测试",
{ "intro": "",
"q": "xxx", "type": "dataset",
"a": "xxxx" "permission": "private",
"canWrite": true,
"isOwner": true,
"vectorModel": {
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"inputPrice": 0,
"defaultToken": 512,
"maxToken": 8000,
"weight": 100
}
} }
], ]
} }
``` ```
{{< /markdownify >}} {{< /markdownify >}}
{{< /tab >}} {{< /tab >}}
{{< /tabs >}}
{{< tab tabName="响应例子" >}} ### 获取知识库详情
{{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```bash
curl --location --request GET 'http://localhost:3000/api/core/dataset/detail?id=6593e137231a2be9c5603ba7' \
--header 'Authorization: Bearer {{authorization}}' \
```
{{< /markdownify >}}
{{< /tab >}}
{{< tab tabName="参数说明" >}}
{{< markdownify >}}
{{% alert icon=" " context="success" %}}
- id: 知识库的ID
{{% /alert %}}
{{< /markdownify >}}
{{< /tab >}}
{{< tab tabName="响应示例" >}}
{{< markdownify >}} {{< markdownify >}}
```json ```json
{ {
"code": 200, "code": 200,
"statusText": "", "statusText": "",
"message": "",
"data": { "data": {
"insertLen": 1, // 最终插入成功的数量 "_id": "6593e137231a2be9c5603ba7",
"overToken": [], // 超出 token 的 "parentId": null,
"teamId": "65422be6aa44b7da77729ec8",
"repeat": [], // 重复的数量 "tmbId": "65422be6aa44b7da77729ec9",
"error": [] // 其他错误 "type": "dataset",
"status": "active",
"avatar": "/icon/logo.svg",
"name": "FastGPT test",
"vectorModel": {
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"inputPrice": 0,
"defaultToken": 512,
"maxToken": 8000,
"weight": 100
},
"agentModel": {
"model": "gpt-3.5-turbo-16k",
"name": "FastAI-16k",
"maxContext": 16000,
"maxResponse": 16000,
"inputPrice": 0,
"outputPrice": 0
},
"intro": "",
"permission": "private",
"updateTime": "2024-01-02T10:11:03.084Z",
"canWrite": true,
"isOwner": true
} }
} }
``` ```
{{< /markdownify >}} {{< /markdownify >}}
{{< /tab >}} {{< /tab >}}
{{< /tabs >}}
{{< tab tabName="QA Prompt 模板" >}} ### 删除一个知识库
{{< markdownify >}}
{{theme}} 里的内容可以换成数据的主题。默认为：它们可能包含多个主题内容 {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```bash
curl --location --request DELETE 'http://localhost:3000/api/core/dataset/delete?id=65abc8729d1448617cba5df6' \
--header 'Authorization: Bearer {{authorization}}' \
``` ```
我会给你一段文本，{{theme}}，学习它们，并整理学习成果，要求为：
1. 提出最多 25 个问题。
2. 给出每个问题的答案。
3. 答案要详细完整，答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 markdown 元素。
4. 按格式返回多个问题和答案:
Q1: 问题。 {{< /markdownify >}}
A1: 答案。 {{< /tab >}}
Q2:
A2:
……
我的文本："""{{text}}""" {{< tab tabName="参数说明" >}}
``` {{< markdownify >}}
{{% alert icon=" " context="success" %}}
- id: 知识库的ID
{{% /alert %}}
{{< /markdownify >}} {{< /markdownify >}}
{{< /tab >}} {{< /tab >}}
{{< tab tabName="响应示例" >}}
{{< markdownify >}}
```json
{
"code": 200,
"statusText": "",
"message": "",
"data": null
}
```
{{< /markdownify >}}
{{< /tab >}}
{{< /tabs >}} {{< /tabs >}}
## 搜索测试 ## 集合
### 创建一个空的集合
{{< tabs tabTotal="3" >}} {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}} {{< tab tabName="请求示例" >}}
{{< markdownify >}} {{< markdownify >}}
```bash ```bash
curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTest' \ curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/create' \
--header 'Authorization: Bearer fastgpt-xxxxx' \ --header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \ --header 'Content-Type: application/json' \
--data-raw '{ --data-raw '{
"datasetId": "知识库的ID", "datasetId":"6593e137231a2be9c5603ba7",
"text": "导演是谁", "parentId": null,
"limit": 5000, "name":"测试",
"similarity": 0, "type":"virtual",
"searchMode": "embedding", "metadata":{
"usingReRank": false "test":111
}
}' }'
``` ```
...@@ -181,12 +310,15 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTe ...@@ -181,12 +310,15 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTe
{{< tab tabName="参数说明" >}} {{< tab tabName="参数说明" >}}
{{< markdownify >}} {{< markdownify >}}
- datasetId - 知识库ID {{% alert icon=" " context="success" %}}
- text - 需要测试的文本 - datasetId: 知识库的ID(必填)
- limit - 最大 tokens 数量 - parentId： 父级ID，不填则默认为根目录
- similarity - 最低相关度（0~1，可选） - name: 集合名称（必填）
- searchMode - 搜索模式：embedding | fullTextRecall | mixedRecall - type:
- usingReRank - 使用重排 - folder：文件夹
- virtual：虚拟集合(手动集合)
- metadata： 元数据（暂时没啥用）
{{% /alert %}}
{{< /markdownify >}} {{< /markdownify >}}
{{< /tab >}} {{< /tab >}}
...@@ -194,71 +326,76 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTe ...@@ -194,71 +326,76 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTe
{{< tab tabName="响应示例" >}} {{< tab tabName="响应示例" >}}
{{< markdownify >}} {{< markdownify >}}
返回 top k 结果， limit 为最大 Tokens 数量，最多 20000 tokens。 data 为集合的 ID。
```bash ```json
{ {
"code": 200, "code": 200,
"statusText": "", "statusText": "",
"data": [ "message": "",
{ "data": "65abcd009d1448617cba5ee1"
"id": "65599c54a5c814fb803363cb",
"q": "你是谁",
"a": "我是FastGPT助手",
"datasetId": "6554684f7f9ed18a39a4d15c",
"collectionId": "6556cd795e4b663e770bb66d",
"sourceName": "GBT 15104-2021 装饰单板贴面人造板.pdf",
"sourceId": "6556cd775e4b663e770bb65c",
"score": 0.8050316572189331
},
......
]
} }
``` ```
{{< /markdownify >}} {{< /markdownify >}}
{{< /tab >}} {{< /tab >}}
{{< /tabs >}} {{< /tabs >}}
# 更多接口 ### 创建一个纯文本集合(商业版)
目前未整理，简陋导出： 传入一段文字，创建一个集合，会根据传入的文字进行分割。
## POST 知识库搜索测试 {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
POST /core/dataset/searchTest ```bash
curl --location --request POST 'http://localhost:3000/api/proApi/core/dataset/collection/create/text' \
--header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"text":"xxxxxxxx",
"datasetId":"6593e137231a2be9c5603ba7",
"parentId": null,
"name":"测试训练",
> Body Parameters "trainingType": "qa",
"chunkSize":8000,
"chunkSplitter":"",
"qaPrompt":"11",
```json "metadata":{}
{ }'
"datasetId": "656c2ccff7f114064daa72f6",
"text": "导演是谁",
"limit": 1500,
"searchMode": "embedding",
"usingReRank": true,
"similarity": 0.5
}
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
{{< tab tabName="参数说明" >}}
{{< markdownify >}}
{{% alert icon=" " context="success" %}}
- text: 原文本
- datasetId: 知识库的ID(必填)
- parentId： 父级ID，不填则默认为根目录
- name: 集合名称（必填）
- metadata： 元数据（暂时没啥用）
- trainingType:（必填）
- chunk: 按文本长度进行分割
- qa: QA拆分
- chunkSize: 每个 chunk 的长度（可选）. chunk模式:100~3000; qa模式: 4000~模型最大token（16k模型通常建议不超过10000）
- chunkSplitter: 自定义最高优先分割符号（可选）
- qaPrompt: qa拆分自定义提示词（可选）
{{% /alert %}}
|Name|Location|Type|Required|Description| {{< /markdownify >}}
|---|---|---|---|---| {{< /tab >}}
|Authorization|header|string| no |none|
|body|body|object| no |none|
|» datasetId|body|string| yes |none|
|» text|body|string| yes |none|
|» limit|body|integer| no |none|
|» searchMode|body|[search mode](#schemasearch%20mode)| yes |none|
|» usingReRank|body|boolean| no |none|
|» similarity|body|[similary](#schemasimilary)| no |none|
> Response Examples {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> 成功 data 为集合的 ID。
```json ```json
{ {
...@@ -266,1110 +403,795 @@ POST /core/dataset/searchTest ...@@ -266,1110 +403,795 @@ POST /core/dataset/searchTest
"statusText": "", "statusText": "",
"message": "", "message": "",
"data": { "data": {
"list": [ "collectionId": "65abcfab9d1448617cba5f0d",
{ "results": {
"id": "65962b23f5fac58e46330dfd", "insertLen": 5, // 分割成多少段
"q": "# 快速了解 FastGPT\nFastGPT 的能力与优势\n\nFastGPT 是一个基于 LLM 大语言模型的知识库问答系统，提供开箱即用的数据处理、模型调用等能力。同时可以通过 Flow 可视化进行工作流编排，从而实现复杂的问答场景！\n\n🤖\n\nFastGPT 在线使用：[https://fastgpt.in](https://fastgpt.in)\n\n| | |\n| --- | --- |\n| ![](https://doc.fastgpt.in/imgs/intro1.png) | ![](https://doc.fastgpt.in/imgs/intro2.png) |\n| ![](https://doc.fastgpt.in/imgs/intro3.png) | ![](https://doc.fastgpt.in/imgs/intro4.png) |\n\n", "overToken": [],
"a": "", "repeat": [],
"chunkIndex": 0, "error": []
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65962b2089642fd209da3b03",
"sourceName": "https://doc.fastgpt.in/docs/intro/",
"sourceId": "https://doc.fastgpt.in/docs/intro/",
"score": [
{
"type": "embedding",
"value": 0.8036568760871887,
"index": 20
},
{
"type": "fullText",
"value": 1.168349443855932,
"index": 2
},
{
"type": "reRank",
"value": 0.9870296135626316,
"index": 0
},
{
"type": "rrf",
"value": 0.04366449476962486,
"index": 0
}
]
},
{
"id": "65962b24f5fac58e46330dff",
"q": "# 快速了解 FastGPT\n## FastGPT 能力\n### 2. 简单易用的可视化界面\nFastGPT 采用直观的可视化界面设计，为各种应用场景提供了丰富实用的功能。通过简洁易懂的操作步骤，可以轻松完成 AI 客服的创建和训练流程。\n\n![](https://doc.fastgpt.in/imgs/ability5.png)\n\n",
"a": "",
"chunkIndex": 2,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65962b2089642fd209da3b03",
"sourceName": "https://doc.fastgpt.in/docs/intro/",
"sourceId": "https://doc.fastgpt.in/docs/intro/",
"score": [
{
"type": "embedding",
"value": 0.8152669668197632,
"index": 3
},
{
"type": "fullText",
"value": 1.0511363636363635,
"index": 8
},
{
"type": "reRank",
"value": 0.9287972729281414,
"index": 14
},
{
"type": "rrf",
"value": 0.04265696347031964,
"index": 1
}
]
},
{
"id": "65962b25f5fac58e46330e00",
"q": "# 快速了解 FastGPT\n## FastGPT 能力\n### 3. 自动数据预处理\n提供手动输入、直接分段、LLM 自动处理和 CSV 等多种数据导入途径，其中“直接分段”支持通过 PDF、WORD、Markdown 和 CSV 文档内容作为上下文。FastGPT 会自动对文本数据进行预处理、向量化和 QA 分割，节省手动训练时间，提升效能。\n\n![](https://doc.fastgpt.in/imgs/ability2.png)\n\n",
"a": "",
"chunkIndex": 3,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65962b2089642fd209da3b03",
"sourceName": "https://doc.fastgpt.in/docs/intro/",
"sourceId": "https://doc.fastgpt.in/docs/intro/",
"score": [
{
"type": "embedding",
"value": 0.8158369064331055,
"index": 2
},
{
"type": "fullText",
"value": 1.014030612244898,
"index": 20
},
{
"type": "reRank",
"value": 0.9064876908461501,
"index": 17
},
{
"type": "rrf",
"value": 0.04045823457588163,
"index": 2
}
]
},
{
"id": "65a7e1e8fc13bdf20fd46d41",
"q": "# 快速了解 FastGPT\n## FastGPT 能力\n### 5. 强大的 API 集成\nFastGPT 对外的 API 接口对齐了 OpenAI 官方接口，可以直接接入现有的 GPT 应用，也可以轻松集成到企业微信、公众号、飞书等平台。\n\n![](https://doc.fastgpt.in/imgs/ability4.png)",
"a": "",
"chunkIndex": 66,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7e1d4fc13bdf20fd46abe",
"sourceName": "dataset - 2024-01-04T151625.388.csv",
"sourceId": "65a7e1d2fc13bdf20fd46abc",
"score": [
{
"type": "embedding",
"value": 0.803692102432251,
"index": 18
},
{
"type": "fullText",
"value": 1.0511363636363635,
"index": 7
},
{
"type": "reRank",
"value": 0.9177460552422909,
"index": 15
},
{
"type": "rrf",
"value": 0.03970501147383226,
"index": 3
}
]
},
{
"id": "65a7be319d96e21823f69c9b",
"q": "FastGPT Flow 的工作流设计方案提供了哪些操作？",
"a": "FastGPT Flow 的工作流设计方案提供了数据预处理、各类 AI 应用设置、调试测试及结果反馈等操作。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.8283981680870056,
"index": 0
},
{
"type": "reRank",
"value": 0.9620363047907355,
"index": 4
},
{
"type": "rrf",
"value": 0.03177805800756621,
"index": 4
}
]
},
{
"id": "65a7be389d96e21823f69d58",
"q": "FastGPT Flow 的实验室预约示例中使用了哪些参数？",
"a": "FastGPT Flow 的实验室预约示例中使用了姓名、时间和实验室名称等参数。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.8143455386161804,
"index": 9
},
{
"type": "reRank",
"value": 0.9806919138043485,
"index": 1
},
{
"type": "rrf",
"value": 0.0304147465437788,
"index": 5
}
]
},
{
"id": "65a7be309d96e21823f69c78",
"q": "FastGPT Flow 是什么？",
"a": "FastGPT Flow 是一款基于大型语言模型的知识库问答系统，通过引入 Flow 可视化工作流编排技术，提供了一个即插即用的解决方案。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.8115077018737793,
"index": 11
},
{
"type": "reRank",
"value": 0.9686195704870232,
"index": 3
},
{
"type": "rrf",
"value": 0.029513888888888888,
"index": 6
}
]
},
{
"id": "65a7be389d96e21823f69d5e",
"q": "FastGPT Flow 的实验室预约示例中的代码实现了哪些功能？",
"a": "FastGPT Flow 的实验室预约示例中的代码实现了预约实验室、修改预约、查询预约和取消预约等功能。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.8166953921318054,
"index": 1
},
{
"type": "reRank",
"value": 0.8350804533361768,
"index": 20
},
{
"type": "rrf",
"value": 0.028474711270410194,
"index": 8
}
]
},
{
"id": "65a7be389d96e21823f69d4f",
"q": "FastGPT Flow 的联网搜索示例中使用了哪些参数？",
"a": "FastGPT Flow 的联网搜索示例中使用了搜索关键词、Google 搜索的 API 密钥和自定义搜索引擎 ID。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.8025297522544861,
"index": 21
},
{
"type": "reRank",
"value": 0.9730876959261983,
"index": 2
},
{
"type": "rrf",
"value": 0.028068137824235385,
"index": 10
}
]
},
{
"id": "65a7e1e8fc13bdf20fd46d55",
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7e1d4fc13bdf20fd46abe",
"sourceName": "dataset - 2024-01-04T151625.388.csv",
"sourceId": "65a7e1d2fc13bdf20fd46abc",
"q": "# 快速了解 FastGPT\n## FastGPT 特点\n1. **项目开源**\n \n FastGPT 遵循附加条件 Apache License 2.0 开源协议，你可以 [Fork](https://github.com/labring/FastGPT/fork) 之后进行二次开发和发布。FastGPT 社区版将保留核心功能，商业版仅在社区版基础上使用 API 的形式进行扩展，不影响学习使用。\n \n2. **独特的 QA 结构**\n \n 针对客服问答场景设计的 QA 结构，提高在大量数据场景中的问答准确性。\n \n3. **可视化工作流**\n \n 通过 Flow 模块展示了从问题输入到模型输出的完整流程，便于调试和设计复杂流程。\n \n4. **无限扩展**\n \n 基于 API 进行扩展，无需修改 FastGPT 源码，也可快速接入现有的程序中。\n \n5. **便于调试**\n \n 提供搜索测试、引用修改、完整对话预览等多种调试途径。\n \n6. **支持多种模型**\n \n 支持 GPT、Claude、文心一言等多种 LLM 模型，未来也将支持自定义的向量模型。",
"a": "",
"chunkIndex": 67,
"score": [
{
"type": "fullText",
"value": 1.0340073529411764,
"index": 12
},
{
"type": "reRank",
"value": 0.9542227274192233,
"index": 9
},
{
"type": "rrf",
"value": 0.027272727272727275,
"index": 11
}
]
},
{
"id": "65a7be319d96e21823f69c8f",
"q": "FastGPT Flow 的工作流设计中，模块之间如何进行组合和组装？",
"a": "FastGPT Flow 允许用户在核心工作流模块中进行自由组合和组装，从而衍生出一个新的模块。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.8098832368850708,
"index": 13
},
{
"type": "reRank",
"value": 0.9478657435317039,
"index": 12
},
{
"type": "rrf",
"value": 0.027212143650499815,
"index": 12
}
]
},
{
"id": "65a7be359d96e21823f69ce0",
"q": "FastGPT Flow 的模块的输入和输出如何连接？",
"a": "FastGPT Flow 的模块的输入和输出通过连接点进行连接，连接点的颜色代表了不同的数据类型。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.8060981035232544,
"index": 16
},
{
"type": "reRank",
"value": 0.9530133603823691,
"index": 10
},
{
"type": "rrf",
"value": 0.027071520029266508,
"index": 13
}
]
},
{
"id": "65a7be319d96e21823f69c98",
"q": "FastGPT Flow 的工作流设计方案能够满足哪些问答场景？",
"a": "FastGPT Flow 的工作流设计方案能够满足基本的 AI 知识库问答需求，并适应各种复杂的问答场景，例如联网搜索、数据库操作、数据实时更新、消息通知等。",
"chunkIndex": 0,
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7be059d96e21823f69af5",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7be059d96e21823f69ae8",
"score": [
{
"type": "embedding",
"value": 0.814436137676239,
"index": 8
},
{
"type": "reRank",
"value": 0.8814109034236719,
"index": 19
},
{
"type": "rrf",
"value": 0.026992753623188405,
"index": 16
}
]
},
{
"id": "65a7e058fc13bdf20fd46577",
"datasetId": "6593e137231a2be9c5603ba7",
"collectionId": "65a7e01efc13bdf20fd45815",
"sourceName": "FastGPT软著.pdf",
"sourceId": "65a7e01dfc13bdf20fd457f3",
"q": "FastGPT Flow 工作流设计112312 3123213123 232321312 21312 23一、介绍FastGPT 作为一款基于大型语言模型（LLM）的知识库问答系统，旨在为用户提供一个即插即用的解决方案。它集成了数据处理、模型调用等多项功能，通过引入 Flow 可视化工作流编排技术，进一步增强了对复杂问答场景的支持能力。本文将重点介绍 FastGPT Flow工作流的设计方案和应用优势。\nFastGPT Flow 工 作 流 采 用 了 React Flow 框 架 作 为 UI 底 座 ， 结 合 自 研 的 FlowController 实现工作流的运行。FastGPT 使用 Flow 模块为用户呈现了一个直观、可视化的界面，从而简化了 AI 应用工作流程的设计和管理方式。React Flow 的应用使得用户能够以图形化的方式组织和编排工作流，这不仅使得工作流的创建过程更为直观，同时也为用户提供了强大且灵活的工作流编辑器。在 FastGPT Flow 工作流设计中，核心工作流模块包括用户引导、问题输入、知识库检索、AI 文本生成、问题分类、结构化内容提取、指定回复、应用调用和 HTTP 扩展，并允许用户在这类模块中进行自由组合和组装，从而衍生出一个新的模块。",
"a": "",
"chunkIndex": 0,
"score": [
{
"type": "fullText",
"value": 1.0229779411764706,
"index": 15
},
{
"type": "reRank",
"value": 0.9577545043363116,
"index": 8
},
{
"type": "rrf",
"value": 0.026992753623188405,
"index": 17
}
]
} }
],
"duration": "2.978s",
"searchMode": "mixedRecall",
"limit": 1500,
"similarity": 0.1,
"usingReRank": true,
"usingSimilarityFilter": true
} }
} }
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
{{< /tabs >}}
### 创建一个链接集合(商业版)
|HTTP Status Code |Meaning|Description|Data schema| 传入一个网络链接，创建一个集合，会先去对应网页抓取内容，再抓取的文字进行分割。
|---|---|---|---|
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
HTTP Status Code **200** ```bash
curl --location --request POST 'http://localhost:3000/api/proApi/core/dataset/collection/create/link' \
--header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"link":"https://doc.fastgpt.in/docs/course/quick-start/",
"datasetId":"6593e137231a2be9c5603ba7",
"parentId": null,
|Name|Type|Required|Restrictions|Title|description| "trainingType": "chunk",
|---|---|---|---|---|---| "chunkSize":512,
|» code|integer|true|none||none| "chunkSplitter":"",
|» statusText|string|true|none||none| "qaPrompt":"",
|» message|string|true|none||none|
|» data|object|true|none||none|
|»» list|[object]|true|none||none|
|»»» id|string|true|none||none|
|»»» q|string|true|none||none|
|»»» a|string|true|none||none|
|»»» chunkIndex|integer|true|none||none|
|»»» datasetId|string|true|none||none|
|»»» collectionId|string|true|none||none|
|»»» sourceName|string|true|none||none|
|»»» sourceId|string|true|none||none|
|»»» score|[object]|true|none||none|
|»»»» type|string|true|none||none|
|»»»» value|number|true|none||none|
|»»»» index|integer|true|none||none|
|»» duration|string|true|none||none|
|»» searchMode|string|true|none||none|
|»» limit|integer|true|none||none|
|»» similarity|number|true|none||none|
|»» usingReRank|boolean|true|none||none|
|»» usingSimilarityFilter|boolean|true|none||none|
# openapi/知识库/知识库crud "metadata":{
"webPageSelector":".docs-content"
}
}'
```
## GET 获取知识库列表 {{< /markdownify >}}
{{< /tab >}}
GET /core/dataset/list {{< tab tabName="参数说明" >}}
{{< markdownify >}}
### Params {{% alert icon=" " context="success" %}}
- link: 网络链接
- datasetId: 知识库的ID(必填)
- parentId： 父级ID，不填则默认为根目录
- metadata.webPageSelector: 网页选择器，用于指定网页中的哪个元素作为文本(可选)
- trainingType:（必填）
- chunk: 按文本长度进行分割
- qa: QA拆分
- chunkSize: 每个 chunk 的长度（可选）. chunk模式:100~3000; qa模式: 4000~模型最大token（16k模型通常建议不超过10000）
- chunkSplitter: 自定义最高优先分割符号（可选）
- qaPrompt: qa拆分自定义提示词（可选）
{{% /alert %}}
|Name|Location|Type|Required|Description| {{< /markdownify >}}
|---|---|---|---|---| {{< /tab >}}
|parentId|query|string| no |父级的ID|
|Authorization|header|string| no |none|
> Response Examples {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> 200 Response data 为集合的 ID。
```json ```json
{} {
"code": 200,
"statusText": "",
"message": "",
"data": {
"collectionId": "65abd0ad9d1448617cba6031"
}
}
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
{{< /tabs >}}
|HTTP Status Code |Meaning|Description|Data schema| ### 创建一个文件集合(商业版)
|---|---|---|---|
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema 传入一个文件，创建一个集合，会读取文件内容进行分割。目前支持：pdf, docx, md, txt, html, csv。
## GET 获取知识库详情 {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
GET /core/dataset/detail ```bash
curl --location --request POST 'http://localhost:3000/api/proApi/core/dataset/collection/create/file' \
--header 'Authorization: Bearer {{authorization}}' \
--form 'file=@"C:\\Users\\user\\Desktop\\fastgpt测试文件\\index.html"' \
--form 'data="{\"datasetId\":\"6593e137231a2be9c5603ba7\",\"parentId\":null,\"trainingType\":\"chunk\",\"chunkSize\":512,\"chunkSplitter\":\"\",\"qaPrompt\":\"\",\"metadata\":{}}"'
```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Description| {{< tab tabName="参数说明" >}}
|---|---|---|---|---| {{< markdownify >}}
|id|query|string| no |知识库id|
|Authorization|header|string| no |none|
> Response Examples 需要使用 POST form-data 的格式上传。包含 file 和 data 两个字段。
{{% alert icon=" " context="success" %}}
- file: 文件
- data: 知识库相关信息（json序列化后传入）
- datasetId: 知识库的ID(必填)
- parentId： 父级ID，不填则默认为根目录
- trainingType:（必填）
- chunk: 按文本长度进行分割
- qa: QA拆分
- chunkSize: 每个 chunk 的长度（可选）. chunk模式:100~3000; qa模式: 4000~模型最大token（16k模型通常建议不超过10000）
- chunkSplitter: 自定义最高优先分割符号（可选）
- qaPrompt: qa拆分自定义提示词（可选）
{{% /alert %}}
> 200 Response {{< /markdownify >}}
{{< /tab >}}
```json {{< tab tabName="响应示例" >}}
{} {{< markdownify >}}
```
### Responses
|HTTP Status Code |Meaning|Description|Data schema|
|---|---|---|---|
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema
# openapi/知识库/集合crud
## POST 获取知识库集合列表
POST /core/dataset/collection/list
> Body Parameters data 为集合的 ID。
```json ```json
{ {
"pageNum": 1, "code": 200,
"pageSize": 10, "statusText": "",
"datasetId": "6597ca43e26f2a90a1501414", "message": "",
"parentId": null, "data": {
"searchText": "", "collectionId": "65abc044e4704bac793fbd81",
"simple": true "results": {
"insertLen": 1,
"overToken": [],
"repeat": [],
"error": []
}
}
} }
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Description| {{< /tabs >}}
|---|---|---|---|---|
|Authorization|header|string| no |none|
|body|body|object| no |none|
|» pageNum|body|integer| no |none|
|» pageSize|body|integer| no |none|
|» datasetId|body|string| yes |none|
|» parentId|body|null| no |none|
|» searchText|body|string| no |none|
|» simple|body|boolean| no |none|
> Response Examples
> 200 Response
```json
{}
```
### Responses
|HTTP Status Code |Meaning|Description|Data schema|
|---|---|---|---|
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema
## GET 获取集合详情
GET /core/dataset/collection/detail
### Params
|Name|Location|Type|Required|Description|
|---|---|---|---|---|
|id|query|string| no |知识库id|
|Authorization|header|string| no |none|
> Response Examples ### 获取集合列表
> 200 Response {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request POST 'http://localhost:3000/api/core/dataset/collection/list' \
--header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"pageNum":1,
"pageSize": 10,
"datasetId":"6593e137231a2be9c5603ba7",
"parentId": null,
"searchText":""
}'
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema| {{< tab tabName="参数说明" >}}
|---|---|---|---| {{< markdownify >}}
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{% alert icon=" " context="success" %}}
- pageNum: 页码（选填）
- pageSize: 每页数量，最大30（选填）
- datasetId: 知识库的ID(必填)
- parentId: 父级Id（选填）
- searchText: 模糊搜索文本（选填）
{{% /alert %}}
## PUT 更新集合 {{< /markdownify >}}
{{< /tab >}}
PUT /core/dataset/collection/update {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> Body Parameters
```json ```json
{ {
"id": "6597ce094e10ee661f0891c8", "code": 200,
"parentId": null, "statusText": "",
"name": "222" "message": "",
"data": {
"pageNum": 1,
"pageSize": 10,
"data": [
{
"_id": "6593e137231a2be9c5603ba9",
"parentId": null,
"tmbId": "65422be6aa44b7da77729ec9",
"type": "virtual",
"name": "手动录入",
"updateTime": "2099-01-01T00:00:00.000Z",
"dataAmount": 3,
"trainingAmount": 0,
"canWrite": true
},
{
"_id": "65abd0ad9d1448617cba6031",
"parentId": null,
"tmbId": "65422be6aa44b7da77729ec9",
"type": "link",
"name": "快速上手 | FastGPT",
"rawLink": "https://doc.fastgpt.in/docs/course/quick-start/",
"updateTime": "2024-01-20T13:54:53.031Z",
"dataAmount": 3,
"trainingAmount": 0,
"canWrite": true
}
],
"total": 93
}
} }
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|Authorization|header|string| no ||none|
|body|body|object| no ||none|
|» id|body|string| yes ||none|
|» parentId|body|null| no | 父级的id|none|
|» name|body|string| no | 名称|none|
> Response Examples ### 获取集合详情
> 200 Response {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request GET 'http://localhost:3000/api/core/dataset/collection/detail?id=65abcfab9d1448617cba5f0d' \
--header 'Authorization: Bearer {{authorization}}' \
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema| {{< tab tabName="参数说明" >}}
|---|---|---|---| {{< markdownify >}}
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{% alert icon=" " context="success" %}}
- id: 集合的ID
{{% /alert %}}
## POST 创建空集合（文件夹或者一个空集合） {{< /markdownify >}}
{{< /tab >}}
POST /core/dataset/collection/create {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> Body Parameters
```json ```json
{ {
"datasetId": "6597ca43e26f2a90a1501414", "code": 200,
"parentId": null, "statusText": "",
"name": "集合名", "message": "",
"type": "folder", "data": {
"metadata": {} "_id": "65abcfab9d1448617cba5f0d",
"parentId": null,
"teamId": "65422be6aa44b7da77729ec8",
"tmbId": "65422be6aa44b7da77729ec9",
"datasetId": {
"_id": "6593e137231a2be9c5603ba7",
"parentId": null,
"teamId": "65422be6aa44b7da77729ec8",
"tmbId": "65422be6aa44b7da77729ec9",
"type": "dataset",
"status": "active",
"avatar": "/icon/logo.svg",
"name": "FastGPT test",
"vectorModel": "text-embedding-ada-002",
"agentModel": "gpt-3.5-turbo-16k",
"intro": "",
"permission": "private",
"updateTime": "2024-01-02T10:11:03.084Z"
},
"type": "virtual",
"name": "测试训练",
"trainingType": "qa",
"chunkSize": 8000,
"chunkSplitter": "",
"qaPrompt": "11",
"rawTextLength": 40466,
"hashRawText": "47270840614c0cc122b29daaddc09c2a48f0ec6e77093611ab12b69cba7fee12",
"createTime": "2024-01-20T13:50:35.838Z",
"updateTime": "2024-01-20T13:50:35.838Z",
"canWrite": true,
"sourceName": "测试训练"
}
} }
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|Authorization|header|string| no ||none|
|body|body|object| no ||none|
|» datasetId|body|string| yes ||none|
|» parentId|body|null| no ||none|
|» name|body|string| yes ||none|
|» type|body|[collection type](#schemacollection%20type)| yes ||none|
|» metadata|body|object| no ||none|
> Response Examples ### 修改集合信息
> 200 Response {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request PUT 'http://localhost:3000/api/core/dataset/collection/update' \
--header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"id":"65abcfab9d1448617cba5f0d",
"parentId":null,
"name":"测2222试"
}'
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema| {{< tab tabName="参数说明" >}}
|---|---|---|---| {{< markdownify >}}
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{% alert icon=" " context="success" %}}
- id: 集合的ID
- parentId: 修改父级ID（可选）
- name: 修改集合名称（可选）
{{% /alert %}}
## POST 创建文本集合 {{< /markdownify >}}
{{< /tab >}}
POST /core/dataset/collection/create/text {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> Body Parameters
```json ```json
{ {
"text": "xxxxxxxxxxxxxx", "code": 200,
"datasetId": "6593e137231a2be9c5603ba7", "statusText": "",
"parentId": null, "message": "",
"name": "测试", "data": null
"trainingType": "qa",
"chunkSize": 8000,
"chunkSplitter": "",
"qaPrompt": "",
"metadata": {}
} }
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|Authorization|header|string| no ||none|
|body|body|object| no ||none|
|» datasetId|body|string| no ||none|
|» parentId|body|null| no ||none|
|» name|body|string| yes ||none|
|» text|body|string| yes | 原文本|none|
|» trainingType|body|[training type](#schematraining%20type)| yes ||none|
|» chunkSize|body|integer| no | 分块大小|none|
|» chunkSplitter|body|string| no | 自定义最高优先级的分段符号|none|
|» qaPrompt|body|string| no ||none|
|» metadata|body|object| no ||none|
> Response Examples ### 删除一个集合
> 200 Response {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request DELETE 'http://localhost:3000/api/core/dataset/collection/delete?id=65aa2a64e6cb9b8ccdc00de8' \
--header 'Authorization: Bearer {{authorization}}' \
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema| {{< tab tabName="参数说明" >}}
|---|---|---|---| {{< markdownify >}}
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{% alert icon=" " context="success" %}}
- id: 集合的ID
{{% /alert %}}
## POST 创建网络链接集合 {{< /markdownify >}}
{{< /tab >}}
POST /core/dataset/collection/create/link {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> Body Parameters
```json ```json
{ {
"link": "https://doc.fastgpt.in/docs/course/quick-start/", "code": 200,
"datasetId": "6593e137231a2be9c5603ba7", "statusText": "",
"parentId": null, "message": "",
"trainingType": "chunk", "data": null
"chunkSize": 512,
"chunkSplitter": "",
"qaPrompt": "",
"metadata": {
"webPageSelector": ".docs-content"
}
} }
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|Authorization|header|string| no ||none|
|body|body|object| no ||none|
|» datasetId|body|string| yes ||none|
|» parentId|body|null| no ||none|
|» link|body|string| yes ||none|
|» trainingType|body|[training type](#schematraining%20type)| yes ||none|
|» chunkSize|body|integer| no ||none|
|» chunkSplitter|body|string| no ||none|
|» qaPrompt|body|string| no ||none|
|» metadata|body|object| no ||none|
|»» webPageSelector|body|string| no | web选择器|none|
> Response Examples
> 200 Response
```json
{}
```
### Responses
|HTTP Status Code |Meaning|Description|Data schema|
|---|---|---|---|
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema
## DELETE 删除一个集合
DELETE /core/dataset/collection/delete
### Params ## 数据
|Name|Location|Type|Required|Title|Description| ### 为集合批量添加添加数据
|---|---|---|---|---|---|
|id|query|string| no ||知识库id|
|Authorization|header|string| no ||none|
> Response Examples 注意，每次最多推送 200 组数据。
> 200 Response {{< tabs tabTotal="4" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/data/pushData' \
--header 'Authorization: Bearer apikey' \
--header 'Content-Type: application/json' \
--data-raw '{
    "collectionId": "64663f451ba1676dbdef0499",
"trainingMode": "chunk",
"prompt": "可选。qa 拆分引导词，chunk 模式下忽略",
"billId": "可选。如果有这个值，本次的数据会被聚合到一个订单中，这个值可以重复使用。可以参考 [创建训练订单] 获取该值。",
    "data": [
{
"q": "你是谁？",
"a": "我是FastGPT助手"
},
{
"q": "你会什么？",
"a": "我什么都会",
"indexes": [{
"defaultIndex": false,
"type":"custom",
"text":"自定义索引，不使用默认索引"
}]
}
]
}'
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema|
|---|---|---|---|
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema
# openapi/知识库/数据crud {{< tab tabName="参数说明" >}}
{{< markdownify >}}
## POST 获取数据列表 {{% alert icon=" " context="success" %}}
- collectionId: 集合ID（必填）
- trainingType:（必填）
- chunk: 按文本长度进行分割
- qa: QA拆分
- prompt: 自定义 QA 拆分提示词，需严格按照模板，建议不要传入。（选填）
- data：（具体数据）
- q: 主要数据（必填）
- a: 辅助数据（选填）
- indexes: 自定义索引（选填），不传入则默认使用q和a构建索引。也可以传入
{{% /alert %}}
POST /core/dataset/data/list {{< /markdownify >}}
{{< /tab >}}
> Body Parameters {{< tab tabName="响应例子" >}}
{{< markdownify >}}
```json ```json
{ {
"pageNum": 1, "code": 200,
"pageSize": 10, "statusText": "",
"collectionId": "65a8d2700d70d3de0bf09186", "data": {
"searchText": "" "insertLen": 1, // 最终插入成功的数量
"overToken": [], // 超出 token 的
"repeat": [], // 重复的数量
"error": [] // 其他错误
}
} }
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Title|Description|
|---|---|---|---|---|---|
|Authorization|header|string| no ||none|
|body|body|object| no ||none|
|» pageNum|body|integer| yes ||none|
|» pageSize|body|integer| yes ||none|
|» searchText|body|string| yes ||none|
|» collectionId|body|string| yes ||none|
> Response Examples {{< tab tabName="QA Prompt 模板" >}}
{{< markdownify >}}
> 200 Response {{theme}} 里的内容可以换成数据的主题。默认为：它们可能包含多个主题内容
```json
{}
``` ```
我会给你一段文本，{{theme}}，学习它们，并整理学习成果，要求为：
1. 提出最多 25 个问题。
2. 给出每个问题的答案。
3. 答案要详细完整，答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 markdown 元素。
4. 按格式返回多个问题和答案:
### Responses Q1: 问题。
A1: 答案。
|HTTP Status Code |Meaning|Description|Data schema| Q2:
|---|---|---|---| A2:
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline| ……
### Responses Data Schema
## GET 获取数据详情
GET /core/dataset/data/detail
### Params
|Name|Location|Type|Required|Title|Description|
|---|---|---|---|---|---|
|id|query|string| yes ||none|
|Authorization|header|string| no ||none|
> Response Examples
> 200 Response
```json 我的文本："""{{text}}"""
{}
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema|
|---|---|---|---|
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema
## DELETE 删除一条数据
DELETE /core/dataset/data/delete
### Params {{< /tabs >}}
|Name|Location|Type|Required|Title|Description|
|---|---|---|---|---|---|
|id|query|string| no ||none|
|Authorization|header|string| no ||none|
> Response Examples ### 获取集合的数据列表
> 200 Response {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request POST 'http://localhost:3000/api/core/dataset/data/list' \
--header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"pageNum":1,
"pageSize": 10,
"collectionId":"65abd4ac9d1448617cba6171",
"searchText":""
}'
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema| {{< tab tabName="参数说明" >}}
|---|---|---|---| {{< markdownify >}}
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{% alert icon=" " context="success" %}}
- pageNum: 页码（选填）
- pageSize: 每页数量，最大30（选填）
- collectionId: 集合的ID（必填）
- searchText: 模糊搜索词（选填）
{{% /alert %}}
## PUT 更新数据 {{< /markdownify >}}
{{< /tab >}}
PUT /core/dataset/data/update {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> Body Parameters
```json ```json
{ {
"id": "6597ce094e10ee661f0891c8", "code": 200,
"parentId": null, "statusText": "",
"name": "222" "message": "",
"data": {
"pageNum": 1,
"pageSize": 10,
"data": [
{
"_id": "65abd4b29d1448617cba61db",
"datasetId": "65abc9bd9d1448617cba5e6c",
"collectionId": "65abd4ac9d1448617cba6171",
"q": "N o . 2 0 2 2 1 2中 国 信 息 通 信 研 究 院京东探索研究院2022年 9月人工智能生成内容（AIGC）白皮书(2022 年)版权声明本白皮书版权属于中国信息通信研究院和京东探索研究院，并受法律保护。转载、摘编或利用其它方式使用本白皮书文字或者观点的，应注明“来源：中国信息通信研究院和京东探索研究院”。违反上述声明者，编者将追究其相关法律责任。前 言习近平总书记曾指出，“数字技术正以新理念、新业态、新模式全面融入人类经济、政治、文化、社会、生态文明建设各领域和全过程”。在当前数字世界和物理世界加速融合的大背景下，人工智能生成内容（Artificial Intelligence Generated Content，简称 AIGC）正在悄然引导着一场深刻的变革，重塑甚至颠覆数字内容的生产方式和消费模式，将极大地丰富人们的数字生活，是未来全面迈向数字文明新时代不可或缺的支撑力量。",
"a": "",
"chunkIndex": 0
},
{
"_id": "65abd4b39d1448617cba624d",
"datasetId": "65abc9bd9d1448617cba5e6c",
"collectionId": "65abd4ac9d1448617cba6171",
"q": "本白皮书重点从 AIGC 技术、应用和治理等维度进行了阐述。在技术层面，梳理提出了 AIGC 技术体系，既涵盖了对现实世界各种内容的数字化呈现和增强，也包括了基于人工智能的自主内容创作。在应用层面，重点分析了 AIGC 在传媒、电商、影视等行业和场景的应用情况，探讨了以虚拟数字人、写作机器人等为代表的新业态和新应用。在治理层面，从政策监管、技术能力、企业应用等视角，分析了AIGC 所暴露出的版权纠纷、虚假信息传播等各种问题。最后，从政府、行业、企业、社会等层面，给出了 AIGC 发展和治理建议。由于人工智能仍处于飞速发展阶段，我们对 AIGC 的认识还有待进一步深化，白皮书中存在不足之处，敬请大家批评指正。目 录一、 人工智能生成内容的发展历程与概念.............................................................. 1（一）AIGC 历史沿革 .......................................................................................... 1（二）AIGC 的概念与内涵 .................................................................................. 4二、人工智能生成内容的技术体系及其演进方向.................................................... 7（一）AIGC 技术升级步入深化阶段 .................................................................. 7（二）AIGC 大模型架构潜力凸显 .................................................................... 10（三）AIGC 技术演化出三大前沿能力 ............................................................ 18三、人工智能生成内容的应用场景.......................................................................... 26（一）AIGC+传媒：人机协同生产，",
"a": "",
"chunkIndex": 1
}
],
"total": 63
}
} }
``` ```
{{< /markdownify >}}
{{< /tab >}}
{{< /tabs >}}
### Params ### 获取单条数据详情
|Name|Location|Type|Required|Title|Description|
|---|---|---|---|---|---|
|Authorization|header|string| no ||none|
|body|body|object| no ||none|
|» id|body|string| yes ||none|
|» q|body|string| yes ||none|
|» a|body|string| no ||none|
|» indexes|body|[[数据自定义向量](#schema%e6%95%b0%e6%8d%ae%e8%87%aa%e5%ae%9a%e4%b9%89%e5%90%91%e9%87%8f)]| no ||none|
> Response Examples
> 200 Response {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request GET 'http://localhost:3000/api/core/dataset/data/detail?id=65abd4b29d1448617cba61db' \
--header 'Authorization: Bearer {{authorization}}' \
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema| {{< tab tabName="参数说明" >}}
|---|---|---|---| {{< markdownify >}}
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{% alert icon=" " context="success" %}}
- id: 数据的id
{{% /alert %}}
## POST 知识库插入记录（批量插入） {{< /markdownify >}}
{{< /tab >}}
POST /core/dataset/data/pushData {{< tab tabName="响应示例" >}}
{{< markdownify >}}
> Body Parameters
```json ```json
{ {
"collectionId": "string", "code": 200,
"data": [ "statusText": "",
{ "message": "",
"a": "string", "data": {
"q": "string", "id": "65abd4b29d1448617cba61db",
"chunkIndex": 1 "q": "N o . 2 0 2 2 1 2中 国 信 息 通 信 研 究 院京东探索研究院2022年 9月人工智能生成内容（AIGC）白皮书(2022 年)版权声明本白皮书版权属于中国信息通信研究院和京东探索研究院，并受法律保护。转载、摘编或利用其它方式使用本白皮书文字或者观点的，应注明“来源：中国信息通信研究院和京东探索研究院”。违反上述声明者，编者将追究其相关法律责任。前 言习近平总书记曾指出，“数字技术正以新理念、新业态、新模式全面融入人类经济、政治、文化、社会、生态文明建设各领域和全过程”。在当前数字世界和物理世界加速融合的大背景下，人工智能生成内容（Artificial Intelligence Generated Content，简称 AIGC）正在悄然引导着一场深刻的变革，重塑甚至颠覆数字内容的生产方式和消费模式，将极大地丰富人们的数字生活，是未来全面迈向数字文明新时代不可或缺的支撑力量。",
"a": "",
"chunkIndex": 0,
"indexes": [
{
"defaultIndex": true,
"type": "chunk",
"dataId": "3720083",
"text": "N o . 2 0 2 2 1 2中 国 信 息 通 信 研 究 院京东探索研究院2022年 9月人工智能生成内容（AIGC）白皮书(2022 年)版权声明本白皮书版权属于中国信息通信研究院和京东探索研究院，并受法律保护。转载、摘编或利用其它方式使用本白皮书文字或者观点的，应注明“来源：中国信息通信研究院和京东探索研究院”。违反上述声明者，编者将追究其相关法律责任。前 言习近平总书记曾指出，“数字技术正以新理念、新业态、新模式全面融入人类经济、政治、文化、社会、生态文明建设各领域和全过程”。在当前数字世界和物理世界加速融合的大背景下，人工智能生成内容（Artificial Intelligence Generated Content，简称 AIGC）正在悄然引导着一场深刻的变革，重塑甚至颠覆数字内容的生产方式和消费模式，将极大地丰富人们的数字生活，是未来全面迈向数字文明新时代不可或缺的支撑力量。",
"_id": "65abd4b29d1448617cba61dc"
}
],
"datasetId": "65abc9bd9d1448617cba5e6c",
"collectionId": "65abd4ac9d1448617cba6171",
"sourceName": "中文-AIGC白皮书2022.pdf",
"sourceId": "65abd4ac9d1448617cba6166",
"isOwner": true,
"canWrite": true
} }
],
"trainingMode": "string",
"promot": "string",
"billId": ""
} }
``` ```
### Params {{< /markdownify >}}
{{< /tab >}}
|Name|Location|Type|Required|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|Authorization|header|string| no ||none|
|body|body|object| no ||none|
|» collectionId|body|string| yes ||none|
|» data|body|[object]| yes ||none|
|»» a|body|string| no ||none|
|»» q|body|string| no ||none|
|»» chunkIndex|body|integer| no ||none|
|» trainingMode|body|[training type](#schematraining%20type)| no ||none|
|» promot|body|string| no ||none|
|» billId|body|string| no ||none|
> Response Examples ### 修改单条数据
> 200 Response {{< tabs tabTotal="3" >}}
{{< tab tabName="请求示例" >}}
{{< markdownify >}}
```json ```bash
{} curl --location --request PUT 'http://localhost:3000/api/core/dataset/data/update' \
--header 'Authorization: Bearer {{authorization}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"id":"65abd4b29d1448617cba61db",
"q":"测试111",
"a":"sss",
"indexes":[]
}'
``` ```
### Responses {{< /markdownify >}}
{{< /tab >}}
|HTTP Status Code |Meaning|Description|Data schema| {{< tab tabName="参数说明" >}}
|---|---|---|---| {{< markdownify >}}
|200|[OK](https://tools.ietf.org/html/rfc7231#section-6.3.1)|成功|Inline|
### Responses Data Schema {{% alert icon=" " context="success" %}}
- id: 数据的id
- q: 主要数据（选填）
- a: 辅助数据（选填）
- indexes: 自定义索引（选填），类型参考`为集合批量添加添加数据`，建议直接不传。更新q,a后，如果有默认索引，则会直接更新默认索引。
{{% /alert %}}
# Data Schema {{< /markdownify >}}
{{< /tab >}}
<h2 id="tocS_similary">similary</h2> {{< tab tabName="响应示例" >}}
{{< markdownify >}}
<a id="schemasimilary"></a>
<a id="schema_similary"></a>
<a id="tocSsimilary"></a>
<a id="tocssimilary"></a>
```json ```json
1 {
"code": 200,
"statusText": "",
"message": "",
"data": null
}
``` ```
### Attribute {{< /markdownify >}}
{{< /tab >}}
|Name|Type|Required|Restrictions|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|*anonymous*|integer|false|none||none|
<h2 id="tocS_search mode">search mode</h2>
<a id="schemasearch mode"></a> ### 删除单条数据
<a id="schema_search mode"></a>
<a id="tocSsearch mode"></a>
<a id="tocssearch mode"></a>
```json {{< tabs tabTotal="3" >}}
"embedding" {{< tab tabName="请求示例" >}}
{{< markdownify >}}
```bash
curl --location --request DELETE 'http://localhost:3000/api/core/dataset/data/delete?id=65abd4b39d1448617cba624d' \
--header 'Authorization: Bearer {{authorization}}' \
``` ```
### Attribute {{< /markdownify >}}
{{< /tab >}}
|Name|Type|Required|Restrictions|Title|Description| {{< tab tabName="参数说明" >}}
|---|---|---|---|---|---| {{< markdownify >}}
|*anonymous*|string|false|none||none|
#### Enum {{% alert icon=" " context="success" %}}
- id: 数据的id
{{% /alert %}}
|Name|Value| {{< /markdownify >}}
|---|---| {{< /tab >}}
|*anonymous*|embedding|
|*anonymous*|fullTextRecall|
|*anonymous*|mixedRecall|
<h2 id="tocS_training type">training type</h2> {{< tab tabName="响应示例" >}}
{{< markdownify >}}
<a id="schematraining type"></a>
<a id="schema_training type"></a>
<a id="tocStraining type"></a>
<a id="tocstraining type"></a>
```json ```json
"chunk" {
"code": 200,
"statusText": "",
"message": "",
"data": "success"
}
``` ```
### Attribute {{< /markdownify >}}
{{< /tab >}}
|Name|Type|Required|Restrictions|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|*anonymous*|string|false|none||none|
#### Enum
|Name|Value|
|---|---|
|*anonymous*|chunk|
|*anonymous*|qa|
<h2 id="tocS_collection type">collection type</h2>
<a id="schemacollection type"></a> ## 搜索测试
<a id="schema_collection type"></a>
<a id="tocScollection type"></a>
<a id="tocscollection type"></a>
```json {{< tabs tabTotal="3" >}}
"folder" {{< tab tabName="请求示例" >}}
{{< markdownify >}}
```bash
curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTest' \
--header 'Authorization: Bearer fastgpt-xxxxx' \
--header 'Content-Type: application/json' \
--data-raw '{
"datasetId": "知识库的ID",
"text": "导演是谁",
"limit": 5000,
"similarity": 0,
"searchMode": "embedding",
"usingReRank": false
}'
``` ```
### Attribute {{< /markdownify >}}
{{< /tab >}}
|Name|Type|Required|Restrictions|Title|Description| {{< tab tabName="参数说明" >}}
|---|---|---|---|---|---| {{< markdownify >}}
|*anonymous*|string|false|none||none|
#### Enum {{% alert icon=" " context="success" %}}
- datasetId - 知识库ID
- text - 需要测试的文本
- limit - 最大 tokens 数量
- similarity - 最低相关度（0~1，可选）
- searchMode - 搜索模式：embedding | fullTextRecall | mixedRecall
- usingReRank - 使用重排
{{% /alert %}}
|Name|Value| {{< /markdownify >}}
|---|---| {{< /tab >}}
|*anonymous*|folder|
|*anonymous*|virtual|
|*anonymous*|link|
|*anonymous*|file|
<h2 id="tocS_数据自定义向量">数据自定义向量</h2> {{< tab tabName="响应示例" >}}
{{< markdownify >}}
<a id="schema数据自定义向量"></a> 返回 top k 结果， limit 为最大 Tokens 数量，最多 20000 tokens。
<a id="schema_数据自定义向量"></a>
<a id="tocS数据自定义向量"></a>
<a id="tocs数据自定义向量"></a>
```json ```json
{ {
"defaultIndex": true, "code": 200,
"type": "string", "statusText": "",
"text": "string" "data": [
{
"id": "65599c54a5c814fb803363cb",
"q": "你是谁",
"a": "我是FastGPT助手",
"datasetId": "6554684f7f9ed18a39a4d15c",
"collectionId": "6556cd795e4b663e770bb66d",
"sourceName": "GBT 15104-2021 装饰单板贴面人造板.pdf",
"sourceId": "6556cd775e4b663e770bb65c",
"score": 0.8050316572189331
},
......
]
} }
``` ```
### Attribute {{< /markdownify >}}
{{< /tab >}}
|Name|Type|Required|Restrictions|Title|Description| {{< /tabs >}}
|---|---|---|---|---|---|
|defaultIndex|boolean|false|none||是否为默认|
|type|string|true|none||none|
|text|string|true|none||索引文本|
docSite/content/docs/development/upgrading/467.md
...@@ -27,7 +27,8 @@ curl --location --request POST 'https://{{host}}/api/admin/initv467' \ ...@@ -27,7 +27,8 @@ curl --location --request POST 'https://{{host}}/api/admin/initv467' \
1. 修改了知识库UI及新的导入交互方式。 1. 修改了知识库UI及新的导入交互方式。
2. 优化知识库和对话的数据索引。 2. 优化知识库和对话的数据索引。
3. 知识库 openAPI，支持通过 API 操作知识库。（文档待补充） 3. 知识库 openAPI，支持通过 [API 操作知识库](/docs/development/openapi/dataset)。
4. 新增 - 输入框变量提示。输入 { 号后将会获得可用变量提示。根据社区针对高级编排的反馈，我们计划于 2 月份的版本中，优化变量内容，支持模块的局部变量以及更多全局变量写入。 4. 新增 - 输入框变量提示。输入 { 号后将会获得可用变量提示。根据社区针对高级编排的反馈，我们计划于 2 月份的版本中，优化变量内容，支持模块的局部变量以及更多全局变量写入。
5. 修复 - API 对话时，chatId 冲突问题。 5. 优化 - 切换团队后会保存记录，下次登录时优先登录该团队。
6. 修复 - Iframe 嵌入网页可能导致的 window.onLoad 冲突。 6. 修复 - API 对话时，chatId 冲突问题。
\ No newline at end of file 7. 修复 - Iframe 嵌入网页可能导致的 window.onLoad 冲突。
\ No newline at end of file
packages/global/common/error/utils.ts
import { replaceSensitiveLink } from '../string/tools';
export const getErrText = (err: any, def = '') => { export const getErrText = (err: any, def = '') => {
const msg: string = typeof err === 'string' ? err : err?.message || def || ''; const msg: string = typeof err === 'string' ? err : err?.message || def || '';
msg && console.log('error =>', msg); msg && console.log('error =>', msg);
return msg; return replaceSensitiveLink(msg);
}; };
packages/global/common/string/tools.ts
...@@ -38,6 +38,12 @@ export function replaceVariable(text: string, obj: Record<string, string | numbe ...@@ -38,6 +38,12 @@ export function replaceVariable(text: string, obj: Record<string, string | numbe
return text || ''; return text || '';
} }
/* replace sensitive link */
export const replaceSensitiveLink = (text: string) => {
const urlRegex = /(?<=https?:\/\/)[^\s]+/g;
return text.replace(urlRegex, 'xxx');
};
export const getNanoid = (size = 12) => { export const getNanoid = (size = 12) => {
return customAlphabet('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890', size)(); return customAlphabet('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890', size)();
}; };
packages/service/common/file/utils.ts
...@@ -4,7 +4,7 @@ export const removeFilesByPaths = (paths: string[]) => { ...@@ -4,7 +4,7 @@ export const removeFilesByPaths = (paths: string[]) => {
paths.forEach((path) => { paths.forEach((path) => {
fs.unlink(path, (err) => { fs.unlink(path, (err) => {
if (err) { if (err) {
console.error(err); // console.error(err);
} }
}); });
}); });
......
packages/service/common/response/index.ts
...@@ -3,6 +3,7 @@ import { sseResponseEventEnum } from './constant'; ...@@ -3,6 +3,7 @@ import { sseResponseEventEnum } from './constant';
import { proxyError, ERROR_RESPONSE, ERROR_ENUM } from '@fastgpt/global/common/error/errorCode'; import { proxyError, ERROR_RESPONSE, ERROR_ENUM } from '@fastgpt/global/common/error/errorCode';
import { addLog } from '../system/log'; import { addLog } from '../system/log';
import { clearCookie } from '../../support/permission/controller'; import { clearCookie } from '../../support/permission/controller';
import { replaceSensitiveLink } from '@fastgpt/global/common/string/tools';
export interface ResponseType<T = any> { export interface ResponseType<T = any> {
code: number; code: number;
...@@ -52,7 +53,7 @@ export const jsonRes = <T = any>( ...@@ -52,7 +53,7 @@ export const jsonRes = <T = any>(
res.status(code).json({ res.status(code).json({
code, code,
statusText: '', statusText: '',
message: message || msg, message: replaceSensitiveLink(message || msg),
data: data !== undefined ? data : null data: data !== undefined ? data : null
}); });
}; };
...@@ -90,7 +91,7 @@ export const sseErrRes = (res: NextApiResponse, error: any) => { ...@@ -90,7 +91,7 @@ export const sseErrRes = (res: NextApiResponse, error: any) => {
responseWrite({ responseWrite({
res, res,
event: sseResponseEventEnum.error, event: sseResponseEventEnum.error,
data: JSON.stringify({ message: msg }) data: JSON.stringify({ message: replaceSensitiveLink(msg) })
}); });
}; };
......
packages/service/support/permission/controller.ts
...@@ -168,6 +168,10 @@ export async function parseHeaderCert({ ...@@ -168,6 +168,10 @@ export async function parseHeaderCert({
return Promise.reject(ERROR_ENUM.unAuthorization); return Promise.reject(ERROR_ENUM.unAuthorization);
})(); })();
if (!authRoot && (!teamId || !tmbId)) {
return Promise.reject(ERROR_ENUM.unAuthorization);
}
return { return {
userId: String(uid), userId: String(uid),
teamId: String(teamId), teamId: String(teamId),
......
packages/web/components/common/Icon/icons/support/account/loginoutLight.svg
<?xml version="1.0" standalone="no"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1689855121257" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="3135" xmlns:xlink="http://www.w3.org/1999/xlink" width="64" height="64"><path d="M952.7 492.1c-1.4-1.8-3.1-3.4-4.8-4.9l-179-178.9c-12.5-12.5-32.9-12.5-45.4 0s-12.5 32.9 0 45.4l126 126H421.3h-0.1c-18.2 0-32.9 14.8-32.9 33s14.7 33 32.9 33c0.3 0.1 0.5 0 0.7 0h427.8l-126 126c-12.3 12.3-12.3 32.4 0 44.7l0.7 0.7c12.3 12.3 32.4 12.3 44.7 0l182-182c11.7-11.7 12.3-30.6 1.6-43z" fill="#515151" p-id="3136"></path><path d="M562.3 799c-18 0-32.7 14.7-32.7 32.7v63.8H129.2V128.7h400.4v63.1c0 18 14.7 32.7 32.7 32.7s32.7-14.7 32.7-32.7V96.3c0-3.5-0.6-6.8-1.6-10-4.2-13.3-16.6-23-31.2-23H96.6c-18 0-32.7 14.7-32.7 32.7v831.9c0 14.2 9.2 26.3 21.8 30.8 3.6 1.4 7.5 2.1 11.5 2.1h463.2c0.6 0 1.3 0.1 1.9 0.1 18 0 32.7-14.7 32.7-32.7v-96.5c0-18-14.7-32.7-32.7-32.7z" fill="#515151" p-id="3137"></path><path d="M256.8 512.7a32.9 33 0 1 0 65.8 0 32.9 33 0 1 0-65.8 0Z" fill="#515151" p-id="3138"></path></svg> <?xml version="1.0" standalone="no"?>
\ No newline at end of file <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1689855121257"
class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="3135"
xmlns:xlink="http://www.w3.org/1999/xlink" width="64" height="64">
<path
d="M952.7 492.1c-1.4-1.8-3.1-3.4-4.8-4.9l-179-178.9c-12.5-12.5-32.9-12.5-45.4 0s-12.5 32.9 0 45.4l126 126H421.3h-0.1c-18.2 0-32.9 14.8-32.9 33s14.7 33 32.9 33c0.3 0.1 0.5 0 0.7 0h427.8l-126 126c-12.3 12.3-12.3 32.4 0 44.7l0.7 0.7c12.3 12.3 32.4 12.3 44.7 0l182-182c11.7-11.7 12.3-30.6 1.6-43z"
p-id="3136"></path>
<path
d="M562.3 799c-18 0-32.7 14.7-32.7 32.7v63.8H129.2V128.7h400.4v63.1c0 18 14.7 32.7 32.7 32.7s32.7-14.7 32.7-32.7V96.3c0-3.5-0.6-6.8-1.6-10-4.2-13.3-16.6-23-31.2-23H96.6c-18 0-32.7 14.7-32.7 32.7v831.9c0 14.2 9.2 26.3 21.8 30.8 3.6 1.4 7.5 2.1 11.5 2.1h463.2c0.6 0 1.3 0.1 1.9 0.1 18 0 32.7-14.7 32.7-32.7v-96.5c0-18-14.7-32.7-32.7-32.7z"
p-id="3137"></path>
<path d="M256.8 512.7a32.9 33 0 1 0 65.8 0 32.9 33 0 1 0-65.8 0Z" p-id="3138"></path>
</svg>
\ No newline at end of file
projects/app/public/docs/versionIntro.md
...@@ -4,8 +4,9 @@ ...@@ -4,8 +4,9 @@
2. 优化知识库和对话的数据索引，加快数据操作。 2. 优化知识库和对话的数据索引，加快数据操作。
3. 知识库 openAPI，支持通过 API 操作知识库。 3. 知识库 openAPI，支持通过 API 操作知识库。
4. 新增 - 输入框变量提示。输入 { 号后将会获得可用变量提示。根据社区针对高级编排的反馈，我们计划于 2 月份的版本中，优化变量内容，支持模块的局部变量以及更多全局变量写入。 4. 新增 - 输入框变量提示。输入 { 号后将会获得可用变量提示。根据社区针对高级编排的反馈，我们计划于 2 月份的版本中，优化变量内容，支持模块的局部变量以及更多全局变量写入。
5. 修复 - API 对话时，chatId 冲突问题。 5. 优化 - 切换团队后会保存记录，下次登录时优先登录该团队。
6. 修复 - Iframe 嵌入网页可能导致的 window.onLoad 冲突。 6. 修复 - API 对话时，chatId 冲突问题。
7. [使用文档](https://doc.fastgpt.in/docs/intro/) 7. 修复 - Iframe 嵌入网页可能导致的 window.onLoad 冲突。
8. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow) 8. [使用文档](https://doc.fastgpt.in/docs/intro/)
9. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/) 9. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow)
10. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/)
projects/app/public/locales/zh/common.json
...@@ -95,7 +95,7 @@ ...@@ -95,7 +95,7 @@
"Last Step": "上一步", "Last Step": "上一步",
"Last use time": "最后使用时间", "Last use time": "最后使用时间",
"Load Failed": "加载失败", "Load Failed": "加载失败",
"Loading": "加载中", "Loading": "加载中...",
"Max credit": "最大金额", "Max credit": "最大金额",
"Max credit tips": "该链接最大可消耗多少金额，超出后链接将被禁止使用。-1 代表无限制。", "Max credit tips": "该链接最大可消耗多少金额，超出后链接将被禁止使用。-1 代表无限制。",
"More settings": "更多设置", "More settings": "更多设置",
...@@ -541,7 +541,8 @@ ...@@ -541,7 +541,8 @@
"success": "开始同步" "success": "开始同步"
} }
}, },
"training": {} "training": {
}
}, },
"data": { "data": {
"Auxiliary Data": "辅助数据", "Auxiliary Data": "辅助数据",
......
projects/app/src/components/ChatBox/MessageInput.tsx
import { useSpeech } from '@/web/common/hooks/useSpeech'; import { useSpeech } from '@/web/common/hooks/useSpeech';
import { useSystemStore } from '@/web/common/system/useSystemStore'; import { useSystemStore } from '@/web/common/system/useSystemStore';
import { Box, Flex, Image, Spinner, Textarea } from '@chakra-ui/react'; import { Box, Flex, Image, Spinner, Textarea } from '@chakra-ui/react';
import React, { useRef, useEffect, useCallback, useState } from 'react'; import React, { useRef, useEffect, useCallback, useState, useTransition } from 'react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import MyTooltip from '../MyTooltip'; import MyTooltip from '../MyTooltip';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
...@@ -37,7 +37,7 @@ const MessageInput = ({ ...@@ -37,7 +37,7 @@ const MessageInput = ({
showFileSelector = false, showFileSelector = false,
resetInputVal resetInputVal
}: { }: {
onChange: (e: string) => void; onChange?: (e: string) => void;
onSendMessage: (e: string) => void; onSendMessage: (e: string) => void;
onStop: () => void; onStop: () => void;
isChatting: boolean; isChatting: boolean;
...@@ -45,6 +45,8 @@ const MessageInput = ({ ...@@ -45,6 +45,8 @@ const MessageInput = ({
TextareaDom: React.MutableRefObject<HTMLTextAreaElement | null>; TextareaDom: React.MutableRefObject<HTMLTextAreaElement | null>;
resetInputVal: (val: string) => void; resetInputVal: (val: string) => void;
}) => { }) => {
const [, startSts] = useTransition();
const { shareId } = useRouter().query as { shareId?: string }; const { shareId } = useRouter().query as { shareId?: string };
const { const {
isSpeaking, isSpeaking,
...@@ -330,17 +332,29 @@ ${images.map((img) => JSON.stringify({ src: img.src })).join('\n')} ...@@ -330,17 +332,29 @@ ${images.map((img) => JSON.stringify({ src: img.src })).join('\n')}
const textarea = e.target; const textarea = e.target;
textarea.style.height = textareaMinH; textarea.style.height = textareaMinH;
textarea.style.height = `${textarea.scrollHeight}px`; textarea.style.height = `${textarea.scrollHeight}px`;
onChange(textarea.value);
startSts(() => {
onChange?.(textarea.value);
});
}} }}
onKeyDown={(e) => { onKeyDown={(e) => {
// enter send.(pc or iframe && enter and unPress shift) // enter send.(pc or iframe && enter and unPress shift)
if ((isPc || window !== parent) && e.keyCode === 13 && !e.shiftKey) { const isEnter = e.keyCode === 13;
handleSend(); if (isEnter && TextareaDom.current && (e.ctrlKey || e.altKey)) {
e.preventDefault(); TextareaDom.current.value += '\n';
TextareaDom.current.style.height = textareaMinH;
TextareaDom.current.style.height = `${TextareaDom.current.scrollHeight}px`;
return;
} }
// 全选内容 // 全选内容
// @ts-ignore // @ts-ignore
e.key === 'a' && e.ctrlKey && e.target?.select(); e.key === 'a' && e.ctrlKey && e.target?.select();
if ((isPc || window !== parent) && e.keyCode === 13 && !e.shiftKey) {
handleSend();
e.preventDefault();
}
}} }}
onPaste={(e) => { onPaste={(e) => {
const clipboardData = e.clipboardData; const clipboardData = e.clipboardData;
......
projects/app/src/components/ChatBox/index.tsx
...@@ -36,7 +36,7 @@ import { adaptChat2GptMessages } from '@fastgpt/global/core/chat/adapt'; ...@@ -36,7 +36,7 @@ import { adaptChat2GptMessages } from '@fastgpt/global/core/chat/adapt';
import { useMarkdown } from '@/web/common/hooks/useMarkdown'; import { useMarkdown } from '@/web/common/hooks/useMarkdown';
import { ModuleItemType } from '@fastgpt/global/core/module/type.d'; import { ModuleItemType } from '@fastgpt/global/core/module/type.d';
import { VariableInputEnum } from '@fastgpt/global/core/module/constants'; import { VariableInputEnum } from '@fastgpt/global/core/module/constants';
import { useForm } from 'react-hook-form'; import { UseFormReturn, useForm } from 'react-hook-form';
import type { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d'; import type { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d';
import { fileDownload } from '@/web/common/file/utils'; import { fileDownload } from '@/web/common/file/utils';
import { htmlTemplate } from '@/constants/common'; import { htmlTemplate } from '@/constants/common';
...@@ -65,7 +65,7 @@ const SelectMarkCollection = dynamic(() => import('./SelectMarkCollection')); ...@@ -65,7 +65,7 @@ const SelectMarkCollection = dynamic(() => import('./SelectMarkCollection'));
import styles from './index.module.scss'; import styles from './index.module.scss';
import { postQuestionGuide } from '@/web/core/ai/api'; import { postQuestionGuide } from '@/web/core/ai/api';
import { splitGuideModule } from '@fastgpt/global/core/module/utils'; import { splitGuideModule } from '@fastgpt/global/core/module/utils';
import type { AppTTSConfigType } from '@fastgpt/global/core/module/type.d'; import type { AppTTSConfigType, VariableItemType } from '@fastgpt/global/core/module/type.d';
import MessageInput from './MessageInput'; import MessageInput from './MessageInput';
import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants'; import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants';
import ChatBoxDivider from '../core/chat/Divider'; import ChatBoxDivider from '../core/chat/Divider';
...@@ -98,6 +98,15 @@ enum FeedbackTypeEnum { ...@@ -98,6 +98,15 @@ enum FeedbackTypeEnum {
hidden = 'hidden' hidden = 'hidden'
} }
const MessageCardStyle: BoxProps = {
px: 4,
py: 3,
borderRadius: '0 8px 8px 8px',
boxShadow: '0 0 8px rgba(0,0,0,0.15)',
display: 'inline-block',
maxW: ['calc(100% - 25px)', 'calc(100% - 40px)']
};
type Props = { type Props = {
feedbackType?: `${FeedbackTypeEnum}`; feedbackType?: `${FeedbackTypeEnum}`;
showMarkIcon?: boolean; // admin mark dataset showMarkIcon?: boolean; // admin mark dataset
...@@ -157,7 +166,6 @@ const ChatBox = ( ...@@ -157,7 +166,6 @@ const ChatBox = (
const isNewChatReplace = useRef(false); const isNewChatReplace = useRef(false);
const [refresh, setRefresh] = useState(false); const [refresh, setRefresh] = useState(false);
const [variables, setVariables] = useState<Record<string, any>>({}); // settings variable
const [chatHistory, setChatHistory] = useState<ChatSiteItemType[]>([]); const [chatHistory, setChatHistory] = useState<ChatSiteItemType[]>([]);
const [feedbackId, setFeedbackId] = useState<string>(); const [feedbackId, setFeedbackId] = useState<string>();
const [readFeedbackData, setReadFeedbackData] = useState<{ const [readFeedbackData, setReadFeedbackData] = useState<{
...@@ -180,7 +188,17 @@ const ChatBox = ( ...@@ -180,7 +188,17 @@ const ChatBox = (
); );
// compute variable input is finish. // compute variable input is finish.
const [variableInputFinish, setVariableInputFinish] = useState(false); const chatForm = useForm<{
variables: Record<string, any>;
}>({
defaultValues: {
variables: {}
}
});
const { setValue, watch, handleSubmit } = chatForm;
const variables = watch('variables');
const [variableInputFinish, setVariableInputFinish] = useState(false); // clicked start chat button
const variableIsFinish = useMemo(() => { const variableIsFinish = useMemo(() => {
if (!variableModules || variableModules.length === 0 || chatHistory.length > 0) return true; if (!variableModules || variableModules.length === 0 || chatHistory.length > 0) return true;
...@@ -194,21 +212,15 @@ const ChatBox = ( ...@@ -194,21 +212,15 @@ const ChatBox = (
return variableInputFinish; return variableInputFinish;
}, [chatHistory.length, variableInputFinish, variableModules, variables]); }, [chatHistory.length, variableInputFinish, variableModules, variables]);
const { register, reset, getValues, setValue, handleSubmit } = useForm<Record<string, any>>({
defaultValues: variables
});
// 滚动到底部 // 滚动到底部
const scrollToBottom = useCallback( const scrollToBottom = (behavior: 'smooth' | 'auto' = 'smooth') => {
(behavior: 'smooth' | 'auto' = 'smooth') => { if (!ChatBoxRef.current) return;
if (!ChatBoxRef.current) return; ChatBoxRef.current.scrollTo({
ChatBoxRef.current.scrollTo({ top: ChatBoxRef.current.scrollHeight,
top: ChatBoxRef.current.scrollHeight, behavior
behavior });
}); };
},
[ChatBoxRef]
);
// 聊天信息生成中……获取当前滚动条位置，判断是否需要滚动到底部 // 聊天信息生成中……获取当前滚动条位置，判断是否需要滚动到底部
const generatingScroll = useCallback( const generatingScroll = useCallback(
throttle(() => { throttle(() => {
...@@ -222,28 +234,31 @@ const ChatBox = ( ...@@ -222,28 +234,31 @@ const ChatBox = (
[] []
); );
// eslint-disable-next-line react-hooks/exhaustive-deps // eslint-disable-next-line react-hooks/exhaustive-deps
const generatingMessage = ({ text = '', status, name }: generatingMessageProps) => { const generatingMessage = useCallback(
setChatHistory((state) => ({ text = '', status, name }: generatingMessageProps) => {
state.map((item, index) => { setChatHistory((state) =>
if (index !== state.length - 1) return item; state.map((item, index) => {
return { if (index !== state.length - 1) return item;
...item, return {
...(text ...item,
? { ...(text
value: item.value + text ? {
} value: item.value + text
: {}), }
...(status && name : {}),
? { ...(status && name
status, ? {
moduleName: name status,
} moduleName: name
: {}) }
}; : {})
}) };
); })
generatingScroll(); );
}; generatingScroll();
},
[generatingScroll]
);
// 重置输入内容 // 重置输入内容
const resetInputVal = useCallback((val: string) => { const resetInputVal = useCallback((val: string) => {
...@@ -284,149 +299,157 @@ const ChatBox = ( ...@@ -284,149 +299,157 @@ const ChatBox = (
} }
} catch (error) {} } catch (error) {}
}, },
[questionGuide, scrollToBottom, shareId] [questionGuide, shareId]
); );
/** /**
* user confirm send prompt * user confirm send prompt
*/ */
const sendPrompt = useCallback( const sendPrompt = useCallback(
async (variables: Record<string, any> = {}, inputVal = '', history = chatHistory) => { ({
if (!onStartChat) return; inputVal = '',
if (isChatting) { history = chatHistory
toast({ }: {
title: '正在聊天中...请等待结束', inputVal?: string;
status: 'warning' history?: ChatSiteItemType[];
}); }) => {
return; handleSubmit(async ({ variables }) => {
} if (!onStartChat) return;
questionGuideController.current?.abort('stop'); if (isChatting) {
// get input value toast({
const val = inputVal.trim(); title: '正在聊天中...请等待结束',
status: 'warning'
if (!val) { });
toast({ return;
title: '内容为空', }
status: 'warning' questionGuideController.current?.abort('stop');
}); // get input value
return; const val = inputVal.trim();
}
if (!val) {
const newChatList: ChatSiteItemType[] = [ toast({
...history, title: '内容为空',
{ status: 'warning'
dataId: nanoid(), });
obj: 'Human', return;
value: val,
status: 'finish'
},
{
dataId: nanoid(),
obj: 'AI',
value: '',
status: 'loading'
} }
];
// 插入内容
setChatHistory(newChatList);
// 清空输入内容 const newChatList: ChatSiteItemType[] = [
resetInputVal(''); ...history,
setQuestionGuide([]); {
setTimeout(() => { dataId: nanoid(),
scrollToBottom(); obj: 'Human',
}, 100); value: val,
try { status: 'finish'
// create abort obj },
const abortSignal = new AbortController(); {
chatController.current = abortSignal; dataId: nanoid(),
obj: 'AI',
const messages = adaptChat2GptMessages({ messages: newChatList, reserveId: true }); value: '',
status: 'loading'
const { }
responseData, ];
responseText,
isNewChat = false
} = await onStartChat({
chatList: newChatList.map((item) => ({
dataId: item.dataId,
obj: item.obj,
value: item.value,
status: item.status,
moduleName: item.moduleName
})),
messages,
controller: abortSignal,
generatingMessage,
variables
});
isNewChatReplace.current = isNewChat; // 插入内容
setChatHistory(newChatList);
// set finish status
setChatHistory((state) =>
state.map((item, index) => {
if (index !== state.length - 1) return item;
return {
...item,
status: 'finish',
responseData
};
})
);
// 清空输入内容
resetInputVal('');
setQuestionGuide([]);
setTimeout(() => { setTimeout(() => {
createQuestionGuide({ scrollToBottom();
history: newChatList.map((item, i) =>
i === newChatList.length - 1
? {
...item,
value: responseText
}
: item
)
});
generatingScroll();
isPc && TextareaDom.current?.focus();
}, 100); }, 100);
} catch (err: any) { try {
toast({ // create abort obj
title: t(getErrText(err, 'core.chat.error.Chat error')), const abortSignal = new AbortController();
status: 'error', chatController.current = abortSignal;
duration: 5000,
isClosable: true const messages = adaptChat2GptMessages({ messages: newChatList, reserveId: true });
});
const {
responseData,
responseText,
isNewChat = false
} = await onStartChat({
chatList: newChatList.map((item) => ({
dataId: item.dataId,
obj: item.obj,
value: item.value,
status: item.status,
moduleName: item.moduleName
})),
messages,
controller: abortSignal,
generatingMessage,
variables
});
if (!err?.responseText) { isNewChatReplace.current = isNewChat;
resetInputVal(inputVal);
setChatHistory(newChatList.slice(0, newChatList.length - 2));
}
// set finish status // set finish status
setChatHistory((state) => setChatHistory((state) =>
state.map((item, index) => { state.map((item, index) => {
if (index !== state.length - 1) return item; if (index !== state.length - 1) return item;
return { return {
...item, ...item,
status: 'finish' status: 'finish',
}; responseData
}) };
); })
} );
setTimeout(() => {
createQuestionGuide({
history: newChatList.map((item, i) =>
i === newChatList.length - 1
? {
...item,
value: responseText
}
: item
)
});
generatingScroll();
isPc && TextareaDom.current?.focus();
}, 100);
} catch (err: any) {
toast({
title: t(getErrText(err, 'core.chat.error.Chat error')),
status: 'error',
duration: 5000,
isClosable: true
});
if (!err?.responseText) {
resetInputVal(inputVal);
setChatHistory(newChatList.slice(0, newChatList.length - 2));
}
// set finish status
setChatHistory((state) =>
state.map((item, index) => {
if (index !== state.length - 1) return item;
return {
...item,
status: 'finish'
};
})
);
}
})();
}, },
[ [
chatHistory, chatHistory,
onStartChat,
isChatting,
resetInputVal,
toast,
scrollToBottom,
generatingMessage,
createQuestionGuide, createQuestionGuide,
generatingMessage,
generatingScroll, generatingScroll,
handleSubmit,
isChatting,
isPc, isPc,
t onStartChat,
resetInputVal,
t,
toast
] ]
); );
...@@ -444,11 +467,14 @@ const ChatBox = ( ...@@ -444,11 +467,14 @@ const ChatBox = (
); );
setChatHistory((state) => (index === 0 ? [] : state.slice(0, index))); setChatHistory((state) => (index === 0 ? [] : state.slice(0, index)));
sendPrompt(variables, delHistory[0].value, chatHistory.slice(0, index)); sendPrompt({
inputVal: delHistory[0].value,
history: chatHistory.slice(0, index)
});
} catch (error) {} } catch (error) {}
setLoading(false); setLoading(false);
}, },
[chatHistory, onDelMessage, sendPrompt, setLoading, variables] [chatHistory, onDelMessage, sendPrompt, setLoading]
); );
// delete one message // delete one message
const delOneMessage = useCallback( const delOneMessage = useCallback(
...@@ -471,27 +497,21 @@ const ChatBox = ( ...@@ -471,27 +497,21 @@ const ChatBox = (
defaultVal[item.key] = ''; defaultVal[item.key] = '';
}); });
reset(e || defaultVal); setValue('variables', e || defaultVal);
setVariables(e || defaultVal);
}, },
resetHistory(e) { resetHistory(e) {
setVariableInputFinish(!!e.length); setVariableInputFinish(!!e.length);
setChatHistory(e); setChatHistory(e);
}, },
scrollToBottom, scrollToBottom,
sendPrompt: (question: string) => handleSubmit((item) => sendPrompt(item, question))() sendPrompt: (question: string) => {
sendPrompt({
inputVal: question
});
}
})); }));
/* style start */ /* style start */
const MessageCardStyle: BoxProps = {
px: 4,
py: 3,
borderRadius: '0 8px 8px 8px',
boxShadow: '0 0 8px rgba(0,0,0,0.15)',
display: 'inline-block',
maxW: ['calc(100% - 25px)', 'calc(100% - 40px)']
};
const showEmpty = useMemo( const showEmpty = useMemo(
() => () =>
feConfigs?.show_emptyChat && feConfigs?.show_emptyChat &&
...@@ -534,14 +554,18 @@ const ChatBox = ( ...@@ -534,14 +554,18 @@ const ChatBox = (
useEffect(() => { useEffect(() => {
const windowMessage = ({ data }: MessageEvent<{ type: 'sendPrompt'; text: string }>) => { const windowMessage = ({ data }: MessageEvent<{ type: 'sendPrompt'; text: string }>) => {
if (data?.type === 'sendPrompt' && data?.text) { if (data?.type === 'sendPrompt' && data?.text) {
handleSubmit((item) => sendPrompt(item, data.text))(); sendPrompt({
inputVal: data.text
});
} }
}; };
window.addEventListener('message', windowMessage); window.addEventListener('message', windowMessage);
eventBus.on(EventNameEnum.sendQuestion, ({ text }: { text: string }) => { eventBus.on(EventNameEnum.sendQuestion, ({ text }: { text: string }) => {
if (!text) return; if (!text) return;
handleSubmit((data) => sendPrompt(data, text))(); sendPrompt({
inputVal: text
});
}); });
eventBus.on(EventNameEnum.editQuestion, ({ text }: { text: string }) => { eventBus.on(EventNameEnum.editQuestion, ({ text }: { text: string }) => {
if (!text) return; if (!text) return;
...@@ -553,140 +577,81 @@ const ChatBox = ( ...@@ -553,140 +577,81 @@ const ChatBox = (
eventBus.off(EventNameEnum.sendQuestion); eventBus.off(EventNameEnum.sendQuestion);
eventBus.off(EventNameEnum.editQuestion); eventBus.off(EventNameEnum.editQuestion);
}; };
}, [handleSubmit, resetInputVal, sendPrompt]); }, [resetInputVal, sendPrompt]);
const onSubmitVariables = useCallback(
(data: Record<string, any>) => {
setVariableInputFinish(true);
onUpdateVariable?.(data);
},
[onUpdateVariable]
);
const HumanChatCard = useCallback(
({ item, index }: { item: ChatSiteItemType; index: number }) => {
return (
<>
{/* control icon */}
<Flex w={'100%'} alignItems={'center'} justifyContent={'flex-end'}>
<ChatControllerComponent
chat={item}
onDelete={
onDelMessage
? () => {
delOneMessage({ dataId: item.dataId, index });
}
: undefined
}
onRetry={useCallback(() => retryInput(index), [index])}
/>
<ChatAvatar src={userAvatar} type={'Human'} />
</Flex>
{/* content */}
<Box mt={['6px', 2]} textAlign={'right'}>
<Card
className="markdown"
{...MessageCardStyle}
bg={'primary.200'}
borderRadius={'8px 0 8px 8px'}
textAlign={'left'}
>
<Markdown source={item.value} isChatting={false} />
</Card>
</Box>
</>
);
},
[]
);
return ( return (
<Flex flexDirection={'column'} h={'100%'}> <Flex flexDirection={'column'} h={'100%'}>
<Script src="/js/html2pdf.bundle.min.js" strategy="lazyOnload"></Script> <Script src="/js/html2pdf.bundle.min.js" strategy="lazyOnload"></Script>
{/* chat box container */} {/* chat box container */}
<Box ref={ChatBoxRef} flex={'1 0 0'} h={0} w={'100%'} overflow={'overlay'} px={[4, 0]} pb={3}> <Box ref={ChatBoxRef} flex={'1 0 0'} h={0} w={'100%'} overflow={'overlay'} px={[4, 0]} pb={3}>
<Box id="chat-container" maxW={['100%', '92%']} h={'100%'} mx={'auto'}> <Box id="chat-container" maxW={['100%', '92%']} h={'100%'} mx={'auto'}>
{showEmpty && <Empty />} {showEmpty && <Empty />}
{!!welcomeText && <WelcomeText appAvatar={appAvatar} welcomeText={welcomeText} />}
{!!welcomeText && (
<Box py={3}>
{/* avatar */}
<ChatAvatar src={appAvatar} type={'AI'} />
{/* message */}
<Box textAlign={'left'}>
<Card order={2} mt={2} {...MessageCardStyle} bg={'white'}>
<Markdown source={`~~~guide \n${welcomeText}`} isChatting={false} />
</Card>
</Box>
</Box>
)}
{/* variable input */} {/* variable input */}
{!!variableModules?.length && ( {!!variableModules?.length && (
<Box py={3}> <VariableInput
{/* avatar */} appAvatar={appAvatar}
<ChatAvatar src={appAvatar} type={'AI'} /> variableModules={variableModules}
{/* message */} variableIsFinish={variableIsFinish}
<Box textAlign={'left'}> chatForm={chatForm}
<Card order={2} mt={2} bg={'white'} w={'400px'} {...MessageCardStyle}> onSubmitVariables={onSubmitVariables}
{variableModules.map((item) => ( />
<Box key={item.id} mb={4}>
<VariableLabel required={item.required}>{item.label}</VariableLabel>
{item.type === VariableInputEnum.input && (
<Input
isDisabled={variableIsFinish}
bg={'myWhite.400'}
{...register(item.key, {
required: item.required
})}
/>
)}
{item.type === VariableInputEnum.textarea && (
<Textarea
isDisabled={variableIsFinish}
bg={'myWhite.400'}
{...register(item.key, {
required: item.required
})}
rows={5}
maxLength={4000}
/>
)}
{item.type === VariableInputEnum.select && (
<MySelect
width={'100%'}
isDisabled={variableIsFinish}
list={(item.enums || []).map((item) => ({
label: item.value,
value: item.value
}))}
{...register(item.key, {
required: item.required
})}
value={getValues(item.key)}
onchange={(e) => {
setValue(item.key, e);
setRefresh(!refresh);
}}
/>
)}
</Box>
))}
{!variableIsFinish && (
<Button
leftIcon={<MyIcon name={'core/chat/chatFill'} w={'16px'} />}
size={'sm'}
maxW={'100px'}
onClick={handleSubmit((data) => {
onUpdateVariable?.(data);
setVariables(data);
setVariableInputFinish(true);
})}
>
{t('core.chat.Start Chat')}
</Button>
)}
</Card>
</Box>
</Box>
)} )}
{/* chat history */} {/* chat history */}
<Box id={'history'}> <Box id={'history'}>
{chatHistory.map((item, index) => ( {chatHistory.map((item, index) => (
<Box key={item.dataId} py={5}> <Box key={item.dataId} py={5}>
{item.obj === 'Human' && ( {item.obj === 'Human' && <HumanChatCard item={item} index={index} />}
<>
{/* control icon */}
<Flex w={'100%'} alignItems={'center'} justifyContent={'flex-end'}>
<ChatController
chat={item}
onDelete={
onDelMessage
? () => {
delOneMessage({ dataId: item.dataId, index });
}
: undefined
}
onRetry={() => retryInput(index)}
/>
<ChatAvatar src={userAvatar} type={'Human'} />
</Flex>
{/* content */}
<Box mt={['6px', 2]} textAlign={'right'}>
<Card
className="markdown"
{...MessageCardStyle}
bg={'primary.200'}
borderRadius={'8px 0 8px 8px'}
textAlign={'left'}
>
<Markdown source={item.value} isChatting={false} />
</Card>
</Box>
</>
)}
{item.obj === 'AI' && ( {item.obj === 'AI' && (
<> <>
{/* control icon */}
<Flex w={'100%'} alignItems={'center'}> <Flex w={'100%'} alignItems={'center'}>
<ChatAvatar src={appAvatar} type={'AI'} /> <ChatAvatar src={appAvatar} type={'AI'} />
<ChatController {/* control icon */}
<ChatControllerComponent
ml={2} ml={2}
chat={item} chat={item}
setChatHistory={setChatHistory} setChatHistory={setChatHistory}
...@@ -723,36 +688,35 @@ const ChatBox = ( ...@@ -723,36 +688,35 @@ const ChatBox = (
} }
: undefined : undefined
} }
onAddUserLike={(() => { onAddUserLike={
if (feedbackType !== FeedbackTypeEnum.user || item.userBadFeedback) { feedbackType !== FeedbackTypeEnum.user || item.userBadFeedback
return; ? undefined
} : () => {
return () => { if (!item.dataId || !chatId || !appId) return;
if (!item.dataId || !chatId || !appId) return;
const isGoodFeedback = !!item.userGoodFeedback;
const isGoodFeedback = !!item.userGoodFeedback; setChatHistory((state) =>
setChatHistory((state) => state.map((chatItem) =>
state.map((chatItem) => chatItem.dataId === item.dataId
chatItem.dataId === item.dataId ? {
? { ...chatItem,
...chatItem, userGoodFeedback: isGoodFeedback ? undefined : 'yes'
userGoodFeedback: isGoodFeedback ? undefined : 'yes' }
} : chatItem
: chatItem )
) );
); try {
try { updateChatUserFeedback({
updateChatUserFeedback({ appId,
appId, chatId,
chatId, chatItemId: item.dataId,
chatItemId: item.dataId, shareId,
shareId, outLinkUid,
outLinkUid, userGoodFeedback: isGoodFeedback ? undefined : 'yes'
userGoodFeedback: isGoodFeedback ? undefined : 'yes' });
}); } catch (error) {}
} catch (error) {} }
}; }
})()}
onCloseUserLike={ onCloseUserLike={
feedbackType === FeedbackTypeEnum.admin feedbackType === FeedbackTypeEnum.admin
? () => { ? () => {
...@@ -931,13 +895,12 @@ const ChatBox = ( ...@@ -931,13 +895,12 @@ const ChatBox = (
</Box> </Box>
</Box> </Box>
{/* message input */} {/* message input */}
{onStartChat && variableIsFinish && active ? ( {onStartChat && variableIsFinish && active && (
<MessageInput <MessageInput
onChange={(e) => { onSendMessage={(inputVal) => {
setRefresh(!refresh); sendPrompt({
}} inputVal
onSendMessage={(e) => { });
handleSubmit((data) => sendPrompt(data, e))();
}} }}
onStop={() => chatController.current?.abort('stop')} onStop={() => chatController.current?.abort('stop')}
isChatting={isChatting} isChatting={isChatting}
...@@ -945,7 +908,7 @@ const ChatBox = ( ...@@ -945,7 +908,7 @@ const ChatBox = (
resetInputVal={resetInputVal} resetInputVal={resetInputVal}
showFileSelector={showFileSelector} showFileSelector={showFileSelector}
/> />
) : null} )}
{/* user feedback modal */} {/* user feedback modal */}
{!!feedbackId && chatId && appId && ( {!!feedbackId && chatId && appId && (
<FeedbackModal <FeedbackModal
...@@ -1115,30 +1078,125 @@ export const useChatBox = () => { ...@@ -1115,30 +1078,125 @@ export const useChatBox = () => {
}; };
}; };
function VariableLabel({ const WelcomeText = React.memo(function Welcome({
required = false, appAvatar,
children welcomeText
}: { }: {
required?: boolean; appAvatar?: string;
children: React.ReactNode | string; welcomeText: string;
}) { }) {
return ( return (
<Box as={'label'} display={'inline-block'} position={'relative'} mb={1}> <Box py={3}>
{children} {/* avatar */}
{required && ( <ChatAvatar src={appAvatar} type={'AI'} />
<Box {/* message */}
position={'absolute'} <Box textAlign={'left'}>
top={'-2px'} <Card order={2} mt={2} {...MessageCardStyle} bg={'white'}>
right={'-10px'} <Markdown source={`~~~guide \n${welcomeText}`} isChatting={false} />
color={'red.500'} </Card>
fontWeight={'bold'} </Box>
>
*
</Box>
)}
</Box> </Box>
); );
} });
const VariableInput = React.memo(function VariableInput({
appAvatar,
variableModules,
variableIsFinish,
chatForm,
onSubmitVariables
}: {
appAvatar?: string;
variableModules: VariableItemType[];
variableIsFinish: boolean;
onSubmitVariables: (e: Record<string, any>) => void;
chatForm: UseFormReturn<{
variables: Record<string, any>;
}>;
}) {
const { t } = useTranslation();
const { register, setValue, handleSubmit: handleSubmitChat, watch } = chatForm;
const variables = watch('variables');
return (
<Box py={3}>
{/* avatar */}
<ChatAvatar src={appAvatar} type={'AI'} />
{/* message */}
<Box textAlign={'left'}>
<Card order={2} mt={2} bg={'white'} w={'400px'} {...MessageCardStyle}>
{variableModules.map((item) => (
<Box key={item.id} mb={4}>
<Box as={'label'} display={'inline-block'} position={'relative'} mb={1}>
{item.label}
{item.required && (
<Box
position={'absolute'}
top={'-2px'}
right={'-10px'}
color={'red.500'}
fontWeight={'bold'}
>
*
</Box>
)}
</Box>
{item.type === VariableInputEnum.input && (
<Input
isDisabled={variableIsFinish}
bg={'myWhite.400'}
{...register(`variables.${item.key}`, {
required: item.required
})}
/>
)}
{item.type === VariableInputEnum.textarea && (
<Textarea
isDisabled={variableIsFinish}
bg={'myWhite.400'}
{...register(`variables.${item.key}`, {
required: item.required
})}
rows={5}
maxLength={4000}
/>
)}
{item.type === VariableInputEnum.select && (
<MySelect
width={'100%'}
isDisabled={variableIsFinish}
list={(item.enums || []).map((item) => ({
label: item.value,
value: item.value
}))}
{...register(`variables.${item.key}`, {
required: item.required
})}
value={variables[item.key]}
onchange={(e) => {
setValue(`variables.${item.key}`, e);
}}
/>
)}
</Box>
))}
{!variableIsFinish && (
<Button
leftIcon={<MyIcon name={'core/chat/chatFill'} w={'16px'} />}
size={'sm'}
maxW={'100px'}
onClick={handleSubmitChat((data) => {
onSubmitVariables(data);
})}
>
{t('core.chat.Start Chat')}
</Button>
)}
</Card>
</Box>
</Box>
);
});
function ChatAvatar({ src, type }: { src?: string; type: 'Human' | 'AI' }) { function ChatAvatar({ src, type }: { src?: string; type: 'Human' | 'AI' }) {
const theme = useTheme(); const theme = useTheme();
return ( return (
...@@ -1173,7 +1231,7 @@ function Empty() { ...@@ -1173,7 +1231,7 @@ function Empty() {
); );
} }
function ChatController({ const ChatControllerComponent = React.memo(function ChatControllerComponent({
chat, chat,
setChatHistory, setChatHistory,
display, display,
...@@ -1226,7 +1284,7 @@ function ChatController({ ...@@ -1226,7 +1284,7 @@ function ChatController({
return ( return (
<Flex {...controlContainerStyle} ml={ml} mr={mr} display={display}> <Flex {...controlContainerStyle} ml={ml} mr={mr} display={display}>
<MyTooltip label={'复制'}> <MyTooltip label={t('common.Copy')}>
<MyIcon <MyIcon
{...controlIconStyle} {...controlIconStyle}
name={'copy'} name={'copy'}
...@@ -1246,7 +1304,7 @@ function ChatController({ ...@@ -1246,7 +1304,7 @@ function ChatController({
/> />
</MyTooltip> </MyTooltip>
)} )}
<MyTooltip label={'删除'}> <MyTooltip label={t('common.Delete')}>
<MyIcon <MyIcon
{...controlIconStyle} {...controlIconStyle}
name={'delete'} name={'delete'}
...@@ -1259,7 +1317,7 @@ function ChatController({ ...@@ -1259,7 +1317,7 @@ function ChatController({
{showVoiceIcon && {showVoiceIcon &&
hasAudio && hasAudio &&
(audioLoading ? ( (audioLoading ? (
<MyTooltip label={'加载中...'}> <MyTooltip label={t('common.Loading')}>
<MyIcon {...controlIconStyle} name={'common/loading'} /> <MyIcon {...controlIconStyle} name={'common/loading'} />
</MyTooltip> </MyTooltip>
) : audioPlaying ? ( ) : audioPlaying ? (
...@@ -1372,4 +1430,4 @@ function ChatController({ ...@@ -1372,4 +1430,4 @@ function ChatController({
)} )}
</Flex> </Flex>
); );
} });
projects/app/src/components/Markdown/index.tsx
...@@ -35,36 +35,79 @@ export enum CodeClassName { ...@@ -35,36 +35,79 @@ export enum CodeClassName {
img = 'img' img = 'img'
} }
function Code({ inline, className, children }: any) { const Markdown = ({ source, isChatting = false }: { source: string; isChatting?: boolean }) => {
const match = /language-(\w+)/.exec(className || ''); const components = useMemo<any>(
const codeType = match?.[1]; () => ({
img: Image,
pre: 'div',
p: (pProps: any) => <p {...pProps} dir="auto" />,
code: Code,
a: A
}),
[]
);
if (codeType === CodeClassName.mermaid) { const formatSource = source
return <MermaidCodeBlock code={String(children)} />; .replace(/\\n/g, '\n&nbsp;')
} .replace(/(http[s]?:\/\/[^\s，。]+)([。，])/g, '$1 $2')
.replace(/\n*(\[QUOTE SIGN\]\(.*\))/g, '$1');
if (codeType === CodeClassName.guide) {
return <ChatGuide text={String(children)} />;
}
if (codeType === CodeClassName.questionGuide) {
return <QuestionGuide text={String(children)} />;
}
if (codeType === CodeClassName.echarts) {
return <EChartsCodeBlock code={String(children)} />;
}
if (codeType === CodeClassName.img) {
return <ImageBlock images={String(children)} />;
}
return ( return (
<CodeLight className={className} inline={inline} match={match}> <ReactMarkdown
{children} className={`markdown ${styles.markdown}
</CodeLight> ${isChatting ? `${formatSource ? styles.waitingAnimation : styles.animation}` : ''}
`}
remarkPlugins={[RemarkMath, RemarkGfm, RemarkBreaks]}
rehypePlugins={[RehypeKatex]}
components={components}
linkTarget={'_blank'}
>
{formatSource}
</ReactMarkdown>
); );
} };
function Image({ src }: { src?: string }) {
export default React.memo(Markdown);
const Code = React.memo(function Code(e: any) {
const { inline, className, children } = e;
const match = /language-(\w+)/.exec(className || '');
const codeType = match?.[1];
const strChildren = String(children);
const Component = useMemo(() => {
if (codeType === CodeClassName.mermaid) {
return <MermaidCodeBlock code={strChildren} />;
}
if (codeType === CodeClassName.guide) {
return <ChatGuide text={strChildren} />;
}
if (codeType === CodeClassName.questionGuide) {
return <QuestionGuide text={strChildren} />;
}
if (codeType === CodeClassName.echarts) {
return <EChartsCodeBlock code={strChildren} />;
}
if (codeType === CodeClassName.img) {
return <ImageBlock images={strChildren} />;
}
return (
<CodeLight className={className} inline={inline} match={match}>
{children}
</CodeLight>
);
}, [codeType, className, inline, match, strChildren]);
return Component;
});
const Image = React.memo(function Image({ src }: { src?: string }) {
return <MdImage src={src} />; return <MdImage src={src} />;
} });
function A({ children, ...props }: any) { const A = React.memo(function A({ children, ...props }: any) {
const { t } = useTranslation(); const { t } = useTranslation();
// empty href link // empty href link
...@@ -109,38 +152,4 @@ function A({ children, ...props }: any) { ...@@ -109,38 +152,4 @@ function A({ children, ...props }: any) {
} }
return <Link {...props}>{children}</Link>; return <Link {...props}>{children}</Link>;
} });
const Markdown = ({ source, isChatting = false }: { source: string; isChatting?: boolean }) => {
const components = useMemo<any>(
() => ({
img: Image,
pre: 'div',
p: (pProps: any) => <p {...pProps} dir="auto" />,
code: Code,
a: A
}),
[]
);
const formatSource = source
.replace(/\\n/g, '\n&nbsp;')
.replace(/(http[s]?:\/\/[^\s，。]+)([。，])/g, '$1 $2')
.replace(/\n*(\[QUOTE SIGN\]\(.*\))/g, '$1');
return (
<ReactMarkdown
className={`markdown ${styles.markdown}
${isChatting ? `${formatSource ? styles.waitingAnimation : styles.animation}` : ''}
`}
remarkPlugins={[RemarkMath, RemarkGfm, RemarkBreaks]}
rehypePlugins={[RehypeKatex]}
components={components}
linkTarget={'_blank'}
>
{formatSource}
</ReactMarkdown>
);
};
export default React.memo(Markdown);
projects/app/src/components/common/Textarea/TagTextarea.tsx
...@@ -79,6 +79,8 @@ const TagTextarea = ({ defaultValues, onUpdate, ...props }: Props) => { ...@@ -79,6 +79,8 @@ const TagTextarea = ({ defaultValues, onUpdate, ...props }: Props) => {
ref={InputRef} ref={InputRef}
variant={'unstyled'} variant={'unstyled'}
display={'inline-block'} display={'inline-block'}
h={'24px'}
borderRadius={'none'}
w="auto" w="auto"
onBlur={(e) => { onBlur={(e) => {
const value = e.target.value; const value = e.target.value;
......
projects/app/src/components/core/module/AIChatSettingsModal.tsx
...@@ -66,7 +66,6 @@ const AIChatSettingsModal = ({ ...@@ -66,7 +66,6 @@ const AIChatSettingsModal = ({
}, [getValues]); }, [getValues]);
const quoteTemplateVariables = (() => [ const quoteTemplateVariables = (() => [
...pickerMenu,
{ {
key: 'q', key: 'q',
label: 'q', label: 'q',
...@@ -91,15 +90,21 @@ const AIChatSettingsModal = ({ ...@@ -91,15 +90,21 @@ const AIChatSettingsModal = ({
key: 'index', key: 'index',
label: t('core.dataset.search.Quote index'), label: t('core.dataset.search.Quote index'),
icon: 'core/app/simpleMode/variable' icon: 'core/app/simpleMode/variable'
} },
...pickerMenu
])(); ])();
const quotePromptVariables = (() => [ const quotePromptVariables = (() => [
...pickerMenu,
{ {
key: 'quote', key: 'quote',
label: t('core.app.Quote templates'), label: t('core.app.Quote templates'),
icon: 'core/app/simpleMode/variable' icon: 'core/app/simpleMode/variable'
} },
{
key: 'question',
label: t('core.module.input.label.user question'),
icon: 'core/app/simpleMode/variable'
},
...pickerMenu
])(); ])();
const LabelStyles: BoxProps = { const LabelStyles: BoxProps = {
......
projects/app/src/components/support/user/team/TeamManageModal/InviteModal.tsx
...@@ -55,11 +55,13 @@ const InviteModal = ({ ...@@ -55,11 +55,13 @@ const InviteModal = ({
openConfirm( openConfirm(
() => onClose(), () => onClose(),
undefined, undefined,
t('user.team.Invite Member Success Tip', { <Box whiteSpace={'pre-wrap'}>
success: res.invite.length, {t('user.team.Invite Member Success Tip', {
inValid: res.inValid.map((item) => item.username).join(', '), success: res.invite.length,
inTeam: res.inTeam.map((item) => item.username).join(', ') inValid: res.inValid.map((item) => item.username).join(', '),
}) inTeam: res.inTeam.map((item) => item.username).join(', ')
})}
</Box>
)(); )();
}, },
errorToast: t('user.team.Invite Member Failed Tip') errorToast: t('user.team.Invite Member Failed Tip')
......
projects/app/src/components/support/user/team/TeamManageModal/index.tsx
...@@ -75,7 +75,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -75,7 +75,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
const { mutate: onSwitchTeam, isLoading: isSwitchTeam } = useRequest({ const { mutate: onSwitchTeam, isLoading: isSwitchTeam } = useRequest({
mutationFn: async (teamId: string) => { mutationFn: async (teamId: string) => {
const token = await putSwitchTeam(teamId); const token = await putSwitchTeam(teamId);
setToken(token); token && setToken(token);
return initUserInfo(); return initUserInfo();
}, },
errorToast: t('user.team.Switch Team Failed') errorToast: t('user.team.Switch Team Failed')
...@@ -286,13 +286,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -286,13 +286,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
size="sm" size="sm"
borderRadius={'md'} borderRadius={'md'}
ml={3} ml={3}
leftIcon={ leftIcon={<MyIcon name={'support/account/loginoutLight'} w={'14px'} />}
<MyIcon
name={'support/account/loginoutLight'}
w={'14px'}
color={'primary.500'}
/>
}
onClick={() => { onClick={() => {
openLeaveConfirm(() => onLeaveTeam(userInfo?.team?.teamId))(); openLeaveConfirm(() => onLeaveTeam(userInfo?.team?.teamId))();
}} }}
......
projects/app/src/pages/account/components/Info.tsx
...@@ -271,28 +271,32 @@ const UserInfo = () => { ...@@ -271,28 +271,32 @@ const UserInfo = () => {
)} )}
</Flex> </Flex>
</Box> </Box>
<Box mt={6} whiteSpace={'nowrap'} w={['85%', '300px']}> {feConfigs?.show_pay && (
<Flex alignItems={'center'}> <Box mt={6} whiteSpace={'nowrap'} w={['85%', '300px']}>
<Box flex={'1 0 0'} fontSize={'md'}> <Flex alignItems={'center'}>
{t('support.user.team.Dataset usage')}:&nbsp;{datasetUsageMap.usedSize}/ <Box flex={'1 0 0'} fontSize={'md'}>
{datasetSub.maxSize} {t('support.user.team.Dataset usage')}:&nbsp;{datasetUsageMap.usedSize}/
{datasetSub.maxSize}
</Box>
{userInfo?.team?.canWrite && (
<Button size={'sm'} onClick={onOpenSubDatasetModal}>
{t('support.wallet.Buy more')}
</Button>
)}
</Flex>
<Box mt={1}>
<Progress
value={datasetUsageMap.value}
colorScheme={datasetUsageMap.colorScheme}
borderRadius={'md'}
isAnimated
hasStripe
borderWidth={'1px'}
borderColor={'borderColor.base'}
/>
</Box> </Box>
<Button size={'sm'} onClick={onOpenSubDatasetModal}>
{t('support.wallet.Buy more')}
</Button>
</Flex>
<Box mt={1}>
<Progress
value={datasetUsageMap.value}
colorScheme={datasetUsageMap.colorScheme}
borderRadius={'md'}
isAnimated
hasStripe
borderWidth={'1px'}
borderColor={'borderColor.base'}
/>
</Box> </Box>
</Box> )}
</> </>
)} )}
......
projects/app/src/pages/api/core/dataset/collection/create/file.ts deleted 100644 → 0
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { uploadFile } from '@fastgpt/service/common/file/gridfs/controller';
import { getUploadModel } from '@fastgpt/service/common/file/multer';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { FileCreateDatasetCollectionParams } from '@fastgpt/global/core/dataset/api';
import { removeFilesByPaths } from '@fastgpt/service/common/file/utils';
import { createOneCollection } from '@fastgpt/service/core/dataset/collection/controller';
import { DatasetCollectionTypeEnum } from '@fastgpt/global/core/dataset/constants';
/**
* Creates the multer uploader
*/
const upload = getUploadModel({
maxSize: 500 * 1024 * 1024
});
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
let filePaths: string[] = [];
const { datasetId } = req.query as { datasetId: string };
try {
await connectToDatabase();
const { teamId, tmbId } = await authDataset({
req,
authToken: true,
authApiKey: true,
per: 'w',
datasetId
});
const { file, bucketName, data } = await upload.doUpload<FileCreateDatasetCollectionParams>(
req,
res
);
filePaths = [file.path];
if (!file || !bucketName) {
throw new Error('file is empty');
}
const { fileMetadata, collectionMetadata, ...collectionData } = data;
// upload file and create collection
const fileId = await uploadFile({
teamId,
tmbId,
bucketName,
path: file.path,
filename: file.originalname,
contentType: file.mimetype,
metadata: fileMetadata
});
// create collection
const collectionId = await createOneCollection({
...collectionData,
metadata: collectionMetadata,
teamId,
tmbId,
type: DatasetCollectionTypeEnum.file,
fileId
});
jsonRes(res, {
data: collectionId
});
} catch (error) {
jsonRes(res, {
code: 500,
error
});
}
removeFilesByPaths(filePaths);
}
export const config = {
api: {
bodyParser: false
}
};
projects/app/src/pages/api/core/dataset/collection/create/link.ts deleted 100644 → 0
/*
Create one dataset collection
*/
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import type { LinkCreateDatasetCollectionParams } from '@fastgpt/global/core/dataset/api.d';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { createOneCollection } from '@fastgpt/service/core/dataset/collection/controller';
import {
TrainingModeEnum,
DatasetCollectionTypeEnum
} from '@fastgpt/global/core/dataset/constants';
import { checkDatasetLimit } from '@fastgpt/service/support/permission/limit/dataset';
import { predictDataLimitLength } from '@fastgpt/global/core/dataset/utils';
import { createTrainingBill } from '@fastgpt/service/support/wallet/bill/controller';
import { BillSourceEnum } from '@fastgpt/global/support/wallet/bill/constants';
import { getQAModel, getVectorModel } from '@/service/core/ai/model';
import { reloadCollectionChunks } from '@fastgpt/service/core/dataset/collection/utils';
import { startQueue } from '@/service/utils/tools';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try {
await connectToDatabase();
const {
link,
trainingType = TrainingModeEnum.chunk,
chunkSize = 512,
chunkSplitter,
qaPrompt,
...body
} = req.body as LinkCreateDatasetCollectionParams;
const { teamId, tmbId, dataset } = await authDataset({
req,
authToken: true,
authApiKey: true,
datasetId: body.datasetId,
per: 'w'
});
// 1. check dataset limit
await checkDatasetLimit({
teamId,
freeSize: global.feConfigs?.subscription?.datasetStoreFreeSize,
insertLen: predictDataLimitLength(trainingType, new Array(10))
});
// 2. create collection
const collectionId = await createOneCollection({
...body,
name: link,
teamId,
tmbId,
type: DatasetCollectionTypeEnum.link,
trainingType,
chunkSize,
chunkSplitter,
qaPrompt,
rawLink: link
});
// 3. create bill and start sync
const { billId } = await createTrainingBill({
teamId,
tmbId,
appName: 'core.dataset.collection.Sync Collection',
billSource: BillSourceEnum.training,
vectorModel: getVectorModel(dataset.vectorModel).name,
agentModel: getQAModel(dataset.agentModel).name
});
await reloadCollectionChunks({
collectionId,
tmbId,
billId
});
startQueue();
jsonRes(res, {
data: { collectionId }
});
} catch (err) {
jsonRes(res, {
code: 500,
error: err
});
}
}
projects/app/src/pages/api/core/dataset/collection/create/text.ts deleted 100644 → 0
/*
Create one dataset collection
*/
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import type { TextCreateDatasetCollectionParams } from '@fastgpt/global/core/dataset/api.d';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { createOneCollection } from '@fastgpt/service/core/dataset/collection/controller';
import {
TrainingModeEnum,
DatasetCollectionTypeEnum
} from '@fastgpt/global/core/dataset/constants';
import { splitText2Chunks } from '@fastgpt/global/common/string/textSplitter';
import { checkDatasetLimit } from '@fastgpt/service/support/permission/limit/dataset';
import { predictDataLimitLength } from '@fastgpt/global/core/dataset/utils';
import { pushDataToTrainingQueue } from '@/service/core/dataset/data/controller';
import { hashStr } from '@fastgpt/global/common/string/tools';
import { createTrainingBill } from '@fastgpt/service/support/wallet/bill/controller';
import { BillSourceEnum } from '@fastgpt/global/support/wallet/bill/constants';
import { getQAModel, getVectorModel } from '@/service/core/ai/model';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try {
await connectToDatabase();
const {
name,
text,
trainingType = TrainingModeEnum.chunk,
chunkSize = 512,
chunkSplitter,
qaPrompt,
...body
} = req.body as TextCreateDatasetCollectionParams;
const { teamId, tmbId, dataset } = await authDataset({
req,
authToken: true,
authApiKey: true,
datasetId: body.datasetId,
per: 'w'
});
// 1. split text to chunks
const { chunks } = splitText2Chunks({
text,
chunkLen: chunkSize,
overlapRatio: trainingType === TrainingModeEnum.chunk ? 0.2 : 0,
customReg: chunkSplitter ? [chunkSplitter] : []
});
// 2. check dataset limit
await checkDatasetLimit({
teamId,
freeSize: global.feConfigs?.subscription?.datasetStoreFreeSize,
insertLen: predictDataLimitLength(trainingType, chunks)
});
// 3. create collection and training bill
const [collectionId, { billId }] = await Promise.all([
createOneCollection({
...body,
teamId,
tmbId,
type: DatasetCollectionTypeEnum.virtual,
name,
trainingType,
chunkSize,
chunkSplitter,
qaPrompt,
hashRawText: hashStr(text),
rawTextLength: text.length
}),
createTrainingBill({
teamId,
tmbId,
appName: name,
billSource: BillSourceEnum.training,
vectorModel: getVectorModel(dataset.vectorModel)?.name,
agentModel: getQAModel(dataset.agentModel)?.name
})
]);
// 4. push chunks to training queue
const insertResults = await pushDataToTrainingQueue({
teamId,
tmbId,
collectionId,
trainingMode: trainingType,
prompt: qaPrompt,
billId,
data: chunks.map((text, index) => ({
q: text,
chunkIndex: index
}))
});
jsonRes(res, {
data: { collectionId, results: insertResults }
});
} catch (err) {
jsonRes(res, {
code: 500,
error: err
});
}
}
export const config = {
api: {
bodyParser: {
sizeLimit: '10mb'
}
}
};
projects/app/src/pages/api/core/dataset/create.ts
...@@ -6,6 +6,7 @@ import type { CreateDatasetParams } from '@/global/core/dataset/api.d'; ...@@ -6,6 +6,7 @@ import type { CreateDatasetParams } from '@/global/core/dataset/api.d';
import { createDefaultCollection } from '@fastgpt/service/core/dataset/collection/controller'; import { createDefaultCollection } from '@fastgpt/service/core/dataset/collection/controller';
import { authUserNotVisitor } from '@fastgpt/service/support/permission/auth/user'; import { authUserNotVisitor } from '@fastgpt/service/support/permission/auth/user';
import { DatasetTypeEnum } from '@fastgpt/global/core/dataset/constants'; import { DatasetTypeEnum } from '@fastgpt/global/core/dataset/constants';
import { getQAModel, getVectorModel } from '@/service/core/ai/model';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) { export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try { try {
...@@ -13,18 +14,18 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse< ...@@ -13,18 +14,18 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
const { const {
parentId, parentId,
name, name,
type, type = DatasetTypeEnum.dataset,
avatar, avatar,
vectorModel = global.vectorModels[0].model, vectorModel = global.vectorModels[0].model,
agentModel = global.qaModels[0].model agentModel = global.qaModels[0].model
} = req.body as CreateDatasetParams; } = req.body as CreateDatasetParams;
// auth // auth
const { teamId, tmbId } = await authUserNotVisitor({ req, authToken: true }); const { teamId, tmbId } = await authUserNotVisitor({ req, authToken: true, authApiKey: true });
// check model valid // check model valid
const vectorModelStore = global.vectorModels.find((item) => item.model === vectorModel); const vectorModelStore = getVectorModel(vectorModel);
const agentModelStore = global.qaModels.find((item) => item.model === agentModel); const agentModelStore = getQAModel(agentModel);
if (!vectorModelStore || !agentModelStore) { if (!vectorModelStore || !agentModelStore) {
throw new Error('vectorModel or qaModel is invalid'); throw new Error('vectorModel or qaModel is invalid');
} }
......
projects/app/src/pages/api/core/dataset/delete.ts
...@@ -18,7 +18,13 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse< ...@@ -18,7 +18,13 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
} }
// auth owner // auth owner
const { teamId } = await authDataset({ req, authToken: true, datasetId, per: 'owner' }); const { teamId } = await authDataset({
req,
authToken: true,
authApiKey: true,
datasetId,
per: 'owner'
});
const datasets = await findDatasetAndAllChildren({ const datasets = await findDatasetAndAllChildren({
teamId, teamId,
......
projects/app/src/pages/api/plusApi/[...path].ts deleted 100644 → 0
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { request } from '@fastgpt/service/common/api/plusRequest';
import type { Method } from 'axios';
import { setCookie } from '@fastgpt/service/support/permission/controller';
import { connectToDatabase } from '@/service/mongo';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
const method = (req.method || 'POST') as Method;
const { path = [], ...query } = req.query as any;
const url = `/${path?.join('/')}?${new URLSearchParams(query).toString()}`;
if (!url) {
throw new Error('url is empty');
}
const data = req.body || query;
const repose = await request(
url,
data,
{
headers: {
...req.headers,
// @ts-ignore
rootkey: undefined
}
},
method
);
/* special response */
// response cookie
if (repose?.cookie) {
setCookie(res, repose.cookie);
return jsonRes(res, {
data: repose?.cookie
});
}
jsonRes(res, {
data: repose
});
} catch (error) {
jsonRes(res, {
code: 500,
error
});
}
}
export const config = {
api: {
bodyParser: {
sizeLimit: '10mb'
},
responseLimit: '10mb'
}
};
projects/app/src/pages/api/proApi/[...path].ts 0 → 100644
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { request } from 'http';
import { FastGPTProUrl } from '@fastgpt/service/common/system/constants';
import url from 'url';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
const { path = [], ...query } = req.query as any;
const requestPath = `/api/${path?.join('/')}?${new URLSearchParams(query).toString()}`;
if (!requestPath) {
throw new Error('url is empty');
}
const parsedUrl = url.parse(FastGPTProUrl);
delete req.headers?.rootkey;
const requestResult = request({
protocol: parsedUrl.protocol,
hostname: parsedUrl.hostname,
port: parsedUrl.port,
path: requestPath,
method: req.method,
headers: req.headers
});
req.pipe(requestResult);
requestResult.on('response', (response) => {
Object.keys(response.headers).forEach((key) => {
// @ts-ignore
res.setHeader(key, response.headers[key]);
});
response.statusCode && res.writeHead(response.statusCode);
response.pipe(res);
});
requestResult.on('error', (e) => {
res.send(e);
res.end();
});
} catch (error) {
jsonRes(res, {
code: 500,
error
});
}
}
export const config = {
api: {
bodyParser: false
}
};
projects/app/src/pages/dataset/detail/index.tsx
...@@ -87,7 +87,7 @@ const Detail = ({ datasetId, currentTab }: { datasetId: string; currentTab: `${T ...@@ -87,7 +87,7 @@ const Detail = ({ datasetId, currentTab }: { datasetId: string; currentTab: `${T
onError(err: any) { onError(err: any) {
router.replace(`/dataset/list`); router.replace(`/dataset/list`);
toast({ toast({
title: getErrText(err, t('common.Load Failed')), title: t(getErrText(err, t('common.Load Failed'))),
status: 'error' status: 'error'
}); });
} }
......
projects/app/src/pages/dataset/list/index.tsx
...@@ -46,13 +46,15 @@ import { PermissionTypeEnum } from '@fastgpt/global/support/permission/constant' ...@@ -46,13 +46,15 @@ import { PermissionTypeEnum } from '@fastgpt/global/support/permission/constant'
import { DatasetItemType } from '@fastgpt/global/core/dataset/type'; import { DatasetItemType } from '@fastgpt/global/core/dataset/type';
import ParentPaths from '@/components/common/ParentPaths'; import ParentPaths from '@/components/common/ParentPaths';
import DatasetTypeTag from '@/components/core/dataset/DatasetTypeTag'; import DatasetTypeTag from '@/components/core/dataset/DatasetTypeTag';
import { useToast } from '@/web/common/hooks/useToast';
import { getErrText } from '@fastgpt/global/common/error/utils';
const CreateModal = dynamic(() => import('./component/CreateModal'), { ssr: false }); const CreateModal = dynamic(() => import('./component/CreateModal'), { ssr: false });
const MoveModal = dynamic(() => import('./component/MoveModal'), { ssr: false }); const MoveModal = dynamic(() => import('./component/MoveModal'), { ssr: false });
const Kb = () => { const Kb = () => {
const { t } = useTranslation(); const { t } = useTranslation();
const theme = useTheme(); const { toast } = useToast();
const router = useRouter(); const router = useRouter();
const { parentId } = router.query as { parentId: string }; const { parentId } = router.query as { parentId: string };
const { setLoading } = useSystemStore(); const { setLoading } = useSystemStore();
...@@ -115,9 +117,20 @@ const Kb = () => { ...@@ -115,9 +117,20 @@ const Kb = () => {
errorToast: t('dataset.Export Dataset Limit Error') errorToast: t('dataset.Export Dataset Limit Error')
}); });
const { data, refetch, isFetching } = useQuery(['loadDataset', parentId], () => { const { data, refetch, isFetching } = useQuery(
return Promise.all([loadDatasets(parentId), getDatasetPaths(parentId)]); ['loadDataset', parentId],
}); () => {
return Promise.all([loadDatasets(parentId), getDatasetPaths(parentId)]);
},
{
onError(err) {
toast({
status: 'error',
title: t(getErrText(err))
});
}
}
);
const paths = data?.[1] || []; const paths = data?.[1] || [];
......
projects/app/src/pages/login/provider.tsx
...@@ -106,9 +106,9 @@ const provider = ({ code, state, error }: { code: string; state: string; error?: ...@@ -106,9 +106,9 @@ const provider = ({ code, state, error }: { code: string; state: string; error?:
export async function getServerSideProps(content: any) { export async function getServerSideProps(content: any) {
return { return {
props: { props: {
code: content?.query?.code, code: content?.query?.code || '',
state: content?.query?.state, state: content?.query?.state || '',
error: content?.query?.error, error: content?.query?.error || '',
...(await serviceSideProps(content)) ...(await serviceSideProps(content))
} }
}; };
......
projects/app/src/web/common/hooks/useConfirm.tsx
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; import React, { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { useDisclosure, Button, ModalBody, ModalFooter } from '@chakra-ui/react'; import { useDisclosure, Button, ModalBody, ModalFooter } from '@chakra-ui/react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import MyModal from '@/components/MyModal'; import MyModal from '@/components/MyModal';
...@@ -35,7 +35,7 @@ export const useConfirm = (props?: { ...@@ -35,7 +35,7 @@ export const useConfirm = (props?: {
content, content,
showCancel = true showCancel = true
} = props || {}; } = props || {};
const [customContent, setCustomContent] = useState(content); const [customContent, setCustomContent] = useState<string | React.ReactNode>(content);
const { isOpen, onOpen, onClose } = useDisclosure(); const { isOpen, onOpen, onClose } = useDisclosure();
...@@ -44,7 +44,7 @@ export const useConfirm = (props?: { ...@@ -44,7 +44,7 @@ export const useConfirm = (props?: {
return { return {
openConfirm: useCallback( openConfirm: useCallback(
(confirm?: any, cancel?: any, customContent?: string) => { (confirm?: any, cancel?: any, customContent?: string | React.ReactNode) => {
confirmCb.current = confirm; confirmCb.current = confirm;
cancelCb.current = cancel; cancelCb.current = cancel;
......
projects/app/src/web/core/dataset/api.ts
...@@ -58,7 +58,7 @@ export const putDatasetById = (data: DatasetUpdateBody) => PUT<void>(`/core/data ...@@ -58,7 +58,7 @@ export const putDatasetById = (data: DatasetUpdateBody) => PUT<void>(`/core/data
export const delDatasetById = (id: string) => DELETE(`/core/dataset/delete?id=${id}`); export const delDatasetById = (id: string) => DELETE(`/core/dataset/delete?id=${id}`);
export const postWebsiteSync = (data: PostWebsiteSyncParams) => export const postWebsiteSync = (data: PostWebsiteSyncParams) =>
POST(`/plusApi/core/dataset/websiteSync`, data, { POST(`/proApi/core/dataset/websiteSync`, data, {
timeout: 600000 timeout: 600000
}).catch(); }).catch();
...@@ -76,7 +76,7 @@ export const getDatasetCollectionById = (id: string) => ...@@ -76,7 +76,7 @@ export const getDatasetCollectionById = (id: string) =>
export const postDatasetCollection = (data: CreateDatasetCollectionParams) => export const postDatasetCollection = (data: CreateDatasetCollectionParams) =>
POST<string>(`/core/dataset/collection/create`, data); POST<string>(`/core/dataset/collection/create`, data);
export const postCreateDatasetLinkCollection = (data: LinkCreateDatasetCollectionParams) => export const postCreateDatasetLinkCollection = (data: LinkCreateDatasetCollectionParams) =>
POST<{ collectionId: string }>(`/core/dataset/collection/create/link`, data); POST<{ collectionId: string }>(`/proApi/core/dataset/collection/create/link`, data);
export const putDatasetCollectionById = (data: UpdateDatasetCollectionParams) => export const putDatasetCollectionById = (data: UpdateDatasetCollectionParams) =>
POST(`/core/dataset/collection/update`, data); POST(`/core/dataset/collection/update`, data);
......
projects/app/src/web/core/dataset/utils.ts
...@@ -27,18 +27,22 @@ export const fileCollectionCreate = ({ ...@@ -27,18 +27,22 @@ export const fileCollectionCreate = ({
form.append('bucketName', BucketNameEnum.dataset); form.append('bucketName', BucketNameEnum.dataset);
form.append('file', file, encodeURIComponent(file.name)); form.append('file', file, encodeURIComponent(file.name));
return POST<string>(`/core/dataset/collection/create/file?datasetId=${data.datasetId}`, form, { return POST<string>(
timeout: 480000, `/proApi/core/dataset/collection/create/emptyFile?datasetId=${data.datasetId}`,
onUploadProgress: (e) => { form,
if (!e.total) return; {
timeout: 480000,
onUploadProgress: (e) => {
if (!e.total) return;
const percent = Math.round((e.loaded / e.total) * 100); const percent = Math.round((e.loaded / e.total) * 100);
percentListen && percentListen(percent); percentListen && percentListen(percent);
}, },
headers: { headers: {
'Content-Type': 'multipart/form-data; charset=utf-8' 'Content-Type': 'multipart/form-data; charset=utf-8'
}
} }
}); );
}; };
export async function chunksUpload({ export async function chunksUpload({
......
projects/app/src/web/support/activity/promotion/api.ts
...@@ -7,8 +7,8 @@ export const getPromotionInitData = () => ...@@ -7,8 +7,8 @@ export const getPromotionInitData = () =>
GET<{ GET<{
invitedAmount: number; invitedAmount: number;
earningsAmount: number; earningsAmount: number;
}>('/plusApi/support/activity/promotion/getPromotionData'); }>('/proApi/support/activity/promotion/getPromotionData');
/* promotion records */ /* promotion records */
export const getPromotionRecords = (data: RequestPaging) => export const getPromotionRecords = (data: RequestPaging) =>
POST<PromotionRecordType>(`/plusApi/support/activity/promotion/getPromotions`, data); POST<PromotionRecordType>(`/proApi/support/activity/promotion/getPromotions`, data);
projects/app/src/web/support/user/api.ts
...@@ -14,14 +14,14 @@ export const sendAuthCode = (data: { ...@@ -14,14 +14,14 @@ export const sendAuthCode = (data: {
username: string; username: string;
type: `${UserAuthTypeEnum}`; type: `${UserAuthTypeEnum}`;
googleToken: string; googleToken: string;
}) => POST(`/plusApi/support/user/inform/sendAuthCode`, data); }) => POST(`/proApi/support/user/inform/sendAuthCode`, data);
export const getTokenLogin = () => export const getTokenLogin = () =>
GET<UserType>('/support/user/account/tokenLogin', {}, { maxQuantity: 1 }); GET<UserType>('/support/user/account/tokenLogin', {}, { maxQuantity: 1 });
export const oauthLogin = (params: OauthLoginProps) => export const oauthLogin = (params: OauthLoginProps) =>
POST<ResLogin>('/plusApi/support/user/account/login/oauth', params); POST<ResLogin>('/proApi/support/user/account/login/oauth', params);
export const postFastLogin = (params: FastLoginProps) => export const postFastLogin = (params: FastLoginProps) =>
POST<ResLogin>('/plusApi/support/user/account/login/fastLogin', params); POST<ResLogin>('/proApi/support/user/account/login/fastLogin', params);
export const postRegister = ({ export const postRegister = ({
username, username,
...@@ -34,7 +34,7 @@ export const postRegister = ({ ...@@ -34,7 +34,7 @@ export const postRegister = ({
password: string; password: string;
inviterId?: string; inviterId?: string;
}) => }) =>
POST<ResLogin>(`/plusApi/support/user/account/register/emailAndPhone`, { POST<ResLogin>(`/proApi/support/user/account/register/emailAndPhone`, {
username, username,
code, code,
inviterId, inviterId,
...@@ -50,7 +50,7 @@ export const postFindPassword = ({ ...@@ -50,7 +50,7 @@ export const postFindPassword = ({
code: string; code: string;
password: string; password: string;
}) => }) =>
POST<ResLogin>(`/plusApi/support/user/account/password/updateByCode`, { POST<ResLogin>(`/proApi/support/user/account/password/updateByCode`, {
username, username,
code, code,
password: hashStr(password) password: hashStr(password)
......
projects/app/src/web/support/user/inform/api.ts
...@@ -3,7 +3,7 @@ import type { PagingData, RequestPaging } from '@/types'; ...@@ -3,7 +3,7 @@ import type { PagingData, RequestPaging } from '@/types';
import type { UserInformSchema } from '@fastgpt/global/support/user/inform/type'; import type { UserInformSchema } from '@fastgpt/global/support/user/inform/type';
export const getInforms = (data: RequestPaging) => export const getInforms = (data: RequestPaging) =>
POST<PagingData<UserInformSchema>>(`/plusApi/support/user/inform/list`, data); POST<PagingData<UserInformSchema>>(`/proApi/support/user/inform/list`, data);
export const getUnreadCount = () => GET<number>(`/plusApi/support/user/inform/countUnread`); export const getUnreadCount = () => GET<number>(`/proApi/support/user/inform/countUnread`);
export const readInform = (id: string) => GET(`/plusApi/support/user/inform/read`, { id }); export const readInform = (id: string) => GET(`/proApi/support/user/inform/read`, { id });
projects/app/src/web/support/user/team/api.ts
...@@ -16,29 +16,29 @@ import { ...@@ -16,29 +16,29 @@ import {
/* --------------- team ---------------- */ /* --------------- team ---------------- */
export const getTeamList = (status: `${TeamMemberSchema['status']}`) => export const getTeamList = (status: `${TeamMemberSchema['status']}`) =>
GET<TeamItemType[]>(`/plusApi/support/user/team/list`, { status }); GET<TeamItemType[]>(`/proApi/support/user/team/list`, { status });
export const postCreateTeam = (data: CreateTeamProps) => export const postCreateTeam = (data: CreateTeamProps) =>
POST<string>(`/plusApi/support/user/team/create`, data); POST<string>(`/proApi/support/user/team/create`, data);
export const putUpdateTeam = (data: UpdateTeamProps) => export const putUpdateTeam = (data: UpdateTeamProps) =>
PUT(`/plusApi/support/user/team/update`, data); PUT(`/proApi/support/user/team/update`, data);
export const putSwitchTeam = (teamId: string) => export const putSwitchTeam = (teamId: string) =>
PUT<string>(`/plusApi/support/user/team/switch`, { teamId }); PUT<string>(`/proApi/support/user/team/switch`, { teamId });
/* --------------- team member ---------------- */ /* --------------- team member ---------------- */
export const getTeamMembers = (teamId: string) => export const getTeamMembers = (teamId: string) =>
GET<TeamMemberItemType[]>(`/plusApi/support/user/team/member/list`, { teamId }); GET<TeamMemberItemType[]>(`/proApi/support/user/team/member/list`, { teamId });
export const postInviteTeamMember = (data: InviteMemberProps) => export const postInviteTeamMember = (data: InviteMemberProps) =>
POST<InviteMemberResponse>(`/plusApi/support/user/team/member/invite`, data); POST<InviteMemberResponse>(`/proApi/support/user/team/member/invite`, data);
export const putUpdateMember = (data: UpdateTeamMemberProps) => export const putUpdateMember = (data: UpdateTeamMemberProps) =>
PUT(`/plusApi/support/user/team/member/update`, data); PUT(`/proApi/support/user/team/member/update`, data);
export const putUpdateMemberName = (name: string) => export const putUpdateMemberName = (name: string) =>
PUT(`/plusApi/support/user/team/member/updateName`, { name }); PUT(`/proApi/support/user/team/member/updateName`, { name });
export const delRemoveMember = (props: DelMemberProps) => export const delRemoveMember = (props: DelMemberProps) =>
DELETE(`/plusApi/support/user/team/member/delete`, props); DELETE(`/proApi/support/user/team/member/delete`, props);
export const updateInviteResult = (data: UpdateInviteProps) => export const updateInviteResult = (data: UpdateInviteProps) =>
PUT('/plusApi/support/user/team/member/updateInvite', data); PUT('/proApi/support/user/team/member/updateInvite', data);
export const delLeaveTeam = (teamId: string) => export const delLeaveTeam = (teamId: string) =>
DELETE('/plusApi/support/user/team/member/leave', { teamId }); DELETE('/proApi/support/user/team/member/leave', { teamId });
/* team limit */ /* team limit */
export const checkTeamExportDatasetLimit = (datasetId: string) => export const checkTeamExportDatasetLimit = (datasetId: string) =>
......
projects/app/src/web/support/wallet/bill/api.ts
...@@ -4,7 +4,7 @@ import type { PagingData, RequestPaging } from '@/types'; ...@@ -4,7 +4,7 @@ import type { PagingData, RequestPaging } from '@/types';
import type { BillItemType } from '@fastgpt/global/support/wallet/bill/type'; import type { BillItemType } from '@fastgpt/global/support/wallet/bill/type';
export const getUserBills = (data: RequestPaging) => export const getUserBills = (data: RequestPaging) =>
POST<PagingData<BillItemType>>(`/plusApi/support/wallet/bill/getBill`, data); POST<PagingData<BillItemType>>(`/proApi/support/wallet/bill/getBill`, data);
export const postCreateTrainingBill = (data: CreateTrainingBillProps) => export const postCreateTrainingBill = (data: CreateTrainingBillProps) =>
POST<string>(`/support/wallet/bill/createTrainingBill`, data); POST<string>(`/support/wallet/bill/createTrainingBill`, data);
projects/app/src/web/support/wallet/pay/api.ts
import { GET } from '@/web/common/api/request'; import { GET } from '@/web/common/api/request';
import type { PaySchema } from '@fastgpt/global/support/wallet/pay/type.d'; import type { PaySchema } from '@fastgpt/global/support/wallet/pay/type.d';
export const getPayOrders = () => GET<PaySchema[]>(`/plusApi/support/wallet/pay/getPayOrders`); export const getPayOrders = () => GET<PaySchema[]>(`/proApi/support/wallet/pay/getPayOrders`);
export const getPayCode = (amount: number) => export const getPayCode = (amount: number) =>
GET<{ GET<{
codeUrl: string; codeUrl: string;
payId: string; payId: string;
}>(`/plusApi/support/wallet/pay/getPayCode`, { amount }); }>(`/proApi/support/wallet/pay/getPayCode`, { amount });
export const checkPayResult = (payId: string) => export const checkPayResult = (payId: string) =>
GET<string>(`/plusApi/support/wallet/pay/checkPayResult`, { payId }).then((data) => { GET<string>(`/proApi/support/wallet/pay/checkPayResult`, { payId }).then((data) => {
try { try {
GET('/common/system/unlockTask'); GET('/common/system/unlockTask');
} catch (error) {} } catch (error) {}
......
projects/app/src/web/support/wallet/sub/api.ts
...@@ -10,4 +10,4 @@ export const getTeamDatasetValidSub = () => ...@@ -10,4 +10,4 @@ export const getTeamDatasetValidSub = () =>
}>(`/support/wallet/sub/getDatasetSub`); }>(`/support/wallet/sub/getDatasetSub`);
export const postExpandTeamDatasetSub = (data: SubDatasetSizeParams) => export const postExpandTeamDatasetSub = (data: SubDatasetSizeParams) =>
POST('/plusApi/support/wallet/sub/datasetSize/expand', data); POST('/proApi/support/wallet/sub/datasetSize/expand', data);
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment