Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit 92ebd6a0 by archer
Options

doc: m3e model

parent 0d26b1d4
Showing with 130 additions and 74 deletions
client/data/config.json
......@@ -49,7 +49,9 @@
{
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"price": 0
"price": 0,
"defaultToken": 500,
"maxToken": 3000
}
],
"QAModel": {
......
client/src/pages/kb/detail/components/Import/Csv.tsx
......@@ -17,6 +17,7 @@ const fileExtension = '.csv';
const CsvImport = ({ kbId }: { kbId: string }) => {
const { kbDetail } = useUserStore();
const maxToken = kbDetail.vectorModel?.maxToken || 2000;
const theme = useTheme();
const router = useRouter();
......@@ -39,7 +40,7 @@ const CsvImport = ({ kbId }: { kbId: string }) => {
mutationFn: async () => {
const chunks = files.map((file) => file.chunks).flat();
const filterChunks = chunks.filter((item) => item.q.length < kbDetail.vectorModel.maxToken);
const filterChunks = chunks.filter((item) => item.q.length < maxToken);
if (filterChunks.length !== chunks.length) {
toast({
......
client/src/pages/kb/detail/components/Import/Manual.tsx
......@@ -12,6 +12,7 @@ type ManualFormType = { q: string; a: string };
const ManualImport = ({ kbId }: { kbId: string }) => {
const { kbDetail } = useUserStore();
const maxToken = kbDetail.vectorModel?.maxToken || 2000;
const { register, handleSubmit, reset } = useForm({
defaultValues: { q: '', a: '' }
......@@ -71,8 +72,8 @@ const ManualImport = ({ kbId }: { kbId: string }) => {
<Box flex={1} mr={[0, 4]} mb={[4, 0]} h={['50%', '100%']} position={'relative'}>
<Box h={'30px'}>{'匹配的知识点'}</Box>
<Textarea
placeholder={`匹配的知识点。这部分内容会被搜索，请把控内容的质量。最多 ${kbDetail.vectorModel.maxToken} 字。`}
maxLength={kbDetail.vectorModel.maxToken}
placeholder={`匹配的知识点。这部分内容会被搜索，请把控内容的质量。最多 ${maxToken} 字。`}
maxLength={maxToken}
h={['250px', '500px']}
{...register(`q`, {
required: true,
......
docSite/assets/jsconfig.json
......@@ -3,8 +3,8 @@
"baseUrl": ".",
"paths": {
"*": [
"../../../../../Library/Caches/hugo_cache/modules/filecache/modules/pkg/mod/github.com/gohugoio/hugo-mod-jslibs-dist/popperjs/v2@v2.21100.20000/package/dist/cjs/popper.js/*",
"../../../../../Library/Caches/hugo_cache/modules/filecache/modules/pkg/mod/github.com/twbs/bootstrap@v5.3.0+incompatible/js/*"
"../../../../../.cache/hugo_cache/modules/filecache/modules/pkg/mod/github.com/gohugoio/hugo-mod-jslibs-dist/popperjs/v2@v2.21100.20000/package/dist/cjs/popper.js/*",
"../../../../../.cache/hugo_cache/modules/filecache/modules/pkg/mod/github.com/twbs/bootstrap@v5.3.0+incompatible/js/*"
]
}
}
......
docSite/content/docs/custom-models/_index.md 0 → 100644
---
weight: 0
title: '本地模型使用'
description: 'FastGPT 对接本地模型'
icon: 'model_training'
draft: false
images: []
---
docSite/content/docs/custom-models/m3e.md 0 → 100644
---
title: '接入 M3E 向量模型'
description: ' 将 FastGPT 接入私有化模型 M3E'
icon: 'model_training'
draft: false
toc: true
weight: 100
---
## 前言
FastGPT 默认使用了 openai 的 embedding 向量模型，如果你想私有部署的话，可以使用 M3E 向量模型进行替换。M3E 向量模型属于小模型，资源使用不高，CPU 也可以运行。下面教程是基于 “睡大觉” 同学提供的一个 CPU 版本的镜像。
## 部署镜像
镜像名: `stawky/m3e-large-api:latest`
端口号: 6008
## 接入 OneAPI
添加一个渠道，参数如下：
![](/imgs/model-m3e1.png)
## 测试
curl 例子：
```bash
curl --location --request POST 'https://domain/v1/embeddings' \
--header 'Authorization: Bearer sk-key' \
--header 'Content-Type: application/json' \
--data-raw '{
"model": "m3e",
"input": ["laf是什么"]
}'
```
Authorization 为 sk-key。model 为刚刚在 OneAPI 填写的自定义模型。
## 接入 FastGPT
修改 config.json 配置文件，在 VectorModels 中加入 M3E 模型：
```json
"VectorModels": [
{
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"price": 0.2,
"defaultToken": 500,
"maxToken": 3000
},
{
"model": "m3e",
"name": "M3E（测试使用）",
"price": 0.1,
"defaultToken": 500,
"maxToken": 1800
}
],
```
## 测试使用
1. 创建知识库时候选择 M3E 模型。
注意，一旦选择后，知识库将无法修改向量模型。
![](/imgs/model-m3e2.png)
2. 导入数据
3. 搜索测试
![](/imgs/model-m3e3.png)
4. 应用绑定知识库
注意，应用只能绑定同一个向量模型的知识库，不能跨模型绑定。并且，需要注意调整相似度，不同向量模型的相似度（距离）会有所区别，需要自行测试实验。
![](/imgs/model-m3e4.png)
docSite/content/docs/installation/docker.md
......@@ -162,72 +162,7 @@ docker-compose up -d
### 如何自定义配置文件？
需要在 `docker-compose.yml` 同级目录创建一个 `config.json` 文件，内容如下：
```json
{
"FeConfig": {
"show_emptyChat": true,
"show_register": false,
"show_appStore": false,
"show_userDetail": false,
"show_git": true,
"systemTitle": "FastGPT",
"authorText": "Made by FastGPT Team.",
"gitLoginKey": "",
"scripts": []
},
"SystemParams": {
"gitLoginSecret": "",
"vectorMaxProcess": 15,
"qaMaxProcess": 15,
"pgIvfflatProbe": 20
},
"plugins": {},
"ChatModels": [
{
"model": "gpt-3.5-turbo",
"name": "GPT35-4k",
"contextMaxToken": 4000,
"quoteMaxToken": 2000,
"maxTemperature": 1.2,
"price": 0,
"defaultSystem": ""
},
{
"model": "gpt-3.5-turbo-16k",
"name": "GPT35-16k",
"contextMaxToken": 16000,
"quoteMaxToken": 8000,
"maxTemperature": 1.2,
"price": 0,
"defaultSystem": ""
},
{
"model": "gpt-4",
"name": "GPT4-8k",
"contextMaxToken": 8000,
"quoteMaxToken": 4000,
"maxTemperature": 1.2,
"price": 0,
"defaultSystem": ""
}
],
"QAModel": {
"model": "gpt-3.5-turbo-16k",
"name": "GPT35-16k",
"maxToken": 16000,
"price": 0
},
"VectorModels": [
{
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"price": 0
}
]
}
```
需要在 `docker-compose.yml` 同级目录创建一个 `config.json` 文件，内容参考: [配置详解](/docs/installation/reference/configuration/)
然后修改 `docker-compose.yml` 中的 `fastgpt` 容器内容，增加挂载选项即可：
......
docSite/content/docs/installation/reference/configuration.md
......@@ -106,7 +106,9 @@ weight: 751
{
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"price": 0
"price": 0,
"defaultToken": 500,
"maxToken": 3000
}
]
}
......
docSite/content/docs/installation/upgrading/421.md 0 → 100644
---
title: '升级到 V4.2.1'
description: 'FastGPT 从旧版本升级到 V4.2.1 操作指南'
icon: 'upgrade'
draft: false
toc: true
weight: 763
---
私有部署，如果添加了配置文件，需要在配置文件中修改 `VectorModels` 字段。增加 defaultToken 和 maxToken，分别对应直接分段时的默认 token 数量和该模型支持的 token 上限（通常不建议超过 3000）
```json
"VectorModels": [
{
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"price": 0,
"defaultToken": 500,
"maxToken": 3000
}
]
```
改动目的是，我们认为不需要留有选择余地，选择一个最合适的模型去进行任务即可。
files/deploy/fastgpt/config.json
......@@ -50,7 +50,9 @@
{
"model": "text-embedding-ada-002",
"name": "Embedding-2",
"price": 0
"price": 0,
"defaultToken": 500,
"maxToken": 3000
}
],
"QAModel": {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment