Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit 42a8184e by Archer Committed by GitHub
Options

v4.6.9-alpha (#918) 

Co-authored-by: Mufei <327958099@qq.com>
Co-authored-by: heheer <71265218+newfish-cmyk@users.noreply.github.com>
parent f9f0b4bf
Showing with 1272 additions and 697 deletions
.vscode/settings.json
...@@ -10,6 +10,6 @@ ...@@ -10,6 +10,6 @@
"i18n-ally.keystyle": "nested", "i18n-ally.keystyle": "nested",
"i18n-ally.sortKeys": true, "i18n-ally.sortKeys": true,
"i18n-ally.keepFulfilled": true, "i18n-ally.keepFulfilled": true,
"i18n-ally.sourceLanguage": "en", // 根据此语言文件翻译其他语言文件的变量和内容 "i18n-ally.sourceLanguage": "zh", // 根据此语言文件翻译其他语言文件的变量和内容
"i18n-ally.displayLanguage": "en", // 显示语言 "i18n-ally.displayLanguage": "zh", // 显示语言
} }
\ No newline at end of file
docSite/content/docs/agreement/open-source.md
...@@ -11,7 +11,7 @@ FastGPT 项目在 Apache License 2.0 许可下开源，同时包含以下附加 ...@@ -11,7 +11,7 @@ FastGPT 项目在 Apache License 2.0 许可下开源，同时包含以下附加
+ FastGPT 允许被用于商业化，例如作为其他应用的“后端即服务”使用，或者作为应用开发平台提供给企业。然而，当满足以下条件时，必须联系作者获得商业许可： + FastGPT 允许被用于商业化，例如作为其他应用的“后端即服务”使用，或者作为应用开发平台提供给企业。然而，当满足以下条件时，必须联系作者获得商业许可：
+ 多租户 SaaS 服务：除非获得 FastGPT 的明确书面授权，否则不得使用 fastgpt.in 的源码来运营与 fastgpt.in 服务版类似的多租户 SaaS 服务。 + 多租户 SaaS 服务：除非获得 FastGPT 的明确书面授权，否则不得使用 fastgpt.in 的源码来运营与 fastgpt.in 服务类似的多租户 SaaS 服务。
+ LOGO 及版权信息：在使用 FastGPT 的过程中，不得移除或修改 FastGPT 控制台内的 LOGO 或版权信息。 + LOGO 及版权信息：在使用 FastGPT 的过程中，不得移除或修改 FastGPT 控制台内的 LOGO 或版权信息。
请通过电子邮件 yujinlong@sealos.io 联系我们咨询许可事宜。 请通过电子邮件 yujinlong@sealos.io 联系我们咨询许可事宜。
......
docSite/content/docs/agreement/privacy.md 0 → 100644
---
title: '隐私政策'
description: ' FastGPT 隐私政策'
icon: 'gavel'
draft: false
toc: true
weight: 1221
---
最后更新时间：2024年3月3日
我们非常重视您的隐私保护，在您使用FastGPT云服务时，我们将按照以下政策收集、使用、披露和保护您的个人信息。请您仔细阅读并充分理解本隐私政策。
**我们可能需要收集的信息**
1. 在您注册或使用本服务时，我们可能收集您的姓名、电话号码、电子邮件地址、地址等个人信息。
2. 在您使用本服务过程中产生的信息，如操作日志、访问IP地址、设备型号等。
3. 我们可能会通过 Cookies 或其他技术收集和存储您访问本服务的相关信息，以便为您提供更好的用户体验。
**我们如何使用收集的信息？**
1. 我们会根据法律法规规定以及与用户之间的约定来处理用户的个人信息。
2. 我们可能会将收集到的信息用于改进服务质量、开发新产品或功能等目的。
3. 我们可能会将收集到的信息用于向您推送与本服务相关的通知或广告。
**信息披露**
1. 我们不会向任何第三方披露您的个人信息，除非：
1. 您事先同意；
2. 法律法规要求；
3. 为维护我们或其他用户的合法权益。
2. 我们可能与关联公司、合作伙伴分享您的个人信息，但我们会采取相应的保密措施，确保信息安全。
**信息保护**
1. 我们采取各种安全措施，包括加密、访问控制等技术手段，以保护您的个人信息免受未经授权的访问、使用或泄露。
2. 我们会定期对收集、存储和处理的个人信息进行安全评估，以确保个人信息安全。
3. 在发生个人信息泄露等安全事件时，我们会立即启动应急预案，并在法律法规规定的范围内向您及时告知。
4. 我们不会使用您的数据进行额外的备份存储或用于模型训练。
5. 您在本服务进行的数据删除均为物理删除，不可恢复。如有有非物理删除的操作，我们会在服务中特别指出。
**用户权利**
1. 您有权随时查阅、更正或删除您的个人信息。
2. 您有权拒绝我们收集您的个人信息，但这可能导致您无法使用本服务的部分功能。
3. 您有权要求我们停止处理您的个人信息，但这可能导致您无法继续使用本服务。
**隐私政策更新**
1. 我们可能会对本隐私政策进行修改。如本隐私政策发生变更，我们将在本服务页面上发布修改后的隐私政策。如您继续使用本服务，则视为同意修改后的隐私政策。
2. 我们鼓励您定期查阅本隐私政策，以了解我们如何保护您的个人信息。
**未成年人保护**
我们非常重视对未成年人个人信息的保护，如您为未成年人，请在监护人指导下使用本服务，并请监护人帮助您在使用本服务过程中正确处理个人信息。
**跨境数据传输**
由于我们的服务器可能位于不同国家或地区，您同意我们可能需要将您的个人信息传输至其他国家或地区，并在该等国家或地区存储和处理以向您提供服务。我们会采取适当措施确保跨境传输的数据仍然受到适当保护。
**联系我们**
1. 如您对本隐私政策有任何疑问、建议或投诉，请通过以下方式与我们联系：yujinlong@sealos.io。
2. 我们将尽快回复并解决您提出的问题。
\ No newline at end of file
docSite/content/docs/agreement/disclaimer.md → docSite/content/docs/agreement/terms.md
--- ---
title: '免责声明' title: '服务协议'
description: ' FastGPT 免责声明' description: ' FastGPT 服务协议'
icon: 'gavel' icon: 'gavel'
draft: false draft: false
toc: true toc: true
weight: 1220 weight: 1220
--- ---
由于生成式 AI 的特性，其在不同国家的管控措施也会有所不同。请所有使用者务必遵守所在地的相关法律。 最后更新时间：2024年3月3日
免责声明：以任何违反 FastGPT 可接受使用政策的方式使用，包括但不限于法律、法规、政府命令或法令禁止的任何用途，或任何侵犯他人权利的使用；由使用者自行承担。我们对由客户使用产生的问题概不负责。 FastGPT 服务协议是您与珠海环界云计算有限公司（以下简称“我们”或“本公司”）之间就FastGPT云服务（以下简称“本服务”）的使用等相关事项所订立的协议。请您仔细阅读并充分理解本协议各条款，特别是免除或者限制我们责任的条款、对您权益的限制条款、争议解决和法律适用条款等。如您不同意本协议任一内容，请勿注册或使用本服务。
下面是各国对生成式AI的管控条例的链接： **第1条 服务内容**
[中国生成式人工智能服务管理办法（征求意见稿）](http://www.cac.gov.cn/2023-04/11/c_1682854275475410.htm) 1. 我们将向您提供存储、计算、网络传输等基于互联网的信息技术服务。
2. 我们将不定期向您通过站内信、电子邮件或短信等形式向您推送最新的动态。
3. 我们将为您提供相关技术支持和客户服务，帮助您更好地使用本服务。
4. 我们将为您提供稳定的在线服务，保证每月服务可用性不低于99%。
**第2条 用户注册与账户管理**
## 内容要求 1. 您在使用本服务前需要注册一个账户。您保证在注册时提供的信息真实、准确、完整，并及时更新。
2. 您应妥善保管账户名和密码，对由此产生的全部行为负责。如发现他人使用您的账户，请及时修改账号密码或与我们进行联系。
3. 我们有权对您的账户进行审查，如发现您的账户存在异常或违法情况，我们有权暂停或终止向您提供服务。
我们禁止使用我们对接的模型服务生成可能对个人或社会造成伤害的内容。保障平台的安全性，是长期稳定运营的关键。如发现任何利用平台接入模型能力进行违规内容生成和使用，将立即封号，账号余额不退。 **第3条 使用规则**
- 剥削和虐待 1. 您不得利用本服务从事任何违法活动或侵犯他人合法权益的行为，包括但不限于侵犯知识产权、泄露他人商业机密等。
2. 您不得通过任何手段恶意注册账户，包括但不限于以牟利、炒作、套现等目的。
3. 您不得利用本服务传播任何违法、有害、恶意软件等信息。
4. 您应遵守相关法律法规及本协议的规定，对在本服务中发布的信息及使用本服务所产生的结果承担全部责任。
5. 我们禁止使用我们对接的模型服务生成可能对个人或社会造成伤害的内容。保障平台的安全性，是长期稳定运营的关键。如发现任何利用平台接入模型能力进行违规内容生成和使用，将立即封号，账号余额不退。违规内容包括但不限于：
- 剥削和虐待
- 禁止描述、展示或宣扬儿童性剥削或性虐待的内容，无论法律是否禁止。这包括涉及儿童或使儿童色情的内容。 - 禁止描述、展示或宣扬儿童性剥削或性虐待的内容，无论法律是否禁止。这包括涉及儿童或使儿童色情的内容。
- 禁止描述或用于培养儿童的内容。修饰是成年人以剥削，特别是性剥削为目的与儿童建立关系的行为。这包括以性剥削、贩运或其他形式剥削为目的与儿童交流。 - 禁止描述或用于培养儿童的内容。修饰是成年人以剥削，特别是性剥削为目的与儿童建立关系的行为。这包括以性剥削、贩运或其他形式剥削为目的与儿童交流。
- 未经同意的私密内容 - 未经同意的私密内容
- 服务禁止描述、提供或宣传未经同意的亲密活动的内容。 - 服务禁止描述、提供或宣传未经同意的亲密活动的内容。
- 禁止描述、提供特征或宣传或用于招揽商业性活动和性服务的内容。这包括鼓励和协调真正的性活动。 - 禁止描述、提供特征或宣传或用于招揽商业性活动和性服务的内容。这包括鼓励和协调真正的性活动。
- 禁止描述或用于人口贩运目的的内容。这包括招募人员、便利交通、支付和助长对人的剥削，如强迫劳动、家庭奴役、役、强迫婚姻和强迫医疗程序。 - 禁止描述或用于人口贩运目的的内容。这包括招募人员、便利交通、支付和助长对人的剥削，如强迫劳动、家庭奴役、役、强迫婚姻和强迫医疗程序。
- 自杀和自残，禁止描述、赞美、支持、促进、美化、鼓励和/或指导个人自残或自杀的内容。 - 自杀和自残，禁止描述、赞美、支持、促进、美化、鼓励和/或指导个人自残或自杀的内容。
- 暴力内容和行为 - 暴力内容和行为
- 禁止描述、展示或宣扬血腥暴力或血腥的内容。 - 禁止描述、展示或宣扬血腥暴力或血腥的内容。
- 禁止描绘恐怖主义行为的内容；赞扬或支持恐怖组织、恐怖行为者或暴力恐怖意识形态；鼓励恐怖活动；向恐怖组织或恐怖事业提供援助；或协助恐怖组织招募成员。 - 禁止描绘恐怖主义行为的内容；赞扬或支持恐怖组织、恐怖行为者或暴力恐怖意识形态；鼓励恐怖活动；向恐怖组织或恐怖事业提供援助；或协助恐怖组织招募成员。
- 禁止通过暴力威胁或煽动来鼓吹或宣扬对他人的暴力行为的内容。 - 禁止通过暴力威胁或煽动来鼓吹或宣扬对他人的暴力行为的内容。
- 仇恨言论和歧视 - 仇恨言论和歧视
- 禁止基于实际或感知的种族、民族、国籍、性别、性别认同、性取向、宗教信仰、年龄、残疾状况、种姓或与系统性偏见或边缘化相关的任何其他特征等特征攻击、诋毁、恐吓、降级、针对或排斥个人或群体的内容。 - 禁止基于实际或感知的种族、民族、国籍、性别、性别认同、性取向、宗教信仰、年龄、残疾状况、种姓或与系统性偏见或边缘化相关的任何其他特征等特征攻击、诋毁、恐吓、降级、针对或排斥个人或群体的内容。
- 禁止针对个人或群体进行威胁、恐吓、侮辱、贬低或贬低的语言或图像、宣扬身体伤害或其他虐待行为（如跟踪）的内容。 - 禁止针对个人或群体进行威胁、恐吓、侮辱、贬低或贬低的语言或图像、宣扬身体伤害或其他虐待行为（如跟踪）的内容。
- 禁止故意欺骗并可能对公共利益产生不利影响的内容，包括与健康、安全、选举诚信或公民参与相关的欺骗性或不真实内容。 - 禁止故意欺骗并可能对公共利益产生不利影响的内容，包括与健康、安全、选举诚信或公民参与相关的欺骗性或不真实内容。
- 直接支持非法主动攻击或造成技术危害的恶意软件活动的内容，例如提供恶意可执行文件、组织拒绝服务攻击或管理命令和控制服务器。 - 直接支持非法主动攻击或造成技术危害的恶意软件活动的内容，例如提供恶意可执行文件、组织拒绝服务攻击或管理命令和控制服务器。
**第4条 费用及支付**
1. 您同意支付与本服务相关的费用，具体费用标准以我们公布的价格为准。
2. 我们可能会根据运营成本和市场情况调整费用标准。最新价格以您付款时刻的价格为准。
**第5条 服务免责与责任限制**
1. 本服务按照现有技术和条件所能达到的水平提供。我们不能保证本服务完全无故障或满足您的所有需求。
2. 对于因您自身误操作导致的数据丢失、损坏等情况，我们不承担责任。
3. 由于生成式 AI 的特性，其在不同国家的管控措施也会有所不同，请所有使用者务必遵守所在地的相关法律。如果您以任何违反 FastGPT 可接受使用政策的方式使用，包括但不限于法律、法规、政府命令或法令禁止的任何用途，或任何侵犯他人权利的使用；由使用者自行承担。我们对由客户使用产生的问题概不负责。下面是各国对生成式AI的管控条例的链接：
[中国生成式人工智能服务管理办法（征求意见稿）](http://www.cac.gov.cn/2023-04/11/c_1682854275475410.htm)
**第6条 知识产权**
1. 我们对本服务及相关软件、技术、文档等拥有全部知识产权，除非经我们明确许可，您不得进行复制、分发、出租、反向工程等行为。
2. 您在使用本服务过程中产生的所有数据和内容（包括但不限于文件、图片等）的知识产权归您所有。我们不会对您的数据和内容进行使用、复制、修改等行为。
3. 在线服务中其他用户的数据和内容的知识产权归原用户所有，未经原用户许可，您不得进行使用、复制、修改等行为。
**第7条 其他条款**
1. 如本协议中部分条款因违反法律法规而被视为无效，不影响其他条款的效力。
2. 本公司保留对本协议及隐私政策的最终解释权。如您对本协议或隐私政策有任何疑问，请联系我们：yujinlong@sealos.io。
\ No newline at end of file
docSite/content/docs/development/openapi/share.md
...@@ -181,7 +181,7 @@ curl --location --request POST '{{host}}/shareAuth/finish' \ ...@@ -181,7 +181,7 @@ curl --location --request POST '{{host}}/shareAuth/finish' \
"totalPoints": 1.5278, "totalPoints": 1.5278,
"query": "导演是谁\n《铃芽之旅》的导演是谁？\n这部电影的导演是谁？\n谁是《铃芽之旅》的导演？", "query": "导演是谁\n《铃芽之旅》的导演是谁？\n这部电影的导演是谁？\n谁是《铃芽之旅》的导演？",
"model": "Embedding-2(旧版，不推荐使用)", "model": "Embedding-2(旧版，不推荐使用)",
"charsLength": 1524, "tokens": 1524,
"similarity": 0.83, "similarity": 0.83,
"limit": 400, "limit": 400,
"searchMode": "embedding", "searchMode": "embedding",
...@@ -195,7 +195,7 @@ curl --location --request POST '{{host}}/shareAuth/finish' \ ...@@ -195,7 +195,7 @@ curl --location --request POST '{{host}}/shareAuth/finish' \
"moduleType": "chatNode", "moduleType": "chatNode",
"totalPoints": 0.593, "totalPoints": 0.593,
"model": "FastAI-4k", "model": "FastAI-4k",
"charsLength": 593, "tokens": 593,
"query": "导演是谁", "query": "导演是谁",
"maxToken": 2000, "maxToken": 2000,
"quoteList": [ "quoteList": [
...@@ -252,7 +252,7 @@ type ResponseType = { ...@@ -252,7 +252,7 @@ type ResponseType = {
query?: string; // 用户问题/检索词 query?: string; // 用户问题/检索词
textOutput?: string; // 文本输出 textOutput?: string; // 文本输出
charsLength?: number; // 上下文总字数 tokens?: number; // 上下文总Tokens
model?: string; // 使用到的模型 model?: string; // 使用到的模型
contextTotalLen?: number; // 上下文总长度 contextTotalLen?: number; // 上下文总长度
totalPoints?: number; // 总消耗AI积分 totalPoints?: number; // 总消耗AI积分
...@@ -268,7 +268,7 @@ type ResponseType = { ...@@ -268,7 +268,7 @@ type ResponseType = {
searchUsingReRank?: boolean; // 是否使用rerank searchUsingReRank?: boolean; // 是否使用rerank
extensionModel?: string; // 问题扩展模型 extensionModel?: string; // 问题扩展模型
extensionResult?: string; // 问题扩展结果 extensionResult?: string; // 问题扩展结果
extensionCharsLength?: number; // 问题扩展总字符长度 extensionTokens?: number; // 问题扩展总字符长度
cqList?: ClassifyQuestionAgentItemType[]; // 分类问题列表 cqList?: ClassifyQuestionAgentItemType[]; // 分类问题列表
cqResult?: string; // 分类问题结果 cqResult?: string; // 分类问题结果
......
docSite/content/docs/development/upgrading/469.md
...@@ -17,14 +17,18 @@ curl --location --request POST 'https://{{host}}/api/init/v469' \ ...@@ -17,14 +17,18 @@ curl --location --request POST 'https://{{host}}/api/init/v469' \
--header 'Content-Type: application/json' --header 'Content-Type: application/json'
``` ```
会重置计量表。 1. 重置计量表。
2. 执行脏数据清理（清理无效的文件、清理无效的图片、清理无效的知识库集合、清理无效的向量）
## V4.6.9 更新说明 ## V4.6.9 更新说明
1. 新增 - 完善了HTTP模块的变量提示。 1. 商业版新增 - 知识库新增“增强处理”训练模式，可生成更多类型索引。
2. 新增 - HTTP模块支持OpenAI单接口导入。 2. 新增 - 完善了HTTP模块的变量提示。
3. 优化 - 问题补全。增加英文类型。同时可以设置为单独模块，方便复用。 3. 新增 - HTTP模块支持OpenAI单接口导入。
4. 优化 - 重写了计量模式 4. 优化 - 问题补全。增加英文类型。同时可以设置为单独模块，方便复用。
5. 修复 - 标注功能。 5. 优化 - 重写了计量模式
6. 修复 - qa生成线程计数错误。 6. 优化 - Token 过滤历史记录，保持偶数条，防止部分模型报错。
7. 优化 - 分享链接SEO，可直接展示应用名和头像。
8. 修复 - 标注功能。
9. 修复 - qa生成线程计数错误。
docSite/content/docs/pricing.md
...@@ -7,14 +7,4 @@ toc: true ...@@ -7,14 +7,4 @@ toc: true
weight: 1200 weight: 1200
--- ---
## Tokens 说明 线上版价格请查看：https://cloud.fastgpt.in/price
[OpenAI 的 API 官方计费模式](https://openai.com/pricing#language-models)为：按每次 API 请求内容和返回内容 tokens 长度来定价。每个模型具有不同的计价方式，以每 1,000 个 tokens 消耗为单位定价。其中 1,000 个 tokens 约为 900 个英文，约 600 个中文（不是很准确，与上下长度有关，相同的词出现越多，词:Tokens 的比例越大）。平台的 tokens 数量计算算法与 OpenAI 一致，您可以随时通过「使用记录」来查看余额消耗明细的说明，来对比计算是否一致。 \ No newline at end of file
![](/imgs/fastgpt-price.png)
## FastGPT 线上计费
[https://fastgpt.in](https://fastgpt.in) 采用按量计费的模式，最新计费标准可在 `账号-计费标准` 查看。同时可以在 `账号-使用记录` 中查看具体使用情况，
![](/imgs/cloud_price1.jpg)
package.json
...@@ -14,11 +14,11 @@ ...@@ -14,11 +14,11 @@
"devDependencies": { "devDependencies": {
"@chakra-ui/cli": "^2.4.1", "@chakra-ui/cli": "^2.4.1",
"husky": "^8.0.3", "husky": "^8.0.3",
"i18next": "^22.5.1", "i18next": "23.10.0",
"lint-staged": "^13.2.1", "lint-staged": "^13.2.1",
"next-i18next": "^13.3.0", "next-i18next": "15.2.0",
"prettier": "3.2.4", "prettier": "3.2.4",
"react-i18next": "^12.3.1", "react-i18next": "13.5.0",
"zhlint": "^0.7.1" "zhlint": "^0.7.1"
}, },
"lint-staged": { "lint-staged": {
......
packages/global/common/file/api.d.ts
import { MongoImageTypeEnum } from './image/constants'; import { MongoImageTypeEnum } from './image/constants';
import { OutLinkChatAuthProps } from '../../support/permission/chat.d';
export type preUploadImgProps = { export type preUploadImgProps = OutLinkChatAuthProps & {
type: `${MongoImageTypeEnum}`; type: `${MongoImageTypeEnum}`;
expiredTime?: Date; expiredTime?: Date;
metadata?: Record<string, any>; metadata?: Record<string, any>;
shareId?: string;
}; };
export type UploadImgProps = preUploadImgProps & { export type UploadImgProps = preUploadImgProps & {
base64Img: string; base64Img: string;
......
packages/global/common/string/tiktoken/index.ts
...@@ -4,6 +4,7 @@ import { Tiktoken } from 'js-tiktoken/lite'; ...@@ -4,6 +4,7 @@ import { Tiktoken } from 'js-tiktoken/lite';
import { adaptChat2GptMessages } from '../../../core/chat/adapt'; import { adaptChat2GptMessages } from '../../../core/chat/adapt';
import { ChatCompletionRequestMessageRoleEnum } from '../../../core/ai/constant'; import { ChatCompletionRequestMessageRoleEnum } from '../../../core/ai/constant';
import encodingJson from './cl100k_base.json'; import encodingJson from './cl100k_base.json';
import { ChatMessageItemType } from '../../../core/ai/type';
/* init tikToken obj */ /* init tikToken obj */
export function getTikTokenEnc() { export function getTikTokenEnc() {
...@@ -29,32 +30,35 @@ export function getTikTokenEnc() { ...@@ -29,32 +30,35 @@ export function getTikTokenEnc() {
/* count one prompt tokens */ /* count one prompt tokens */
export function countPromptTokens( export function countPromptTokens(
prompt = '', prompt = '',
role: '' | `${ChatCompletionRequestMessageRoleEnum}` = '' role: '' | `${ChatCompletionRequestMessageRoleEnum}` = '',
tools?: any
) { ) {
const enc = getTikTokenEnc(); const enc = getTikTokenEnc();
const text = `${role}\n${prompt}`; const toolText = tools
? JSON.stringify(tools)
.replace('"', '')
.replace('\n', '')
.replace(/( ){2,}/g, ' ')
: '';
const text = `${role}\n${prompt}\n${toolText}`.trim();
try { try {
const encodeText = enc.encode(text); const encodeText = enc.encode(text);
return encodeText.length + role.length; // 补充 role 估算值 const supplementaryToken = role ? 4 : 0;
return encodeText.length + supplementaryToken;
} catch (error) { } catch (error) {
return text.length; return text.length;
} }
} }
/* count messages tokens */ /* count messages tokens */
export function countMessagesTokens({ messages }: { messages: ChatItemType[] }) { export const countMessagesTokens = (messages: ChatItemType[], tools?: any) => {
const adaptMessages = adaptChat2GptMessages({ messages, reserveId: true }); const adaptMessages = adaptChat2GptMessages({ messages, reserveId: true });
let totalTokens = 0; return countGptMessagesTokens(adaptMessages, tools);
for (let i = 0; i < adaptMessages.length; i++) { };
const item = adaptMessages[i]; export const countGptMessagesTokens = (messages: ChatMessageItemType[], tools?: any) =>
const tokens = countPromptTokens(item.content, item.role); messages.reduce((sum, item) => sum + countPromptTokens(item.content, item.role, tools), 0);
totalTokens += tokens;
}
return totalTokens;
}
/* slice messages from top to bottom by maxTokens */ /* slice messages from top to bottom by maxTokens */
export function sliceMessagesTB({ export function sliceMessagesTB({
......
packages/global/common/system/types/index.d.ts
...@@ -39,9 +39,12 @@ export type FastGPTFeConfigsType = { ...@@ -39,9 +39,12 @@ export type FastGPTFeConfigsType = {
systemTitle?: string; systemTitle?: string;
googleClientVerKey?: string; googleClientVerKey?: string;
isPlus?: boolean; isPlus?: boolean;
show_phoneLogin?: boolean;
show_emailLogin?: boolean;
oauth?: { oauth?: {
github?: string; github?: string;
google?: string; google?: string;
wechat?: string;
}; };
limit?: { limit?: {
exportDatasetLimitMinutes?: number; exportDatasetLimitMinutes?: number;
......
packages/global/core/app/type.d.ts
...@@ -5,7 +5,7 @@ import type { AIChatModuleProps, DatasetModuleProps } from '../module/node/type. ...@@ -5,7 +5,7 @@ import type { AIChatModuleProps, DatasetModuleProps } from '../module/node/type.
import { VariableInputEnum } from '../module/constants'; import { VariableInputEnum } from '../module/constants';
import { SelectedDatasetType } from '../module/api'; import { SelectedDatasetType } from '../module/api';
import { DatasetSearchModeEnum } from '../dataset/constants'; import { DatasetSearchModeEnum } from '../dataset/constants';
import { TeamTagsSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d'; import { TeamTagSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d';
export interface AppSchema { export interface AppSchema {
_id: string; _id: string;
userId: string; userId: string;
...@@ -20,7 +20,7 @@ export interface AppSchema { ...@@ -20,7 +20,7 @@ export interface AppSchema {
modules: ModuleItemType[]; modules: ModuleItemType[];
permission: `${PermissionTypeEnum}`; permission: `${PermissionTypeEnum}`;
inited?: boolean; inited?: boolean;
teamTags: [string]; teamTags: string[];
} }
export type AppListItemType = { export type AppListItemType = {
......
packages/global/core/chat/type.d.ts
...@@ -26,23 +26,6 @@ export type ChatSchema = { ...@@ -26,23 +26,6 @@ export type ChatSchema = {
metadata?: Record<string, any>; metadata?: Record<string, any>;
}; };
export type teamInfoType = {
avatar: string;
balance: number;
createTime: string;
maxSize: number;
name: string;
ownerId: string;
tagsUrl: string;
_id: string;
};
export type chatAppListSchema = {
apps: AppType[];
teamInfo: teamInfoSchema;
uid?: string;
};
export type ChatWithAppSchema = Omit<ChatSchema, 'appId'> & { export type ChatWithAppSchema = Omit<ChatSchema, 'appId'> & {
appId: AppSchema; appId: AppSchema;
}; };
...@@ -90,6 +73,13 @@ export type ChatSiteItemType = ChatItemType & { ...@@ -90,6 +73,13 @@ export type ChatSiteItemType = ChatItemType & {
ttsBuffer?: Uint8Array; ttsBuffer?: Uint8Array;
}; };
/* --------- team chat --------- */
export type ChatAppListSchema = {
apps: AppType[];
teamInfo: teamInfoSchema;
uid?: string;
};
/* ---------- history ------------- */ /* ---------- history ------------- */
export type HistoryItemType = { export type HistoryItemType = {
chatId: string; chatId: string;
...@@ -111,7 +101,7 @@ export type moduleDispatchResType = { ...@@ -111,7 +101,7 @@ export type moduleDispatchResType = {
textOutput?: string; textOutput?: string;
// bill // bill
charsLength?: number; tokens?: number;
model?: string; model?: string;
contextTotalLen?: number; contextTotalLen?: number;
totalPoints?: number; totalPoints?: number;
...@@ -129,7 +119,7 @@ export type moduleDispatchResType = { ...@@ -129,7 +119,7 @@ export type moduleDispatchResType = {
searchUsingReRank?: boolean; searchUsingReRank?: boolean;
extensionModel?: string; extensionModel?: string;
extensionResult?: string; extensionResult?: string;
extensionCharsLength?: number; extensionTokens?: number;
// cq // cq
cqList?: ClassifyQuestionAgentItemType[]; cqList?: ClassifyQuestionAgentItemType[];
......
packages/global/core/dataset/constants.ts
...@@ -75,17 +75,25 @@ export const DatasetCollectionSyncResultMap = { ...@@ -75,17 +75,25 @@ export const DatasetCollectionSyncResultMap = {
/* ------------ training -------------- */ /* ------------ training -------------- */
export enum TrainingModeEnum { export enum TrainingModeEnum {
chunk = 'chunk', chunk = 'chunk',
auto = 'auto',
qa = 'qa' qa = 'qa'
} }
export const TrainingTypeMap = { export const TrainingTypeMap = {
[TrainingModeEnum.chunk]: { [TrainingModeEnum.chunk]: {
label: 'core.dataset.training.Chunk mode', label: 'core.dataset.training.Chunk mode',
tooltip: 'core.dataset.import.Chunk Split Tip' tooltip: 'core.dataset.import.Chunk Split Tip',
isPlus: true
},
[TrainingModeEnum.auto]: {
label: 'core.dataset.training.Auto mode',
tooltip: 'core.dataset.training.Auto mode Tip',
isPlus: true
}, },
[TrainingModeEnum.qa]: { [TrainingModeEnum.qa]: {
label: 'core.dataset.training.QA mode', label: 'core.dataset.training.QA mode',
tooltip: 'core.dataset.import.QA Import Tip' tooltip: 'core.dataset.import.QA Import Tip',
isPlus: true
} }
}; };
......
packages/global/core/dataset/utils.ts
...@@ -48,5 +48,6 @@ export function getDefaultIndex(props?: { q?: string; a?: string; dataId?: strin ...@@ -48,5 +48,6 @@ export function getDefaultIndex(props?: { q?: string; a?: string; dataId?: strin
export const predictDataLimitLength = (mode: `${TrainingModeEnum}`, data: any[]) => { export const predictDataLimitLength = (mode: `${TrainingModeEnum}`, data: any[]) => {
if (mode === TrainingModeEnum.qa) return data.length * 20; if (mode === TrainingModeEnum.qa) return data.length * 20;
if (mode === TrainingModeEnum.auto) return data.length * 5;
return data.length; return data.length;
}; };
packages/global/core/module/type.d.ts
...@@ -8,7 +8,7 @@ import { ...@@ -8,7 +8,7 @@ import {
import { FlowNodeInputItemType, FlowNodeOutputItemType } from './node/type'; import { FlowNodeInputItemType, FlowNodeOutputItemType } from './node/type';
import { UserModelSchema } from 'support/user/type'; import { UserModelSchema } from 'support/user/type';
import { moduleDispatchResType } from '..//chat/type'; import { moduleDispatchResType } from '..//chat/type';
import { ChatModuleBillType } from '../../support/wallet/bill/type'; import { ChatModuleUsageType } from '../../support/wallet/bill/type';
export type FlowModuleTemplateType = { export type FlowModuleTemplateType = {
id: string; // module id, unique id: string; // module id, unique
...@@ -129,5 +129,5 @@ export type ModuleDispatchProps<T> = ChatDispatchProps & { ...@@ -129,5 +129,5 @@ export type ModuleDispatchProps<T> = ChatDispatchProps & {
}; };
export type ModuleDispatchResponse<T> = T & { export type ModuleDispatchResponse<T> = T & {
[ModuleOutputKeyEnum.responseData]?: moduleDispatchResType; [ModuleOutputKeyEnum.responseData]?: moduleDispatchResType;
[ModuleOutputKeyEnum.moduleDispatchBills]?: ChatModuleBillType[]; [ModuleOutputKeyEnum.moduleDispatchBills]?: ChatModuleUsageType[];
}; };
packages/global/support/outLink/type.d.ts
import { AppSchema } from 'core/app/type';
import { OutLinkTypeEnum } from './constant'; import { OutLinkTypeEnum } from './constant';
export type OutLinkSchema = { export type OutLinkSchema = {
...@@ -18,6 +19,9 @@ export type OutLinkSchema = { ...@@ -18,6 +19,9 @@ export type OutLinkSchema = {
hookUrl?: string; hookUrl?: string;
}; };
}; };
export type OutLinkWithAppType = Omit<OutLinkSchema, 'appId'> & {
appId: AppSchema;
};
export type OutLinkEditType = { export type OutLinkEditType = {
_id?: string; _id?: string;
......
packages/global/support/permission/chat.d.ts 0 → 100644
type ShareChatAuthProps = {
shareId?: string;
outLinkUid?: string;
};
type TeamChatAuthProps = {
teamId?: string;
teamToken?: string;
};
export type OutLinkChatAuthProps = ShareChatAuthProps & TeamChatAuthProps;
packages/global/support/permission/constant.ts
...@@ -2,7 +2,8 @@ export enum AuthUserTypeEnum { ...@@ -2,7 +2,8 @@ export enum AuthUserTypeEnum {
token = 'token', token = 'token',
root = 'root', root = 'root',
apikey = 'apikey', apikey = 'apikey',
outLink = 'outLink' outLink = 'outLink',
teamDomain = 'teamDomain'
} }
export enum PermissionTypeEnum { export enum PermissionTypeEnum {
......
packages/global/support/user/api.d.ts
...@@ -10,7 +10,11 @@ export type OauthLoginProps = { ...@@ -10,7 +10,11 @@ export type OauthLoginProps = {
code: string; code: string;
callbackUrl: string; callbackUrl: string;
inviterId?: string; inviterId?: string;
tmbId?: string; };
export type WxLoginProps = {
inviterId?: string;
code: string;
}; };
export type FastLoginProps = { export type FastLoginProps = {
......
packages/global/support/user/auth/constants.ts 0 → 100644
export enum UserAuthTypeEnum {
register = 'register',
findPassword = 'findPassword',
wxLogin = 'wxLogin'
}
export const userAuthTypeMap = {
[UserAuthTypeEnum.register]: 'register',
[UserAuthTypeEnum.findPassword]: 'findPassword',
[UserAuthTypeEnum.wxLogin]: 'wxLogin'
};
packages/global/support/user/constant.ts
...@@ -13,10 +13,6 @@ export const userStatusMap = { ...@@ -13,10 +13,6 @@ export const userStatusMap = {
export enum OAuthEnum { export enum OAuthEnum {
github = 'github', github = 'github',
google = 'google' google = 'google',
} wechat = 'wechat'
export enum UserAuthTypeEnum {
register = 'register',
findPassword = 'findPassword'
} }
packages/global/support/user/login/api.d.ts 0 → 100644
export type GetWXLoginQRResponse = {
code: string;
codeUrl: string;
};
packages/global/support/user/team/controller.d.ts
...@@ -15,7 +15,7 @@ export type UpdateTeamProps = { ...@@ -15,7 +15,7 @@ export type UpdateTeamProps = {
teamId: string; teamId: string;
name?: string; name?: string;
avatar?: string; avatar?: string;
tagsUrl?: string; teamDomain?: string;
}; };
/* ------------- member ----------- */ /* ------------- member ----------- */
......
packages/global/support/user/team/tag.d.ts 0 → 100644
export type AuthTeamTagTokenProps = {
teamId: string;
teamToken: string;
};
export type AuthTokenFromTeamDomainResponse = {
success: boolean;
msg?: string;
message?: string;
data: {
uid: string;
tags: string[];
};
};
packages/global/support/user/team/type.d.ts
...@@ -8,22 +8,19 @@ export type TeamSchema = { ...@@ -8,22 +8,19 @@ export type TeamSchema = {
avatar: string; avatar: string;
createTime: Date; createTime: Date;
balance: number; balance: number;
maxSize: number; teamDomain: string;
tagsUrl: string;
limit: { limit: {
lastExportDatasetTime: Date; lastExportDatasetTime: Date;
lastWebsiteSyncTime: Date; lastWebsiteSyncTime: Date;
}; };
}; };
export type tagsType = { export type tagsType = {
label: string,
key: string
}
export type TeamTagsSchema = {
_id: string;
label: string; label: string;
teamId: string;
key: string; key: string;
};
export type TeamTagSchema = TeamTagItemType & {
_id: string;
teamId: string;
createTime: Date; createTime: Date;
}; };
...@@ -56,11 +53,11 @@ export type TeamItemType = { ...@@ -56,11 +53,11 @@ export type TeamItemType = {
avatar: string; avatar: string;
balance: number; balance: number;
tmbId: string; tmbId: string;
teamDomain: string;
defaultTeam: boolean; defaultTeam: boolean;
role: `${TeamMemberRoleEnum}`; role: `${TeamMemberRoleEnum}`;
status: `${TeamMemberStatusEnum}`; status: `${TeamMemberStatusEnum}`;
canWrite: boolean; canWrite: boolean;
maxSize: number;
}; };
export type TeamMemberItemType = { export type TeamMemberItemType = {
...@@ -72,3 +69,8 @@ export type TeamMemberItemType = { ...@@ -72,3 +69,8 @@ export type TeamMemberItemType = {
role: `${TeamMemberRoleEnum}`; role: `${TeamMemberRoleEnum}`;
status: `${TeamMemberStatusEnum}`; status: `${TeamMemberStatusEnum}`;
}; };
export type TeamTagItemType = {
label: string;
key: string;
};
packages/global/support/wallet/bill/type.d.ts
...@@ -21,9 +21,9 @@ export type BillSchemaType = { ...@@ -21,9 +21,9 @@ export type BillSchemaType = {
}; };
}; };
export type ChatModuleBillType = { export type ChatModuleUsageType = {
tokens?: number;
totalPoints: number; totalPoints: number;
moduleName: string; moduleName: string;
model?: string; model?: string;
charsLength?: number;
}; };
packages/global/support/wallet/usage/api.d.ts
...@@ -20,7 +20,6 @@ export type CreateUsageProps = { ...@@ -20,7 +20,6 @@ export type CreateUsageProps = {
appName: string; appName: string;
appId?: string; appId?: string;
totalPoints: number; totalPoints: number;
// inputTokens: number;
source: `${UsageSourceEnum}`; source: `${UsageSourceEnum}`;
list: UsageListItemType[]; list: UsageListItemType[];
}; };
packages/global/support/wallet/usage/type.d.ts
...@@ -2,6 +2,7 @@ import { CreateUsageProps } from './api'; ...@@ -2,6 +2,7 @@ import { CreateUsageProps } from './api';
import { UsageSourceEnum } from './constants'; import { UsageSourceEnum } from './constants';
export type UsageListItemCountType = { export type UsageListItemCountType = {
tokens?: number;
charsLength?: number; charsLength?: number;
duration?: number; duration?: number;
}; };
......
packages/service/common/mongo/init.ts
...@@ -34,6 +34,12 @@ export async function connectMongo({ ...@@ -34,6 +34,12 @@ export async function connectMongo({
retryReads: true retryReads: true
}); });
mongoose.connection.on('error', (error) => {
console.log('mongo error', error);
global.mongodb?.disconnect();
global.mongodb = undefined;
});
console.log('mongo connected'); console.log('mongo connected');
afterHook && (await afterHook()); afterHook && (await afterHook());
......
packages/service/common/system/log.ts
...@@ -3,9 +3,12 @@ import dayjs from 'dayjs'; ...@@ -3,9 +3,12 @@ import dayjs from 'dayjs';
/* add logger */ /* add logger */
export const addLog = { export const addLog = {
log(level: 'info' | 'warn' | 'error', msg: string, obj: Record<string, any> = {}) { log(level: 'info' | 'warn' | 'error', msg: string, obj: Record<string, any> = {}) {
const stringifyObj = JSON.stringify(obj);
const isEmpty = Object.keys(obj).length === 0;
console.log( console.log(
`[${level.toLocaleUpperCase()}] ${dayjs().format('YYYY-MM-DD HH:mm:ss')} ${msg} ${ `[${level.toLocaleUpperCase()}] ${dayjs().format('YYYY-MM-DD HH:mm:ss')} ${msg} ${
level !== 'error' ? JSON.stringify(obj) : '' level !== 'error' && !isEmpty ? stringifyObj : ''
}` }`
); );
......
packages/service/common/system/tools.ts
...@@ -11,6 +11,7 @@ export const initFastGPTConfig = (config?: FastGPTConfigFileType) => { ...@@ -11,6 +11,7 @@ export const initFastGPTConfig = (config?: FastGPTConfigFileType) => {
if (!config) return; if (!config) return;
global.feConfigs = config.feConfigs; global.feConfigs = config.feConfigs;
global.systemEnv = config.systemEnv;
global.subPlans = config.subPlans; global.subPlans = config.subPlans;
global.llmModels = config.llmModels; global.llmModels = config.llmModels;
......
packages/service/common/vectorStore/controller.ts
...@@ -22,7 +22,7 @@ export const insertDatasetDataVector = async ({ ...@@ -22,7 +22,7 @@ export const insertDatasetDataVector = async ({
query: string; query: string;
model: VectorModelItemType; model: VectorModelItemType;
}) => { }) => {
const { vectors, charsLength } = await getVectorsByText({ const { vectors, tokens } = await getVectorsByText({
model, model,
input: query input: query
}); });
...@@ -32,27 +32,7 @@ export const insertDatasetDataVector = async ({ ...@@ -32,27 +32,7 @@ export const insertDatasetDataVector = async ({
}); });
return { return {
charsLength, tokens,
insertId insertId
}; };
}; };
// export const updateDatasetDataVector = async ({
// id,
// ...props
// }: InsertVectorProps & {
// id: string;
// query: string;
// model: VectorModelItemType;
// }) => {
// // insert new vector
// const { charsLength, insertId } = await insertDatasetDataVector(props);
// // delete old vector
// await deleteDatasetDataVector({
// teamId: props.teamId,
// id
// });
// return { charsLength, insertId };
// };
packages/service/core/ai/embedding/index.ts
import { VectorModelItemType } from '@fastgpt/global/core/ai/model.d'; import { VectorModelItemType } from '@fastgpt/global/core/ai/model.d';
import { getAIApi } from '../config'; import { getAIApi } from '../config';
import { replaceValidChars } from '../../chat/utils'; import { countPromptTokens } from '@fastgpt/global/common/string/tiktoken';
type GetVectorProps = { type GetVectorProps = {
model: VectorModelItemType; model: VectorModelItemType;
...@@ -37,7 +37,7 @@ export async function getVectorsByText({ model, input }: GetVectorProps) { ...@@ -37,7 +37,7 @@ export async function getVectorsByText({ model, input }: GetVectorProps) {
} }
return { return {
charsLength: replaceValidChars(input).length, tokens: countPromptTokens(input),
vectors: await Promise.all(res.data.map((item) => unityDimensional(item.embedding))) vectors: await Promise.all(res.data.map((item) => unityDimensional(item.embedding)))
}; };
}); });
......
packages/service/core/ai/functions/createQuestionGuide.ts
import type { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d'; import type { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d';
import { getAIApi } from '../config'; import { getAIApi } from '../config';
import { countGptMessagesChars } from '../../chat/utils'; import { countGptMessagesTokens } from '@fastgpt/global/common/string/tiktoken';
export const Prompt_QuestionGuide = `我不太清楚问你什么问题，请帮我生成 3 个问题，引导我继续提问。问题的长度应小于20个字符，按 JSON 格式返回: ["问题1", "问题2", "问题3"]`; export const Prompt_QuestionGuide = `我不太清楚问你什么问题，请帮我生成 3 个问题，引导我继续提问。问题的长度应小于20个字符，按 JSON 格式返回: ["问题1", "问题2", "问题3"]`;
...@@ -34,12 +34,12 @@ export async function createQuestionGuide({ ...@@ -34,12 +34,12 @@ export async function createQuestionGuide({
const start = answer.indexOf('['); const start = answer.indexOf('[');
const end = answer.lastIndexOf(']'); const end = answer.lastIndexOf(']');
const charsLength = countGptMessagesChars(concatMessages); const tokens = countGptMessagesTokens(concatMessages);
if (start === -1 || end === -1) { if (start === -1 || end === -1) {
return { return {
result: [], result: [],
charsLength: 0 tokens: 0
}; };
} }
...@@ -51,12 +51,12 @@ export async function createQuestionGuide({ ...@@ -51,12 +51,12 @@ export async function createQuestionGuide({
try { try {
return { return {
result: JSON.parse(jsonStr), result: JSON.parse(jsonStr),
charsLength tokens
}; };
} catch (error) { } catch (error) {
return { return {
result: [], result: [],
charsLength: 0 tokens: 0
}; };
} }
} }
packages/service/core/ai/functions/queryExtension.ts
import { replaceVariable } from '@fastgpt/global/common/string/tools'; import { replaceVariable } from '@fastgpt/global/common/string/tools';
import { getAIApi } from '../config'; import { getAIApi } from '../config';
import { ChatItemType } from '@fastgpt/global/core/chat/type'; import { ChatItemType } from '@fastgpt/global/core/chat/type';
import { countGptMessagesChars } from '../../chat/utils'; import { countGptMessagesTokens } from '@fastgpt/global/common/string/tiktoken';
/* /*
query extension - 问题扩展 query extension - 问题扩展
...@@ -106,7 +106,7 @@ export const queryExtension = async ({ ...@@ -106,7 +106,7 @@ export const queryExtension = async ({
rawQuery: string; rawQuery: string;
extensionQueries: string[]; extensionQueries: string[];
model: string; model: string;
charsLength: number; tokens: number;
}> => { }> => {
const systemFewShot = chatBg const systemFewShot = chatBg
? `Q: 对话背景。 ? `Q: 对话背景。
...@@ -148,7 +148,7 @@ A: ${chatBg} ...@@ -148,7 +148,7 @@ A: ${chatBg}
rawQuery: query, rawQuery: query,
extensionQueries: [], extensionQueries: [],
model, model,
charsLength: 0 tokens: 0
}; };
} }
...@@ -161,7 +161,7 @@ A: ${chatBg} ...@@ -161,7 +161,7 @@ A: ${chatBg}
rawQuery: query, rawQuery: query,
extensionQueries: queries, extensionQueries: queries,
model, model,
charsLength: countGptMessagesChars(messages) tokens: countGptMessagesTokens(messages)
}; };
} catch (error) { } catch (error) {
console.log(error); console.log(error);
...@@ -169,7 +169,7 @@ A: ${chatBg} ...@@ -169,7 +169,7 @@ A: ${chatBg}
rawQuery: query, rawQuery: query,
extensionQueries: [], extensionQueries: [],
model, model,
charsLength: 0 tokens: 0
}; };
} }
}; };
packages/service/core/chat/utils.ts
import type { ChatItemType } from '@fastgpt/global/core/chat/type.d'; import type { ChatItemType } from '@fastgpt/global/core/chat/type.d';
import { ChatRoleEnum, IMG_BLOCK_KEY } from '@fastgpt/global/core/chat/constants'; import { ChatRoleEnum, IMG_BLOCK_KEY } from '@fastgpt/global/core/chat/constants';
import { countMessagesTokens, countPromptTokens } from '@fastgpt/global/common/string/tiktoken'; import { countMessagesTokens } from '@fastgpt/global/common/string/tiktoken';
import { adaptRole_Chat2Message } from '@fastgpt/global/core/chat/adapt'; import type { ChatCompletionContentPart } from '@fastgpt/global/core/ai/type.d';
import type {
ChatCompletionContentPart,
ChatMessageItemType
} from '@fastgpt/global/core/ai/type.d';
import axios from 'axios'; import axios from 'axios';
/* slice chat context by tokens */ /* slice chat context by tokens */
...@@ -32,26 +28,34 @@ export function ChatContextFilter({ ...@@ -32,26 +28,34 @@ export function ChatContextFilter({
const chatPrompts: ChatItemType[] = messages.slice(chatStartIndex); const chatPrompts: ChatItemType[] = messages.slice(chatStartIndex);
// reduce token of systemPrompt // reduce token of systemPrompt
maxTokens -= countMessagesTokens({ maxTokens -= countMessagesTokens(systemPrompts);
messages: systemPrompts
});
// 根据 tokens 截断内容 // Save the last chat prompt(question)
const chats: ChatItemType[] = []; const question = chatPrompts.pop();
if (!question) {
return systemPrompts;
}
const chats: ChatItemType[] = [question];
// 从后往前截取对话内容 // 从后往前截取对话内容, 每次需要截取2个
for (let i = chatPrompts.length - 1; i >= 0; i--) { while (1) {
const item = chatPrompts[i]; const assistant = chatPrompts.pop();
chats.unshift(item); const user = chatPrompts.pop();
if (!assistant || !user) {
break;
}
const tokens = countPromptTokens(item.value, adaptRole_Chat2Message(item.obj)); const tokens = countMessagesTokens([assistant, user]);
maxTokens -= tokens; maxTokens -= tokens;
/* 整体 tokens 超出范围，截断 */
/* 整体 tokens 超出范围, system必须保留 */ if (maxTokens < 0) {
if (maxTokens <= 0) { break;
if (chats.length > 1) {
chats.shift();
} }
chats.unshift(assistant);
chats.unshift(user);
if (chatPrompts.length === 0) {
break; break;
} }
} }
...@@ -59,16 +63,6 @@ export function ChatContextFilter({ ...@@ -59,16 +63,6 @@ export function ChatContextFilter({
return [...systemPrompts, ...chats]; return [...systemPrompts, ...chats];
} }
export const replaceValidChars = (str: string) => {
const reg = /[\s\r\n]+/g;
return str.replace(reg, '');
};
export const countMessagesChars = (messages: ChatItemType[]) => {
return messages.reduce((sum, item) => sum + replaceValidChars(item.value).length, 0);
};
export const countGptMessagesChars = (messages: ChatMessageItemType[]) =>
messages.reduce((sum, item) => sum + replaceValidChars(item.content).length, 0);
/** /**
string to vision model. Follow the markdown code block rule for interception: string to vision model. Follow the markdown code block rule for interception:
......
packages/service/core/dataset/schema.ts
...@@ -4,6 +4,7 @@ import { DatasetSchemaType } from '@fastgpt/global/core/dataset/type.d'; ...@@ -4,6 +4,7 @@ import { DatasetSchemaType } from '@fastgpt/global/core/dataset/type.d';
import { import {
DatasetStatusEnum, DatasetStatusEnum,
DatasetStatusMap, DatasetStatusMap,
DatasetTypeEnum,
DatasetTypeMap DatasetTypeMap
} from '@fastgpt/global/core/dataset/constants'; } from '@fastgpt/global/core/dataset/constants';
import { import {
...@@ -39,7 +40,7 @@ const DatasetSchema = new Schema({ ...@@ -39,7 +40,7 @@ const DatasetSchema = new Schema({
type: String, type: String,
enum: Object.keys(DatasetTypeMap), enum: Object.keys(DatasetTypeMap),
required: true, required: true,
default: 'dataset' default: DatasetTypeEnum.dataset
}, },
status: { status: {
type: String, type: String,
......
packages/service/core/dataset/training/controller.ts
...@@ -46,12 +46,16 @@ export async function pushDataListToTrainingQueue({ ...@@ -46,12 +46,16 @@ export async function pushDataListToTrainingQueue({
} = await getCollectionWithDataset(collectionId); } = await getCollectionWithDataset(collectionId);
const checkModelValid = async () => { const checkModelValid = async () => {
if (trainingMode === TrainingModeEnum.chunk) { const agentModelData = datasetModelList?.find((item) => item.model === agentModel);
if (!agentModelData) {
return Promise.reject(`Vector model ${agentModel} is inValid`);
}
const vectorModelData = vectorModelList?.find((item) => item.model === vectorModel); const vectorModelData = vectorModelList?.find((item) => item.model === vectorModel);
if (!vectorModelData) { if (!vectorModelData) {
return Promise.reject(`File model ${vectorModel} is inValid`); return Promise.reject(`File model ${vectorModel} is inValid`);
} }
if (trainingMode === TrainingModeEnum.chunk) {
return { return {
maxToken: vectorModelData.maxToken * 1.3, maxToken: vectorModelData.maxToken * 1.3,
model: vectorModelData.model, model: vectorModelData.model,
...@@ -59,17 +63,14 @@ export async function pushDataListToTrainingQueue({ ...@@ -59,17 +63,14 @@ export async function pushDataListToTrainingQueue({
}; };
} }
if (trainingMode === TrainingModeEnum.qa) { if (trainingMode === TrainingModeEnum.qa || trainingMode === TrainingModeEnum.auto) {
const qaModelData = datasetModelList?.find((item) => item.model === agentModel);
if (!qaModelData) {
return Promise.reject(`Vector model ${agentModel} is inValid`);
}
return { return {
maxToken: qaModelData.maxContext * 0.8, maxToken: agentModelData.maxContext * 0.8,
model: qaModelData.model, model: agentModelData.model,
weight: 0 weight: 0
}; };
} }
return Promise.reject(`Training mode "${trainingMode}" is inValid`); return Promise.reject(`Training mode "${trainingMode}" is inValid`);
}; };
......
packages/service/core/dataset/training/utils.ts 0 → 100644
import { DatasetTrainingSchemaType } from '@fastgpt/global/core/dataset/type';
import { addLog } from '../../../common/system/log';
import { getErrText } from '@fastgpt/global/common/error/utils';
import { MongoDatasetTraining } from './schema';
export const checkInvalidChunkAndLock = async ({
err,
errText,
data
}: {
err: any;
errText: string;
data: DatasetTrainingSchemaType;
}) => {
if (err?.response) {
addLog.error(`openai error: ${errText}`, {
status: err.response?.status,
statusText: err.response?.statusText,
data: err.response?.data
});
} else {
addLog.error(getErrText(err, errText), err);
}
if (
err?.message === 'invalid message format' ||
err?.type === 'invalid_request_error' ||
err?.code === 500
) {
addLog.info('Lock training data');
console.log(err);
try {
await MongoDatasetTraining.findByIdAndUpdate(data._id, {
lockTime: new Date('2998/5/5')
});
} catch (error) {}
return true;
}
return false;
};
packages/service/support/outLink/schema.ts
...@@ -6,6 +6,7 @@ import { ...@@ -6,6 +6,7 @@ import {
TeamCollectionName, TeamCollectionName,
TeamMemberCollectionName TeamMemberCollectionName
} from '@fastgpt/global/support/user/team/constant'; } from '@fastgpt/global/support/user/team/constant';
import { appCollectionName } from '../../core/app/schema';
const OutLinkSchema = new Schema({ const OutLinkSchema = new Schema({
shareId: { shareId: {
...@@ -24,7 +25,7 @@ const OutLinkSchema = new Schema({ ...@@ -24,7 +25,7 @@ const OutLinkSchema = new Schema({
}, },
appId: { appId: {
type: Schema.Types.ObjectId, type: Schema.Types.ObjectId,
ref: 'model', ref: appCollectionName,
required: true required: true
}, },
type: { type: {
......
packages/service/support/permission/auth/common.ts
import { AuthUserTypeEnum } from '@fastgpt/global/support/permission/constant';
import { parseHeaderCert } from '../controller'; import { parseHeaderCert } from '../controller';
import { AuthModeType } from '../type'; import { AuthModeType } from '../type';
import { authOutLinkValid } from './outLink';
import { SERVICE_LOCAL_HOST } from '../../../common/system/tools'; import { SERVICE_LOCAL_HOST } from '../../../common/system/tools';
export const authCert = async (props: AuthModeType) => { export const authCert = async (props: AuthModeType) => {
...@@ -13,25 +11,6 @@ export const authCert = async (props: AuthModeType) => { ...@@ -13,25 +11,6 @@ export const authCert = async (props: AuthModeType) => {
canWrite: true canWrite: true
}; };
}; };
export async function authCertOrShareId({
shareId,
...props
}: AuthModeType & { shareId?: string }) {
if (!shareId) {
return authCert(props);
}
const { shareChat } = await authOutLinkValid({ shareId });
return {
teamId: String(shareChat.teamId),
tmbId: String(shareChat.tmbId),
authType: AuthUserTypeEnum.outLink,
apikey: '',
isOwner: false,
canWrite: false
};
}
/* auth the request from local service */ /* auth the request from local service */
export const authRequestFromLocal = ({ req }: AuthModeType) => { export const authRequestFromLocal = ({ req }: AuthModeType) => {
......
packages/service/support/user/team/controller.ts
...@@ -23,11 +23,11 @@ async function getTeamMember(match: Record<string, any>): Promise<TeamItemType> ...@@ -23,11 +23,11 @@ async function getTeamMember(match: Record<string, any>): Promise<TeamItemType>
avatar: tmb.teamId.avatar, avatar: tmb.teamId.avatar,
balance: tmb.teamId.balance, balance: tmb.teamId.balance,
tmbId: String(tmb._id), tmbId: String(tmb._id),
teamDomain: tmb.teamId?.teamDomain,
role: tmb.role, role: tmb.role,
status: tmb.status, status: tmb.status,
defaultTeam: tmb.defaultTeam, defaultTeam: tmb.defaultTeam,
canWrite: tmb.role !== TeamMemberRoleEnum.visitor, canWrite: tmb.role !== TeamMemberRoleEnum.visitor
maxSize: tmb.teamId.maxSize
}; };
} }
...@@ -55,14 +55,12 @@ export async function createDefaultTeam({ ...@@ -55,14 +55,12 @@ export async function createDefaultTeam({
teamName = 'My Team', teamName = 'My Team',
avatar = '/icon/logo.svg', avatar = '/icon/logo.svg',
balance, balance,
maxSize = 5,
session session
}: { }: {
userId: string; userId: string;
teamName?: string; teamName?: string;
avatar?: string; avatar?: string;
balance?: number; balance?: number;
maxSize?: number;
session: ClientSession; session: ClientSession;
}) { }) {
// auth default team // auth default team
...@@ -82,7 +80,6 @@ export async function createDefaultTeam({ ...@@ -82,7 +80,6 @@ export async function createDefaultTeam({
name: teamName, name: teamName,
avatar, avatar,
balance, balance,
maxSize,
createTime: new Date() createTime: new Date()
} }
], ],
...@@ -106,8 +103,7 @@ export async function createDefaultTeam({ ...@@ -106,8 +103,7 @@ export async function createDefaultTeam({
console.log('default team exist', userId); console.log('default team exist', userId);
await MongoTeam.findByIdAndUpdate(tmb.teamId, { await MongoTeam.findByIdAndUpdate(tmb.teamId, {
$set: { $set: {
...(balance !== undefined && { balance }), ...(balance !== undefined && { balance })
maxSize
} }
}); });
} }
......
packages/service/support/user/team/teamSchema.ts
...@@ -25,11 +25,7 @@ const TeamSchema = new Schema({ ...@@ -25,11 +25,7 @@ const TeamSchema = new Schema({
type: Number, type: Number,
default: 0 default: 0
}, },
maxSize: { teamDomain: {
type: Number,
default: 1
},
tagsUrl: {
type: String type: String
}, },
limit: { limit: {
......
packages/service/support/user/team/teamTagsSchema.ts
import { connectionMongo, type Model } from '../../../common/mongo'; import { connectionMongo, type Model } from '../../../common/mongo';
const { Schema, model, models } = connectionMongo; const { Schema, model, models } = connectionMongo;
import { TeamTagsSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d'; import { TeamTagSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d';
import { import {
TeamCollectionName, TeamCollectionName,
TeamTagsCollectionName TeamTagsCollectionName
} from '@fastgpt/global/support/user/team/constant'; } from '@fastgpt/global/support/user/team/constant';
const TeamTagsSchema = new Schema({ const TeamTagSchema = new Schema({
label: {
type: String,
required: true
},
teamId: { teamId: {
type: Schema.Types.ObjectId, type: Schema.Types.ObjectId,
ref: TeamCollectionName, ref: TeamCollectionName,
required: true required: true
}, },
key: { key: {
type: String type: String,
required: true
},
label: {
type: String,
required: true
}, },
createTime: { createTime: {
type: Date, type: Date,
...@@ -26,10 +27,10 @@ const TeamTagsSchema = new Schema({ ...@@ -26,10 +27,10 @@ const TeamTagsSchema = new Schema({
}); });
try { try {
TeamTagsSchema.index({ teamId: 1 }); TeamTagSchema.index({ teamId: 1 });
} catch (error) { } catch (error) {
console.log(error); console.log(error);
} }
export const MongoTeamTags: Model<TeamTagsSchemaType> = export const MongoTeamTags: Model<TeamTagsSchemaType> =
models[TeamTagsCollectionName] || model(TeamTagsCollectionName, TeamTagsSchema); models[TeamTagsCollectionName] || model(TeamTagsCollectionName, TeamTagSchema);
packages/service/support/wallet/usage/controller.ts
...@@ -31,13 +31,19 @@ export const createTrainingUsage = async ({ ...@@ -31,13 +31,19 @@ export const createTrainingUsage = async ({
{ {
moduleName: 'support.wallet.moduleName.index', moduleName: 'support.wallet.moduleName.index',
model: vectorModel, model: vectorModel,
charsLength: 0, tokens: 0,
amount: 0 amount: 0
}, },
{ {
moduleName: 'support.wallet.moduleName.qa', moduleName: 'support.wallet.moduleName.qa',
model: agentModel, model: agentModel,
charsLength: 0, tokens: 0,
amount: 0
},
{
moduleName: 'core.dataset.training.Auto mode',
model: agentModel,
tokens: 0,
amount: 0 amount: 0
} }
] ]
......
packages/service/support/wallet/usage/utils.ts 0 → 100644
import { ModelTypeEnum, getModelMap } from '../../../core/ai/model';
export const formatModelChars2Points = ({
model,
tokens = 0,
modelType,
multiple = 1000
}: {
model: string;
tokens: number;
modelType: `${ModelTypeEnum}`;
multiple?: number;
}) => {
const modelData = getModelMap?.[modelType]?.(model);
if (!modelData) {
return {
totalPoints: 0,
modelName: ''
};
}
const totalPoints = (modelData.charsPointsPrice || 0) * (tokens / multiple);
return {
modelName: modelData.name,
totalPoints
};
};
packages/service/type.d.ts
import { FastGPTFeConfigsType } from '@fastgpt/global/common/system/types'; import { FastGPTFeConfigsType, SystemEnvType } from '@fastgpt/global/common/system/types';
import { import {
AudioSpeechModelType, AudioSpeechModelType,
ReRankModelItemType, ReRankModelItemType,
...@@ -10,6 +10,7 @@ import { SubPlanType } from '@fastgpt/global/support/wallet/sub/type'; ...@@ -10,6 +10,7 @@ import { SubPlanType } from '@fastgpt/global/support/wallet/sub/type';
declare global { declare global {
var feConfigs: FastGPTFeConfigsType; var feConfigs: FastGPTFeConfigsType;
var systemEnv: SystemEnvType;
var subPlans: SubPlanType | undefined; var subPlans: SubPlanType | undefined;
var llmModels: LLMModelItemType[]; var llmModels: LLMModelItemType[];
......
packages/web/components/common/Icon/constants.ts
...@@ -52,6 +52,7 @@ export const iconPaths = { ...@@ -52,6 +52,7 @@ export const iconPaths = {
'common/uploadFileFill': () => import('./icons/common/uploadFileFill.svg'), 'common/uploadFileFill': () => import('./icons/common/uploadFileFill.svg'),
'common/viewLight': () => import('./icons/common/viewLight.svg'), 'common/viewLight': () => import('./icons/common/viewLight.svg'),
'common/voiceLight': () => import('./icons/common/voiceLight.svg'), 'common/voiceLight': () => import('./icons/common/voiceLight.svg'),
'common/wechatFill': () => import('./icons/common/wechatFill.svg'),
copy: () => import('./icons/copy.svg'), copy: () => import('./icons/copy.svg'),
'core/app/aiFill': () => import('./icons/core/app/aiFill.svg'), 'core/app/aiFill': () => import('./icons/core/app/aiFill.svg'),
'core/app/aiLight': () => import('./icons/core/app/aiLight.svg'), 'core/app/aiLight': () => import('./icons/core/app/aiLight.svg'),
...@@ -144,6 +145,7 @@ export const iconPaths = { ...@@ -144,6 +145,7 @@ export const iconPaths = {
save: () => import('./icons/save.svg'), save: () => import('./icons/save.svg'),
stop: () => import('./icons/stop.svg'), stop: () => import('./icons/stop.svg'),
'support/account/loginoutLight': () => import('./icons/support/account/loginoutLight.svg'), 'support/account/loginoutLight': () => import('./icons/support/account/loginoutLight.svg'),
'support/account/passwordLogin': () => import('./icons/support/account/passwordLogin.svg'),
'support/account/plans': () => import('./icons/support/account/plans.svg'), 'support/account/plans': () => import('./icons/support/account/plans.svg'),
'support/account/promotionLight': () => import('./icons/support/account/promotionLight.svg'), 'support/account/promotionLight': () => import('./icons/support/account/promotionLight.svg'),
'support/bill/extraDatasetsize': () => import('./icons/support/bill/extraDatasetsize.svg'), 'support/bill/extraDatasetsize': () => import('./icons/support/bill/extraDatasetsize.svg'),
......
packages/web/components/common/Icon/icons/common/wechatFill.svg 0 → 100644
<?xml version="1.0" standalone="no"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1708675975750" class="icon" viewBox="0 0 1025 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="4218" xmlns:xlink="http://www.w3.org/1999/xlink" width="200.1953125" height="200"><path d="M1024.16 694.816c0-149.92-143.104-271.392-319.584-271.392-176.576 0-319.68 121.504-319.68 271.392S528 966.208 704.576 966.208c55.456 0 107.648-12.096 153.184-33.248l125.984 54.528-14.592-140.544c34.784-43.392 55.04-95.808 55.04-152.128zM596.832 621.28c-25.152 0-45.472-20.352-45.472-45.472s20.32-45.472 45.472-45.472c25.12 0 45.44 20.384 45.44 45.472s-20.384 45.472-45.44 45.472z m215.392 0c-25.056 0-45.44-20.352-45.44-45.472s20.384-45.472 45.44-45.472c25.184 0 45.536 20.384 45.536 45.472s-20.352 45.472-45.536 45.472zM704.576 387.488c49.376 0 96.416 8.8 139.264 24.64 0.32-5.728 0.992-11.232 0.992-16.992 0-198.08-189.152-358.624-422.432-358.624C189.184 36.512 0.032 197.024 0.032 395.136c0 74.496 26.816 143.776 72.704 201.12L53.472 781.92l166.432-72.096c41.216 19.2 86.784 32.16 134.88 38.784-3.616-17.504-5.824-35.424-5.824-53.792 0.032-169.44 159.552-307.296 355.616-307.296z m-139.808-209.6c33.184 0 60 26.88 60 60 0 33.184-26.816 60.064-60 60.064s-60.032-26.88-60.032-60.064c0-33.152 26.88-60 60.032-60zM280.032 297.952c-33.184 0-60-26.88-60-60.064 0-33.152 26.848-60 60-60 33.184 0 60.032 26.88 60.032 60s-26.88 60.064-60.032 60.064z" fill="#51C332" p-id="4219"></path></svg>
\ No newline at end of file
packages/web/components/common/Icon/icons/support/account/passwordLogin.svg 0 → 100644
<svg t="1709471698048" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="4242"
width="128" height="128">
<path
d="M855.158154 945.664H168.999385c-28.081231 0-50.845538-22.843077-50.845539-51.003077V486.833231C118.153846 458.673231 129.457231 433.230769 157.538462 433.230769h708.923076c28.081231 0 39.502769 25.442462 39.50277 53.602462v407.827692c0 28.16-22.764308 51.003077-50.806154 51.003077z m-340.913231-376.595692a99.761231 99.761231 0 0 0-99.603692 99.958154c0 40.251077 23.827692 74.712615 57.974154 90.54523V827.076923a39.384615 39.384615 0 0 0 78.76923 0v-65.417846a99.879385 99.879385 0 0 0 62.424616-92.632615 99.761231 99.761231 0 0 0-99.564308-99.958154z m0.551385-396.524308c-104.841846 0-189.794462 81.329231-197.159385 184.123077H217.718154C229.060923 201.334154 358.321231 78.769231 516.489846 78.769231s287.428923 122.564923 298.732308 277.897846h-103.266462c-7.364923-102.793846-92.317538-184.123077-197.159384-184.123077z"
fill="#3B9BF8" p-id="4243"></path>
</svg>
\ No newline at end of file
packages/web/components/common/Input/HttpInput/Editor.tsx
...@@ -64,14 +64,17 @@ export default function Editor({ ...@@ -64,14 +64,17 @@ export default function Editor({
useEffect(() => { useEffect(() => {
setKey(getNanoid(6)); setKey(getNanoid(6));
setFocus(false);
}, [updateTrigger]); }, [updateTrigger]);
const dropdownVariables = useMemo( const dropdownVariables = useMemo(
() => () =>
variables.filter((item) => { variables.filter((item) => {
return item.key.includes(currentValue || '') && item.key !== currentValue; const key = item.key.toLowerCase();
const current = currentValue?.toLowerCase();
return key.includes(current || '') && item.key !== currentValue;
}), }),
[currentValue] [currentValue, variables]
); );
return ( return (
...@@ -115,7 +118,7 @@ export default function Editor({ ...@@ -115,7 +118,7 @@ export default function Editor({
}} }}
/> />
{hasVariablePlugin ? <VariablePickerPlugin variables={variables} /> : ''} {hasVariablePlugin ? <VariablePickerPlugin variables={variables} /> : ''}
{hasVariablePlugin ? <VariablePlugin variables={variables} /> : ''} <VariablePlugin variables={variables} />
<OnBlurPlugin onBlur={onBlur} /> <OnBlurPlugin onBlur={onBlur} />
<SingleLinePlugin /> <SingleLinePlugin />
</LexicalComposer> </LexicalComposer>
......
packages/web/components/common/Textarea/JsonEditor/index.tsx
...@@ -21,6 +21,7 @@ type Props = Omit<BoxProps, 'resize' | 'onChange'> & { ...@@ -21,6 +21,7 @@ type Props = Omit<BoxProps, 'resize' | 'onChange'> & {
value?: string; value?: string;
onChange?: (e: string) => void; onChange?: (e: string) => void;
variables?: EditorVariablePickerType[]; variables?: EditorVariablePickerType[];
defaultHeight?: number;
}; };
const options = { const options = {
...@@ -52,11 +53,12 @@ const JSONEditor = ({ ...@@ -52,11 +53,12 @@ const JSONEditor = ({
resize, resize,
variables = [], variables = [],
placeholder, placeholder,
defaultHeight = 100,
...props ...props
}: Props) => { }: Props) => {
const { toast } = useToast(); const { toast } = useToast();
const { t } = useTranslation(); const { t } = useTranslation();
const [height, setHeight] = useState(props.height || 100); const [height, setHeight] = useState(defaultHeight);
const [placeholderDisplay, setPlaceholderDisplay] = useState('block'); const [placeholderDisplay, setPlaceholderDisplay] = useState('block');
const initialY = useRef(0); const initialY = useRef(0);
const completionRegisterRef = useRef<any>(); const completionRegisterRef = useRef<any>();
...@@ -205,7 +207,15 @@ const JSONEditor = ({ ...@@ -205,7 +207,15 @@ const JSONEditor = ({
}, []); }, []);
return ( return (
<Box position={'relative'}> <Box
borderWidth={'1px'}
borderRadius={'md'}
borderColor={'myGray.200'}
py={2}
height={height}
position={'relative'}
{...props}
>
{resize && ( {resize && (
<Box <Box
position={'absolute'} position={'absolute'}
...@@ -219,18 +229,8 @@ const JSONEditor = ({ ...@@ -219,18 +229,8 @@ const JSONEditor = ({
<MyIcon name={'common/editor/resizer'} width={'16px'} height={'16px'} /> <MyIcon name={'common/editor/resizer'} width={'16px'} height={'16px'} />
</Box> </Box>
)} )}
<Box
borderWidth={'1px'}
borderRadius={'md'}
borderColor={'myGray.200'}
py={2}
height={'auto'}
position={'relative'}
{...props}
>
<Editor <Editor
height={height} height={'100%'}
defaultLanguage="json" defaultLanguage="json"
options={options as any} options={options as any}
theme="JSONEditorTheme" theme="JSONEditorTheme"
...@@ -268,7 +268,6 @@ const JSONEditor = ({ ...@@ -268,7 +268,6 @@ const JSONEditor = ({
{placeholder} {placeholder}
</Box> </Box>
</Box> </Box>
</Box>
); );
}; };
......
packages/web/components/common/Textarea/PromptEditor/modules/DropDownMenu/index.tsx
...@@ -44,6 +44,9 @@ export default function DropDownMenu({ ...@@ -44,6 +44,9 @@ export default function DropDownMenu({
position={'fixed'} position={'fixed'}
w={'auto'} w={'auto'}
zIndex={99999} zIndex={99999}
maxH={'300px'}
overflow={'auto'}
className="nowheel"
> >
{variables.map((item, index) => ( {variables.map((item, index) => (
<Flex <Flex
......
packages/web/package.json
...@@ -14,13 +14,13 @@ ...@@ -14,13 +14,13 @@
"@fingerprintjs/fingerprintjs": "^4.2.1", "@fingerprintjs/fingerprintjs": "^4.2.1",
"@monaco-editor/react": "^4.6.0", "@monaco-editor/react": "^4.6.0",
"mammoth": "^1.6.0", "mammoth": "^1.6.0",
"i18next": "^22.5.1", "i18next": "23.10.0",
"joplin-turndown-plugin-gfm": "^1.0.12", "joplin-turndown-plugin-gfm": "^1.0.12",
"next-i18next": "^13.3.0", "next-i18next": "15.2.0",
"pdfjs-dist": "4.0.269", "pdfjs-dist": "4.0.269",
"react": "18.2.0", "react": "18.2.0",
"react-dom": "18.2.0", "react-dom": "18.2.0",
"react-i18next": "^12.3.1", "react-i18next": "13.5.0",
"turndown": "^7.1.2", "turndown": "^7.1.2",
"lexical": "0.12.6", "lexical": "0.12.6",
"@lexical/react": "0.12.6", "@lexical/react": "0.12.6",
......
pnpm-lock.yaml
This source diff could not be displayed because it is too large. You can view the blob instead.
projects/app/package.json
{ {
"name": "app", "name": "app",
"version": "4.6.8", "version": "4.6.9",
"private": false, "private": false,
"scripts": { "scripts": {
"dev": "next dev", "dev": "next dev",
...@@ -33,7 +33,7 @@ ...@@ -33,7 +33,7 @@
"formidable": "^2.1.1", "formidable": "^2.1.1",
"framer-motion": "^9.0.6", "framer-motion": "^9.0.6",
"hyperdown": "^2.4.29", "hyperdown": "^2.4.29",
"i18next": "^22.5.1", "i18next": "23.10.0",
"immer": "^9.0.19", "immer": "^9.0.19",
"js-yaml": "^4.1.0", "js-yaml": "^4.1.0",
"jschardet": "^3.0.0", "jschardet": "^3.0.0",
...@@ -42,13 +42,13 @@ ...@@ -42,13 +42,13 @@
"mermaid": "^10.2.3", "mermaid": "^10.2.3",
"nanoid": "^4.0.1", "nanoid": "^4.0.1",
"next": "13.5.2", "next": "13.5.2",
"next-i18next": "^13.3.0", "next-i18next": "15.2.0",
"nprogress": "^0.2.0", "nprogress": "^0.2.0",
"react": "18.2.0", "react": "18.2.0",
"react-day-picker": "^8.7.1", "react-day-picker": "^8.7.1",
"react-dom": "18.2.0", "react-dom": "18.2.0",
"react-hook-form": "^7.43.1", "react-hook-form": "^7.43.1",
"react-i18next": "^12.3.1", "react-i18next": "13.5.0",
"react-markdown": "^8.0.7", "react-markdown": "^8.0.7",
"react-syntax-highlighter": "^15.5.0", "react-syntax-highlighter": "^15.5.0",
"reactflow": "^11.7.4", "reactflow": "^11.7.4",
......
projects/app/public/docs/versionIntro.md
### Fast GPT V4.6.8 ### Fast GPT V4.6.8
1. 新增 - 知识库搜索合并模块。 1. 新增 - 知识库新增“增强处理”训练模式，可生成更多类型索引。
2. 新增 - 新的 Http 模块，支持更加灵活的参数传入。同时支持了输入输出自动数据类型转化，例如：接口输出的 JSON 类型会自动转成字符串类型，直接给其他模块使用。此外，还补充了一些例子，可在文档中查看。 2. 新增 - 完善了HTTP模块的变量提示。
3. 优化 - 问题优化并入知识库搜索模块，无需单独配置。并且问题优化的同时，实现了问题扩展，丰富搜索的语义。（知识库模块会看到有2个参数配置，有一个是多余的，如果想让它消失，可以删除模块，重新增加一个） 3. 新增 - HTTP模块支持OpenAI单接口导入。
4. 修复 - 语音输入文件无法上传。 4. 优化 - 问题补全。增加英文类型。同时可以设置为单独模块，方便复用。
5. 修复 - 对话框重新生成无法使用。 5. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow/intro)
6. [点击查看高级编排介绍文档](https://doc.fastgpt.in/docs/workflow/intro) 6. [使用文档](https://doc.fastgpt.in/docs/intro/)
7. [使用文档](https://doc.fastgpt.in/docs/intro/) 7. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/)
8. [点击查看商业版](https://doc.fastgpt.in/docs/commercial/) \ No newline at end of file
projects/app/public/locales/en/common.json
...@@ -268,6 +268,7 @@ ...@@ -268,6 +268,7 @@
"Quote templates": "Quote templates", "Quote templates": "Quote templates",
"Random": "Random", "Random": "Random",
"Save and preview": "Save", "Save and preview": "Save",
"Search team tags": "Search tags",
"Select TTS": "Select TTS", "Select TTS": "Select TTS",
"Select app from template": "Select from the template", "Select app from template": "Select from the template",
"Select quote template": "Select quote template", "Select quote template": "Select quote template",
...@@ -278,6 +279,7 @@ ...@@ -278,6 +279,7 @@
"Simple Config Tip": "Only basic functions are included. For complex agent functions, use advanced orchestration.", "Simple Config Tip": "Only basic functions are included. For complex agent functions, use advanced orchestration.",
"TTS": "Audio Speech", "TTS": "Audio Speech",
"TTS Tip": "After this function is enabled, the voice playback function can be used after each conversation. Use of this feature may incur additional charges.", "TTS Tip": "After this function is enabled, the voice playback function can be used after each conversation. Use of this feature may incur additional charges.",
"Team tags": "Team tags",
"Temperature": "Temperature", "Temperature": "Temperature",
"Welcome Text": "Welcome Text", "Welcome Text": "Welcome Text",
"create app": "Create App", "create app": "Create App",
...@@ -409,7 +411,7 @@ ...@@ -409,7 +411,7 @@
"Stop Speak": "Stop Speak", "Stop Speak": "Stop Speak",
"Type a message": "Input problem", "Type a message": "Input problem",
"Unpin": "Unpin", "Unpin": "Unpin",
"You need to a chat app": "You need to a chat app", "You need to a chat app": "You don't have apps available",
"error": { "error": {
"Chat error": "Chat error", "Chat error": "Chat error",
"Messages empty": "Interface content is empty, maybe the text is too long ~", "Messages empty": "Interface content is empty, maybe the text is too long ~",
...@@ -468,7 +470,7 @@ ...@@ -468,7 +470,7 @@
"module similarity": "Similarity", "module similarity": "Similarity",
"module temperature": "Temperature", "module temperature": "Temperature",
"module time": "Running Time", "module time": "Running Time",
"module tokens": "Tokens", "module tokens": "Total Tokens",
"plugin output": "Plugin Output", "plugin output": "Plugin Output",
"search using reRank": "ReRank", "search using reRank": "ReRank",
"text output": "Text Output" "text output": "Text Output"
...@@ -594,6 +596,7 @@ ...@@ -594,6 +596,7 @@
"file": "File", "file": "File",
"folder": "Folder", "folder": "Folder",
"import": { "import": {
"Auto mode Estimated Price Tips": "Enhanced processing calls the file processing model: {{price}} integral /1k Tokens",
"Auto process": "Auto", "Auto process": "Auto",
"Auto process desc": "Automatically set segmentation and preprocessing rules", "Auto process desc": "Automatically set segmentation and preprocessing rules",
"CSV Import": "CSV QA Import", "CSV Import": "CSV QA Import",
...@@ -615,9 +618,9 @@ ...@@ -615,9 +618,9 @@
"Data file progress": "Data upload progress", "Data file progress": "Data upload progress",
"Data process params": "Data process params", "Data process params": "Data process params",
"Down load csv template": "Down load csv template", "Down load csv template": "Down load csv template",
"Embedding Estimated Price Tips": "Index billing: {{price}}/1k chars", "Embedding Estimated Price Tips": "Index billing: {{price}}/1k Tokens",
"Estimated Price": "Estimated Price: : {{amount}}{{unit}}", "Estimated Price": "Estimated Price: : {{amount}}{{unit}}",
"Estimated Price Tips": "QA charges\nInput: 1k chars={{charsPointsPrice}} points", "Estimated Price Tips": "QA charges: {{charsPointsPrice}} points/1k Tokens",
"Estimated points": "About {{points}} points", "Estimated points": "About {{points}} points",
"Fetch Error": "Get link failed", "Fetch Error": "Get link failed",
"Fetch Url": "Url", "Fetch Url": "Url",
...@@ -638,7 +641,7 @@ ...@@ -638,7 +641,7 @@
"Preview chunks": "Chunks", "Preview chunks": "Chunks",
"Preview raw text": "Preview file text (max show 10000 words)", "Preview raw text": "Preview file text (max show 10000 words)",
"Process way": "Process way", "Process way": "Process way",
"QA Estimated Price Tips": "QA billing: {{price}}/1k characters (including input and output)", "QA Estimated Price Tips": "QA billing: {{price}}/1k Tokens (including input and output)",
"QA Import": "QA Split", "QA Import": "QA Split",
"QA Import Tip": "According to certain rules, the text is broken into a larger paragraph, and the AI is invoked to generate a question and answer pair for the paragraph.", "QA Import Tip": "According to certain rules, the text is broken into a larger paragraph, and the AI is invoked to generate a question and answer pair for the paragraph.",
"Re Preview": "RePreview", "Re Preview": "RePreview",
...@@ -737,6 +740,8 @@ ...@@ -737,6 +740,8 @@
}, },
"training": { "training": {
"Agent queue": "QA wait list", "Agent queue": "QA wait list",
"Auto mode": "Enhancement process",
"Auto mode Tip": "Subindex and call model are used to generate relevant questions and abstracts to increase the semantic richness of data blocks and facilitate retrieval. It consumes more storage space and increases the number of AI calls.",
"Chunk mode": "Chunk split", "Chunk mode": "Chunk split",
"Full": "Expect more than 5 minutes", "Full": "Expect more than 5 minutes",
"Leisure": "Leisure", "Leisure": "Leisure",
...@@ -1217,13 +1222,33 @@ ...@@ -1217,13 +1222,33 @@
"Sending Code": "Sending" "Sending Code": "Sending"
}, },
"login": { "login": {
"And": "&",
"Email": "Email",
"Forget Password": "Forget Password?",
"Github": "Github", "Github": "Github",
"Google": "Google", "Google": "Google",
"Provider error": "Login exception, please try again" "Password": "Password",
"Password login": "Password login",
"Phone": "Phone Login",
"Phone number": "Phone",
"Policy tip": "By using it, you agree with us",
"Privacy": "Privacy",
"Provider error": "Login exception, please try again",
"Register": "Register",
"Root login": "Log in as the root user",
"Root password placeholder": "root password is the environment variable you set",
"Terms": "Terms",
"Username": "Username",
"Wechat": "Wechat",
"Wx qr login": "Wechat scan code login"
}, },
"team": { "team": {
"Dataset usage": "Dataset usage", "Dataset usage": "Dataset usage",
"member": "Member" "Team Tags Async Success": "Team async success",
"member": "Member",
"tag": {
"Have not opened": "Team chat is not enabled"
}
} }
}, },
"wallet": { "wallet": {
......
projects/app/public/locales/zh/common.json
...@@ -268,6 +268,7 @@ ...@@ -268,6 +268,7 @@
"Quote templates": "引用内容模板", "Quote templates": "引用内容模板",
"Random": "发散", "Random": "发散",
"Save and preview": "保存并预览", "Save and preview": "保存并预览",
"Search team tags": "搜索标签",
"Select TTS": "选择语音播放模式", "Select TTS": "选择语音播放模式",
"Select app from template": "从模板中选择", "Select app from template": "从模板中选择",
"Select quote template": "选择引用提示模板", "Select quote template": "选择引用提示模板",
...@@ -278,6 +279,7 @@ ...@@ -278,6 +279,7 @@
"Simple Config Tip": "仅包含基础功能，复杂 agent 功能请使用高级编排。", "Simple Config Tip": "仅包含基础功能，复杂 agent 功能请使用高级编排。",
"TTS": "语音播报", "TTS": "语音播报",
"TTS Tip": "开启后，每次对话后可使用语音播放功能。使用该功能可能产生额外费用。", "TTS Tip": "开启后，每次对话后可使用语音播放功能。使用该功能可能产生额外费用。",
"Team tags": "团队标签",
"Temperature": "温度", "Temperature": "温度",
"Welcome Text": "对话开场白", "Welcome Text": "对话开场白",
"create app": "创建属于你的 AI 应用", "create app": "创建属于你的 AI 应用",
...@@ -409,7 +411,7 @@ ...@@ -409,7 +411,7 @@
"Stop Speak": "停止录音", "Stop Speak": "停止录音",
"Type a message": "输入问题", "Type a message": "输入问题",
"Unpin": "取消置顶", "Unpin": "取消置顶",
"You need to a chat app": "鉴权失败，暂无权限访问应用", "You need to a chat app": "你没有可用的应用",
"error": { "error": {
"Chat error": "对话出现异常", "Chat error": "对话出现异常",
"Messages empty": "接口内容为空，可能文本超长了~", "Messages empty": "接口内容为空，可能文本超长了~",
...@@ -468,7 +470,7 @@ ...@@ -468,7 +470,7 @@
"module similarity": "相似度", "module similarity": "相似度",
"module temperature": "温度", "module temperature": "温度",
"module time": "运行时长", "module time": "运行时长",
"module tokens": "Tokens", "module tokens": "总Tokens",
"plugin output": "插件输出值", "plugin output": "插件输出值",
"search using reRank": "结果重排", "search using reRank": "结果重排",
"text output": "文本输出" "text output": "文本输出"
...@@ -596,13 +598,14 @@ ...@@ -596,13 +598,14 @@
"file": "文件", "file": "文件",
"folder": "目录", "folder": "目录",
"import": { "import": {
"Auto mode Estimated Price Tips": "增强处理需调用文件处理模型: {{price}}积分/1k Tokens",
"Auto process": "自动", "Auto process": "自动",
"Auto process desc": "自动设置分割和预处理规则", "Auto process desc": "自动设置分割和预处理规则",
"CSV Import": "CSV 导入", "CSV Import": "CSV 导入",
"CSV Import Tip": "通过批量导入问答对，要求提前整理好数据", "CSV Import Tip": "通过批量导入问答对，要求提前整理好数据",
"Chunk Range": "范围: {{min}}~{{max}}", "Chunk Range": "范围: {{min}}~{{max}}",
"Chunk Split": "直接分段", "Chunk Split": "直接分段",
"Chunk Split Tip": "将文本按一定的规则进行分段处理后，转成可进行语义搜索的格式，适合绝大多数场景。", "Chunk Split Tip": "将文本按一定的规则进行分段处理后，转成可进行语义搜索的格式，适合绝大多数场景。不需要调用模型额外处理，成本低。",
"Chunk length": "分块总量", "Chunk length": "分块总量",
"Csv format error": "csv 文件格式有误,请确保 index 和 content 两列", "Csv format error": "csv 文件格式有误,请确保 index 和 content 两列",
"Custom file": "自定义文本", "Custom file": "自定义文本",
...@@ -617,9 +620,9 @@ ...@@ -617,9 +620,9 @@
"Data file progress": "数据上传进度", "Data file progress": "数据上传进度",
"Data process params": "数据处理参数", "Data process params": "数据处理参数",
"Down load csv template": "点击下载 CSV 模板", "Down load csv template": "点击下载 CSV 模板",
"Embedding Estimated Price Tips": "索引计费: {{price}}积分/1k字符", "Embedding Estimated Price Tips": "索引计费: {{price}}积分/1k Tokens",
"Estimated Price": "预估价格: {{amount}}{{unit}}", "Estimated Price": "预估价格: {{amount}}{{unit}}",
"Estimated Price Tips": "QA计费为\n输入: 1k字符 = {{charsPointsPrice}}积分", "Estimated Price Tips": "QA计费为\n输入: {{charsPointsPrice}}积分/1k Tokens",
"Estimated points": "预估消耗 {{points}} 积分", "Estimated points": "预估消耗 {{points}} 积分",
"Fetch Error": "获取链接失败", "Fetch Error": "获取链接失败",
"Fetch Url": "网络链接", "Fetch Url": "网络链接",
...@@ -640,9 +643,9 @@ ...@@ -640,9 +643,9 @@
"Preview chunks": "分段预览", "Preview chunks": "分段预览",
"Preview raw text": "预览源文本（最多展示10000字）", "Preview raw text": "预览源文本（最多展示10000字）",
"Process way": "处理方式", "Process way": "处理方式",
"QA Estimated Price Tips": "QA计费为: {{price}}积分/1k 字符(包含输入和输出)", "QA Estimated Price Tips": "QA计费为: {{price}}积分/1k Tokens(包含输入和输出)",
"QA Import": "QA拆分", "QA Import": "QA拆分",
"QA Import Tip": "根据一定规则，将文本拆成一段较大的段落，调用 AI 为该段落生成问答对。", "QA Import Tip": "根据一定规则，将文本拆成一段较大的段落，调用 AI 为该段落生成问答对。有非常高的检索精度，但是会丢失很多内容细节。",
"Re Preview": "重新生成预览", "Re Preview": "重新生成预览",
"Select file": "选择文件", "Select file": "选择文件",
"Select source": "选择来源", "Select source": "选择来源",
...@@ -739,6 +742,8 @@ ...@@ -739,6 +742,8 @@
}, },
"training": { "training": {
"Agent queue": "QA训练排队", "Agent queue": "QA训练排队",
"Auto mode": "增强处理(实验)",
"Auto mode Tip": "通过子索引以及调用模型生成相关问题与摘要，来增加数据块的语义丰富度，更利于检索。需要消耗更多的存储空间和增加AI调用次数。",
"Chunk mode": "直接分段", "Chunk mode": "直接分段",
"Full": "预计5分钟以上", "Full": "预计5分钟以上",
"Leisure": "空闲", "Leisure": "空闲",
...@@ -1219,13 +1224,33 @@ ...@@ -1219,13 +1224,33 @@
"Sending Code": "正在发送" "Sending Code": "正在发送"
}, },
"login": { "login": {
"And": "和",
"Email": "邮箱",
"Forget Password": "忘记密码?",
"Github": "Github 登录", "Github": "Github 登录",
"Google": "Google 登录", "Google": "Google 登录",
"Provider error": "登录异常，请重试" "Password": "密码",
"Password login": "密码登录",
"Phone": "手机号登录",
"Phone number": "手机号",
"Policy tip": "使用即代表你同意我们的",
"Privacy": "隐私政策",
"Provider error": "登录异常，请重试",
"Register": "注册账号",
"Root login": "使用root用户登录",
"Root password placeholder": "root密码为你设置的环境变量",
"Terms": "服务协议",
"Username": "用户名",
"Wechat": "微信登录",
"Wx qr login": "微信扫码登录"
}, },
"team": { "team": {
"Dataset usage": "知识库容量", "Dataset usage": "知识库容量",
"member": "成员" "Team Tags Async Success": "同步完成",
"member": "成员",
"tag": {
"Have not opened": "未开通团队聊天功能"
}
} }
}, },
"wallet": { "wallet": {
......
projects/app/src/components/ChatBox/MessageInput.tsx
...@@ -13,6 +13,7 @@ import { IMG_BLOCK_KEY } from '@fastgpt/global/core/chat/constants'; ...@@ -13,6 +13,7 @@ import { IMG_BLOCK_KEY } from '@fastgpt/global/core/chat/constants';
import { addDays } from 'date-fns'; import { addDays } from 'date-fns';
import { useRequest } from '@/web/common/hooks/useRequest'; import { useRequest } from '@/web/common/hooks/useRequest';
import { MongoImageTypeEnum } from '@fastgpt/global/common/file/image/constants'; import { MongoImageTypeEnum } from '@fastgpt/global/common/file/image/constants';
import { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 6); const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 6);
enum FileTypeEnum { enum FileTypeEnum {
...@@ -35,8 +36,12 @@ const MessageInput = ({ ...@@ -35,8 +36,12 @@ const MessageInput = ({
isChatting, isChatting,
TextareaDom, TextareaDom,
showFileSelector = false, showFileSelector = false,
resetInputVal resetInputVal,
}: { shareId,
outLinkUid,
teamId,
teamToken
}: OutLinkChatAuthProps & {
onChange?: (e: string) => void; onChange?: (e: string) => void;
onSendMessage: (e: string) => void; onSendMessage: (e: string) => void;
onStop: () => void; onStop: () => void;
...@@ -47,7 +52,6 @@ const MessageInput = ({ ...@@ -47,7 +52,6 @@ const MessageInput = ({
}) => { }) => {
const [, startSts] = useTransition(); const [, startSts] = useTransition();
const { shareId } = useRouter().query as { shareId?: string };
const { const {
isSpeaking, isSpeaking,
isTransCription, isTransCription,
...@@ -56,7 +60,7 @@ const MessageInput = ({ ...@@ -56,7 +60,7 @@ const MessageInput = ({
speakingTimeString, speakingTimeString,
renderAudioGraph, renderAudioGraph,
stream stream
} = useSpeech({ shareId }); } = useSpeech({ shareId, outLinkUid, teamId, teamToken });
const { isPc } = useSystemStore(); const { isPc } = useSystemStore();
const canvasRef = useRef<HTMLCanvasElement>(null); const canvasRef = useRef<HTMLCanvasElement>(null);
const { t } = useTranslation(); const { t } = useTranslation();
...@@ -82,7 +86,10 @@ const MessageInput = ({ ...@@ -82,7 +86,10 @@ const MessageInput = ({
maxSize: 1024 * 1024 * 5, maxSize: 1024 * 1024 * 5,
// 30 day expired. // 30 day expired.
expiredTime: addDays(new Date(), 7), expiredTime: addDays(new Date(), 7),
shareId shareId,
outLinkUid,
teamId,
teamToken
}); });
setFileList((state) => setFileList((state) =>
state.map((item) => state.map((item) =>
...@@ -320,7 +327,7 @@ ${images.map((img) => JSON.stringify({ src: img.src })).join('\n')} ...@@ -320,7 +327,7 @@ ${images.map((img) => JSON.stringify({ src: img.src })).join('\n')}
rows={1} rows={1}
height={'22px'} height={'22px'}
lineHeight={'22px'} lineHeight={'22px'}
maxHeight={'150px'} maxHeight={'50vh'}
maxLength={-1} maxLength={-1}
overflowY={'auto'} overflowY={'auto'}
whiteSpace={'pre-wrap'} whiteSpace={'pre-wrap'}
......
projects/app/src/components/ChatBox/QuoteModal.tsx
import React, { useMemo, useState } from 'react'; import React, { useMemo } from 'react';
import { ModalBody, Box, useTheme } from '@chakra-ui/react'; import { ModalBody, Box, useTheme } from '@chakra-ui/react';
import MyModal from '../MyModal'; import MyModal from '../MyModal';
...@@ -10,12 +10,12 @@ import RawSourceBox from '../core/dataset/RawSourceBox'; ...@@ -10,12 +10,12 @@ import RawSourceBox from '../core/dataset/RawSourceBox';
const QuoteModal = ({ const QuoteModal = ({
rawSearch = [], rawSearch = [],
onClose, onClose,
isShare, showDetail,
metadata metadata
}: { }: {
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseItemType[];
onClose: () => void; onClose: () => void;
isShare: boolean; showDetail: boolean;
metadata?: { metadata?: {
collectionId: string; collectionId: string;
sourceId?: string; sourceId?: string;
...@@ -57,7 +57,7 @@ const QuoteModal = ({ ...@@ -57,7 +57,7 @@ const QuoteModal = ({
} }
> >
<ModalBody> <ModalBody>
<QuoteList rawSearch={filterResults} isShare={isShare} /> <QuoteList rawSearch={filterResults} showDetail={showDetail} />
</ModalBody> </ModalBody>
</MyModal> </MyModal>
</> </>
...@@ -68,10 +68,10 @@ export default QuoteModal; ...@@ -68,10 +68,10 @@ export default QuoteModal;
export const QuoteList = React.memo(function QuoteList({ export const QuoteList = React.memo(function QuoteList({
rawSearch = [], rawSearch = [],
isShare showDetail
}: { }: {
rawSearch: SearchDataResponseItemType[]; rawSearch: SearchDataResponseItemType[];
isShare: boolean; showDetail: boolean;
}) { }) {
const theme = useTheme(); const theme = useTheme();
...@@ -88,7 +88,7 @@ export const QuoteList = React.memo(function QuoteList({ ...@@ -88,7 +88,7 @@ export const QuoteList = React.memo(function QuoteList({
_hover={{ '& .hover-data': { display: 'flex' } }} _hover={{ '& .hover-data': { display: 'flex' } }}
bg={i % 2 === 0 ? 'white' : 'myWhite.500'} bg={i % 2 === 0 ? 'white' : 'myWhite.500'}
> >
<QuoteItem quoteItem={item} canViewSource={!isShare} linkToDataset={!isShare} /> <QuoteItem quoteItem={item} canViewSource={showDetail} linkToDataset={showDetail} />
</Box> </Box>
))} ))}
</> </>
......
projects/app/src/components/ChatBox/ResponseTags.tsx
import React, { useMemo, useState } from 'react'; import React, { useMemo, useState } from 'react';
import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type.d'; import type { ChatHistoryItemResType } from '@fastgpt/global/core/chat/type.d';
import type { ChatItemType } from '@fastgpt/global/core/chat/type'; import type { ChatItemType } from '@fastgpt/global/core/chat/type';
import { Flex, BoxProps, useDisclosure, Image, useTheme, Box } from '@chakra-ui/react'; import { Flex, BoxProps, useDisclosure, useTheme, Box } from '@chakra-ui/react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { useSystemStore } from '@/web/common/system/useSystemStore'; import { useSystemStore } from '@/web/common/system/useSystemStore';
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type'; import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type';
...@@ -20,10 +20,10 @@ const WholeResponseModal = dynamic(() => import('./WholeResponseModal'), { ssr: ...@@ -20,10 +20,10 @@ const WholeResponseModal = dynamic(() => import('./WholeResponseModal'), { ssr:
const ResponseTags = ({ const ResponseTags = ({
responseData = [], responseData = [],
isShare showDetail
}: { }: {
responseData?: ChatHistoryItemResType[]; responseData?: ChatHistoryItemResType[];
isShare: boolean; showDetail: boolean;
}) => { }) => {
const theme = useTheme(); const theme = useTheme();
const { isPc } = useSystemStore(); const { isPc } = useSystemStore();
...@@ -76,13 +76,13 @@ const ResponseTags = ({ ...@@ -76,13 +76,13 @@ const ResponseTags = ({
sourceName: item.sourceName, sourceName: item.sourceName,
sourceId: item.sourceId, sourceId: item.sourceId,
icon: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }), icon: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }),
canReadQuote: !isShare || strIsLink(item.sourceId), canReadQuote: showDetail || strIsLink(item.sourceId),
collectionId: item.collectionId collectionId: item.collectionId
})), })),
historyPreview: chatData?.historyPreview, historyPreview: chatData?.historyPreview,
runningTime: +responseData.reduce((sum, item) => sum + (item.runningTime || 0), 0).toFixed(2) runningTime: +responseData.reduce((sum, item) => sum + (item.runningTime || 0), 0).toFixed(2)
}; };
}, [isShare, responseData]); }, [showDetail, responseData]);
const TagStyles: BoxProps = { const TagStyles: BoxProps = {
mr: 2, mr: 2,
...@@ -134,7 +134,7 @@ const ResponseTags = ({ ...@@ -134,7 +134,7 @@ const ResponseTags = ({
</Flex> </Flex>
</> </>
)} )}
{!isShare && ( {showDetail && (
<Flex alignItems={'center'} mt={3} flexWrap={'wrap'}> <Flex alignItems={'center'} mt={3} flexWrap={'wrap'}>
{quoteList.length > 0 && ( {quoteList.length > 0 && (
<MyTooltip label="查看引用"> <MyTooltip label="查看引用">
...@@ -187,7 +187,7 @@ const ResponseTags = ({ ...@@ -187,7 +187,7 @@ const ResponseTags = ({
{!!quoteModalData && ( {!!quoteModalData && (
<QuoteModal <QuoteModal
{...quoteModalData} {...quoteModalData}
isShare={isShare} showDetail={showDetail}
onClose={() => setQuoteModalData(undefined)} onClose={() => setQuoteModalData(undefined)}
/> />
)} )}
...@@ -195,7 +195,11 @@ const ResponseTags = ({ ...@@ -195,7 +195,11 @@ const ResponseTags = ({
<ContextModal context={contextModalData} onClose={() => setContextModalData(undefined)} /> <ContextModal context={contextModalData} onClose={() => setContextModalData(undefined)} />
)} )}
{isOpenWholeModal && ( {isOpenWholeModal && (
<WholeResponseModal response={responseData} isShare={isShare} onClose={onCloseWholeModal} /> <WholeResponseModal
response={responseData}
showDetail={showDetail}
onClose={onCloseWholeModal}
/>
)} )}
</> </>
); );
......
projects/app/src/components/ChatBox/WholeResponseModal.tsx
...@@ -51,11 +51,11 @@ function Row({ ...@@ -51,11 +51,11 @@ function Row({
const WholeResponseModal = ({ const WholeResponseModal = ({
response, response,
isShare, showDetail,
onClose onClose
}: { }: {
response: ChatHistoryItemResType[]; response: ChatHistoryItemResType[];
isShare: boolean; showDetail: boolean;
onClose: () => void; onClose: () => void;
}) => { }) => {
const { t } = useTranslation(); const { t } = useTranslation();
...@@ -78,7 +78,7 @@ const WholeResponseModal = ({ ...@@ -78,7 +78,7 @@ const WholeResponseModal = ({
} }
> >
<Flex h={'100%'} flexDirection={'column'}> <Flex h={'100%'} flexDirection={'column'}>
<ResponseBox response={response} isShare={isShare} /> <ResponseBox response={response} showDetail={showDetail} />
</Flex> </Flex>
</MyModal> </MyModal>
); );
...@@ -88,10 +88,10 @@ export default WholeResponseModal; ...@@ -88,10 +88,10 @@ export default WholeResponseModal;
const ResponseBox = React.memo(function ResponseBox({ const ResponseBox = React.memo(function ResponseBox({
response, response,
isShare showDetail
}: { }: {
response: ChatHistoryItemResType[]; response: ChatHistoryItemResType[];
isShare: boolean; showDetail: boolean;
}) { }) {
const theme = useTheme(); const theme = useTheme();
const { t } = useTranslation(); const { t } = useTranslation();
...@@ -142,10 +142,7 @@ const ResponseBox = React.memo(function ResponseBox({ ...@@ -142,10 +142,7 @@ const ResponseBox = React.memo(function ResponseBox({
value={`${activeModule?.runningTime || 0}s`} value={`${activeModule?.runningTime || 0}s`}
/> />
<Row label={t('core.chat.response.module model')} value={activeModule?.model} /> <Row label={t('core.chat.response.module model')} value={activeModule?.model} />
<Row <Row label={t('core.chat.response.module tokens')} value={`${activeModule?.tokens}`} />
label={t('support.wallet.usage.Chars length')}
value={`${activeModule?.charsLength}`}
/>
<Row label={t('core.chat.response.module query')} value={activeModule?.query} /> <Row label={t('core.chat.response.module query')} value={activeModule?.query} />
<Row <Row
label={t('core.chat.response.context total length')} label={t('core.chat.response.context total length')}
...@@ -188,7 +185,7 @@ const ResponseBox = React.memo(function ResponseBox({ ...@@ -188,7 +185,7 @@ const ResponseBox = React.memo(function ResponseBox({
{activeModule.quoteList && activeModule.quoteList.length > 0 && ( {activeModule.quoteList && activeModule.quoteList.length > 0 && (
<Row <Row
label={t('core.chat.response.module quoteList')} label={t('core.chat.response.module quoteList')}
rawDom={<QuoteList isShare={isShare} rawSearch={activeModule.quoteList} />} rawDom={<QuoteList showDetail={showDetail} rawSearch={activeModule.quoteList} />}
/> />
)} )}
</> </>
...@@ -280,7 +277,7 @@ const ResponseBox = React.memo(function ResponseBox({ ...@@ -280,7 +277,7 @@ const ResponseBox = React.memo(function ResponseBox({
{activeModule?.pluginDetail && activeModule?.pluginDetail.length > 0 && ( {activeModule?.pluginDetail && activeModule?.pluginDetail.length > 0 && (
<Row <Row
label={t('core.chat.response.Plugin Resonse Detail')} label={t('core.chat.response.Plugin Resonse Detail')}
rawDom={<ResponseBox response={activeModule.pluginDetail} isShare={isShare} />} rawDom={<ResponseBox response={activeModule.pluginDetail} showDetail={showDetail} />}
/> />
)} )}
</> </>
......
projects/app/src/components/ChatBox/index.tsx
...@@ -68,6 +68,7 @@ import type { AppTTSConfigType, VariableItemType } from '@fastgpt/global/core/mo ...@@ -68,6 +68,7 @@ import type { AppTTSConfigType, VariableItemType } from '@fastgpt/global/core/mo
import MessageInput from './MessageInput'; import MessageInput from './MessageInput';
import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants'; import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants';
import ChatBoxDivider from '../core/chat/Divider'; import ChatBoxDivider from '../core/chat/Divider';
import { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 24); const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 24);
...@@ -106,7 +107,7 @@ const MessageCardStyle: BoxProps = { ...@@ -106,7 +107,7 @@ const MessageCardStyle: BoxProps = {
maxW: ['calc(100% - 25px)', 'calc(100% - 40px)'] maxW: ['calc(100% - 25px)', 'calc(100% - 40px)']
}; };
type Props = { type Props = OutLinkChatAuthProps & {
feedbackType?: `${FeedbackTypeEnum}`; feedbackType?: `${FeedbackTypeEnum}`;
showMarkIcon?: boolean; // admin mark dataset showMarkIcon?: boolean; // admin mark dataset
showVoiceIcon?: boolean; showVoiceIcon?: boolean;
...@@ -120,9 +121,6 @@ type Props = { ...@@ -120,9 +121,6 @@ type Props = {
// not chat test params // not chat test params
appId?: string; appId?: string;
chatId?: string; chatId?: string;
shareId?: string;
shareTeamId?: string;
outLinkUid?: string;
onUpdateVariable?: (e: Record<string, any>) => void; onUpdateVariable?: (e: Record<string, any>) => void;
onStartChat?: (e: StartChatFnProps) => Promise<{ onStartChat?: (e: StartChatFnProps) => Promise<{
...@@ -147,8 +145,9 @@ const ChatBox = ( ...@@ -147,8 +145,9 @@ const ChatBox = (
appId, appId,
chatId, chatId,
shareId, shareId,
shareTeamId,
outLinkUid, outLinkUid,
teamId,
teamToken,
onUpdateVariable, onUpdateVariable,
onStartChat, onStartChat,
onDelMessage onDelMessage
...@@ -288,7 +287,10 @@ const ChatBox = ( ...@@ -288,7 +287,10 @@ const ChatBox = (
const result = await postQuestionGuide( const result = await postQuestionGuide(
{ {
messages: adaptChat2GptMessages({ messages: history, reserveId: false }).slice(-6), messages: adaptChat2GptMessages({ messages: history, reserveId: false }).slice(-6),
shareId shareId,
outLinkUid,
teamId,
teamToken
}, },
abortSignal abortSignal
); );
...@@ -300,7 +302,7 @@ const ChatBox = ( ...@@ -300,7 +302,7 @@ const ChatBox = (
} }
} catch (error) {} } catch (error) {}
}, },
[questionGuide, shareId] [questionGuide, shareId, outLinkUid, teamId, teamToken]
); );
/** /**
...@@ -398,7 +400,6 @@ const ChatBox = ( ...@@ -398,7 +400,6 @@ const ChatBox = (
}; };
}) })
); );
if (!shareTeamId) {
setTimeout(() => { setTimeout(() => {
createQuestionGuide({ createQuestionGuide({
history: newChatList.map((item, i) => history: newChatList.map((item, i) =>
...@@ -413,7 +414,6 @@ const ChatBox = ( ...@@ -413,7 +414,6 @@ const ChatBox = (
generatingScroll(); generatingScroll();
isPc && TextareaDom.current?.focus(); isPc && TextareaDom.current?.focus();
}, 100); }, 100);
}
} catch (err: any) { } catch (err: any) {
toast({ toast({
title: t(getErrText(err, 'core.chat.error.Chat error')), title: t(getErrText(err, 'core.chat.error.Chat error')),
...@@ -622,6 +622,7 @@ const ChatBox = ( ...@@ -622,6 +622,7 @@ const ChatBox = (
{/* control icon */} {/* control icon */}
<Flex w={'100%'} alignItems={'center'} justifyContent={'flex-end'}> <Flex w={'100%'} alignItems={'center'} justifyContent={'flex-end'}>
<ChatControllerComponent <ChatControllerComponent
isChatting={isChatting}
chat={item} chat={item}
onDelete={ onDelete={
onDelMessage onDelMessage
...@@ -654,12 +655,17 @@ const ChatBox = ( ...@@ -654,12 +655,17 @@ const ChatBox = (
<ChatAvatar src={appAvatar} type={'AI'} /> <ChatAvatar src={appAvatar} type={'AI'} />
{/* control icon */} {/* control icon */}
<ChatControllerComponent <ChatControllerComponent
isChatting={isChatting}
ml={2} ml={2}
chat={item} chat={item}
setChatHistory={setChatHistory} setChatHistory={setChatHistory}
display={index === chatHistory.length - 1 && isChatting ? 'none' : 'flex'} display={index === chatHistory.length - 1 && isChatting ? 'none' : 'flex'}
showVoiceIcon={showVoiceIcon} showVoiceIcon={showVoiceIcon}
ttsConfig={ttsConfig} ttsConfig={ttsConfig}
shareId={shareId}
outLinkUid={outLinkUid}
teamId={teamId}
teamToken={teamToken}
onDelete={ onDelete={
onDelMessage onDelMessage
? () => { ? () => {
...@@ -829,7 +835,10 @@ const ChatBox = ( ...@@ -829,7 +835,10 @@ const ChatBox = (
isChatting={index === chatHistory.length - 1 && isChatting} isChatting={index === chatHistory.length - 1 && isChatting}
/> />
<ResponseTags responseData={item.responseData} isShare={!!shareId} /> <ResponseTags
responseData={item.responseData}
showDetail={!shareId && !teamId}
/>
{/* custom feedback */} {/* custom feedback */}
{item.customFeedbacks && item.customFeedbacks.length > 0 && ( {item.customFeedbacks && item.customFeedbacks.length > 0 && (
...@@ -909,6 +918,10 @@ const ChatBox = ( ...@@ -909,6 +918,10 @@ const ChatBox = (
TextareaDom={TextareaDom} TextareaDom={TextareaDom}
resetInputVal={resetInputVal} resetInputVal={resetInputVal}
showFileSelector={showFileSelector} showFileSelector={showFileSelector}
shareId={shareId}
outLinkUid={outLinkUid}
teamId={teamId}
teamToken={teamToken}
/> />
)} )}
{/* user feedback modal */} {/* user feedback modal */}
...@@ -1236,6 +1249,7 @@ function Empty() { ...@@ -1236,6 +1249,7 @@ function Empty() {
} }
const ChatControllerComponent = React.memo(function ChatControllerComponent({ const ChatControllerComponent = React.memo(function ChatControllerComponent({
isChatting,
chat, chat,
setChatHistory, setChatHistory,
display, display,
...@@ -1249,8 +1263,13 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({ ...@@ -1249,8 +1263,13 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({
onAddUserDislike, onAddUserDislike,
onAddUserLike, onAddUserLike,
ml, ml,
mr mr,
}: { shareId,
outLinkUid,
teamId,
teamToken
}: OutLinkChatAuthProps & {
isChatting: boolean;
chat: ChatSiteItemType; chat: ChatSiteItemType;
setChatHistory?: React.Dispatch<React.SetStateAction<ChatSiteItemType[]>>; setChatHistory?: React.Dispatch<React.SetStateAction<ChatSiteItemType[]>>;
showVoiceIcon?: boolean; showVoiceIcon?: boolean;
...@@ -1267,7 +1286,11 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({ ...@@ -1267,7 +1286,11 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({
const { t } = useTranslation(); const { t } = useTranslation();
const { copyData } = useCopyData(); const { copyData } = useCopyData();
const { audioLoading, audioPlaying, hasAudio, playAudio, cancelAudio } = useAudioPlay({ const { audioLoading, audioPlaying, hasAudio, playAudio, cancelAudio } = useAudioPlay({
ttsConfig ttsConfig,
shareId,
outLinkUid,
teamId,
teamToken
}); });
const controlIconStyle = { const controlIconStyle = {
w: '14px', w: '14px',
...@@ -1296,7 +1319,7 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({ ...@@ -1296,7 +1319,7 @@ const ChatControllerComponent = React.memo(function ChatControllerComponent({
onClick={() => copyData(chat.value)} onClick={() => copyData(chat.value)}
/> />
</MyTooltip> </MyTooltip>
{!!onDelete && ( {!!onDelete && !isChatting && (
<> <>
{onRetry && ( {onRetry && (
<MyTooltip label={t('core.chat.retry')}> <MyTooltip label={t('core.chat.retry')}>
......
projects/app/src/components/TagEdit/index.tsx deleted 100644 → 0
import React, { useEffect, useMemo, useState } from 'react';
import {
Menu,
MenuButton,
MenuList,
MenuItemOption,
MenuOptionGroup,
Flex,
TagLabel,
TagCloseButton,
HStack,
Tag,
Input
} from '@chakra-ui/react';
import type { TeamTagsSchema } from '@fastgpt/global/support/user/team/type';
const TagEdit = ({
defaultValues,
teamsTags,
setSelectedTags
}: {
defaultValues: [];
teamsTags: Array<TeamTagsSchema>;
setSelectedTags: (item: Array<string>) => void;
}) => {
const [teamTagsOptions, setTeamTagsOptions] = useState(teamsTags);
const setSelectTeamsTags = (item: any) => {
setSelectedTags(item);
};
useMemo(() => {
setTeamTagsOptions(teamsTags);
}, [teamsTags]);
return (
<>
<Menu closeOnSelect={false}>
<MenuButton className="menu-btn" maxHeight={'250'} minWidth={'80%'}>
<HStack
style={{
border: 'solid 2px #f3f3f3',
borderRadius: '5px',
padding: '3px',
flexWrap: 'wrap',
minHeight: '40px'
}}
>
{teamsTags.map((item: TeamTagsSchema, index: number) => {
const key: string = item?.key;
if (defaultValues.indexOf(key as never) > -1) {
return (
<Tag
key={index}
size={'md'}
colorScheme="red"
// maxWidth={"100px"}
borderRadius="full"
>
<TagLabel> {item.label}</TagLabel>
<TagCloseButton />
</Tag>
);
}
})}
</HStack>
</MenuButton>
<MenuList style={{ height: '300px', overflow: 'scroll' }}>
<Input
style={{ border: 'none', borderBottom: 'solid 1px #f6f6f6' }}
placeholder="pleace "
onChange={(e: any) => {
// 对用户输入的搜索文本进行小写转换，以实现不区分大小写的搜索
const searchLower: string = e?.nativeEvent?.data || '';
// 使用filter方法来过滤列表，只返回包含搜索文本的项
const resultList = teamsTags.filter((item) => {
const searchValue = item.label || '';
// 对列表中的每一项也进行小写转换
return searchValue.includes(searchLower);
});
!searchLower ? setTeamTagsOptions(teamsTags) : setTeamTagsOptions(resultList);
}}
/>
<MenuOptionGroup
defaultValue={defaultValues}
type="checkbox"
style={{ height: '300px', overflow: 'scroll' }}
onChange={(e) => {
setSelectTeamsTags(e);
}}
>
{teamTagsOptions.map((item, index) => {
return (
<MenuItemOption key={index} value={item.key}>
{item?.label}
</MenuItemOption>
);
})}
</MenuOptionGroup>
</MenuList>
</Menu>
</>
);
};
export default TagEdit;
projects/app/src/components/core/dataset/DatasetTypeTag.tsx
...@@ -8,7 +8,7 @@ import { DatasetTypeMap } from '@fastgpt/global/core/dataset/constants'; ...@@ -8,7 +8,7 @@ import { DatasetTypeMap } from '@fastgpt/global/core/dataset/constants';
const DatasetTypeTag = ({ type, ...props }: { type: `${DatasetTypeEnum}` } & FlexProps) => { const DatasetTypeTag = ({ type, ...props }: { type: `${DatasetTypeEnum}` } & FlexProps) => {
const { t } = useTranslation(); const { t } = useTranslation();
const item = DatasetTypeMap[type]; const item = DatasetTypeMap[type] || DatasetTypeMap['dataset'];
return ( return (
<Flex <Flex
......
projects/app/src/components/core/module/Flow/components/nodes/NodeHttp/index.tsx
...@@ -301,10 +301,22 @@ function RenderHttpProps({ ...@@ -301,10 +301,22 @@ function RenderHttpProps({
headers && headers &&
jsonBody && jsonBody &&
{ {
[TabEnum.params]: <RenderForm moduleId={moduleId} input={params} variables={variables} />, [TabEnum.params]: (
<RenderForm
moduleId={moduleId}
input={params}
variables={variables}
tabType={TabEnum.params}
/>
),
[TabEnum.body]: <RenderJson moduleId={moduleId} variables={variables} input={jsonBody} />, [TabEnum.body]: <RenderJson moduleId={moduleId} variables={variables} input={jsonBody} />,
[TabEnum.headers]: ( [TabEnum.headers]: (
<RenderForm moduleId={moduleId} input={headers} variables={variables} /> <RenderForm
moduleId={moduleId}
input={headers}
variables={variables}
tabType={TabEnum.headers}
/>
) )
}[selectedTab]} }[selectedTab]}
</Box> </Box>
...@@ -313,11 +325,13 @@ function RenderHttpProps({ ...@@ -313,11 +325,13 @@ function RenderHttpProps({
const RenderForm = ({ const RenderForm = ({
moduleId, moduleId,
input, input,
variables variables,
tabType
}: { }: {
moduleId: string; moduleId: string;
input: FlowNodeInputItemType; input: FlowNodeInputItemType;
variables: EditorVariablePickerType[]; variables: EditorVariablePickerType[];
tabType?: TabEnum;
}) => { }) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { toast } = useToast(); const { toast } = useToast();
...@@ -327,11 +341,52 @@ const RenderForm = ({ ...@@ -327,11 +341,52 @@ const RenderForm = ({
const [shouldUpdateNode, setShouldUpdateNode] = useState(false); const [shouldUpdateNode, setShouldUpdateNode] = useState(false);
const leftVariables = useMemo(() => { const leftVariables = useMemo(() => {
return variables.filter((variable) => { const HttpHeaders = [
{ key: 'A-IM', label: 'A-IM' },
{ key: 'Accept', label: 'Accept' },
{ key: 'Accept-Charset', label: 'Accept-Charset' },
{ key: 'Accept-Encoding', label: 'Accept-Encoding' },
{ key: 'Accept-Language', label: 'Accept-Language' },
{ key: 'Accept-Datetime', label: 'Accept-Datetime' },
{ key: 'Access-Control-Request-Method', label: 'Access-Control-Request-Method' },
{ key: 'Access-Control-Request-Headers', label: 'Access-Control-Request-Headers' },
{ key: 'Authorization', label: 'Authorization' },
{ key: 'Cache-Control', label: 'Cache-Control' },
{ key: 'Connection', label: 'Connection' },
{ key: 'Content-Length', label: 'Content-Length' },
{ key: 'Content-Type', label: 'Content-Type' },
{ key: 'Cookie', label: 'Cookie' },
{ key: 'Date', label: 'Date' },
{ key: 'Expect', label: 'Expect' },
{ key: 'Forwarded', label: 'Forwarded' },
{ key: 'From', label: 'From' },
{ key: 'Host', label: 'Host' },
{ key: 'If-Match', label: 'If-Match' },
{ key: 'If-Modified-Since', label: 'If-Modified-Since' },
{ key: 'If-None-Match', label: 'If-None-Match' },
{ key: 'If-Range', label: 'If-Range' },
{ key: 'If-Unmodified-Since', label: 'If-Unmodified-Since' },
{ key: 'Max-Forwards', label: 'Max-Forwards' },
{ key: 'Origin', label: 'Origin' },
{ key: 'Pragma', label: 'Pragma' },
{ key: 'Proxy-Authorization', label: 'Proxy-Authorization' },
{ key: 'Range', label: 'Range' },
{ key: 'Referer', label: 'Referer' },
{ key: 'TE', label: 'TE' },
{ key: 'User-Agent', label: 'User-Agent' },
{ key: 'Upgrade', label: 'Upgrade' },
{ key: 'Via', label: 'Via' },
{ key: 'Warning', label: 'Warning' },
{ key: 'Dnt', label: 'Dnt' },
{ key: 'X-Requested-With', label: 'X-Requested-With' },
{ key: 'X-CSRF-Token', label: 'X-CSRF-Token' }
];
return (tabType === TabEnum.headers ? HttpHeaders : variables).filter((variable) => {
const existVariables = list.map((item) => item.key); const existVariables = list.map((item) => item.key);
return !existVariables.includes(variable.key); return !existVariables.includes(variable.key);
}); });
}, [list, variables]); }, [list, tabType, variables]);
useEffect(() => { useEffect(() => {
setList(input.value || []); setList(input.value || []);
...@@ -378,16 +433,23 @@ const RenderForm = ({ ...@@ -378,16 +433,23 @@ const RenderForm = ({
}; };
const handleAddNewProps = (key: string, value: string = '') => { const handleAddNewProps = (key: string, value: string = '') => {
const checkExist = list.find((item) => item.key === key); setList((prevList) => {
if (!key) {
return prevList;
}
const checkExist = prevList.find((item) => item.key === key);
if (checkExist) { if (checkExist) {
return toast({ setUpdateTrigger((prev) => !prev);
toast({
status: 'warning', status: 'warning',
title: t('core.module.http.Key already exists') title: t('core.module.http.Key already exists')
}); });
return prevList;
} }
if (!key) return; return [...prevList, { key, type: 'string', value }];
});
setList((prevList) => [...prevList, { key, type: 'string', value }]);
setShouldUpdateNode(true); setShouldUpdateNode(true);
}; };
...@@ -406,7 +468,7 @@ const RenderForm = ({ ...@@ -406,7 +468,7 @@ const RenderForm = ({
<Td p={0} w={'150px'}> <Td p={0} w={'150px'}>
<HttpInput <HttpInput
hasVariablePlugin={false} hasVariablePlugin={false}
hasDropDownPlugin={true} hasDropDownPlugin={tabType === TabEnum.headers}
setDropdownValue={(value) => { setDropdownValue={(value) => {
handleKeyChange(index, value); handleKeyChange(index, value);
setUpdateTrigger((prev) => !prev); setUpdateTrigger((prev) => !prev);
...@@ -450,16 +512,19 @@ const RenderForm = ({ ...@@ -450,16 +512,19 @@ const RenderForm = ({
<Tr> <Tr>
<Td p={0} w={'150px'}> <Td p={0} w={'150px'}>
<HttpInput <HttpInput
hasDropDownPlugin={true} hasVariablePlugin={false}
hasDropDownPlugin={tabType === TabEnum.headers}
setDropdownValue={(val) => { setDropdownValue={(val) => {
handleAddNewProps(val); handleAddNewProps(val);
setUpdateTrigger((prev) => !prev);
}} }}
placeholder={t('core.module.http.Add props')} placeholder={t('core.module.http.Add props')}
value={''} value={''}
h={40}
variables={leftVariables} variables={leftVariables}
updateTrigger={updateTrigger}
onBlur={(val) => { onBlur={(val) => {
handleAddNewProps(val); handleAddNewProps(val);
setUpdateTrigger((prev) => !prev);
}} }}
/> />
</Td> </Td>
...@@ -490,7 +555,7 @@ const RenderJson = ({ ...@@ -490,7 +555,7 @@ const RenderJson = ({
<Box mt={1}> <Box mt={1}>
<JSONEditor <JSONEditor
bg={'myGray.50'} bg={'myGray.50'}
height={200} defaultHeight={200}
resize resize
value={input.value} value={input.value}
placeholder={t('core.module.template.http body placeholder')} placeholder={t('core.module.template.http body placeholder')}
......
projects/app/src/components/support/user/team/TeamManageModal/index.tsx
...@@ -9,9 +9,7 @@ import { ...@@ -9,9 +9,7 @@ import {
putSwitchTeam, putSwitchTeam,
putUpdateMember, putUpdateMember,
delRemoveMember, delRemoveMember,
delLeaveTeam, delLeaveTeam
getTeamsTags,
insertTeamsTags
} from '@/web/support/user/team/api'; } from '@/web/support/user/team/api';
import { import {
Box, Box,
...@@ -49,7 +47,7 @@ import { useSystemStore } from '@/web/common/system/useSystemStore'; ...@@ -49,7 +47,7 @@ import { useSystemStore } from '@/web/common/system/useSystemStore';
const EditModal = dynamic(() => import('./EditModal')); const EditModal = dynamic(() => import('./EditModal'));
const InviteModal = dynamic(() => import('./InviteModal')); const InviteModal = dynamic(() => import('./InviteModal'));
const TeamTagsAsync = dynamic(() => import('../TeamTagsAsync')); const TeamTagModal = dynamic(() => import('../TeamTagModal'));
const TeamManageModal = ({ onClose }: { onClose: () => void }) => { const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
const { t } = useTranslation(); const { t } = useTranslation();
...@@ -57,7 +55,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -57,7 +55,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
const { toast } = useToast(); const { toast } = useToast();
const { teamPlanStatus } = useUserStore(); const { teamPlanStatus } = useUserStore();
const { feConfigs } = useSystemStore(); const { feConfigs } = useSystemStore();
const [teamsTags, setTeamTags] = useState<any>();
const { ConfirmModal: ConfirmRemoveMemberModal, openConfirm: openRemoveMember } = useConfirm(); const { ConfirmModal: ConfirmRemoveMemberModal, openConfirm: openRemoveMember } = useConfirm();
const { ConfirmModal: ConfirmLeaveTeamModal, openConfirm: openLeaveConfirm } = useConfirm({ const { ConfirmModal: ConfirmLeaveTeamModal, openConfirm: openLeaveConfirm } = useConfirm({
...@@ -87,8 +84,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -87,8 +84,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
mutationFn: async (teamId: string) => { mutationFn: async (teamId: string) => {
const token = await putSwitchTeam(teamId); const token = await putSwitchTeam(teamId);
token && setToken(token); token && setToken(token);
// get team tags
await getTeamsTags(teamId);
return initUserInfo(); return initUserInfo();
}, },
errorToast: t('user.team.Switch Team Failed') errorToast: t('user.team.Switch Team Failed')
...@@ -99,11 +94,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -99,11 +94,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
['getMembers', userInfo?.team?.teamId], ['getMembers', userInfo?.team?.teamId],
() => { () => {
if (!userInfo?.team?.teamId) return []; if (!userInfo?.team?.teamId) return [];
// get team tags
getTeamsTags(userInfo.team.teamId).then((res: any) => {
setTeamTags(res);
});
return getTeamMembers(userInfo.team.teamId); return getTeamMembers(userInfo.team.teamId);
} }
); );
...@@ -217,17 +207,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -217,17 +207,6 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
: {})} : {})}
> >
{team.teamName} {team.teamName}
{/* {userInfo?.team?.teamId === team.teamId && (
<HStack spacing={1}>
{teamsTags.slice(0, 3).map((item: any, index) => {
return (
<Tag key={index} size={'sm'} variant="outline" colorScheme="blue">
{item.label}
</Tag>
);
})}
</HStack>
)} */}
</Box> </Box>
{userInfo?.team?.teamId === team.teamId ? ( {userInfo?.team?.teamId === team.teamId ? (
<MyIcon name={'common/tickFill'} w={'16px'} color={'primary.500'} /> <MyIcon name={'common/tickFill'} w={'16px'} color={'primary.500'} />
...@@ -290,22 +269,23 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -290,22 +269,23 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
<Box ml={2} bg={'myGray.100'} borderRadius={'20px'} px={3} fontSize={'xs'}> <Box ml={2} bg={'myGray.100'} borderRadius={'20px'} px={3} fontSize={'xs'}>
{members.length} {members.length}
</Box> </Box>
{userInfo.team.role === TeamMemberRoleEnum.owner && {userInfo.team.role === TeamMemberRoleEnum.owner && (
teamPlanStatus?.standardConstants &&
teamPlanStatus.standardConstants.maxTeamMember > members.length && (
<Button <Button
variant={'whitePrimary'} variant={'whitePrimary'}
size="sm" size="sm"
borderRadius={'md'} borderRadius={'md'}
ml={3} ml={3}
leftIcon={ leftIcon={<MyIcon name={'common/inviteLight'} w={'14px'} color={'primary.500'} />}
<MyIcon name={'common/inviteLight'} w={'14px'} color={'primary.500'} />
}
onClick={() => { onClick={() => {
if (userInfo.team.maxSize <= members.length) { if (
teamPlanStatus?.standardConstants?.maxTeamMember &&
teamPlanStatus.standardConstants.maxTeamMember <= members.length
) {
toast({ toast({
status: 'warning', status: 'warning',
title: t('user.team.Over Max Member Tip', { max: userInfo.team.maxSize }) title: t('user.team.Over Max Member Tip', {
max: teamPlanStatus.standardConstants.maxTeamMember
})
}); });
} else { } else {
onOpenInvite(); onOpenInvite();
...@@ -323,14 +303,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -323,14 +303,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
ml={3} ml={3}
leftIcon={<DragHandleIcon w={'14px'} color={'primary.500'} />} leftIcon={<DragHandleIcon w={'14px'} color={'primary.500'} />}
onClick={() => { onClick={() => {
if (userInfo.team.maxSize <= members.length) {
toast({
status: 'warning',
title: t('user.team.Team Tags Async', { max: userInfo.team.maxSize })
});
} else {
onOpenTeamTagsAsync(); onOpenTeamTagsAsync();
}
}} }}
> >
{t('user.team.Team Tags Async')} {t('user.team.Team Tags Async')}
...@@ -492,13 +465,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => { ...@@ -492,13 +465,7 @@ const TeamManageModal = ({ onClose }: { onClose: () => void }) => {
onSuccess={refetchMembers} onSuccess={refetchMembers}
/> />
)} )}
{isOpenTeamTagsAsync && ( {isOpenTeamTagsAsync && <TeamTagModal onClose={onCloseTeamTagsAsync} />}
<TeamTagsAsync
teamInfo={teamsTags?.tagsUrl}
teamsTags={teamsTags?.list || []}
onClose={onCloseTeamTagsAsync}
/>
)}
<ConfirmRemoveMemberModal /> <ConfirmRemoveMemberModal />
<ConfirmLeaveTeamModal /> <ConfirmLeaveTeamModal />
</> </>
......
projects/app/src/components/support/user/team/TeamTagsAsync/index.tsx → projects/app/src/components/support/user/team/TeamTagModal/index.tsx
import React, { useEffect, useMemo, useState } from 'react'; import React from 'react';
import MyModal from '@/components/MyModal'; import MyModal from '@/components/MyModal';
import { import {
Box, Box,
...@@ -11,61 +11,74 @@ import { ...@@ -11,61 +11,74 @@ import {
HStack, HStack,
Avatar Avatar
} from '@chakra-ui/react'; } from '@chakra-ui/react';
import { AttachmentIcon, CopyIcon, DragHandleIcon } from '@chakra-ui/icons'; import { putUpdateTeam } from '@/web/support/user/team/api';
import { putUpdateTeamTags, updateTags } from '@/web/support/user/team/api'; import { useFieldArray, useForm } from 'react-hook-form';
import { useForm } from 'react-hook-form';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import type { TeamTagsSchema } from '@fastgpt/global/support/user/team/type'; import type { TeamTagItemType } from '@fastgpt/global/support/user/team/type';
import { useRequest } from '@/web/common/hooks/useRequest'; import { useRequest } from '@/web/common/hooks/useRequest';
import { RepeatIcon } from '@chakra-ui/icons'; import { RepeatIcon } from '@chakra-ui/icons';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
import { useToast } from '@fastgpt/web/hooks/useToast';
import { useCopyData } from '@/web/common/hooks/useCopyData'; import { useCopyData } from '@/web/common/hooks/useCopyData';
import { useUserStore } from '@/web/support/user/useUserStore';
import { useQuery } from '@tanstack/react-query';
import { getTeamsTags, loadTeamTagsByDomain } from '@/web/support/user/team/api';
const TeamTagsAsync = ({ type FormType = {
teamsTags, teamDomain: string;
teamInfo, tags: TeamTagItemType[];
onClose };
}: {
teamsTags: Array<TeamTagsSchema>; const TeamTagsAsync = ({ onClose }: { onClose: () => void }) => {
teamInfo: any;
onClose: () => void;
}) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { toast } = useToast(); const { userInfo, initUserInfo } = useUserStore();
const [_teamsTags, setTeamTags] = useState<Array<TeamTagsSchema>>(teamsTags); const { copyData } = useCopyData();
const { register, setValue, getValues, handleSubmit } = useForm<any>({ const teamInfo = userInfo?.team;
defaultValues: { ...teamInfo }
if (!teamInfo) {
onClose();
return null;
}
const { register, control, handleSubmit } = useForm<FormType>({
defaultValues: {
teamDomain: teamInfo.teamDomain,
tags: []
}
}); });
const { copyData } = useCopyData(); const { fields: teamTags, replace: replaceTeamTags } = useFieldArray({
control,
name: 'tags'
});
const baseUrl = global.feConfigs?.customSharePageDomain || location?.origin; const baseUrl = global.feConfigs?.customSharePageDomain || location?.origin;
const linkUrl = `${baseUrl}/chat/team?shareTeamId=${teamInfo?._id}${ const linkUrl = `${baseUrl}/chat/team?teamId=${teamInfo.teamId}&teamToken=`;
getValues('showHistory') ? '' : '&showHistory=0'
}`;
// tags Async // tags Async
const { mutate: onclickAsync, isLoading: creating } = useRequest({ const { mutate: onclickUpdate, isLoading: isUpdating } = useRequest({
mutationFn: async (data: any) => { mutationFn: async (data: FormType) => {
return putUpdateTeamTags({ tagsUrl: data.tagsUrl, teamId: teamInfo?._id }); return putUpdateTeam({ teamDomain: data.teamDomain, teamId: teamInfo?.teamId });
}, },
onSuccess(id: string) { onSuccess() {
initUserInfo();
onClose(); onClose();
}, },
successToast: t('user.team.Team Tags Async Success'),
errorToast: t('common.Create Failed') errorToast: t('common.Create Failed')
}); });
const asyncTags = async () => { const { mutate: onclickTagAsync, isLoading: isSyncing } = useRequest({
console.log('getValues', getValues()); mutationFn: (data: FormType) => loadTeamTagsByDomain(data.teamDomain),
const res: Array<TeamTagsSchema> = await updateTags(teamInfo?._id, getValues().tagsUrl); onSuccess(res) {
setTeamTags(res); replaceTeamTags(res);
toast({ status: 'success', title: '团队标签同步成功' }); },
}; successToast: t('support.user.team.Team Tags Async Success')
useEffect(() => { });
console.log('teamInfo', teamInfo);
}, []); useQuery(['getTeamsTags'], getTeamsTags, {
onSuccess: (data) => {
replaceTeamTags(data);
}
});
// 获取
return ( return (
<> <>
<MyModal <MyModal
...@@ -80,7 +93,7 @@ const TeamTagsAsync = ({ ...@@ -80,7 +93,7 @@ const TeamTagsAsync = ({
overflow={'hidden'} overflow={'hidden'}
title={ title={
<Box> <Box>
<Box>{teamInfo?.name}</Box> <Box>{teamInfo?.teamName}</Box>
<Box color={'myGray.500'} fontSize={'xs'} fontWeight={'normal'}> <Box color={'myGray.500'} fontSize={'xs'} fontWeight={'normal'}>
{'填写标签同步链接，点击同步按钮即可同步'} {'填写标签同步链接，点击同步按钮即可同步'}
</Box> </Box>
...@@ -98,8 +111,8 @@ const TeamTagsAsync = ({ ...@@ -98,8 +111,8 @@ const TeamTagsAsync = ({
autoFocus autoFocus
bg={'myWhite.600'} bg={'myWhite.600'}
placeholder="请输入同步标签" placeholder="请输入同步标签"
{...register('tagsUrl', { {...register('teamDomain', {
required: t('core.app.error.App name can not be empty') required: true
})} })}
/> />
</Flex> </Flex>
...@@ -146,7 +159,7 @@ const TeamTagsAsync = ({ ...@@ -146,7 +159,7 @@ const TeamTagsAsync = ({
}} }}
spacing={1} spacing={1}
> >
{_teamsTags.map((item, index) => { {teamTags.map((item, index) => {
return ( return (
<Tag key={index} mt={2} size={'md'} colorScheme="red" borderRadius="full"> <Tag key={index} mt={2} size={'md'} colorScheme="red" borderRadius="full">
<Avatar <Avatar
...@@ -161,7 +174,13 @@ const TeamTagsAsync = ({ ...@@ -161,7 +174,13 @@ const TeamTagsAsync = ({
); );
})} })}
</HStack> </HStack>
<Button ml={4} size="md" leftIcon={<RepeatIcon />} onClick={asyncTags}> <Button
isLoading={isSyncing}
ml={4}
size="md"
leftIcon={<RepeatIcon />}
onClick={handleSubmit((data) => onclickTagAsync(data))}
>
立即同步 立即同步
</Button> </Button>
</Flex> </Flex>
...@@ -170,7 +189,7 @@ const TeamTagsAsync = ({ ...@@ -170,7 +189,7 @@ const TeamTagsAsync = ({
<Button variant={'whiteBase'} mr={3} onClick={onClose}> <Button variant={'whiteBase'} mr={3} onClick={onClose}>
{t('common.Close')} {t('common.Close')}
</Button> </Button>
<Button isLoading={creating} onClick={handleSubmit((data) => onclickAsync(data))}> <Button isLoading={isUpdating} onClick={handleSubmit((data) => onclickUpdate(data))}>
{t('user.team.Tags Async')} {t('user.team.Tags Async')}
</Button> </Button>
</ModalFooter> </ModalFooter>
......
projects/app/src/constants/user.ts
export enum PageTypeEnum { export enum LoginPageTypeEnum {
login = 'login', passwordLogin = 'passwordLogin',
register = 'register', register = 'register',
forgetPassword = 'forgetPassword' forgetPassword = 'forgetPassword',
wechat = 'wechat'
} }
export enum PromotionEnum { export enum PromotionEnum {
......
projects/app/src/global/core/ai/api.d.ts
import { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d'; import { ChatMessageItemType } from '@fastgpt/global/core/ai/type.d';
import type { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat.d';
export type CreateQuestionGuideParams = { export type CreateQuestionGuideParams = OutLinkChatAuthProps & {
messages: ChatMessageItemType[]; messages: ChatMessageItemType[];
shareId?: string;
}; };
projects/app/src/global/core/chat/api.d.ts
import type { AppTTSConfigType } from '@fastgpt/global/core/module/type.d'; import type { AppTTSConfigType } from '@fastgpt/global/core/module/type.d';
import { ModuleItemType } from '../module/type'; import { ModuleItemType } from '../module/type';
import { AdminFbkType, ChatItemType, moduleDispatchResType } from '@fastgpt/global/core/chat/type'; import { AdminFbkType, ChatItemType, moduleDispatchResType } from '@fastgpt/global/core/chat/type';
import type { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat.d';
export type GetChatSpeechProps = { export type GetChatSpeechProps = {
ttsConfig: AppTTSConfigType; ttsConfig: AppTTSConfigType;
...@@ -14,16 +15,16 @@ export type InitChatProps = { ...@@ -14,16 +15,16 @@ export type InitChatProps = {
chatId?: string; chatId?: string;
loadCustomFeedbacks?: boolean; loadCustomFeedbacks?: boolean;
}; };
/* ---------- chat ----------- */
export type chatByTeamProps = {
teamId?: string;
appId?: string;
outLinkUid?: string;
};
export type InitOutLinkChatProps = { export type InitOutLinkChatProps = {
chatId?: string; chatId?: string;
shareId?: string; shareId: string;
outLinkUid?: string; outLinkUid: string;
};
export type InitTeamChatProps = {
teamId: string;
appId: string;
chatId?: string;
teamToken: string;
}; };
export type InitChatResponse = { export type InitChatResponse = {
chatId?: string; chatId?: string;
...@@ -43,42 +44,30 @@ export type InitChatResponse = { ...@@ -43,42 +44,30 @@ export type InitChatResponse = {
}; };
/* ---------- history ----------- */ /* ---------- history ----------- */
export type getHistoriesProps = { export type GetHistoriesProps = OutLinkChatAuthProps & {
appId?: string; appId?: string;
authToken?: string;
// share chat
shareId?: string;
outLinkUid?: string; // authToken/uid
}; };
export type UpdateHistoryProps = { export type UpdateHistoryProps = OutLinkChatAuthProps & {
appId: string; appId: string;
chatId: string; chatId: string;
customTitle?: string; customTitle?: string;
top?: boolean; top?: boolean;
shareId?: string;
outLinkUid?: string;
}; };
export type DelHistoryProps = { export type DelHistoryProps = OutLinkChatAuthProps & {
appId: string; appId: string;
chatId: string; chatId: string;
shareId?: string;
outLinkUid?: string;
}; };
export type ClearHistoriesProps = { export type ClearHistoriesProps = OutLinkChatAuthProps & {
appId?: string; appId?: string;
shareId?: string;
outLinkUid?: string;
}; };
/* -------- chat item ---------- */ /* -------- chat item ---------- */
export type DeleteChatItemProps = { export type DeleteChatItemProps = OutLinkChatAuthProps & {
appId: string; appId: string;
chatId: string; chatId: string;
contentId?: string; contentId?: string;
shareId?: string;
outLinkUid?: string;
}; };
export type AdminUpdateFeedbackParams = AdminFbkType & { export type AdminUpdateFeedbackParams = AdminFbkType & {
......
projects/app/src/global/core/prompt/agent.ts
export const Prompt_AgentQA = { export const Prompt_AgentQA = {
description: `<context></context> 标记中是一段文本，学习和分析它，并整理学习成果： description: `<Context></Context> 标记中是一段文本，学习和分析它，并整理学习成果：
- 提出问题并给出每个问题的答案。 - 提出问题并给出每个问题的答案。
- 答案需详细完整，给出相关原文描述。 - 答案需详细完整，尽可能保留原文描述。
- 答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 markdown 元素。 - 答案可以包含普通文字、链接、代码、表格、公示、媒体链接等 Markdown 元素。
- 最多提出 30 个问题。 - 最多提出 30 个问题。
`, `,
fixedText: `最后，你需要按下面的格式返回多个问题和答案: fixedText: `请按以下格式整理学习成果:
<Context>
文本
</Context>
Q1: 问题。 Q1: 问题。
A1: 答案。 A1: 答案。
Q2: Q2:
A2: A2:
……
<context> ------
我们开始吧!
<Context>
{{text}} {{text}}
<context/> <Context/>
` `
}; };
......
projects/app/src/pages/account/components/UpdatePswModal.tsx
...@@ -56,8 +56,8 @@ const UpdatePswModal = ({ onClose }: { onClose: () => void }) => { ...@@ -56,8 +56,8 @@ const UpdatePswModal = ({ onClose }: { onClose: () => void }) => {
{...register('newPsw', { {...register('newPsw', {
required: true, required: true,
maxLength: { maxLength: {
value: 20, value: 60,
message: '密码最少 4 位最多 20 位' message: '密码最少 4 位最多 60 位'
} }
})} })}
></Input> ></Input>
...@@ -70,8 +70,8 @@ const UpdatePswModal = ({ onClose }: { onClose: () => void }) => { ...@@ -70,8 +70,8 @@ const UpdatePswModal = ({ onClose }: { onClose: () => void }) => {
{...register('confirmPsw', { {...register('confirmPsw', {
required: true, required: true,
maxLength: { maxLength: {
value: 20, value: 60,
message: '密码最少 4 位最多 20 位' message: '密码最少 4 位最多 60 位'
} }
})} })}
></Input> ></Input>
......
projects/app/src/pages/account/components/UsageDetail.tsx
...@@ -25,8 +25,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () => ...@@ -25,8 +25,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
[usage.list] [usage.list]
); );
const { hasModel, hasCharsLen, hasDuration } = useMemo(() => { const { hasModel, hasToken, hasCharsLen, hasDuration } = useMemo(() => {
let hasModel = false; let hasModel = false;
let hasToken = false;
let hasCharsLen = false; let hasCharsLen = false;
let hasDuration = false; let hasDuration = false;
let hasDataLen = false; let hasDataLen = false;
...@@ -36,6 +37,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () => ...@@ -36,6 +37,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
hasModel = true; hasModel = true;
} }
if (typeof item.tokens === 'number') {
hasToken = true;
}
if (typeof item.charsLength === 'number') { if (typeof item.charsLength === 'number') {
hasCharsLen = true; hasCharsLen = true;
} }
...@@ -46,6 +50,7 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () => ...@@ -46,6 +50,7 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
return { return {
hasModel, hasModel,
hasToken,
hasCharsLen, hasCharsLen,
hasDuration, hasDuration,
hasDataLen hasDataLen
...@@ -91,9 +96,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () => ...@@ -91,9 +96,9 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
<Tr> <Tr>
<Th>{t('support.wallet.usage.Module name')}</Th> <Th>{t('support.wallet.usage.Module name')}</Th>
{hasModel && <Th>{t('support.wallet.usage.Ai model')}</Th>} {hasModel && <Th>{t('support.wallet.usage.Ai model')}</Th>}
{hasToken && <Th>{t('support.wallet.usage.Token Length')}</Th>}
{hasCharsLen && <Th>{t('support.wallet.usage.Text Length')}</Th>} {hasCharsLen && <Th>{t('support.wallet.usage.Text Length')}</Th>}
{hasDuration && <Th>{t('support.wallet.usage.Duration')}</Th>} {hasDuration && <Th>{t('support.wallet.usage.Duration')}</Th>}
<Th>{t('support.wallet.usage.Total points')}</Th> <Th>{t('support.wallet.usage.Total points')}</Th>
</Tr> </Tr>
</Thead> </Thead>
...@@ -102,6 +107,7 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () => ...@@ -102,6 +107,7 @@ const UsageDetail = ({ usage, onClose }: { usage: UsageItemType; onClose: () =>
<Tr key={i}> <Tr key={i}>
<Td>{t(item.moduleName)}</Td> <Td>{t(item.moduleName)}</Td>
{hasModel && <Td>{item.model ?? '-'}</Td>} {hasModel && <Td>{item.model ?? '-'}</Td>}
{hasToken && <Td>{item.tokens ?? '-'}</Td>}
{hasCharsLen && <Td>{item.charsLength ?? '-'}</Td>} {hasCharsLen && <Td>{item.charsLength ?? '-'}</Td>}
{hasDuration && <Td>{item.duration ?? '-'}</Td>} {hasDuration && <Td>{item.duration ?? '-'}</Td>}
<Td>{formatNumber(item.amount)}</Td> <Td>{formatNumber(item.amount)}</Td>
......
projects/app/src/pages/api/admin/initv469.ts
...@@ -4,6 +4,11 @@ import { connectToDatabase } from '@/service/mongo'; ...@@ -4,6 +4,11 @@ import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common'; import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { MongoUsage } from '@fastgpt/service/support/wallet/usage/schema'; import { MongoUsage } from '@fastgpt/service/support/wallet/usage/schema';
import { connectionMongo } from '@fastgpt/service/common/mongo'; import { connectionMongo } from '@fastgpt/service/common/mongo';
import { checkFiles } from '../timerTask/dataset/checkInValidDatasetFiles';
import { addHours } from 'date-fns';
import { checkInvalid as checkInvalidImg } from '../timerTask/dataset/checkInvalidDatasetImage';
import { checkInvalidCollection } from '../timerTask/dataset/checkInvalidMongoCollection';
import { checkInvalidVector } from '../timerTask/dataset/checkInvalidVector';
/* pg 中的数据搬到 mongo dataset.datas 中，并做映射 */ /* pg 中的数据搬到 mongo dataset.datas 中，并做映射 */
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
...@@ -21,6 +26,21 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -21,6 +26,21 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
}); });
} }
(async () => {
try {
console.log('执行脏数据清理任务');
const end = addHours(new Date(), -1);
const start = addHours(new Date(), -360 * 24);
await checkFiles(start, end);
await checkInvalidImg(start, end);
await checkInvalidCollection(start, end);
await checkInvalidVector(start, end);
console.log('执行脏数据清理任务完毕');
} catch (error) {
console.log('执行脏数据清理任务出错了');
}
})();
jsonRes(res, { jsonRes(res, {
message: 'success' message: 'success'
}); });
......
projects/app/src/pages/api/common/file/uploadImage.ts
import type { NextApiRequest, NextApiResponse } from 'next'; import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response'; import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo'; import { connectToDatabase } from '@/service/mongo';
import { authCertOrShareId } from '@fastgpt/service/support/permission/auth/common'; import { authChatCert } from '@/service/support/permission/auth/chat';
import { uploadMongoImg } from '@fastgpt/service/common/file/image/controller'; import { uploadMongoImg } from '@fastgpt/service/common/file/image/controller';
import { UploadImgProps } from '@fastgpt/global/common/file/api'; import { UploadImgProps } from '@fastgpt/global/common/file/api';
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try { try {
await connectToDatabase(); await connectToDatabase();
const { shareId, ...body } = req.body as UploadImgProps; const body = req.body as UploadImgProps;
const { teamId } = await authCertOrShareId({ req, shareId, authToken: true }); const { teamId } = await authChatCert({ req, authToken: true });
const data = await uploadMongoImg({ const data = await uploadMongoImg({
teamId, teamId,
......
projects/app/src/pages/api/common/system/getInitData.ts
...@@ -4,8 +4,6 @@ import { jsonRes } from '@fastgpt/service/common/response'; ...@@ -4,8 +4,6 @@ import { jsonRes } from '@fastgpt/service/common/response';
import { readFileSync, readdirSync } from 'fs'; import { readFileSync, readdirSync } from 'fs';
import type { InitDateResponse } from '@/global/common/api/systemRes'; import type { InitDateResponse } from '@/global/common/api/systemRes';
import type { FastGPTConfigFileType } from '@fastgpt/global/common/system/types/index.d'; import type { FastGPTConfigFileType } from '@fastgpt/global/common/system/types/index.d';
import { getTikTokenEnc } from '@fastgpt/global/common/string/tiktoken';
import { initHttpAgent } from '@fastgpt/service/common/middle/httpAgent';
import { PluginSourceEnum } from '@fastgpt/global/core/plugin/constants'; import { PluginSourceEnum } from '@fastgpt/global/core/plugin/constants';
import { getFastGPTConfigFromDB } from '@fastgpt/service/common/system/config/controller'; import { getFastGPTConfigFromDB } from '@fastgpt/service/common/system/config/controller';
import { connectToDatabase } from '@/service/mongo'; import { connectToDatabase } from '@/service/mongo';
...@@ -63,7 +61,6 @@ export async function getInitConfig() { ...@@ -63,7 +61,6 @@ export async function getInitConfig() {
await connectToDatabase(); await connectToDatabase();
await Promise.all([ await Promise.all([
initGlobal(),
initSystemConfig(), initSystemConfig(),
// getSimpleModeTemplates(), // getSimpleModeTemplates(),
getSystemVersion(), getSystemVersion(),
...@@ -84,18 +81,6 @@ export async function getInitConfig() { ...@@ -84,18 +81,6 @@ export async function getInitConfig() {
} }
} }
export function initGlobal() {
if (global.communityPlugins) return;
global.communityPlugins = [];
global.simpleModeTemplates = [];
global.qaQueueLen = global.qaQueueLen ?? 0;
global.vectorQueueLen = global.vectorQueueLen ?? 0;
// init tikToken
getTikTokenEnc();
initHttpAgent();
}
export async function initSystemConfig() { export async function initSystemConfig() {
// load config // load config
const [dbConfig, fileConfig] = await Promise.all([ const [dbConfig, fileConfig] = await Promise.all([
...@@ -125,7 +110,6 @@ export async function initSystemConfig() { ...@@ -125,7 +110,6 @@ export async function initSystemConfig() {
// set config // set config
initFastGPTConfig(config); initFastGPTConfig(config);
global.systemEnv = config.systemEnv;
console.log({ console.log({
feConfigs: global.feConfigs, feConfigs: global.feConfigs,
......
projects/app/src/pages/api/common/system/unlockTask.ts
import type { NextApiRequest, NextApiResponse } from 'next'; import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response'; import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo'; import { connectToDatabase } from '@/service/mongo';
import { startQueue } from '@/service/utils/tools';
import { authCert } from '@fastgpt/service/support/permission/auth/common'; import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { startTrainingQueue } from '@/service/core/dataset/training/utils';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) { export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try { try {
await connectToDatabase(); await connectToDatabase();
await authCert({ req, authToken: true }); await authCert({ req, authToken: true });
startQueue(); startTrainingQueue();
} catch (error) {} } catch (error) {}
jsonRes(res); jsonRes(res);
} }
projects/app/src/pages/api/core/ai/agent/createQuestionGuide.ts
...@@ -4,22 +4,21 @@ import { connectToDatabase } from '@/service/mongo'; ...@@ -4,22 +4,21 @@ import { connectToDatabase } from '@/service/mongo';
import type { CreateQuestionGuideParams } from '@/global/core/ai/api.d'; import type { CreateQuestionGuideParams } from '@/global/core/ai/api.d';
import { pushQuestionGuideUsage } from '@/service/support/wallet/usage/push'; import { pushQuestionGuideUsage } from '@/service/support/wallet/usage/push';
import { createQuestionGuide } from '@fastgpt/service/core/ai/functions/createQuestionGuide'; import { createQuestionGuide } from '@fastgpt/service/core/ai/functions/createQuestionGuide';
import { authCertOrShareId } from '@fastgpt/service/support/permission/auth/common'; import { authChatCert } from '@/service/support/permission/auth/chat';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) { export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try { try {
await connectToDatabase(); await connectToDatabase();
const { messages, shareId } = req.body as CreateQuestionGuideParams; const { messages } = req.body as CreateQuestionGuideParams;
const { tmbId, teamId } = await authCertOrShareId({ const { tmbId, teamId } = await authChatCert({
req, req,
authToken: true, authToken: true
shareId
}); });
const qgModel = global.llmModels[0]; const qgModel = global.llmModels[0];
const { result, charsLength } = await createQuestionGuide({ const { result, tokens } = await createQuestionGuide({
messages, messages,
model: qgModel.model model: qgModel.model
}); });
...@@ -29,7 +28,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse< ...@@ -29,7 +28,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
}); });
pushQuestionGuideUsage({ pushQuestionGuideUsage({
charsLength, tokens,
teamId, teamId,
tmbId tmbId
}); });
......
projects/app/src/pages/api/core/chat/clearHistories.ts
...@@ -7,12 +7,13 @@ import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema'; ...@@ -7,12 +7,13 @@ import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { ClearHistoriesProps } from '@/global/core/chat/api'; import { ClearHistoriesProps } from '@/global/core/chat/api';
import { authOutLink } from '@/service/support/permission/auth/outLink'; import { authOutLink } from '@/service/support/permission/auth/outLink';
import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants'; import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants';
import { authTeamSpaceToken } from '@/service/support/permission/auth/team';
/* clear chat history */ /* clear chat history */
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try { try {
await connectToDatabase(); await connectToDatabase();
const { appId, shareId, outLinkUid } = req.query as ClearHistoriesProps; const { appId, shareId, outLinkUid, teamId, teamToken } = req.query as ClearHistoriesProps;
let chatAppId = appId; let chatAppId = appId;
...@@ -26,6 +27,14 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -26,6 +27,14 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
outLinkUid: uid outLinkUid: uid
}; };
} }
if (teamId && teamToken) {
const { uid } = await authTeamSpaceToken({ teamId, teamToken });
return {
teamId,
appId,
outLinkUid: uid
};
}
if (appId) { if (appId) {
const { tmbId } = await authCert({ req, authToken: true }); const { tmbId } = await authCert({ req, authToken: true });
......
projects/app/src/pages/api/core/chat/getHistories.ts
...@@ -4,14 +4,15 @@ import { connectToDatabase } from '@/service/mongo'; ...@@ -4,14 +4,15 @@ import { connectToDatabase } from '@/service/mongo';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema'; import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import type { ChatHistoryItemType } from '@fastgpt/global/core/chat/type.d'; import type { ChatHistoryItemType } from '@fastgpt/global/core/chat/type.d';
import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants'; import { ChatSourceEnum } from '@fastgpt/global/core/chat/constants';
import { getHistoriesProps } from '@/global/core/chat/api'; import { GetHistoriesProps } from '@/global/core/chat/api';
import { authOutLink } from '@/service/support/permission/auth/outLink'; import { authOutLink } from '@/service/support/permission/auth/outLink';
import { authCert } from '@fastgpt/service/support/permission/auth/common'; import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { authTeamSpaceToken } from '@/service/support/permission/auth/team';
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try { try {
await connectToDatabase(); await connectToDatabase();
const { appId, shareId, outLinkUid } = req.body as getHistoriesProps; const { appId, shareId, outLinkUid, teamId, teamToken } = req.body as GetHistoriesProps;
const limit = shareId && outLinkUid ? 20 : 30; const limit = shareId && outLinkUid ? 20 : 30;
...@@ -28,10 +29,12 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -28,10 +29,12 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
} }
}; };
} }
if (appId && outLinkUid) { if (appId && teamId && teamToken) {
const { uid } = await authTeamSpaceToken({ teamId, teamToken });
return { return {
shareId, teamId,
outLinkUid: outLinkUid, appId,
outLinkUid: uid,
source: ChatSourceEnum.team source: ChatSourceEnum.team
}; };
} }
......
projects/app/src/pages/api/core/chat/item/getSpeech.ts
...@@ -4,7 +4,7 @@ import { connectToDatabase } from '@/service/mongo'; ...@@ -4,7 +4,7 @@ import { connectToDatabase } from '@/service/mongo';
import { GetChatSpeechProps } from '@/global/core/chat/api.d'; import { GetChatSpeechProps } from '@/global/core/chat/api.d';
import { text2Speech } from '@fastgpt/service/core/ai/audio/speech'; import { text2Speech } from '@fastgpt/service/core/ai/audio/speech';
import { pushAudioSpeechUsage } from '@/service/support/wallet/usage/push'; import { pushAudioSpeechUsage } from '@/service/support/wallet/usage/push';
import { authCertOrShareId } from '@fastgpt/service/support/permission/auth/common'; import { authChatCert } from '@/service/support/permission/auth/chat';
import { authType2UsageSource } from '@/service/support/wallet/usage/utils'; import { authType2UsageSource } from '@/service/support/wallet/usage/utils';
import { getAudioSpeechModel } from '@fastgpt/service/core/ai/model'; import { getAudioSpeechModel } from '@fastgpt/service/core/ai/model';
import { MongoTTSBuffer } from '@fastgpt/service/common/buffer/tts/schema'; import { MongoTTSBuffer } from '@fastgpt/service/common/buffer/tts/schema';
...@@ -19,13 +19,13 @@ import { MongoTTSBuffer } from '@fastgpt/service/common/buffer/tts/schema'; ...@@ -19,13 +19,13 @@ import { MongoTTSBuffer } from '@fastgpt/service/common/buffer/tts/schema';
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try { try {
await connectToDatabase(); await connectToDatabase();
const { ttsConfig, input, shareId } = req.body as GetChatSpeechProps; const { ttsConfig, input } = req.body as GetChatSpeechProps;
if (!ttsConfig.model || !ttsConfig.voice) { if (!ttsConfig.model || !ttsConfig.voice) {
throw new Error('model or voice not found'); throw new Error('model or voice not found');
} }
const { teamId, tmbId, authType } = await authCertOrShareId({ req, authToken: true, shareId }); const { teamId, tmbId, authType } = await authChatCert({ req, authToken: true });
const ttsModel = getAudioSpeechModel(ttsConfig.model); const ttsModel = getAudioSpeechModel(ttsConfig.model);
const voiceData = ttsModel.voices?.find((item) => item.value === ttsConfig.voice); const voiceData = ttsModel.voices?.find((item) => item.value === ttsConfig.voice);
......
projects/app/src/pages/api/core/chat/outLink/getInforByTeamId.ts deleted 100644 → 0
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import type { chatByTeamProps } from '@/global/core/chat/api.d';
import axios from 'axios';
import { MongoChatItem } from '@fastgpt/service/core/chat/chatItemSchema';
import { getChatItems } from '@fastgpt/service/core/chat/controller';
import { selectShareResponse } from '@/utils/service/core/chat';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
await connectToDatabase();
let { teamId, appId, outLinkUid } = req.query as chatByTeamProps;
const history = await MongoChatItem.find({
appId: appId,
outLinkUid: outLinkUid,
teamId: teamId
});
jsonRes(res, {
data: history
});
} catch (err) {
jsonRes(res, {
code: 500,
data: req.query,
error: err
});
}
}
export const config = {
api: {
responseLimit: '10mb'
}
};
projects/app/src/pages/api/core/chat/outLink/init.ts
...@@ -9,7 +9,7 @@ import { getChatItems } from '@fastgpt/service/core/chat/controller'; ...@@ -9,7 +9,7 @@ import { getChatItems } from '@fastgpt/service/core/chat/controller';
import { MongoTeamMember } from '@fastgpt/service/support/user/team/teamMemberSchema'; import { MongoTeamMember } from '@fastgpt/service/support/user/team/teamMemberSchema';
import { authOutLink } from '@/service/support/permission/auth/outLink'; import { authOutLink } from '@/service/support/permission/auth/outLink';
import { MongoApp } from '@fastgpt/service/core/app/schema'; import { MongoApp } from '@fastgpt/service/core/app/schema';
import { selectShareResponse } from '@/utils/service/core/chat'; import { selectSimpleChatResponse } from '@/utils/service/core/chat';
import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; import { AppErrEnum } from '@fastgpt/global/common/error/code/app';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema'; import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import { ChatErrEnum } from '@fastgpt/global/common/error/code/chat'; import { ChatErrEnum } from '@fastgpt/global/common/error/code/chat';
...@@ -50,7 +50,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -50,7 +50,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
// pick share response field // pick share response field
history.forEach((item) => { history.forEach((item) => {
item.responseData = selectShareResponse({ responseData: item.responseData }); item.responseData = selectSimpleChatResponse({ responseData: item.responseData });
}); });
jsonRes<InitChatResponse>(res, { jsonRes<InitChatResponse>(res, {
......
projects/app/src/pages/api/core/chat/team/init.ts
...@@ -4,59 +4,57 @@ import { connectToDatabase } from '@/service/mongo'; ...@@ -4,59 +4,57 @@ import { connectToDatabase } from '@/service/mongo';
import { getGuideModule } from '@fastgpt/global/core/module/utils'; import { getGuideModule } from '@fastgpt/global/core/module/utils';
import { getChatModelNameListByModules } from '@/service/core/app/module'; import { getChatModelNameListByModules } from '@/service/core/app/module';
import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants'; import { ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants';
import type { InitChatProps, InitChatResponse } from '@/global/core/chat/api.d'; import type { InitChatResponse, InitTeamChatProps } from '@/global/core/chat/api.d';
import { MongoChat } from '@fastgpt/service/core/chat/chatSchema'; import { MongoChat } from '@fastgpt/service/core/chat/chatSchema';
import { MongoApp } from '@fastgpt/service/core/app/schema'; import { MongoApp } from '@fastgpt/service/core/app/schema';
import { getChatItems } from '@fastgpt/service/core/chat/controller'; import { getChatItems } from '@fastgpt/service/core/chat/controller';
import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; import { AppErrEnum } from '@fastgpt/global/common/error/code/app';
import { authTeamSpaceToken } from '@/service/support/permission/auth/team';
import { MongoTeam } from '@fastgpt/service/support/user/team/teamSchema';
import { ChatErrEnum } from '@fastgpt/global/common/error/code/chat';
import { selectSimpleChatResponse } from '@/utils/service/core/chat';
export default async function handler(req: NextApiRequest, res: NextApiResponse) { export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try { try {
await connectToDatabase(); await connectToDatabase();
let { appId, chatId, outLinkUid } = req.query as { let { teamId, appId, chatId, teamToken } = req.query as InitTeamChatProps;
chatId?: string;
appId?: string;
outLinkUid?: string;
};
if (!appId) { if (!teamId || !appId || !teamToken) {
return jsonRes(res, { throw new Error('teamId, appId, teamToken are required');
code: 501,
message: "You don't have an app yet"
});
} }
// auth app permission const { uid } = await authTeamSpaceToken({
const [chat, app] = await Promise.all([ teamId,
// authApp({ teamToken
// req, });
// authToken: false,
// appId, const [team, chat, app] = await Promise.all([
// per: 'r' MongoTeam.findById(teamId, 'name avatar').lean(),
// }), MongoChat.findOne({ teamId, appId, chatId }).lean(),
chatId ? MongoChat.findOne({ appId, chatId }) : undefined,
MongoApp.findById(appId).lean() MongoApp.findById(appId).lean()
]); ]);
if (!app) { if (!app) {
throw new Error(AppErrEnum.unExist); throw new Error(AppErrEnum.unExist);
} }
// auth chat permission // auth chat permission
// if (chat && chat.outLinkUid !== outLinkUid) { if (chat && chat.outLinkUid !== uid) {
// throw new Error(ChatErrEnum.unAuthChat); throw new Error(ChatErrEnum.unAuthChat);
// } }
// // auth chat permission
// if (chat && !app.canWrite && String(tmbId) !== String(chat?.tmbId)) {
// throw new Error(ChatErrEnum.unAuthChat);
// }
// get app and history // get app and history
const { history } = await getChatItems({ const { history } = await getChatItems({
appId, appId,
chatId, chatId,
limit: 30, limit: 30,
field: `dataId obj value adminFeedback userBadFeedback userGoodFeedback ${ModuleOutputKeyEnum.responseData}` field: `dataId obj value userGoodFeedback userBadFeedback adminFeedback ${ModuleOutputKeyEnum.responseData}`
});
// pick share response field
history.forEach((item) => {
item.responseData = selectSimpleChatResponse({ responseData: item.responseData });
}); });
jsonRes<InitChatResponse>(res, { jsonRes<InitChatResponse>(res, {
...@@ -64,7 +62,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse) ...@@ -64,7 +62,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)
chatId, chatId,
appId, appId,
title: chat?.title || '新对话', title: chat?.title || '新对话',
userAvatar: undefined, userAvatar: team?.avatar,
variables: chat?.variables || {}, variables: chat?.variables || {},
history, history,
app: { app: {
......
projects/app/src/pages/api/core/dataset/collection/list.ts
...@@ -8,9 +8,9 @@ import type { GetDatasetCollectionsProps } from '@/global/core/api/datasetReq'; ...@@ -8,9 +8,9 @@ import type { GetDatasetCollectionsProps } from '@/global/core/api/datasetReq';
import { PagingData } from '@/types'; import { PagingData } from '@/types';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema'; import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { DatasetCollectionTypeEnum } from '@fastgpt/global/core/dataset/constants'; import { DatasetCollectionTypeEnum } from '@fastgpt/global/core/dataset/constants';
import { startQueue } from '@/service/utils/tools';
import { authDataset } from '@fastgpt/service/support/permission/auth/dataset'; import { authDataset } from '@fastgpt/service/support/permission/auth/dataset';
import { DatasetDataCollectionName } from '@fastgpt/service/core/dataset/data/schema'; import { DatasetDataCollectionName } from '@fastgpt/service/core/dataset/data/schema';
import { startTrainingQueue } from '@/service/core/dataset/training/utils';
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) { export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
try { try {
...@@ -158,7 +158,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse< ...@@ -158,7 +158,7 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse<
); );
if (data.find((item) => item.trainingAmount > 0)) { if (data.find((item) => item.trainingAmount > 0)) {
startQueue(); startTrainingQueue();
} }
// count collections // count collections
......
projects/app/src/pages/api/core/dataset/data/insertData.ts
...@@ -75,7 +75,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -75,7 +75,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
a: formatA a: formatA
}); });
const { insertId, charsLength } = await insertData2Dataset({ const { insertId, tokens } = await insertData2Dataset({
teamId, teamId,
tmbId, tmbId,
datasetId, datasetId,
...@@ -90,7 +90,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -90,7 +90,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
pushGenerateVectorUsage({ pushGenerateVectorUsage({
teamId, teamId,
tmbId, tmbId,
charsLength, tokens,
model: vectorModelData.model model: vectorModelData.model
}); });
......
projects/app/src/pages/api/core/dataset/data/update.ts
...@@ -34,7 +34,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -34,7 +34,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
insertLen: 1 insertLen: 1
}); });
const { charsLength } = await updateData2Dataset({ const { tokens } = await updateData2Dataset({
dataId: id, dataId: id,
q, q,
a, a,
...@@ -45,7 +45,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -45,7 +45,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
pushGenerateVectorUsage({ pushGenerateVectorUsage({
teamId, teamId,
tmbId, tmbId,
charsLength, tokens,
model: vectorModel model: vectorModel
}); });
......
projects/app/src/pages/api/core/dataset/searchTest.ts
...@@ -58,7 +58,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -58,7 +58,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
extensionBg: datasetSearchExtensionBg extensionBg: datasetSearchExtensionBg
}); });
const { searchRes, charsLength, ...result } = await searchDatasetData({ const { searchRes, tokens, ...result } = await searchDatasetData({
teamId, teamId,
reRankQuery: rewriteQuery, reRankQuery: rewriteQuery,
queries: concatQueries, queries: concatQueries,
...@@ -74,14 +74,14 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -74,14 +74,14 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
const { totalPoints } = pushGenerateVectorUsage({ const { totalPoints } = pushGenerateVectorUsage({
teamId, teamId,
tmbId, tmbId,
charsLength, tokens,
model: dataset.vectorModel, model: dataset.vectorModel,
source: apikey ? UsageSourceEnum.api : UsageSourceEnum.fastgpt, source: apikey ? UsageSourceEnum.api : UsageSourceEnum.fastgpt,
...(aiExtensionResult && ...(aiExtensionResult &&
extensionModel && { extensionModel && {
extensionModel: extensionModel.name, extensionModel: extensionModel.name,
extensionCharsLength: aiExtensionResult.charsLength extensionTokens: aiExtensionResult.tokens
}) })
}); });
if (apikey) { if (apikey) {
......
projects/app/src/pages/api/timerTask/dataset/checkInValidDatasetFiles.ts 0 → 100644
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import {
delFileByFileIdList,
getGFSCollection
} from '@fastgpt/service/common/file/gridfs/controller';
import { addLog } from '@fastgpt/service/common/system/log';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { addHours } from 'date-fns';
/*
check dataset.files data. If there is no match in dataset.collections, delete it
可能异常情况
1. 上传了文件，未成功创建集合
*/
let deleteFileAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const { startHour = 24, endHour = 1 } = req.body as {
startHour?: number;
endHour?: number;
limit?: number;
};
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - 3 day
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deleteFileAmount = 0;
console.log(start, end);
await checkFiles(start, end);
jsonRes(res, {
data: deleteFileAmount,
message: 'success'
});
} catch (error) {
addLog.error(`check valid dataset files error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkFiles(start: Date, end: Date) {
const collection = getGFSCollection('dataset');
const where = {
uploadDate: { $gte: start, $lte: end }
};
// 1. get all file _id
const files = await collection
.find(where, {
projection: {
metadata: 1,
_id: 1
}
})
.toArray();
console.log('total files', files.length);
let index = 0;
for await (const file of files) {
try {
// 2. find fileId in dataset.collections
const hasCollection = await MongoDatasetCollection.countDocuments({
teamId: file.metadata.teamId,
fileId: file._id
});
// 3. if not found, delete file
if (hasCollection === 0) {
await delFileByFileIdList({ bucketName: 'dataset', fileIdList: [String(file._id)] });
console.log('delete file', file._id);
deleteFileAmount++;
}
index++;
index % 100 === 0 && console.log(index);
} catch (error) {
console.log(error);
}
}
console.log(`检测完成，共删除 ${deleteFileAmount} 个无效文件`);
}
projects/app/src/pages/api/timerTask/dataset/checkInvalidDatasetImage.ts 0 → 100644
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { addLog } from '@fastgpt/service/common/system/log';
import { addHours } from 'date-fns';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { MongoImage } from '@fastgpt/service/common/file/image/schema';
/*
检测无效的数据集图片
可能异常情况：
1. 上传文件过程中，上传了图片，但是最终没有创建数据集。
*/
let deleteImageAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const {
startHour = 72,
endHour = 24,
limit = 10
} = req.body as { startHour?: number; endHour?: number; limit?: number };
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - 3 day
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deleteImageAmount = 0;
await checkInvalid(start, end, limit);
jsonRes(res, {
data: deleteImageAmount
});
} catch (error) {
addLog.error(`check Invalid user error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkInvalid(start: Date, end: Date, limit = 50) {
const images = await MongoImage.find(
{
createTime: {
$gte: start,
$lte: end
},
'metadata.relatedId': { $exists: true }
},
'_id teamId metadata'
);
console.log('total images', images.length);
let index = 0;
for await (const image of images) {
try {
// 1. 检测是否有对应的集合
const collection = await MongoDatasetCollection.findOne(
{
teamId: image.teamId,
'metadata.relatedImgId': image.metadata?.relatedId
},
'_id'
);
if (!collection) {
await image.deleteOne();
deleteImageAmount++;
}
index++;
index % 100 === 0 && console.log(index);
} catch (error) {
console.log(error);
}
}
console.log(`检测完成，共删除 ${deleteImageAmount} 个无效图片`);
}
projects/app/src/pages/api/timerTask/dataset/checkInvalidMongoCollection.ts 0 → 100644
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { addLog } from '@fastgpt/service/common/system/log';
import { deleteDatasetDataVector } from '@fastgpt/service/common/vectorStore/controller';
import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema';
import { addHours } from 'date-fns';
import { MongoDatasetCollection } from '@fastgpt/service/core/dataset/collection/schema';
import { MongoDatasetTraining } from '@fastgpt/service/core/dataset/training/schema';
/*
检测无效的 Mongo 数据
异常情况：
1. 训练过程删除知识库，可能导致还会有新的数据插入，导致无效。
*/
let deleteAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const { startHour = 3, endHour = 1 } = req.body as { startHour?: number; endHour?: number };
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - endHour
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deleteAmount = 0;
await checkInvalidCollection(start, end);
jsonRes(res, {
data: deleteAmount,
message: 'success'
});
} catch (error) {
addLog.error(`check Invalid user error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkInvalidCollection(start: Date, end: Date) {
// 1. 获取时间范围的所有data
const rows = await MongoDatasetData.find(
{
updateTime: {
$gte: start,
$lte: end
}
},
'_id teamId collectionId'
).lean();
// 2. 合并所有的collectionId
const map = new Map<string, { teamId: string; collectionId: string }>();
for (const item of rows) {
const collectionId = String(item.collectionId);
if (!map.has(collectionId)) {
map.set(collectionId, { teamId: item.teamId, collectionId });
}
}
const list = Array.from(map.values());
console.log('total collections', list.length);
let index = 0;
for await (const item of list) {
try {
// 3. 查看该collection是否存在，不存在，则删除对应的数据
const collection = await MongoDatasetCollection.findOne({ _id: item.collectionId });
if (!collection) {
const result = await Promise.all([
MongoDatasetTraining.deleteMany({
teamId: item.teamId,
collectionId: item.collectionId
}),
MongoDatasetData.deleteMany({
teamId: item.teamId,
collectionId: item.collectionId
}),
deleteDatasetDataVector({
teamId: item.teamId,
collectionIds: [String(item.collectionId)]
})
]);
console.log(result);
console.log('collection is not found', item);
continue;
}
} catch (error) {}
console.log(++index);
}
}
projects/app/src/pages/api/timerTask/dataset/checkInvalidVector.ts 0 → 100644
import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response';
import { connectToDatabase } from '@/service/mongo';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { addLog } from '@fastgpt/service/common/system/log';
import {
deleteDatasetDataVector,
getVectorDataByTime
} from '@fastgpt/service/common/vectorStore/controller';
import { MongoDatasetData } from '@fastgpt/service/core/dataset/data/schema';
import { addHours } from 'date-fns';
/*
检测无效的 Vector 数据.
异常情况：
1. 插入数据时，vector成功，mongo失败
2. 更新数据，也会有插入 vector
*/
let deletedVectorAmount = 0;
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
const { startHour = 5, endHour = 1 } = req.body as { startHour?: number; endHour?: number };
await authCert({ req, authRoot: true });
await connectToDatabase();
// start: now - maxDay, end: now - endHour
const start = addHours(new Date(), -startHour);
const end = addHours(new Date(), -endHour);
deletedVectorAmount = 0;
await checkInvalidVector(start, end);
jsonRes(res, {
data: deletedVectorAmount,
message: 'success'
});
} catch (error) {
addLog.error(`check Invalid user error`, error);
jsonRes(res, {
code: 500,
error
});
}
}
export async function checkInvalidVector(start: Date, end: Date) {
// 1. get all vector data
const rows = await getVectorDataByTime(start, end);
console.log('total data', rows.length);
let index = 0;
for await (const item of rows) {
if (!item.teamId || !item.datasetId || !item.id) {
console.log('error data', item);
continue;
}
try {
// 2. find dataset.data
const hasData = await MongoDatasetData.countDocuments({
teamId: item.teamId,
datasetId: item.datasetId,
'indexes.dataId': item.id
});
// 3. if not found, delete vector
if (hasData === 0) {
await deleteDatasetDataVector({
teamId: item.teamId,
id: item.id
});
console.log('delete vector data', item.id);
deletedVectorAmount++;
}
index++;
index % 100 === 0 && console.log(index);
} catch (error) {
console.log(error);
}
}
console.log(`检测完成，共删除 ${deletedVectorAmount} 个无效 向量 数据`);
}
projects/app/src/pages/api/v1/audio/transcriptions.ts
import type { NextApiRequest, NextApiResponse } from 'next'; import type { NextApiRequest, NextApiResponse } from 'next';
import { jsonRes } from '@fastgpt/service/common/response'; import { jsonRes } from '@fastgpt/service/common/response';
import { authCert } from '@fastgpt/service/support/permission/auth/common';
import { withNextCors } from '@fastgpt/service/common/middle/cors'; import { withNextCors } from '@fastgpt/service/common/middle/cors';
import { getUploadModel } from '@fastgpt/service/common/file/multer'; import { getUploadModel } from '@fastgpt/service/common/file/multer';
import { removeFilesByPaths } from '@fastgpt/service/common/file/utils'; import { removeFilesByPaths } from '@fastgpt/service/common/file/utils';
import fs from 'fs'; import fs from 'fs';
import { getAIApi } from '@fastgpt/service/core/ai/config'; import { getAIApi } from '@fastgpt/service/core/ai/config';
import { pushWhisperUsage } from '@/service/support/wallet/usage/push'; import { pushWhisperUsage } from '@/service/support/wallet/usage/push';
import { authChatCert } from '@/service/support/permission/auth/chat';
const upload = getUploadModel({ const upload = getUploadModel({
maxSize: 2 maxSize: 2
...@@ -18,12 +18,20 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -18,12 +18,20 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
try { try {
const { const {
file, file,
data: { duration } data: { duration, teamId: spaceTeamId, teamToken }
} = await upload.doUpload<{ duration: number; shareId?: string }>(req, res); } = await upload.doUpload<{
duration: number;
shareId?: string;
teamId?: string;
teamToken?: string;
}>(req, res);
req.body.teamId = spaceTeamId;
req.body.teamToken = teamToken;
filePaths = [file.path]; filePaths = [file.path];
const { teamId, tmbId } = await authCert({ req, authToken: true }); const { teamId, tmbId } = await authChatCert({ req, authToken: true });
if (!global.whisperModel) { if (!global.whisperModel) {
throw new Error('whisper model not found'); throw new Error('whisper model not found');
......
projects/app/src/pages/api/v1/embeddings.ts
...@@ -36,7 +36,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -36,7 +36,7 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
await checkTeamAIPoints(teamId); await checkTeamAIPoints(teamId);
const { charsLength, vectors } = await getVectorsByText({ const { tokens, vectors } = await getVectorsByText({
input: query, input: query,
model: getVectorModel(model) model: getVectorModel(model)
}); });
...@@ -50,15 +50,15 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex ...@@ -50,15 +50,15 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
})), })),
model, model,
usage: { usage: {
prompt_tokens: charsLength, prompt_tokens: tokens,
total_tokens: charsLength total_tokens: tokens
} }
}); });
const { totalPoints } = pushGenerateVectorUsage({ const { totalPoints } = pushGenerateVectorUsage({
teamId, teamId,
tmbId, tmbId,
charsLength, tokens,
model, model,
billId, billId,
source: getUsageSourceByAuthType({ authType }) source: getUsageSourceByAuthType({ authType })
......
projects/app/src/pages/app/detail/components/OutLink/SelectUsingWayModal.tsx
...@@ -136,7 +136,7 @@ const SelectUsingWayModal = ({ share, onClose }: { share: OutLinkSchema; onClose ...@@ -136,7 +136,7 @@ const SelectUsingWayModal = ({ share, onClose }: { share: OutLinkSchema; onClose
/> />
{/* config */} {/* config */}
<Grid gridTemplateColumns={['repeat(3,1fr)']} gridGap={4} my={5}> <Grid gridTemplateColumns={['repeat(2,1fr)', 'repeat(3,1fr)']} gridGap={4} my={5}>
<Flex {...gridItemStyle}> <Flex {...gridItemStyle}>
<Box flex={1}>{t('core.app.outLink.Show History')}</Box> <Box flex={1}>{t('core.app.outLink.Show History')}</Box>
<Switch {...register('showHistory')} /> <Switch {...register('showHistory')} />
......
projects/app/src/pages/app/detail/components/OutLink/Share.tsx
...@@ -14,10 +14,6 @@ import { ...@@ -14,10 +14,6 @@ import {
ModalBody, ModalBody,
Input, Input,
Switch, Switch,
Menu,
MenuButton,
MenuList,
MenuItem,
Link Link
} from '@chakra-ui/react'; } from '@chakra-ui/react';
import { QuestionOutlineIcon } from '@chakra-ui/icons'; import { QuestionOutlineIcon } from '@chakra-ui/icons';
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment