Commit 44e9299d by Archer Committed by GitHub

V4.13.2 features (#5792)

* add manual create http toolset (#5743)

* add manual create http toolset

* optimize code

* optimize

* fix

* fix

* rename filename

* feat: integrate ts-rest (#5741)

* feat: integrate ts-rest

* chore: classify core contract and pro contract

* chore: update lockfile

* chore: tweak dir structure

* chore: tweak dir structure

* update tsrest code (#5755)

* doc

* update tsrest code

* fix http toolset (#5753)

* fix http toolset

* fix

* perf: http toolset

* fix: toolresponse result (#5760)

* doc

* fix: toolresponse result

* fix: mongo watch

* remove log

* feat: integrated to minio (#5748)

* feat: migrate to minio

* feat: migrate apps' and dataset's avatar to minio

* feat: migrate more avatars to minio

* fix: lock file

* feat: migrate copyright settings' logo to minio

* feat: integrate minio

* chore: improve code

* chore: rename variables

* refactor: s3 class

* fix: s3 and mongo operations

* chore: add session for avatar source

* fix: init s3 buckets

* fix: bugbot issues

* expired time code

* perf: avatar code

* union type

* export favouriteContract

* empty bucket check

---------

Co-authored-by: archer <545436317@qq.com>

* refactor: zod schema to generate OpenAPI instead (#5771)

* doc

* fix: text split code (#5773)

* fix: toolresponse result

* remove log

* stream remove

* fix: text split code

* fix: workflow (#5779)

* fix: toolresponse result

* remove log

* fix: value check

* fix: workflow

* openapi doc

* perf: bucket delete cron

* doc

* feat: apikey health

* feat: export variables

* api code move

* perf: workflow performance (#5783)

* perf: reactflow context

* perf: workflow context split

* perf: nodeList computed map

* perf: nodes dependen

* perf: workflow performance

* workflow performance

* removel og

* lock

* version

* loop drag

* reactflow size

* reactflow size

* fix: s3init (#5784)

* doc

* fix: s3init

* perf: dynamic import

* remove moongose dep

* worker build

* worker code

* perf: worker build

* fix: error throw

* doc

* doc

* fix: build

* fix: dockerfile

* nextjs config

* fix: worker

* fix: build (#5791)

* fix: build

* vector cache code

* fix: app info modal avatar upload method replace (#5787)

* fix: app info modal avatar upload method replace

* chore: replace all useSelectFile with useUploadAvatar

* remove invalid code

* add size

* Update projects/app/src/pageComponents/app/detail/WorkflowComponents/Flow/nodes/render/RenderInput/templates/CommonInputForm.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update projects/app/src/pageComponents/app/detail/WorkflowComponents/context/workflowInitContext.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
parent ca3053f0
# 服务端资源版本 ID 缓存方案
## 背景
FastGPT 会采用多节点部署方式,有部分数据缓存会存储在内存里。当需要使用这部分数据时(不管是通过 API 获取,还是后端服务自己获取),都是直接拉取内存数据,这可能会导致数据不一致问题,尤其是用户通过 API 更新数据后再获取,就容易获取未修改数据的节点。
## 解决方案
1. 给每一个缓存数据加上一个版本 ID。
2. 获取该数据时候,不直接引用该数据,而是通过一个 function 获取,该 function 可选的传入一个 versionId。
3. 获取数据时,先检查该 versionId 与 redis 中,资源版本id 与传入的 versionId 是否一致。
4. 如果数据一致,则直接返回数据。
5. 如果数据不一致,则重新获取数据,并返回最新的 versionId。调用方则需要更新其缓存的 versionId。
## 代码方案
* 获取和更新缓存的代码,直接复用 FastGPT/packages/service/common/redis/cache.ts
* 每个资源,自己维护一个 cacheKey
* 每次更新资源/触发拉取最新资源时,都需要更新 cacheKey 的值。
## 涉及的业务
* [ ] FastGPT/projects/app/src/pages/api/common/system/getInitData.ts,获取初始数据
\ No newline at end of file
# 背景
一个通用表格多选 hook/component, 它可以实现表格每一行数据的选择,并且在触发一次选择后,会有特殊的按键进行批量操作。
# 具体描述
当有一行被选中时,底部会出现悬浮层,可以进行批量操作(具体有哪些批量操作由外部决定)
![alt text](./image-1.png)
# 预期封装
1. 选中的值存储在 hook 里,便于判断是否触发底部悬浮层
2. 悬浮层外层 Box 在 hook 里,child 由调用组件实现
3. FastGPT/packages/web/hooks/useTableMultipleSelect.tsx 在这个文件下实现
...@@ -9,4 +9,5 @@ README.md ...@@ -9,4 +9,5 @@ README.md
.yalc/ .yalc/
yalc.lock yalc.lock
testApi/ testApi/
*.local.* *.local.*
\ No newline at end of file *.local
\ No newline at end of file
...@@ -37,4 +37,6 @@ files/helm/fastgpt/charts/*.tgz ...@@ -37,4 +37,6 @@ files/helm/fastgpt/charts/*.tgz
tmp/ tmp/
coverage coverage
document/.source document/.source
\ No newline at end of file
projects/app/worker/
\ No newline at end of file
# CLAUDE.md # CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. 本文件为 Claude Code (claude.ai/code) 在本仓库中工作时提供指导说明。
## Project Overview ## 输出要求
FastGPT is an AI Agent construction platform providing out-of-the-box data processing, model invocation capabilities, and visual workflow orchestration through Flow. This is a full-stack TypeScript application built on NextJS with MongoDB/PostgreSQL backends. 1. 输出语言:中文
2. 输出的设计文档位置:.claude/design,以 Markdown 文件为主。
**Tech Stack**: NextJS + TypeScript + ChakraUI + MongoDB + PostgreSQL (PG Vector)/Milvus 3. 输出 Plan 时,均需写入 .claude/plan 目录下,以 Markdown 文件为主。
## Architecture ## 项目概述
This is a monorepo using pnpm workspaces with the following key structure: FastGPT 是一个 AI Agent 构建平台,通过 Flow 提供开箱即用的数据处理、模型调用能力和可视化工作流编排。这是一个基于 NextJS 构建的全栈 TypeScript 应用,后端使用 MongoDB/PostgreSQL。
### Packages (Library Code) **技术栈**: NextJS + TypeScript + ChakraUI + MongoDB + PostgreSQL (PG Vector)/Milvus
- `packages/global/` - Shared types, constants, utilities used across all projects
- `packages/service/` - Backend services, database schemas, API controllers, workflow engine ## 架构
- `packages/web/` - Shared frontend components, hooks, styles, i18n
- `packages/templates/` - Application templates for the template market 这是一个使用 pnpm workspaces 的 monorepo,主要结构如下:
### Projects (Applications) ### Packages (库代码)
- `projects/app/` - Main NextJS web application (frontend + API routes) - `packages/global/` - 所有项目共享的类型、常量、工具函数
- `projects/sandbox/` - NestJS code execution sandbox service - `packages/service/` - 后端服务、数据库模型、API 控制器、工作流引擎
- `projects/mcp_server/` - Model Context Protocol server implementation - `packages/web/` - 共享的前端组件、hooks、样式、国际化
- `packages/templates/` - 模板市场的应用模板
### Key Directories
- `document/` - Documentation site (NextJS app with content) ### Projects (应用程序)
- `plugins/` - External plugins (models, crawlers, etc.) - `projects/app/` - 主 NextJS Web 应用(前端 + API 路由)
- `deploy/` - Docker and Helm deployment configurations - `projects/sandbox/` - NestJS 代码执行沙箱服务
- `test/` - Centralized test files and utilities - `projects/mcp_server/` - Model Context Protocol 服务器实现
## Development Commands ### 关键目录
- `document/` - 文档站点(NextJS 应用及内容)
### Main Commands (run from project root) - `plugins/` - 外部插件(模型、爬虫等)
- `pnpm dev` - Start development for all projects (uses package.json workspace scripts) - `deploy/` - Docker 和 Helm 部署配置
- `pnpm build` - Build all projects - `test/` - 集中的测试文件和工具
- `pnpm test` - Run tests using Vitest
- `pnpm test:workflow` - Run workflow-specific tests ## 开发命令
- `pnpm lint` - Run ESLint across all TypeScript files with auto-fix
- `pnpm format-code` - Format code using Prettier ### 主要命令(从项目根目录运行)
- `pnpm dev` - 启动所有项目的开发环境(使用 package.json 的 workspace 脚本)
### Project-Specific Commands - `pnpm build` - 构建所有项目
**Main App (projects/app/)**: - `pnpm test` - 使用 Vitest 运行测试
- `cd projects/app && pnpm dev` - Start NextJS dev server - `pnpm test:workflow` - 运行工作流相关测试
- `cd projects/app && pnpm build` - Build NextJS app - `pnpm lint` - 对所有 TypeScript 文件运行 ESLint 并自动修复
- `cd projects/app && pnpm start` - Start production server - `pnpm format-code` - 使用 Prettier 格式化代码
**Sandbox (projects/sandbox/)**: ### 项目专用命令
- `cd projects/sandbox && pnpm dev` - Start NestJS dev server with watch mode **主应用 (projects/app/)**:
- `cd projects/sandbox && pnpm build` - Build NestJS app - `cd projects/app && pnpm dev` - 启动 NextJS 开发服务器
- `cd projects/sandbox && pnpm test` - Run Jest tests - `cd projects/app && pnpm build` - 构建 NextJS 应用
- `cd projects/app && pnpm start` - 启动生产服务器
**MCP Server (projects/mcp_server/)**:
- `cd projects/mcp_server && bun dev` - Start with Bun in watch mode **沙箱 (projects/sandbox/)**:
- `cd projects/mcp_server && bun build` - Build MCP server - `cd projects/sandbox && pnpm dev` - 以监视模式启动 NestJS 开发服务器
- `cd projects/mcp_server && bun start` - Start MCP server - `cd projects/sandbox && pnpm build` - 构建 NestJS 应用
- `cd projects/sandbox && pnpm test` - 运行 Jest 测试
### Utility Commands
- `pnpm create:i18n` - Generate i18n translation files **MCP 服务器 (projects/mcp_server/)**:
- `pnpm api:gen` - Generate OpenAPI documentation - `cd projects/mcp_server && bun dev` - 使用 Bun 以监视模式启动
- `pnpm initIcon` - Initialize icon assets - `cd projects/mcp_server && bun build` - 构建 MCP 服务器
- `pnpm gen:theme-typings` - Generate Chakra UI theme typings - `cd projects/mcp_server && bun start` - 启动 MCP 服务器
## Testing ### 工具命令
- `pnpm create:i18n` - 生成国际化翻译文件
The project uses Vitest for testing with coverage reporting. Key test commands: - `pnpm api:gen` - 生成 OpenAPI 文档
- `pnpm test` - Run all tests - `pnpm initIcon` - 初始化图标资源
- `pnpm test:workflow` - Run workflow tests specifically - `pnpm gen:theme-typings` - 生成 Chakra UI 主题类型定义
- Test files are located in `test/` directory and `projects/app/test/`
- Coverage reports are generated in `coverage/` directory ## 测试
## Code Organization Patterns 项目使用 Vitest 进行测试并生成覆盖率报告。主要测试命令:
- `pnpm test` - 运行所有测试
### Monorepo Structure - `pnpm test:workflow` - 专门运行工作流测试
- Shared code lives in `packages/` and is imported using workspace references - 测试文件位于 `test/` 目录和 `projects/app/test/`
- Each project in `projects/` is a standalone application - 覆盖率报告生成在 `coverage/` 目录
- Use `@fastgpt/global`, `@fastgpt/service`, `@fastgpt/web` imports for shared packages
## 代码组织模式
### API Structure
- NextJS API routes in `projects/app/src/pages/api/` ### Monorepo 结构
- Core business logic in `packages/service/core/` - 共享代码存放在 `packages/` 中,通过 workspace 引用导入
- Database schemas in `packages/service/` with MongoDB/Mongoose - `projects/` 中的每个项目都是独立的应用程序
- 使用 `@fastgpt/global`、`@fastgpt/service`、`@fastgpt/web` 导入共享包
### Frontend Architecture
- React components in `projects/app/src/components/` and `packages/web/components/` ### API 结构
- Chakra UI for styling with custom theme in `packages/web/styles/theme.ts` - NextJS API 路由在 `projects/app/src/pages/api/`
- i18n support with files in `packages/web/i18n/` - API 路由合约定义在`packages/global/openapi/`, 对应的
- State management using React Context and Zustand - 通用服务端业务逻辑在 `packages/service/`和`projects/app/src/service`
- 数据库模型在 `packages/service/` 中,使用 MongoDB/Mongoose
### Workflow System
- Visual workflow editor using ReactFlow ### 前端架构
- Workflow engine in `packages/service/core/workflow/` - React 组件在 `projects/app/src/components/` 和 `packages/web/components/`
- Node definitions in `packages/global/core/workflow/template/` - 使用 Chakra UI 进行样式设计,自定义主题在 `packages/web/styles/theme.ts`
- Dispatch system for executing workflow nodes - 国际化支持文件在 `packages/web/i18n/`
- 使用 React Context 和 Zustand 进行状态管理
## Development Notes
## 开发注意事项
- **Package Manager**: Uses pnpm with workspace configuration
- **Node Version**: Requires Node.js >=18.16.0, pnpm >=9.0.0 - **包管理器**: 使用 pnpm 及 workspace 配置
- **Database**: Supports MongoDB, PostgreSQL with pgvector, or Milvus for vector storage - **Node 版本**: 需要 Node.js >=18.16.0, pnpm >=9.0.0
- **AI Integration**: Supports multiple AI providers through unified interface - **数据库**: 支持 MongoDB、带 pgvector 的 PostgreSQL 或 Milvus 向量存储
- **Internationalization**: Full i18n support for Chinese, English, and Japanese - **AI 集成**: 通过统一接口支持多个 AI 提供商
- **国际化**: 完整支持中文、英文和日文
## Key File Patterns
## 关键文件模式
- `.ts` and `.tsx` files use TypeScript throughout
- Database schemas use Mongoose with TypeScript - `.ts` 和 `.tsx` 文件全部使用 TypeScript
- API routes follow NextJS conventions - 数据库模型使用 Mongoose 配合 TypeScript
- Component files use React functional components with hooks - API 路由遵循 NextJS 约定
- Shared types defined in `packages/global/` with `.d.ts` files - 组件文件使用 React 函数式组件和 hooks
- 共享类型定义在 `packages/global/` 的 `.d.ts` 文件中
## Environment Configuration
## 环境配置
- Configuration files in `projects/app/data/config.json`
- Environment-specific configs supported - 配置文件在 `projects/app/data/config.json`
- Model configurations in `packages/service/core/ai/config/` - 支持特定环境配置
\ No newline at end of file - 模型配置在 `packages/service/core/ai/config/`
## 代码规范
- 尽可能使用 type 进行类型声明,而不是 interface。
\ No newline at end of file
...@@ -21,5 +21,5 @@ ifeq ($(proxy), taobao) ...@@ -21,5 +21,5 @@ ifeq ($(proxy), taobao)
else ifeq ($(proxy), clash) else ifeq ($(proxy), clash)
docker build -f $(filePath) -t $(image) . --network host --build-arg HTTP_PROXY=http://127.0.0.1:7890 --build-arg HTTPS_PROXY=http://127.0.0.1:7890 docker build -f $(filePath) -t $(image) . --network host --build-arg HTTP_PROXY=http://127.0.0.1:7890 --build-arg HTTPS_PROXY=http://127.0.0.1:7890
else else
docker build -f $(filePath) -t $(image) . docker build --progress=plain -f $(filePath) -t $(image) .
endif endif
\ No newline at end of file
...@@ -3,18 +3,47 @@ title: 'V4.13.2(进行中)' ...@@ -3,18 +3,47 @@ title: 'V4.13.2(进行中)'
description: 'FastGPT V4.13.2 更新说明' description: 'FastGPT V4.13.2 更新说明'
--- ---
# 更新指南
## 增加 FastGPT/FastGPT-pro 环境变量
```
S3_PUBLIC_BUCKET=S3公开桶名称(公开读私有写)
```
## 🚀 新增内容 ## 🚀 新增内容
1. HTTP 工具集支持手动创建模式。
2. 项目 OpenAPI 框架引入。
3. APIKey 有效性检测接口。
4. 导出对话日志,末尾跟随当前版本全局变量。
## ⚙️ 优化 ## ⚙️ 优化
1. 非管理员无法看到团队审计日志。 1. 非管理员无法看到团队审计日志。
2. 引入 S3 用于存储应用头像。
3. 工作流画布性能。
## 🐛 修复 ## 🐛 修复
1. LLM 模型默认支持图片,导致请求错误。 1. LLM 模型默认支持图片,导致请求错误。
2. Mongo 多副本切换时候,watch 未重新触发。
3. 文本分块,所有策略用完后,未处理 LastText 数据。
4. 变量输入框,number=0 时,无法通过校验。
5. 工作流复杂循环并行判断异常。
## 🔨 插件更新 ## 🔨 插件更新
1. Perplexity search 工具。 1. 新增:Perplexity search 工具。
2. 新增:Base64转文件工具。
3. 新增:MiniMax TTS 文件生成工具。
4. 新增:Openrouter nano banana 绘图工具。
5. 新增:Redis 缓存操作工具。
6. 新增:Tavily search 工具。
7. 新增:硅基流动 qwen-image 和 qwen-image-edit 工具。
8. 新增:飞书多维表格操作套件。
9. 新增:Youtube 字幕提取。
10. 新增:阿里百炼 qwen image edit。
11. 新增:Markdown 转 PPT 工具。
12. 新增:whisper 语音转文字工具。
13. 系统工具支持配置是否需要在 Worker 中运行。
...@@ -113,7 +113,7 @@ ...@@ -113,7 +113,7 @@
"document/content/docs/upgrading/4-12/4124.mdx": "2025-09-17T22:29:56+08:00", "document/content/docs/upgrading/4-12/4124.mdx": "2025-09-17T22:29:56+08:00",
"document/content/docs/upgrading/4-13/4130.mdx": "2025-09-30T16:00:10+08:00", "document/content/docs/upgrading/4-13/4130.mdx": "2025-09-30T16:00:10+08:00",
"document/content/docs/upgrading/4-13/4131.mdx": "2025-09-30T15:47:06+08:00", "document/content/docs/upgrading/4-13/4131.mdx": "2025-09-30T15:47:06+08:00",
"document/content/docs/upgrading/4-13/4132.mdx": "2025-10-09T15:10:19+08:00", "document/content/docs/upgrading/4-13/4132.mdx": "2025-10-17T21:40:12+08:00",
"document/content/docs/upgrading/4-8/40.mdx": "2025-08-02T19:38:37+08:00", "document/content/docs/upgrading/4-8/40.mdx": "2025-08-02T19:38:37+08:00",
"document/content/docs/upgrading/4-8/41.mdx": "2025-08-02T19:38:37+08:00", "document/content/docs/upgrading/4-8/41.mdx": "2025-08-02T19:38:37+08:00",
"document/content/docs/upgrading/4-8/42.mdx": "2025-08-02T19:38:37+08:00", "document/content/docs/upgrading/4-8/42.mdx": "2025-08-02T19:38:37+08:00",
......
...@@ -4,9 +4,6 @@ export type preUploadImgProps = OutLinkChatAuthProps & { ...@@ -4,9 +4,6 @@ export type preUploadImgProps = OutLinkChatAuthProps & {
// expiredTime?: Date; // expiredTime?: Date;
metadata?: Record<string, any>; metadata?: Record<string, any>;
}; };
export type UploadImgProps = preUploadImgProps & {
base64Img: string;
};
export type UrlFetchParams = { export type UrlFetchParams = {
urlList: string[]; urlList: string[];
......
export type S3TtlSchemaType = {
_id: string;
bucketName: string;
minioKey: string;
expiredTime: Date;
};
...@@ -295,17 +295,21 @@ const commonSplit = (props: SplitProps): SplitResponse => { ...@@ -295,17 +295,21 @@ const commonSplit = (props: SplitProps): SplitResponse => {
const isMarkdownStep = checkIsMarkdownSplit(step); const isMarkdownStep = checkIsMarkdownSplit(step);
const isCustomStep = checkIsCustomStep(step); const isCustomStep = checkIsCustomStep(step);
const forbidConcat = isCustomStep; // forbid=true时候,lastText肯定为空 const forbidConcat = isCustomStep; // forbid=true时候,lastText肯定为空
const textLength = getTextValidLength(text);
// Over step // Over step
if (step >= stepReges.length) { if (step >= stepReges.length) {
if (textLength < maxSize) { // Merge lastText with current text to prevent data loss
return [text]; const combinedText = lastText + text;
const combinedLength = getTextValidLength(combinedText);
if (combinedLength < maxSize) {
return [combinedText];
} }
// use slice-chunkSize to split text // use slice-chunkSize to split text
// Note: Use combinedText.length for slicing, not combinedLength
const chunks: string[] = []; const chunks: string[] = [];
for (let i = 0; i < textLength; i += chunkSize - overlapLen) { for (let i = 0; i < combinedText.length; i += chunkSize - overlapLen) {
chunks.push(text.slice(i, i + chunkSize)); chunks.push(combinedText.slice(i, i + chunkSize));
} }
return chunks; return chunks;
} }
......
export const getTextValidLength = (chunk: string) => { export const getTextValidLength = (chunk: string) => {
return chunk.replaceAll(/[\s\n]/g, '').length; return chunk.replaceAll(/[\s\n]/g, '').length;
}; };
export const isObjectId = (str: string) => {
return /^[0-9a-fA-F]{24}$/.test(str);
};
import { z } from 'zod';
export const ObjectIdSchema = z
.string()
.regex(/^[0-9a-fA-F]{24}$/)
.meta({ example: '68ee0bd23d17260b7829b137', description: 'ObjectId' });
...@@ -13,9 +13,9 @@ import { i18nT } from '../../../../web/i18n/utils'; ...@@ -13,9 +13,9 @@ import { i18nT } from '../../../../web/i18n/utils';
export const getHTTPToolSetRuntimeNode = ({ export const getHTTPToolSetRuntimeNode = ({
name, name,
avatar, avatar,
baseUrl = '', baseUrl,
customHeaders = '', customHeaders,
apiSchemaStr = '', apiSchemaStr,
toolList = [], toolList = [],
headerSecret headerSecret
}: { }: {
...@@ -34,12 +34,11 @@ export const getHTTPToolSetRuntimeNode = ({ ...@@ -34,12 +34,11 @@ export const getHTTPToolSetRuntimeNode = ({
intro: 'HTTP Tools', intro: 'HTTP Tools',
toolConfig: { toolConfig: {
httpToolSet: { httpToolSet: {
baseUrl,
toolList, toolList,
headerSecret, ...(baseUrl !== undefined && { baseUrl }),
customHeaders, ...(apiSchemaStr !== undefined && { apiSchemaStr }),
apiSchemaStr, ...(customHeaders !== undefined && { customHeaders }),
toolId: '' ...(headerSecret !== undefined && { headerSecret })
} }
}, },
inputs: [], inputs: [],
......
...@@ -5,6 +5,7 @@ import SwaggerParser from '@apidevtools/swagger-parser'; ...@@ -5,6 +5,7 @@ import SwaggerParser from '@apidevtools/swagger-parser';
import yaml from 'js-yaml'; import yaml from 'js-yaml';
import type { OpenAPIV3 } from 'openapi-types'; import type { OpenAPIV3 } from 'openapi-types';
import type { OpenApiJsonSchema } from './httpTools/type'; import type { OpenApiJsonSchema } from './httpTools/type';
import { i18nT } from '../../../web/i18n/utils';
type SchemaInputValueType = 'string' | 'number' | 'integer' | 'boolean' | 'array' | 'object'; type SchemaInputValueType = 'string' | 'number' | 'integer' | 'boolean' | 'array' | 'object';
export type JsonSchemaPropertiesItemType = { export type JsonSchemaPropertiesItemType = {
...@@ -180,7 +181,7 @@ export const str2OpenApiSchema = async (yamlStr = ''): Promise<OpenApiJsonSchema ...@@ -180,7 +181,7 @@ export const str2OpenApiSchema = async (yamlStr = ''): Promise<OpenApiJsonSchema
.filter(Boolean) as OpenApiJsonSchema['pathData']; .filter(Boolean) as OpenApiJsonSchema['pathData'];
return { pathData, serverPath }; return { pathData, serverPath };
} catch (err) { } catch (err) {
throw new Error('Invalid Schema'); return Promise.reject(i18nT('common:plugin.Invalid Schema'));
} }
}; };
export const getSchemaValueType = (schema: { type: string; items?: { type: string } }) => { export const getSchemaValueType = (schema: { type: string; items?: { type: string } }) => {
......
...@@ -2,6 +2,7 @@ import type { FlowNodeTemplateType, StoreNodeItemType } from '../workflow/type/n ...@@ -2,6 +2,7 @@ import type { FlowNodeTemplateType, StoreNodeItemType } from '../workflow/type/n
import type { AppTypeEnum } from './constants'; import type { AppTypeEnum } from './constants';
import { PermissionTypeEnum } from '../../support/permission/constant'; import { PermissionTypeEnum } from '../../support/permission/constant';
import type { import type {
ContentTypes,
NodeInputKeyEnum, NodeInputKeyEnum,
VariableInputEnum, VariableInputEnum,
WorkflowIOValueTypeEnum WorkflowIOValueTypeEnum
...@@ -127,6 +128,16 @@ export type HttpToolConfigType = { ...@@ -127,6 +128,16 @@ export type HttpToolConfigType = {
outputSchema: JSONSchemaOutputType; outputSchema: JSONSchemaOutputType;
path: string; path: string;
method: string; method: string;
// manual
staticParams?: Array<{ key: string; value: string }>;
staticHeaders?: Array<{ key: string; value: string }>;
staticBody?: {
type: ContentTypes;
content?: string;
formData?: Array<{ key: string; value: string }>;
};
headerSecret?: StoreSecretValueType;
}; };
/* app chat config type */ /* app chat config type */
......
export type ChatFavouriteAppSchema = {
_id: string;
teamId: string;
appId: string;
favouriteTags: string[]; // tag id list
order: number;
};
export type ChatFavouriteAppUpdateParams = {
appId: string;
order: number;
};
export type ChatFavouriteApp = ChatFavouriteAppSchema & {
name: string;
avatar: string;
intro: string;
};
import { z } from 'zod';
import { ObjectIdSchema } from '../../../common/type/mongo';
export const ChatFavouriteTagSchema = z.object({
id: z.string().meta({ example: 'ptqn6v4I', description: '精选应用标签 ID' }),
name: z.string().meta({ example: '效率', description: '精选应用标签名称' })
});
export type ChatFavouriteTagType = z.infer<typeof ChatFavouriteTagSchema>;
export const ChatFavouriteAppModelSchema = z.object({
_id: ObjectIdSchema,
teamId: ObjectIdSchema,
appId: ObjectIdSchema,
favouriteTags: z
.array(z.string())
.meta({ example: ['ptqn6v4I', 'jHLWiqff'], description: '精选应用标签' }),
order: z.number().meta({ example: 1, description: '排序' })
});
export type ChatFavouriteAppModelType = z.infer<typeof ChatFavouriteAppModelSchema>;
export const ChatFavouriteAppSchema = z.object({
...ChatFavouriteAppModelSchema.shape,
name: z.string().meta({ example: 'Jina 网页阅读', description: '精选应用名称' }),
intro: z.string().optional().meta({ example: '', description: '精选应用简介' }),
avatar: z.string().optional().meta({
example: '/api/system/img/avatar/68ad85a7463006c963799a05/79183cf9face95d336816f492409ed29',
description: '精选应用头像'
})
});
export type ChatFavouriteAppType = z.infer<typeof ChatFavouriteAppSchema>;
export type ChatSettingSchema = {
_id: string;
appId: string;
teamId: string;
slogan: string;
dialogTips: string;
enableHome: boolean;
homeTabTitle: string;
wideLogoUrl?: string;
squareLogoUrl?: string;
selectedTools: {
pluginId: string;
inputs?: Record<`${NodeInputKeyEnum}` | string, any>;
}[];
quickAppIds: string[];
favouriteTags: {
id: string;
name: string;
}[];
};
export type ChatSettingUpdateParams = Partial<Omit<ChatSettingSchema, '_id' | 'appId' | 'teamId'>>;
export type QuickAppType = { _id: string; name: string; avatar: string };
export type ChatFavouriteTagType = ChatSettingSchema['favouriteTags'][number];
export type SelectedToolType = ChatSettingSchema['selectedTools'][number] & {
name: string;
avatar: string;
};
export type ChatSettingReturnType =
| (Omit<ChatSettingSchema, 'quickAppIds' | 'selectedTools'> & {
quickAppList: QuickAppType[];
selectedTools: SelectedToolType[];
})
| undefined;
import { ObjectIdSchema } from '../../../common/type/mongo';
import { z } from 'zod';
import { ChatFavouriteTagSchema } from '../favouriteApp/type';
export const ChatSelectedToolSchema = z.object({
pluginId: ObjectIdSchema,
inputs: z.record(z.string(), z.any()).meta({ example: null, description: '工具输入参数' }),
name: z.string().meta({ example: '测试应用', description: '工具名称' }),
avatar: z.string().meta({ example: '测试应用', description: '工具头像' })
});
export type ChatSelectedToolType = z.infer<typeof ChatSelectedToolSchema>;
export const ChatQuickAppSchema = z.object({
_id: ObjectIdSchema,
name: z.string().meta({ example: '测试应用', description: '快捷应用名称' }),
avatar: z.string().meta({ example: '测试应用', description: '快捷应用头像' })
});
export type ChatQuickAppType = z.infer<typeof ChatQuickAppSchema>;
export const ChatSettingModelSchema = z.object({
_id: ObjectIdSchema,
appId: ObjectIdSchema,
teamId: ObjectIdSchema,
slogan: z
.string()
.optional()
.meta({ example: '你好👋,我是 FastGPT ! 请问有什么可以帮你?', description: 'Slogan' }),
dialogTips: z
.string()
.optional()
.meta({ example: '你可以问我任何问题', description: '对话提示' }),
enableHome: z.boolean().optional().meta({ example: true, description: '是否启用首页' }),
homeTabTitle: z.string().optional().meta({ example: 'FastGPT', description: '首页标签' }),
wideLogoUrl: z.string().optional().meta({
example: '/api/system/img/avatar/68ad85a7463006c963799a05/79183cf9face95d336816f492409ed29',
description: '宽 LOGO'
}),
squareLogoUrl: z.string().optional().meta({
example: '/api/system/img/avatar/68ad85a7463006c963799a05/79183cf9face95d336816f492409ed29',
description: '方 LOGO'
}),
quickAppIds: z
.array(ObjectIdSchema)
.meta({ example: ['68ad85a7463006c963799a05'], description: '快捷应用 ID 列表' }),
selectedTools: z.array(ChatSelectedToolSchema.pick({ pluginId: true, inputs: true })).meta({
example: [{ pluginId: '68ad85a7463006c963799a05', inputs: {} }],
description: '已选工具列表'
}),
favouriteTags: z.array(ChatFavouriteTagSchema).meta({
example: [
{ id: 'ptqn6v4I', name: '效率' },
{ id: 'jHLWiqff', name: '学习' }
],
description: '精选应用标签列表'
})
});
export type ChatSettingModelType = z.infer<typeof ChatSettingModelSchema>;
export const ChatSettingSchema = z.object({
...ChatSettingModelSchema.omit({ quickAppIds: true }).shape,
quickAppList: z.array(ChatQuickAppSchema).meta({
example: [{ _id: '68ad85a7463006c963799a05', name: '测试应用', avatar: '测试应用' }],
description: '快捷应用列表'
}),
selectedTools: z.array(ChatSelectedToolSchema).meta({
example: [
{
pluginId: '68ad85a7463006c963799a05',
inputs: {},
name: '获取当前应用',
avatar: '/icon/logo.svg'
}
],
description: '已选工具列表'
})
});
export type ChatSettingType = z.infer<typeof ChatSettingSchema>;
...@@ -477,6 +477,19 @@ export enum ContentTypes { ...@@ -477,6 +477,19 @@ export enum ContentTypes {
raw = 'raw-text' raw = 'raw-text'
} }
export const contentTypeMap = {
[ContentTypes.none]: '',
[ContentTypes.formData]: '',
[ContentTypes.xWwwFormUrlencoded]: 'application/x-www-form-urlencoded',
[ContentTypes.json]: 'application/json',
[ContentTypes.xml]: 'application/xml',
[ContentTypes.raw]: 'text/plain'
};
// http request methods
export const HTTP_METHODS = ['GET', 'POST', 'PUT', 'DELETE', 'PATCH'] as const;
export type HttpMethod = (typeof HTTP_METHODS)[number];
export const ArrayTypeMap: Record<WorkflowIOValueTypeEnum, WorkflowIOValueTypeEnum> = { export const ArrayTypeMap: Record<WorkflowIOValueTypeEnum, WorkflowIOValueTypeEnum> = {
[WorkflowIOValueTypeEnum.string]: WorkflowIOValueTypeEnum.arrayString, [WorkflowIOValueTypeEnum.string]: WorkflowIOValueTypeEnum.arrayString,
[WorkflowIOValueTypeEnum.number]: WorkflowIOValueTypeEnum.arrayNumber, [WorkflowIOValueTypeEnum.number]: WorkflowIOValueTypeEnum.arrayNumber,
......
...@@ -21,17 +21,6 @@ import { isValidReferenceValueFormat } from '../utils'; ...@@ -21,17 +21,6 @@ import { isValidReferenceValueFormat } from '../utils';
import type { RuntimeEdgeItemType, RuntimeNodeItemType } from './type'; import type { RuntimeEdgeItemType, RuntimeNodeItemType } from './type';
import { isSecretValue } from '../../../common/secret/utils'; import { isSecretValue } from '../../../common/secret/utils';
export const checkIsBranchNode = (node: RuntimeNodeItemType) => {
if (node.catchError) return true;
const map: Record<any, boolean> = {
[FlowNodeTypeEnum.classifyQuestion]: true,
[FlowNodeTypeEnum.userSelect]: true,
[FlowNodeTypeEnum.ifElseNode]: true
};
return !!map[node.flowNodeType];
};
export const extractDeepestInteractive = ( export const extractDeepestInteractive = (
interactive: WorkflowInteractiveResponseType interactive: WorkflowInteractiveResponseType
): WorkflowInteractiveResponseType => { ): WorkflowInteractiveResponseType => {
...@@ -306,45 +295,50 @@ export const checkNodeRunStatus = ({ ...@@ -306,45 +295,50 @@ export const checkNodeRunStatus = ({
}) => { }) => {
const filterRuntimeEdges = filterWorkflowEdges(runtimeEdges); const filterRuntimeEdges = filterWorkflowEdges(runtimeEdges);
const isStartNode = (nodeType: string) => {
const map: Record<any, boolean> = {
[FlowNodeTypeEnum.workflowStart]: true,
[FlowNodeTypeEnum.pluginInput]: true
};
return !!map[nodeType];
};
const splitNodeEdges = (targetNode: RuntimeNodeItemType) => { const splitNodeEdges = (targetNode: RuntimeNodeItemType) => {
const commonEdges: RuntimeEdgeItemType[] = []; const commonEdges: RuntimeEdgeItemType[] = [];
const recursiveEdgeGroupsMap = new Map<string, RuntimeEdgeItemType[]>(); const recursiveEdgeGroupsMap = new Map<string, RuntimeEdgeItemType[]>();
const getEdgeLastBranchHandle = ({ const sourceEdges = filterRuntimeEdges.filter((item) => item.target === targetNode.nodeId);
startEdge,
targetNodeId sourceEdges.forEach((sourceEdge) => {
}: {
startEdge: RuntimeEdgeItemType;
targetNodeId: string;
}): string | '' | undefined => {
const stack: Array<{ const stack: Array<{
edge: RuntimeEdgeItemType; edge: RuntimeEdgeItemType;
visited: Set<string>; visited: Set<string>;
lasestBranchHandle?: string;
}> = [ }> = [
{ {
edge: startEdge, edge: sourceEdge,
visited: new Set([targetNodeId]) visited: new Set([targetNode.nodeId])
} }
]; ];
const MAX_DEPTH = 3000; const MAX_DEPTH = 3000;
let iterations = 0; let iterations = 0;
while (stack.length > 0 && iterations < MAX_DEPTH) { while (stack.length > 0 && iterations < MAX_DEPTH) {
iterations++; iterations++;
const { edge, visited, lasestBranchHandle } = stack.pop()!; const { edge, visited } = stack.pop()!;
// Circle // Start node
const sourceNode = nodesMap.get(edge.source);
if (!sourceNode) continue;
if (isStartNode(sourceNode.flowNodeType)) {
commonEdges.push(sourceEdge);
continue;
}
// Circle detected
if (edge.source === targetNode.nodeId) { if (edge.source === targetNode.nodeId) {
// 检查自身是否为分支节点 recursiveEdgeGroupsMap.set(edge.target, [
const node = nodesMap.get(edge.source); ...(recursiveEdgeGroupsMap.get(edge.target) || []),
if (!node) return ''; sourceEdge
const isBranch = checkIsBranchNode(node); ]);
if (isBranch) return edge.sourceHandle; continue;
// 检测到环,并且环中包含当前节点. 空字符代表是一个无分支循环,属于死循环,则忽略这个边。
return lasestBranchHandle ?? '';
} }
if (visited.has(edge.source)) { if (visited.has(edge.source)) {
...@@ -357,42 +351,12 @@ export const checkNodeRunStatus = ({ ...@@ -357,42 +351,12 @@ export const checkNodeRunStatus = ({
// 查找目标节点的 source edges 并加入栈中 // 查找目标节点的 source edges 并加入栈中
const nextEdges = filterRuntimeEdges.filter((item) => item.target === edge.source); const nextEdges = filterRuntimeEdges.filter((item) => item.target === edge.source);
for (const nextEdge of nextEdges) { for (const nextEdge of nextEdges) {
const node = nodesMap.get(nextEdge.target);
if (!node) continue;
const isBranch = checkIsBranchNode(node);
stack.push({ stack.push({
edge: nextEdge, edge: nextEdge,
visited: newVisited, visited: newVisited
lasestBranchHandle: isBranch ? edge.sourceHandle : lasestBranchHandle
}); });
} }
} }
return;
};
const sourceEdges = filterRuntimeEdges.filter((item) => item.target === targetNode.nodeId);
sourceEdges.forEach((edge) => {
const lastBranchHandle = getEdgeLastBranchHandle({
startEdge: edge,
targetNodeId: targetNode.nodeId
});
// 无效的循环,这条边则忽略
if (lastBranchHandle === '') return;
// 有效循环,则加入递归组
if (lastBranchHandle) {
recursiveEdgeGroupsMap.set(lastBranchHandle, [
...(recursiveEdgeGroupsMap.get(lastBranchHandle) || []),
edge
]);
}
// 无循环的连线,则加入普通组
else {
commonEdges.push(edge);
}
}); });
return { commonEdges, recursiveEdgeGroups: Array.from(recursiveEdgeGroupsMap.values()) }; return { commonEdges, recursiveEdgeGroups: Array.from(recursiveEdgeGroupsMap.values()) };
......
...@@ -52,11 +52,10 @@ export type NodeToolConfigType = { ...@@ -52,11 +52,10 @@ export type NodeToolConfigType = {
}[]; }[];
}; };
httpToolSet?: { httpToolSet?: {
toolId: string;
baseUrl: string;
toolList: HttpToolConfigType[]; toolList: HttpToolConfigType[];
apiSchemaStr: string; baseUrl?: string;
customHeaders: string; apiSchemaStr?: string;
customHeaders?: string;
headerSecret?: StoreSecretValueType; headerSecret?: StoreSecretValueType;
}; };
httpTool?: { httpTool?: {
......
...@@ -69,8 +69,8 @@ export const checkInputIsReference = (input: FlowNodeInputItemType) => { ...@@ -69,8 +69,8 @@ export const checkInputIsReference = (input: FlowNodeInputItemType) => {
}; };
/* node */ /* node */
export const getGuideModule = (modules: StoreNodeItemType[]) => export const getGuideModule = (nodes: StoreNodeItemType[]) =>
modules.find( nodes.find(
(item) => (item) =>
item.flowNodeType === FlowNodeTypeEnum.systemConfig || item.flowNodeType === FlowNodeTypeEnum.systemConfig ||
// @ts-ignore (adapt v1) // @ts-ignore (adapt v1)
......
import { z } from 'zod';
import { ObjectIdSchema } from '../../../../common/type/mongo';
export const GetChatFavouriteListParamsSchema = z.object({
name: z.string().optional().meta({ example: '测试应用', description: '精选应用名称' }),
tag: z.string().optional().meta({ example: '效率', description: '精选应用标签' })
});
export type GetChatFavouriteListParamsType = z.infer<typeof GetChatFavouriteListParamsSchema>;
export const UpdateFavouriteAppTagsParamsSchema = z.object({
id: ObjectIdSchema.meta({ example: '68ad85a7463006c963799a05', description: '精选应用 ID' }),
tags: z.array(z.string()).meta({ example: ['效率', '工具'], description: '精选应用标签' })
});
export const UpdateFavouriteAppParamsSchema = z.object({
appId: ObjectIdSchema.meta({ example: '68ad85a7463006c963799a05', description: '精选应用 ID' }),
order: z.number().meta({ example: 1, description: '排序' })
});
export type UpdateFavouriteAppParamsType = z.infer<typeof UpdateFavouriteAppParamsSchema>;
import { z } from 'zod';
import type { OpenAPIPath } from '../../../type';
import { ChatFavouriteAppSchema } from '../../../../core/chat/favouriteApp/type';
import {
GetChatFavouriteListParamsSchema,
UpdateFavouriteAppParamsSchema,
UpdateFavouriteAppTagsParamsSchema
} from './api';
import { ObjectIdSchema } from '../../../../common/type/mongo';
export const ChatFavouriteAppPath: OpenAPIPath = {
'/proApi/core/chat/setting/favourite/list': {
get: {
summary: '获取精选应用列表',
description: '获取团队配置的精选应用列表,支持按名称和标签筛选',
tags: ['对话页配置'],
requestParams: {
query: GetChatFavouriteListParamsSchema
},
responses: {
200: {
description: '成功返回精选应用列表',
content: {
'application/json': {
schema: z.array(ChatFavouriteAppSchema)
}
}
}
}
}
},
'/proApi/core/chat/setting/favourite/update': {
post: {
summary: '更新精选应用',
description: '批量创建或更新精选应用配置,包括应用 ID、标签和排序信息',
tags: ['对话页配置'],
requestBody: {
content: {
'application/json': {
schema: z.array(UpdateFavouriteAppParamsSchema)
}
}
},
responses: {
200: {
description: '成功更新精选应用',
content: {
'application/json': {
schema: z.array(ChatFavouriteAppSchema)
}
}
}
}
}
},
'/proApi/core/chat/setting/favourite/order': {
put: {
summary: '更新精选应用排序',
description: '批量更新精选应用的显示顺序',
tags: ['对话页配置'],
requestBody: {
content: {
'application/json': {
schema: z.array(
z.object({
id: ObjectIdSchema.meta({
example: '68ad85a7463006c963799a05',
description: '精选应用 ID'
}),
order: z.number().meta({ example: 1, description: '排序' })
})
)
}
}
},
responses: {
200: {
description: '成功更新精选应用排序',
content: {
'application/json': {
schema: z.null()
}
}
}
}
}
},
'/proApi/core/chat/setting/favourite/tags': {
put: {
summary: '更新精选应用标签',
description: '批量更新精选应用的标签分类',
tags: ['对话页配置'],
requestBody: {
content: {
'application/json': {
schema: z.array(UpdateFavouriteAppTagsParamsSchema)
}
}
},
responses: {
200: {
description: '成功更新精选应用标签',
content: {
'application/json': {
schema: z.null()
}
}
}
}
}
},
'/proApi/core/chat/setting/favourite/delete': {
delete: {
summary: '删除精选应用',
description: '根据 ID 删除指定的精选应用配置',
tags: ['对话页配置'],
requestParams: {
query: z.object({
id: ObjectIdSchema
})
},
responses: {
200: {
description: '成功删除精选应用',
content: {
'application/json': {
schema: z.null()
}
}
}
}
}
}
};
import { ChatSettingPath } from './setting';
import { ChatFavouriteAppPath } from './favourite/index';
export const ChatPath = {
...ChatSettingPath,
...ChatFavouriteAppPath
};
import type { OpenAPIPath } from '../../../type';
import { ChatSettingSchema, ChatSettingModelSchema } from '../../../../core/chat/setting/type';
export const ChatSettingPath: OpenAPIPath = {
'/proApi/core/chat/setting/detail': {
get: {
summary: '获取对话页设置',
description:
'获取当前团队的对话页设置,包括 slogan、对话提示、Logo、快捷应用、已选工具和精选应用标签等配置信息',
tags: ['对话页配置'],
responses: {
200: {
description: '成功返回对话页设置信息',
content: {
'application/json': {
schema: ChatSettingSchema
}
}
}
}
}
},
'/proApi/core/chat/setting/update': {
post: {
summary: '更新对话页设置',
description:
'更新团队的对话页设置配置,包括 slogan、对话提示、Logo、快捷应用、已选工具和精选应用标签等信息',
tags: ['对话页配置'],
requestBody: {
content: {
'application/json': {
schema: ChatSettingModelSchema.partial()
}
}
},
responses: {
200: {
description: '成功更新对话页设置',
content: {
'application/json': {
schema: ChatSettingSchema
}
}
}
}
}
}
};
import { createDocument } from 'zod-openapi';
import { ChatPath } from './core/chat';
import { ApiKeyPath } from './support/openapi';
export const openAPIDocument = createDocument({
openapi: '3.1.0',
info: {
title: 'FastGPT API',
version: '0.1.0',
description: 'FastGPT API 文档'
},
paths: {
...ChatPath,
...ApiKeyPath
},
servers: [{ url: '/api' }]
});
import { z } from 'zod';
export const ApiKeyHealthParamsSchema = z.object({
apiKey: z.string().nonempty()
});
export type ApiKeyHealthParamsType = z.infer<typeof ApiKeyHealthParamsSchema>;
export const ApiKeyHealthResponseSchema = z.object({
appId: z.string().optional().meta({
description: '如果有关联的应用,会返回应用ID'
})
});
export type ApiKeyHealthResponseType = z.infer<typeof ApiKeyHealthResponseSchema>;
import { z } from 'zod';
import { formatSuccessResponse, getErrorResponse, type OpenAPIPath } from '../../type';
import { ApiKeyHealthParamsSchema, ApiKeyHealthResponseSchema } from './api';
export const ApiKeyPath: OpenAPIPath = {
'/support/openapi/health': {
get: {
summary: '检查 API Key 是否健康',
tags: ['APIKey'],
requestParams: {
query: ApiKeyHealthParamsSchema
},
responses: {
200: {
description: 'API Key 可用',
content: {
'application/json': {
schema: ApiKeyHealthResponseSchema
}
}
},
500: {
description: 'ApiKey错误',
content: {
'application/json': {
schema: z.object({ message: z.literal('APIKey invalid') })
}
}
}
}
}
}
};
import { z } from 'zod';
import type { createDocument } from 'zod-openapi';
export type OpenAPIPath = Parameters<typeof createDocument>[0]['paths'];
export const getErrorResponse = ({
code = 500,
statusText = 'error',
message = ''
}: {
code?: number;
statusText?: string;
message?: string;
}) => {
return z.object({
code: z.literal(code),
statusText: z.literal(statusText),
message: z.literal(message),
data: z.null().optional().default(null)
});
};
export const formatSuccessResponse = <T>(data: T) => {
return z.object({
code: z.literal(200),
statusText: z.string().optional().default(''),
message: z.string().optional().default(''),
data
});
};
...@@ -17,7 +17,9 @@ ...@@ -17,7 +17,9 @@
"openai": "4.61.0", "openai": "4.61.0",
"openapi-types": "^12.1.3", "openapi-types": "^12.1.3",
"timezones-list": "^3.0.2", "timezones-list": "^3.0.2",
"lodash": "^4.17.21" "lodash": "^4.17.21",
"zod": "^4.1.12",
"zod-openapi": "^5.4.3"
}, },
"devDependencies": { "devDependencies": {
"@types/lodash": "^4.14.191", "@types/lodash": "^4.14.191",
......
import { type UploadImgProps } from '@fastgpt/global/common/file/api'; import { type preUploadImgProps } from '@fastgpt/global/common/file/api';
import { imageBaseUrl } from '@fastgpt/global/common/file/image/constants'; import { imageBaseUrl } from '@fastgpt/global/common/file/image/constants';
import { MongoImage } from './schema'; import { MongoImage } from './schema';
import { type ClientSession, Types } from '../../../common/mongo'; import { type ClientSession, Types } from '../../../common/mongo';
...@@ -18,7 +18,8 @@ export async function uploadMongoImg({ ...@@ -18,7 +18,8 @@ export async function uploadMongoImg({
metadata, metadata,
shareId, shareId,
forever = false forever = false
}: UploadImgProps & { }: preUploadImgProps & {
base64Img: string;
teamId: string; teamId: string;
forever?: Boolean; forever?: Boolean;
}) { }) {
...@@ -109,23 +110,6 @@ const getIdFromPath = (path?: string) => { ...@@ -109,23 +110,6 @@ const getIdFromPath = (path?: string) => {
return id; return id;
}; };
// 删除旧的头像,新的头像去除过期时间
export const refreshSourceAvatar = async (
path?: string,
oldPath?: string,
session?: ClientSession
) => {
const newId = getIdFromPath(path);
const oldId = getIdFromPath(oldPath);
if (!newId || newId === oldId) return;
await MongoImage.updateOne({ _id: newId }, { $unset: { expiredTime: 1 } }, { session });
if (oldId) {
await MongoImage.deleteOne({ _id: oldId }, { session });
}
};
export const removeImageByPath = (path?: string, session?: ClientSession) => { export const removeImageByPath = (path?: string, session?: ClientSession) => {
if (!path) return; if (!path) return;
......
...@@ -123,12 +123,19 @@ export const getMongoLogModel = <T>(name: string, schema: mongoose.Schema) => { ...@@ -123,12 +123,19 @@ export const getMongoLogModel = <T>(name: string, schema: mongoose.Schema) => {
}; };
const syncMongoIndex = async (model: Model<any>) => { const syncMongoIndex = async (model: Model<any>) => {
if (process.env.SYNC_INDEX !== '0' && process.env.NODE_ENV !== 'test') { if (
try { process.env.NODE_ENV === 'test' ||
model.syncIndexes({ background: true }); process.env.SYNC_INDEX === '0' ||
} catch (error) { process.env.NEXT_PHASE === 'phase-production-build' ||
addLog.error('Create index error', error); !MONGO_URL
} ) {
return;
}
try {
await model.syncIndexes({ background: true });
} catch (error) {
addLog.error('Create index error', error);
} }
}; };
......
...@@ -7,7 +7,13 @@ const maxConnecting = Math.max(30, Number(process.env.DB_MAX_LINK || 20)); ...@@ -7,7 +7,13 @@ const maxConnecting = Math.max(30, Number(process.env.DB_MAX_LINK || 20));
/** /**
* connect MongoDB and init data * connect MongoDB and init data
*/ */
export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose> { export async function connectMongo(props: {
db: Mongoose;
url: string;
connectedCb?: () => void;
}): Promise<Mongoose> {
const { db, url, connectedCb } = props;
/* Connecting, connected will return */ /* Connecting, connected will return */
if (db.connection.readyState !== 0) { if (db.connection.readyState !== 0) {
return db; return db;
...@@ -31,7 +37,7 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose> ...@@ -31,7 +37,7 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose>
RemoveListeners(); RemoveListeners();
await db.disconnect(); await db.disconnect();
await delay(1000); await delay(1000);
await connectMongo(db, url); await connectMongo(props);
} }
} catch (error) {} } catch (error) {}
}); });
...@@ -42,7 +48,7 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose> ...@@ -42,7 +48,7 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose>
RemoveListeners(); RemoveListeners();
await db.disconnect(); await db.disconnect();
await delay(1000); await delay(1000);
await connectMongo(db, url); await connectMongo(props);
} }
} catch (error) {} } catch (error) {}
}); });
...@@ -60,9 +66,11 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose> ...@@ -60,9 +66,11 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose>
retryReads: true retryReads: true
}; };
db.connect(url, options); await db.connect(url, options);
console.log('mongo connected'); console.log('mongo connected');
connectedCb?.();
return db; return db;
} catch (error) { } catch (error) {
addLog.error('Mongo connect error', error); addLog.error('Mongo connect error', error);
...@@ -70,6 +78,6 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose> ...@@ -70,6 +78,6 @@ export async function connectMongo(db: Mongoose, url: string): Promise<Mongoose>
await db.disconnect(); await db.disconnect();
await delay(1000); await delay(1000);
return connectMongo(db, url); return connectMongo(props);
} }
} }
...@@ -12,6 +12,72 @@ export interface ResponseType<T = any> { ...@@ -12,6 +12,72 @@ export interface ResponseType<T = any> {
data: T; data: T;
} }
export interface ProcessedError {
code: number;
statusText: string;
message: string;
data?: any;
shouldClearCookie: boolean;
}
/**
* 通用错误处理函数,提取错误信息并分类记录日志
* @param params - 包含错误对象、URL和默认状态码的参数
* @returns 处理后的错误对象
*/
export function processError(params: {
error: any;
url?: string;
defaultCode?: number;
}): ProcessedError {
const { error, url, defaultCode = 500 } = params;
const errResponseKey = typeof error === 'string' ? error : error?.message;
// 1. 处理特定的业务错误(ERROR_RESPONSE)
if (ERROR_RESPONSE[errResponseKey]) {
const shouldClearCookie = errResponseKey === ERROR_ENUM.unAuthorization;
// 记录业务侧错误日志
addLog.info(`Api response error: ${url}`, ERROR_RESPONSE[errResponseKey]);
return {
code: ERROR_RESPONSE[errResponseKey].code || defaultCode,
statusText: ERROR_RESPONSE[errResponseKey].statusText || 'error',
message: ERROR_RESPONSE[errResponseKey].message,
data: ERROR_RESPONSE[errResponseKey].data,
shouldClearCookie
};
}
// 2. 提取通用错误消息
let msg = error?.response?.statusText || error?.message || '请求错误';
if (typeof error === 'string') {
msg = error;
} else if (proxyError[error?.code]) {
msg = '网络连接异常';
} else if (error?.response?.data?.error?.message) {
msg = error?.response?.data?.error?.message;
} else if (error?.error?.message) {
msg = error?.error?.message;
}
// 3. 根据错误类型记录不同级别的日志
if (error instanceof UserError) {
addLog.info(`Request error: ${url}, ${msg}`);
} else {
addLog.error(`System unexpected error: ${url}, ${msg}`, error);
}
// 4. 返回处理后的错误信息
return {
code: defaultCode,
statusText: 'error',
message: replaceSensitiveText(msg),
shouldClearCookie: false
};
}
export const jsonRes = <T = any>( export const jsonRes = <T = any>(
res: NextApiResponse, res: NextApiResponse,
props?: { props?: {
...@@ -24,53 +90,30 @@ export const jsonRes = <T = any>( ...@@ -24,53 +90,30 @@ export const jsonRes = <T = any>(
) => { ) => {
const { code = 200, message = '', data = null, error, url } = props || {}; const { code = 200, message = '', data = null, error, url } = props || {};
const errResponseKey = typeof error === 'string' ? error : error?.message; // 如果有错误,使用统一的错误处理逻辑
// Specified error if (error) {
if (ERROR_RESPONSE[errResponseKey]) { const processedError = processError({ error, url, defaultCode: code });
// login is expired
if (errResponseKey === ERROR_ENUM.unAuthorization) { // 如果需要清除 cookie
if (processedError.shouldClearCookie) {
clearCookie(res); clearCookie(res);
} }
// Bussiness Side Error res.status(500).json({
addLog.info(`Api response error: ${url}`, ERROR_RESPONSE[errResponseKey]); code: processedError.code,
statusText: processedError.statusText,
res.status(code); message: message || processedError.message,
data: processedError.data !== undefined ? processedError.data : null
if (message) { });
res.send(message);
} else {
res.json(ERROR_RESPONSE[errResponseKey]);
}
return; return;
} }
// another error // 成功响应
let msg = '';
if ((code < 200 || code >= 400) && !message) {
msg = error?.response?.statusText || error?.message || '请求错误';
if (typeof error === 'string') {
msg = error;
} else if (proxyError[error?.code]) {
msg = '网络连接异常';
} else if (error?.response?.data?.error?.message) {
msg = error?.response?.data?.error?.message;
} else if (error?.error?.message) {
msg = error?.error?.message;
}
if (error instanceof UserError) {
addLog.info(`Request error: ${url}, ${msg}`);
} else {
addLog.error(`System unexpected error: ${url}, ${msg}`, error);
}
}
res.status(code).json({ res.status(code).json({
code, code,
statusText: '', statusText: '',
message: replaceSensitiveText(message || msg), message: replaceSensitiveText(message),
data: data !== undefined ? data : null data: data !== undefined ? data : null
}); });
}; };
......
import { Client, type RemoveOptions, type CopyConditions, type LifecycleConfig } from 'minio';
import {
type ExtensionType,
type CreatePostPresignedUrlOptions,
type CreatePostPresignedUrlParams,
type CreatePostPresignedUrlResult,
type S3OptionsType
} from '../type';
import { defaultS3Options, Mimes } from '../constants';
import path from 'node:path';
import { MongoS3TTL } from '../schema';
import { UserError } from '@fastgpt/global/common/error/utils';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { addHours } from 'date-fns';
export class S3BaseBucket {
private _client: Client;
private _externalClient: Client | undefined;
/**
*
* @param bucketName the bucket you want to operate
* @param options the options for the s3 client
*/
constructor(
private readonly bucketName: string,
public options: Partial<S3OptionsType> = defaultS3Options
) {
options = { ...defaultS3Options, ...options };
this.options = options;
this._client = new Client(options as S3OptionsType);
if (this.options.externalBaseURL) {
const externalBaseURL = new URL(this.options.externalBaseURL);
const endpoint = externalBaseURL.hostname;
const useSSL = externalBaseURL.protocol === 'https:';
const externalPort = externalBaseURL.port
? parseInt(externalBaseURL.port)
: useSSL
? 443
: undefined; // https 默认 443,其他情况让 MinIO 客户端使用默认端口
this._externalClient = new Client({
useSSL: useSSL,
endPoint: endpoint,
port: externalPort,
accessKey: options.accessKey,
secretKey: options.secretKey,
transportAgent: options.transportAgent
});
}
const init = async () => {
if (!(await this.exist())) {
await this.client.makeBucket(this.bucketName);
}
await this.options.afterInit?.();
};
init();
}
get name(): string {
return this.bucketName;
}
protected get client(): Client {
return this._externalClient ?? this._client;
}
move(src: string, dst: string, options?: CopyConditions): Promise<void> {
const bucket = this.name;
this.client.copyObject(bucket, dst, `/${bucket}/${src}`, options);
return this.delete(src);
}
copy(src: string, dst: string, options?: CopyConditions): ReturnType<Client['copyObject']> {
return this.client.copyObject(this.name, src, dst, options);
}
exist(): Promise<boolean> {
return this.client.bucketExists(this.name);
}
delete(objectKey: string, options?: RemoveOptions): Promise<void> {
return this.client.removeObject(this.name, objectKey, options);
}
async createPostPresignedUrl(
params: CreatePostPresignedUrlParams,
options: CreatePostPresignedUrlOptions = {}
): Promise<CreatePostPresignedUrlResult> {
try {
const { expiredHours } = options;
const filename = params.filename;
const ext = path.extname(filename).toLowerCase() as ExtensionType;
const contentType = Mimes[ext] ?? 'application/octet-stream';
const maxFileSize = this.options.maxFileSize as number;
const key = (() => {
if ('rawKey' in params) return params.rawKey;
return `${params.source}/${params.teamId}/${getNanoid(6)}-${filename}`;
})();
const policy = this.client.newPostPolicy();
policy.setKey(key);
policy.setBucket(this.name);
policy.setContentType(contentType);
policy.setContentLengthRange(1, maxFileSize);
policy.setExpires(new Date(Date.now() + 10 * 60 * 1000));
policy.setUserMetaData({
'content-type': contentType,
'content-disposition': `attachment; filename="${encodeURIComponent(filename)}"`,
'origin-filename': encodeURIComponent(filename),
'upload-time': new Date().toISOString()
});
const { formData, postURL } = await this.client.presignedPostPolicy(policy);
if (expiredHours) {
await MongoS3TTL.create({
minioKey: key,
bucketName: this.name,
expiredTime: addHours(new Date(), expiredHours)
});
}
return {
url: postURL,
fields: formData
};
} catch (error) {
return Promise.reject(error);
}
}
}
import { S3BaseBucket } from './base';
import { S3Buckets } from '../constants';
import { type S3OptionsType } from '../type';
export class S3PrivateBucket extends S3BaseBucket {
constructor(options?: Partial<S3OptionsType>) {
super(S3Buckets.private, options);
}
}
import { S3BaseBucket } from './base';
import { S3Buckets } from '../constants';
import { type S3OptionsType } from '../type';
export class S3PublicBucket extends S3BaseBucket {
constructor(options?: Partial<S3OptionsType>) {
super(S3Buckets.public, {
...options,
afterInit: async () => {
const bucket = this.name;
const policy = JSON.stringify({
Version: '2012-10-17',
Statement: [
{
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${bucket}/*`
}
]
});
try {
await this.client.setBucketPolicy(bucket, policy);
} catch (error) {
// NOTE: maybe it was a cloud S3 that doesn't allow us to set the policy, so that cause the error,
// maybe we can ignore the error, or we have other plan to handle this.
console.error('Failed to set bucket policy:', error);
}
}
});
}
createPublicUrl(objectKey: string): string {
const protocol = this.options.useSSL ? 'https' : 'http';
const hostname = this.options.endPoint;
const port = this.options.port;
const bucket = this.name;
const url = new URL(`${protocol}://${hostname}:${port}/${bucket}/${objectKey}`);
if (this.options.externalBaseURL) {
const externalBaseURL = new URL(this.options.externalBaseURL);
url.port = externalBaseURL.port;
url.hostname = externalBaseURL.hostname;
url.protocol = externalBaseURL.protocol;
}
return url.toString();
}
}
import type { S3ServiceConfig } from './type';
export const defualtS3Config: Omit<S3ServiceConfig, 'bucket'> = {
endPoint: process.env.S3_ENDPOINT || 'localhost',
port: process.env.S3_PORT ? parseInt(process.env.S3_PORT) : 9000,
useSSL: process.env.S3_USE_SSL === 'true',
accessKey: process.env.S3_ACCESS_KEY || 'minioadmin',
secretKey: process.env.S3_SECRET_KEY || 'minioadmin',
externalBaseURL: process.env.S3_EXTERNAL_BASE_URL
};
export const mimeMap: Record<string, string> = { import type { S3PrivateBucket } from './buckets/private';
import type { S3PublicBucket } from './buckets/public';
import { HttpProxyAgent } from 'http-proxy-agent';
import { HttpsProxyAgent } from 'https-proxy-agent';
import type { ClientOptions } from 'minio';
export const Mimes = {
'.gif': 'image/gif',
'.png': 'image/png',
'.jpg': 'image/jpeg', '.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg', '.jpeg': 'image/jpeg',
'.png': 'image/png',
'.gif': 'image/gif',
'.webp': 'image/webp', '.webp': 'image/webp',
'.svg': 'image/svg+xml', '.svg': 'image/svg+xml',
'.pdf': 'application/pdf',
'.txt': 'text/plain',
'.json': 'application/json',
'.csv': 'text/csv', '.csv': 'text/csv',
'.txt': 'text/plain',
'.pdf': 'application/pdf',
'.zip': 'application/zip', '.zip': 'application/zip',
'.docx': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', '.json': 'application/json',
'.xlsx': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
'.pptx': 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
'.doc': 'application/msword', '.doc': 'application/msword',
'.js': 'application/javascript',
'.xls': 'application/vnd.ms-excel', '.xls': 'application/vnd.ms-excel',
'.ppt': 'application/vnd.ms-powerpoint', '.ppt': 'application/vnd.ms-powerpoint',
'.js': 'application/javascript' '.xlsx': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
'.docx': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
'.pptx': 'application/vnd.openxmlformats-officedocument.presentationml.presentation'
} as const;
export const defaultS3Options: {
externalBaseURL?: string;
maxFileSize?: number;
afterInit?: () => Promise<void> | void;
} & ClientOptions = {
maxFileSize: 1024 ** 3, // 1GB
useSSL: process.env.S3_USE_SSL === 'true',
endPoint: process.env.S3_ENDPOINT || 'localhost',
externalBaseURL: process.env.S3_EXTERNAL_BASE_URL,
accessKey: process.env.S3_ACCESS_KEY || 'minioadmin',
secretKey: process.env.S3_SECRET_KEY || 'minioadmin',
port: process.env.S3_PORT ? parseInt(process.env.S3_PORT) : 9000,
transportAgent: process.env.HTTP_PROXY
? new HttpProxyAgent(process.env.HTTP_PROXY)
: process.env.HTTPS_PROXY
? new HttpsProxyAgent(process.env.HTTPS_PROXY)
: undefined
}; };
export const S3Buckets = {
public: process.env.S3_PUBLIC_BUCKET || 'fastgpt-public',
private: process.env.S3_PRIVATE_BUCKET || 'fastgpt-private'
} as const;
import { Client } from 'minio'; import { MongoS3TTL } from './schema';
import { import { addLog } from '../system/log';
type FileMetadataType, import { setCron } from '../system/cron';
type PresignedUrlInput as UploadPresignedURLProps, import { checkTimerLock } from '../system/timerLock/utils';
type UploadPresignedURLResponse, import { TimerIdEnum } from '../system/timerLock/constants';
type S3ServiceConfig
} from './type'; export async function clearExpiredMinioFiles() {
import { defualtS3Config } from './config'; try {
import { randomBytes } from 'crypto'; const expiredFiles = await MongoS3TTL.find({
import { HttpProxyAgent } from 'http-proxy-agent'; expiredTime: { $lte: new Date() }
import { HttpsProxyAgent } from 'https-proxy-agent'; }).lean();
import { extname } from 'path'; if (expiredFiles.length === 0) {
import { addLog } from '../../common/system/log'; addLog.info('No expired minio files to clean');
import { getErrText } from '@fastgpt/global/common/error/utils'; return;
import { mimeMap } from './const';
export class S3Service {
private client: Client;
private config: S3ServiceConfig;
private initialized: boolean = false;
initFunction?: () => Promise<any>;
constructor(config?: Partial<S3ServiceConfig>) {
this.config = { ...defualtS3Config, ...config } as S3ServiceConfig;
this.client = new Client({
endPoint: this.config.endPoint,
port: this.config.port,
useSSL: this.config.useSSL,
accessKey: this.config.accessKey,
secretKey: this.config.secretKey,
transportAgent: process.env.HTTP_PROXY
? new HttpProxyAgent(process.env.HTTP_PROXY)
: process.env.HTTPS_PROXY
? new HttpsProxyAgent(process.env.HTTPS_PROXY)
: undefined
});
this.initFunction = config?.initFunction;
}
public async init() {
if (!this.initialized) {
if (!(await this.client.bucketExists(this.config.bucket))) {
addLog.debug(`Creating bucket: ${this.config.bucket}`);
await this.client.makeBucket(this.config.bucket);
}
await this.initFunction?.();
this.initialized = true;
}
}
private generateFileId(): string {
return randomBytes(16).toString('hex');
}
private generateAccessUrl(filename: string): string {
const protocol = this.config.useSSL ? 'https' : 'http';
const port =
this.config.port && this.config.port !== (this.config.useSSL ? 443 : 80)
? `:${this.config.port}`
: '';
const externalBaseURL = this.config.externalBaseURL;
return externalBaseURL
? `${externalBaseURL}/${this.config.bucket}/${encodeURIComponent(filename)}`
: `${protocol}://${this.config.endPoint}${port}/${this.config.bucket}/${encodeURIComponent(filename)}`;
}
uploadFile = async (fileBuffer: Buffer, originalFilename: string): Promise<FileMetadataType> => {
await this.init();
const inferContentType = (filename: string) => {
const ext = extname(filename).toLowerCase();
return mimeMap[ext] || 'application/octet-stream';
};
if (this.config.maxFileSize && fileBuffer.length > this.config.maxFileSize) {
return Promise.reject(
`File size ${fileBuffer.length} exceeds limit ${this.config.maxFileSize}`
);
} }
const fileId = this.generateFileId(); addLog.info(`Found ${expiredFiles.length} expired minio files to clean`);
const objectName = `${fileId}-${originalFilename}`;
const uploadTime = new Date(); let success = 0;
let fail = 0;
const contentType = inferContentType(originalFilename);
await this.client.putObject(this.config.bucket, objectName, fileBuffer, fileBuffer.length, { for (const file of expiredFiles) {
'Content-Type': contentType, try {
'Content-Disposition': `attachment; filename="${encodeURIComponent(originalFilename)}"`, const bucketName = file.bucketName;
'x-amz-meta-original-filename': encodeURIComponent(originalFilename), const bucket = global.s3BucketMap[bucketName];
'x-amz-meta-upload-time': uploadTime.toISOString()
}); if (bucket) {
await bucket.delete(file.minioKey);
const metadata: FileMetadataType = { await MongoS3TTL.deleteOne({ _id: file._id });
fileId,
originalFilename, success++;
contentType, addLog.info(
size: fileBuffer.length, `Deleted expired minio file: ${file.minioKey} from bucket: ${file.bucketName}`
uploadTime, );
accessUrl: this.generateAccessUrl(objectName) } else {
}; addLog.warn(`Bucket not found: ${file.bucketName}`);
}
return metadata; } catch (error) {
}; fail++;
addLog.error(`Failed to delete minio file: ${file.minioKey}`, error);
generateUploadPresignedURL = async ({
filepath,
contentType,
metadata,
filename
}: UploadPresignedURLProps): Promise<UploadPresignedURLResponse> => {
await this.init();
const objectName = `${filepath}/${filename}`;
try {
const policy = this.client.newPostPolicy();
policy.setBucket(this.config.bucket);
policy.setKey(objectName);
if (contentType) {
policy.setContentType(contentType);
}
if (this.config.maxFileSize) {
policy.setContentLengthRange(1, this.config.maxFileSize);
} }
policy.setExpires(new Date(Date.now() + 10 * 60 * 1000)); // 10 mins
policy.setUserMetaData({
'original-filename': encodeURIComponent(filename),
'upload-time': new Date().toISOString(),
...metadata
});
const { postURL, formData } = await this.client.presignedPostPolicy(policy);
const response: UploadPresignedURLResponse = {
objectName,
uploadUrl: postURL,
formData
};
return response;
} catch (error) {
addLog.error('Failed to generate Upload Presigned URL', error);
return Promise.reject(`Failed to generate Upload Presigned URL: ${getErrText(error)}`);
} }
};
generateDownloadUrl = (objectName: string): string => { addLog.info(`Minio TTL cleanup completed. Success: ${success}, Failed: ${fail}`);
const pathParts = objectName.split('/'); } catch (error) {
const encodedParts = pathParts.map((part) => encodeURIComponent(part)); addLog.error('Error in clearExpiredMinioFiles', error);
const encodedObjectName = encodedParts.join('/'); }
return `${this.config.bucket}/${encodedObjectName}`; }
};
getFile = async (objectName: string): Promise<string> => {
const stat = await this.client.statObject(this.config.bucket, objectName);
if (stat.size > 0) { export function clearExpiredS3FilesCron() {
const accessUrl = this.generateDownloadUrl(objectName); // 每小时执行一次
return accessUrl; setCron('0 */1 * * *', async () => {
if (
await checkTimerLock({
timerId: TimerIdEnum.clearExpiredMinioFiles,
lockMinuted: 59
})
) {
await clearExpiredMinioFiles();
} }
});
return Promise.reject(`File ${objectName} not found`);
};
} }
import { S3Service } from './controller'; import { S3PublicBucket } from './buckets/public';
import { S3PrivateBucket } from './buckets/private';
export const PluginS3Service = (() => { export function initS3Buckets() {
if (!global.pluginS3Service) { const publicBucket = new S3PublicBucket();
global.pluginS3Service = new S3Service({ const privateBucket = new S3PrivateBucket();
bucket: process.env.S3_PLUGIN_BUCKET || 'fastgpt-plugin',
maxFileSize: 50 * 1024 * 1024 // 50MB
});
}
return global.pluginS3Service; global.s3BucketMap = {
})(); [publicBucket.name]: publicBucket,
[privateBucket.name]: privateBucket
declare global { };
var pluginS3Service: S3Service;
} }
import { Schema, getMongoModel } from '../mongo';
import { type S3TtlSchemaType } from '@fastgpt/global/common/file/s3TTL/type';
const collectionName = 's3_ttls';
const S3TTLSchema = new Schema({
bucketName: {
type: String,
required: true
},
minioKey: {
type: String,
required: true
},
expiredTime: {
type: Date,
required: true
}
});
S3TTLSchema.index({ expiredTime: 1 });
S3TTLSchema.index({ bucketName: 1, minioKey: 1 });
export const MongoS3TTL = getMongoModel<S3TtlSchemaType>(collectionName, S3TTLSchema);
import { S3Sources } from '../type';
import { MongoS3TTL } from '../schema';
import { S3PublicBucket } from '../buckets/public';
import { imageBaseUrl } from '@fastgpt/global/common/file/image/constants';
import type { ClientSession } from 'mongoose';
class S3AvatarSource {
private bucket: S3PublicBucket;
private static instance: S3AvatarSource;
constructor() {
this.bucket = new S3PublicBucket();
}
static getInstance() {
return (this.instance ??= new S3AvatarSource());
}
get prefix(): string {
return imageBaseUrl;
}
async createUploadAvatarURL({
filename,
teamId,
autoExpired = true
}: {
filename: string;
teamId: string;
autoExpired?: boolean;
}) {
return this.bucket.createPostPresignedUrl(
{ filename, teamId, source: S3Sources.avatar },
{ expiredHours: autoExpired ? 1 : undefined } // 1 Hourse
);
}
createPublicUrl(objectKey: string): string {
return this.bucket.createPublicUrl(objectKey);
}
async removeAvatarTTL(avatar: string, session?: ClientSession): Promise<void> {
const key = avatar.slice(this.prefix.length);
await MongoS3TTL.deleteOne({ minioKey: key, bucketName: this.bucket.name }, session);
}
async deleteAvatar(avatar: string, session?: ClientSession): Promise<void> {
const key = avatar.slice(this.prefix.length);
await MongoS3TTL.deleteOne({ minioKey: key, bucketName: this.bucket.name }, session);
await this.bucket.delete(key);
}
async refreshAvatar(newAvatar?: string, oldAvatar?: string, session?: ClientSession) {
if (!newAvatar || newAvatar === oldAvatar) return;
// remove the TTL for the new avatar
await this.removeAvatarTTL(newAvatar, session);
if (oldAvatar) {
// delete the old avatar
// 1. delete the TTL record if it exists
// 2. delete the avatar in S3
await this.deleteAvatar(oldAvatar, session);
}
}
}
export function getS3AvatarSource() {
return S3AvatarSource.getInstance();
}
import type { ClientOptions } from 'minio'; import { z } from 'zod';
import type { defaultS3Options, Mimes } from './constants';
export type S3ServiceConfig = { import type { S3BaseBucket } from './buckets/base';
bucket: string;
externalBaseURL?: string; export const S3MetadataSchema = z.object({
/** filename: z.string(),
* Unit: Byte uploadedAt: z.date(),
*/ accessUrl: z.string(),
maxFileSize?: number; contentType: z.string(),
/** id: z.string().length(32),
* for executing some init function for the s3 service size: z.number().positive()
*/ });
initFunction?: () => Promise<any>; export type S3Metadata = z.infer<typeof S3MetadataSchema>;
} & ClientOptions;
export type ContentType = (typeof Mimes)[keyof typeof Mimes];
export type FileMetadataType = { export type ExtensionType = keyof typeof Mimes;
fileId: string;
originalFilename: string; export type S3OptionsType = typeof defaultS3Options;
contentType: string;
size: number; export const S3SourcesSchema = z.enum(['avatar']);
uploadTime: Date; export const S3Sources = S3SourcesSchema.enum;
accessUrl: string; export type S3SourceType = z.infer<typeof S3SourcesSchema>;
};
export type PresignedUrlInput = {
filepath: string;
filename: string;
contentType?: string;
metadata?: Record<string, string>;
};
export type UploadPresignedURLResponse = {
objectName: string;
uploadUrl: string;
formData: Record<string, string>;
};
export type FileUploadInput = {
buffer: Buffer;
filename: string;
};
export enum PluginTypeEnum {
tool = 'tool'
}
export const PluginFilePath = { export const CreatePostPresignedUrlParamsSchema = z.union([
[PluginTypeEnum.tool]: 'plugin/tools' // Option 1: Only rawKey
}; z.object({
filename: z.string().min(1),
rawKey: z.string().min(1)
}),
// Option 2: filename with optional source and teamId
z.object({
filename: z.string().min(1),
source: S3SourcesSchema.optional(),
teamId: z.string().length(16).optional()
})
]);
export type CreatePostPresignedUrlParams = z.infer<typeof CreatePostPresignedUrlParamsSchema>;
export const CreatePostPresignedUrlOptionsSchema = z.object({
expiredHours: z.number().positive().optional() // TTL in Hours, default 7 * 24
});
export type CreatePostPresignedUrlOptions = z.infer<typeof CreatePostPresignedUrlOptionsSchema>;
export const CreatePostPresignedUrlResultSchema = z.object({
url: z.string().min(1),
fields: z.record(z.string(), z.string())
});
export type CreatePostPresignedUrlResult = z.infer<typeof CreatePostPresignedUrlResultSchema>;
declare global {
var s3BucketMap: {
[key: string]: S3BaseBucket;
};
}
...@@ -8,7 +8,8 @@ export enum TimerIdEnum { ...@@ -8,7 +8,8 @@ export enum TimerIdEnum {
notification = 'notification', notification = 'notification',
clearExpiredRawTextBuffer = 'clearExpiredRawTextBuffer', clearExpiredRawTextBuffer = 'clearExpiredRawTextBuffer',
clearExpiredDatasetImage = 'clearExpiredDatasetImage' clearExpiredDatasetImage = 'clearExpiredDatasetImage',
clearExpiredMinioFiles = 'clearExpiredMinioFiles'
} }
export enum LockNotificationEnum { export enum LockNotificationEnum {
......
...@@ -26,12 +26,36 @@ const getVectorObj = () => { ...@@ -26,12 +26,36 @@ const getVectorObj = () => {
return new PgVectorCtrl(); return new PgVectorCtrl();
}; };
const getChcheKey = (teamId: string) => `${CacheKeyEnum.team_vector_count}:${teamId}`; const teamVectorCache = {
const onDelCache = throttle((teamId: string) => delRedisCache(getChcheKey(teamId)), 30000, { getKey: function (teamId: string) {
leading: true, return `${CacheKeyEnum.team_vector_count}:${teamId}`;
trailing: true },
}); get: async function (teamId: string) {
const onIncrCache = (teamId: string) => incrValueToCache(getChcheKey(teamId), 1); const countStr = await getRedisCache(teamVectorCache.getKey(teamId));
if (countStr) {
return Number(countStr);
}
return undefined;
},
set: function ({ teamId, count }: { teamId: string; count: number }) {
retryFn(() =>
setRedisCache(teamVectorCache.getKey(teamId), count, CacheKeyEnumTime.team_vector_count)
).catch();
},
delete: throttle(
function (teamId: string) {
return retryFn(() => delRedisCache(teamVectorCache.getKey(teamId))).catch();
},
30000,
{
leading: true,
trailing: true
}
),
incr: function (teamId: string, count: number) {
retryFn(() => incrValueToCache(teamVectorCache.getKey(teamId), count)).catch();
}
};
const Vector = getVectorObj(); const Vector = getVectorObj();
...@@ -41,16 +65,17 @@ export const recallFromVectorStore = (props: EmbeddingRecallCtrlProps) => ...@@ -41,16 +65,17 @@ export const recallFromVectorStore = (props: EmbeddingRecallCtrlProps) =>
export const getVectorDataByTime = Vector.getVectorDataByTime; export const getVectorDataByTime = Vector.getVectorDataByTime;
export const getVectorCountByTeamId = async (teamId: string) => { export const getVectorCountByTeamId = async (teamId: string) => {
const key = getChcheKey(teamId); const cacheCount = await teamVectorCache.get(teamId);
if (cacheCount !== undefined) {
const countStr = await getRedisCache(key); return cacheCount;
if (countStr) {
return Number(countStr);
} }
const count = await Vector.getVectorCountByTeamId(teamId); const count = await Vector.getVectorCountByTeamId(teamId);
await setRedisCache(key, count, CacheKeyEnumTime.team_vector_count); teamVectorCache.set({
teamId,
count
});
return count; return count;
}; };
...@@ -78,7 +103,7 @@ export const insertDatasetDataVector = async ({ ...@@ -78,7 +103,7 @@ export const insertDatasetDataVector = async ({
}) })
); );
onIncrCache(props.teamId); teamVectorCache.incr(props.teamId, insertIds.length);
return { return {
tokens, tokens,
...@@ -88,6 +113,6 @@ export const insertDatasetDataVector = async ({ ...@@ -88,6 +113,6 @@ export const insertDatasetDataVector = async ({
export const deleteDatasetDataVector = async (props: DelDatasetVectorCtrlProps) => { export const deleteDatasetDataVector = async (props: DelDatasetVectorCtrlProps) => {
const result = await retryFn(() => Vector.delete(props)); const result = await retryFn(() => Vector.delete(props));
onDelCache(props.teamId); teamVectorCache.delete(props.teamId);
return result; return result;
}; };
...@@ -80,7 +80,12 @@ export const loadSystemModels = async (init = false, language = 'en') => { ...@@ -80,7 +80,12 @@ export const loadSystemModels = async (init = false, language = 'en') => {
if (!init && global.systemModelList) return; if (!init && global.systemModelList) return;
await preloadModelProviders(); try {
await preloadModelProviders();
} catch (error) {
console.log('Load systen model error, please check fastgpt-plugin', error);
return Promise.reject(error);
}
global.systemModelList = []; global.systemModelList = [];
global.systemActiveModelList = []; global.systemActiveModelList = [];
...@@ -236,7 +241,7 @@ export const getSystemModelConfig = async (model: string): Promise<SystemModelIt ...@@ -236,7 +241,7 @@ export const getSystemModelConfig = async (model: string): Promise<SystemModelIt
export const watchSystemModelUpdate = () => { export const watchSystemModelUpdate = () => {
const changeStream = MongoSystemModel.watch(); const changeStream = MongoSystemModel.watch();
changeStream.on( return changeStream.on(
'change', 'change',
debounce(async () => { debounce(async () => {
try { try {
......
...@@ -3,6 +3,9 @@ import { getSecretValue } from '../../common/secret/utils'; ...@@ -3,6 +3,9 @@ import { getSecretValue } from '../../common/secret/utils';
import axios from 'axios'; import axios from 'axios';
import { getErrText } from '@fastgpt/global/common/error/utils'; import { getErrText } from '@fastgpt/global/common/error/utils';
import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils'; import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
import type { HttpToolConfigType } from '@fastgpt/global/core/app/type';
import { contentTypeMap, ContentTypes } from '@fastgpt/global/core/workflow/constants';
import { replaceEditorVariable } from '@fastgpt/global/core/workflow/runtime/utils';
export type RunHTTPToolParams = { export type RunHTTPToolParams = {
baseUrl: string; baseUrl: string;
...@@ -11,6 +14,9 @@ export type RunHTTPToolParams = { ...@@ -11,6 +14,9 @@ export type RunHTTPToolParams = {
params: Record<string, any>; params: Record<string, any>;
headerSecret?: StoreSecretValueType; headerSecret?: StoreSecretValueType;
customHeaders?: Record<string, string>; customHeaders?: Record<string, string>;
staticParams?: HttpToolConfigType['staticParams'];
staticHeaders?: HttpToolConfigType['staticHeaders'];
staticBody?: HttpToolConfigType['staticBody'];
}; };
export type RunHTTPToolResult = RequireOnlyOne<{ export type RunHTTPToolResult = RequireOnlyOne<{
...@@ -18,41 +24,143 @@ export type RunHTTPToolResult = RequireOnlyOne<{ ...@@ -18,41 +24,143 @@ export type RunHTTPToolResult = RequireOnlyOne<{
errorMsg?: string; errorMsg?: string;
}>; }>;
export async function runHTTPTool({ const buildHttpRequest = ({
method,
params,
headerSecret,
customHeaders,
staticParams,
staticHeaders,
staticBody
}: Omit<RunHTTPToolParams, 'baseUrl' | 'toolPath'>) => {
const replaceVariables = (text: string) => {
return replaceEditorVariable({
text,
nodes: [],
variables: params
});
};
const body = (() => {
if (!staticBody || staticBody.type === ContentTypes.none) {
return ['POST', 'PUT', 'PATCH'].includes(method.toUpperCase()) ? {} : undefined;
}
if (staticBody.type === ContentTypes.json) {
const contentWithReplacedVars = staticBody.content
? replaceVariables(staticBody.content)
: '{}';
const staticContent = JSON.parse(contentWithReplacedVars);
return { ...staticContent };
}
if (staticBody.type === ContentTypes.formData) {
const formData = new (require('form-data'))();
staticBody.formData?.forEach(({ key, value }) => {
const replacedKey = replaceVariables(key);
const replacedValue = replaceVariables(value);
formData.append(replacedKey, replacedValue);
});
return formData;
}
if (staticBody.type === ContentTypes.xWwwFormUrlencoded) {
const urlencoded = new URLSearchParams();
staticBody.formData?.forEach(({ key, value }) => {
const replacedKey = replaceVariables(key);
const replacedValue = replaceVariables(value);
urlencoded.append(replacedKey, replacedValue);
});
return urlencoded.toString();
}
if (staticBody.type === ContentTypes.xml || staticBody.type === ContentTypes.raw) {
return replaceVariables(staticBody.content || '');
}
return undefined;
})();
const contentType = contentTypeMap[staticBody?.type || ContentTypes.none];
const headers = {
...(contentType && { 'Content-Type': contentType }),
...(customHeaders || {}),
...(headerSecret ? getSecretValue({ storeSecret: headerSecret }) : {}),
...(staticHeaders?.reduce(
(acc, { key, value }) => {
const replacedKey = replaceVariables(key);
const replacedValue = replaceVariables(value);
acc[replacedKey] = replacedValue;
return acc;
},
{} as Record<string, string>
) || {})
};
const queryParams = (() => {
const staticParamsObj =
staticParams?.reduce(
(acc, { key, value }) => {
const replacedKey = replaceVariables(key);
const replacedValue = replaceVariables(value);
acc[replacedKey] = replacedValue;
return acc;
},
{} as Record<string, any>
) || {};
const mergedParams =
method.toUpperCase() === 'GET' || staticParams
? { ...staticParamsObj, ...params }
: staticParamsObj;
return Object.keys(mergedParams).length > 0 ? mergedParams : undefined;
})();
return {
headers,
body,
queryParams
};
};
export const runHTTPTool = async ({
baseUrl, baseUrl,
toolPath, toolPath,
method = 'POST', method = 'POST',
params, params,
headerSecret, headerSecret,
customHeaders customHeaders,
}: RunHTTPToolParams): Promise<RunHTTPToolResult> { staticParams,
staticHeaders,
staticBody
}: RunHTTPToolParams): Promise<RunHTTPToolResult> => {
try { try {
const headers = { const { headers, body, queryParams } = buildHttpRequest({
'Content-Type': 'application/json', method,
...(customHeaders || {}), params,
...(headerSecret ? getSecretValue({ storeSecret: headerSecret }) : {}) headerSecret,
}; customHeaders,
staticParams,
staticHeaders,
staticBody
});
const { data } = await axios({ const { data } = await axios({
method: method.toUpperCase(), method: method.toUpperCase(),
baseURL: baseUrl.startsWith('https://') ? baseUrl : `https://${baseUrl}`, baseURL: baseUrl.startsWith('https://') ? baseUrl : `https://${baseUrl}`,
url: toolPath, url: toolPath,
headers, headers,
data: params, data: body,
params, params: queryParams,
timeout: 300000, timeout: 300000,
httpsAgent: new (require('https').Agent)({ httpsAgent: new (require('https').Agent)({
rejectUnauthorized: false rejectUnauthorized: false
}) })
}); });
return { return { data };
data
};
} catch (error: any) { } catch (error: any) {
console.log(error); return { errorMsg: getErrText(error) };
return {
errorMsg: getErrText(error)
};
} }
} };
import { connectionMongo, getMongoModel } from '../../../common/mongo'; import { connectionMongo, getMongoModel } from '../../../common/mongo';
import { type ChatFavouriteAppSchema as ChatFavouriteAppType } from '@fastgpt/global/core/chat/favouriteApp/type'; import { type ChatFavouriteAppType } from '@fastgpt/global/core/chat/favouriteApp/type';
import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant'; import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
import { AppCollectionName } from '../../app/schema'; import { AppCollectionName } from '../../app/schema';
......
import { connectionMongo, getMongoModel } from '../../../common/mongo'; import { connectionMongo, getMongoModel } from '../../../common/mongo';
import { type ChatSettingSchema as ChatSettingType } from '@fastgpt/global/core/chat/setting/type'; import { type ChatSettingModelType } from '@fastgpt/global/core/chat/setting/type';
import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant'; import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
import { AppCollectionName } from '../../app/schema'; import { AppCollectionName } from '../../app/schema';
...@@ -55,7 +55,7 @@ ChatSettingSchema.virtual('quickAppList', { ...@@ -55,7 +55,7 @@ ChatSettingSchema.virtual('quickAppList', {
ChatSettingSchema.index({ teamId: 1 }); ChatSettingSchema.index({ teamId: 1 });
export const MongoChatSetting = getMongoModel<ChatSettingType>( export const MongoChatSetting = getMongoModel<ChatSettingModelType>(
ChatSettingCollectionName, ChatSettingCollectionName,
ChatSettingSchema ChatSettingSchema
); );
...@@ -236,16 +236,19 @@ export const dispatchRunTool = async (props: RunToolProps): Promise<RunToolRespo ...@@ -236,16 +236,19 @@ export const dispatchRunTool = async (props: RunToolProps): Promise<RunToolRespo
} }
const { data, errorMsg } = await runHTTPTool({ const { data, errorMsg } = await runHTTPTool({
baseUrl: baseUrl, baseUrl: baseUrl || '',
toolPath: httpTool.path, toolPath: httpTool.path,
method: httpTool.method, method: httpTool.method,
params, params,
headerSecret, headerSecret: httpTool.headerSecret || headerSecret,
customHeaders: customHeaders customHeaders: customHeaders
? typeof customHeaders === 'string' ? typeof customHeaders === 'string'
? JSON.parse(customHeaders) ? JSON.parse(customHeaders)
: customHeaders : customHeaders
: undefined : undefined,
staticParams: httpTool.staticParams,
staticHeaders: httpTool.staticHeaders,
staticBody: httpTool.staticBody
}); });
if (errorMsg) { if (errorMsg) {
......
...@@ -633,14 +633,13 @@ export const runWorkflow = async (data: RunWorkflowProps): Promise<DispatchFlowR ...@@ -633,14 +633,13 @@ export const runWorkflow = async (data: RunWorkflowProps): Promise<DispatchFlowR
this.chatNodeUsages = this.chatNodeUsages.concat(nodeDispatchUsages); this.chatNodeUsages = this.chatNodeUsages.concat(nodeDispatchUsages);
} }
if (toolResponses !== undefined && toolResponses !== null) { if (
if (Array.isArray(toolResponses) && toolResponses.length === 0) return; (toolResponses !== undefined && toolResponses !== null) ||
if ( (Array.isArray(toolResponses) && toolResponses.length > 0) ||
!Array.isArray(toolResponses) && (!Array.isArray(toolResponses) &&
typeof toolResponses === 'object' && typeof toolResponses === 'object' &&
Object.keys(toolResponses).length === 0 Object.keys(toolResponses).length > 0)
) ) {
return;
this.toolRunResponse = toolResponses; this.toolRunResponse = toolResponses;
} }
......
import { getErrText } from '@fastgpt/global/common/error/utils'; import { getErrText } from '@fastgpt/global/common/error/utils';
import { import {
contentTypeMap,
ContentTypes, ContentTypes,
NodeInputKeyEnum, NodeInputKeyEnum,
NodeOutputKeyEnum, NodeOutputKeyEnum,
...@@ -59,15 +60,6 @@ type HttpResponse = DispatchNodeResultType< ...@@ -59,15 +60,6 @@ type HttpResponse = DispatchNodeResultType<
const UNDEFINED_SIGN = 'UNDEFINED_SIGN'; const UNDEFINED_SIGN = 'UNDEFINED_SIGN';
const contentTypeMap = {
[ContentTypes.none]: '',
[ContentTypes.formData]: '',
[ContentTypes.xWwwFormUrlencoded]: 'application/x-www-form-urlencoded',
[ContentTypes.json]: 'application/json',
[ContentTypes.xml]: 'application/xml',
[ContentTypes.raw]: 'text/plain'
};
export const dispatchHttp468Request = async (props: HttpRequestProps): Promise<HttpResponse> => { export const dispatchHttp468Request = async (props: HttpRequestProps): Promise<HttpResponse> => {
let { let {
runningAppInfo: { id: appId, teamId, tmbId }, runningAppInfo: { id: appId, teamId, tmbId },
......
...@@ -54,7 +54,8 @@ ...@@ -54,7 +54,8 @@
"tiktoken": "1.0.17", "tiktoken": "1.0.17",
"tunnel": "^0.0.6", "tunnel": "^0.0.6",
"turndown": "^7.1.2", "turndown": "^7.1.2",
"winston": "^3.17.0" "winston": "^3.17.0",
"zod": "^3.24.2"
}, },
"devDependencies": { "devDependencies": {
"@types/cookie": "^0.5.2", "@types/cookie": "^0.5.2",
......
...@@ -17,7 +17,7 @@ import { mongoSessionRun } from '../../../common/mongo/sessionRun'; ...@@ -17,7 +17,7 @@ import { mongoSessionRun } from '../../../common/mongo/sessionRun';
import { DefaultGroupName } from '@fastgpt/global/support/user/team/group/constant'; import { DefaultGroupName } from '@fastgpt/global/support/user/team/group/constant';
import { getAIApi } from '../../../core/ai/config'; import { getAIApi } from '../../../core/ai/config';
import { createRootOrg } from '../../permission/org/controllers'; import { createRootOrg } from '../../permission/org/controllers';
import { refreshSourceAvatar } from '../../../common/file/image/controller'; import { getS3AvatarSource } from '../../../common/s3/sources/avatar';
async function getTeamMember(match: Record<string, any>): Promise<TeamTmbItemType> { async function getTeamMember(match: Record<string, any>): Promise<TeamTmbItemType> {
const tmb = await MongoTeamMember.findOne(match).populate<{ team: TeamSchema }>('team').lean(); const tmb = await MongoTeamMember.findOne(match).populate<{ team: TeamSchema }>('team').lean();
...@@ -244,7 +244,7 @@ export async function updateTeam({ ...@@ -244,7 +244,7 @@ export async function updateTeam({
{ session } { session }
); );
await refreshSourceAvatar(avatar, team?.avatar, session); await getS3AvatarSource().refreshAvatar(avatar, team?.avatar, session);
} }
}); });
} }
...@@ -22,7 +22,7 @@ export const getSafeEnv = () => { ...@@ -22,7 +22,7 @@ export const getSafeEnv = () => {
}; };
export const getWorker = (name: `${WorkerNameEnum}`) => { export const getWorker = (name: `${WorkerNameEnum}`) => {
const workerPath = path.join(process.cwd(), '.next', 'server', 'worker', `${name}.js`); const workerPath = path.join(process.cwd(), 'worker', `${name}.js`);
return new Worker(workerPath, { return new Worker(workerPath, {
env: getSafeEnv() env: getSafeEnv()
}); });
......
import { base64ToFile, fileToBase64 } from '../utils';
import { compressBase64Img } from '../img';
import { useToast } from '../../../hooks/useToast';
import { useCallback, useRef, useTransition } from 'react';
import { useTranslation } from 'next-i18next';
import { type CreatePostPresignedUrlResult } from '../../../../service/common/s3/type';
import { imageBaseUrl } from '@fastgpt/global/common/file/image/constants';
export const useUploadAvatar = (
api: (params: { filename: string }) => Promise<CreatePostPresignedUrlResult>,
{ onSuccess }: { onSuccess?: (avatar: string) => void } = {}
) => {
const { toast } = useToast();
const { t } = useTranslation();
const [uploading, startUpload] = useTransition();
const uploadAvatarRef = useRef<HTMLInputElement>(null);
const handleFileSelectorOpen = useCallback(() => {
if (!uploadAvatarRef.current) return;
uploadAvatarRef.current.click();
}, []);
// manually upload avatar
const handleUploadAvatar = useCallback(
async (file: File) => {
if (!file.name.match(/\.(jpg|png|jpeg)$/)) {
toast({ title: t('account_info:avatar_can_only_select_jpg_png'), status: 'warning' });
return;
}
startUpload(async () => {
const compressed = base64ToFile(
await compressBase64Img({
base64Img: await fileToBase64(file),
maxW: 300,
maxH: 300
}),
file.name
);
const { url, fields } = await api({ filename: file.name });
const formData = new FormData();
Object.entries(fields).forEach(([k, v]) => formData.set(k, v));
formData.set('file', compressed);
const res = await fetch(url, { method: 'POST', body: formData }); // 204
if (res.ok && res.status === 204) {
onSuccess?.(`${imageBaseUrl}${fields.key}`);
}
});
},
[t, toast, api, onSuccess]
);
const onUploadAvatarChange = useCallback(
async (e: React.ChangeEvent<HTMLInputElement>) => {
const files = e.target.files;
if (!files || files.length === 0) {
e.target.value = '';
return;
}
if (files.length > 1) {
toast({ title: t('account_info:avatar_can_only_select_one'), status: 'warning' });
e.target.value = '';
return;
}
const file = files[0]!;
handleUploadAvatar(file);
},
[t, toast, handleUploadAvatar]
);
const Component = useCallback(() => {
return (
<input
hidden
type="file"
multiple={false}
accept=".jpg,.png,.jpeg"
ref={uploadAvatarRef}
onChange={onUploadAvatarChange}
/>
);
}, [onUploadAvatarChange]);
return {
uploading,
Component,
handleFileSelectorOpen,
handleUploadAvatar
};
};
...@@ -99,3 +99,24 @@ async function detectFileEncoding(file: File): Promise<string> { ...@@ -99,3 +99,24 @@ async function detectFileEncoding(file: File): Promise<string> {
return encoding || 'utf-8'; return encoding || 'utf-8';
} }
export const fileToBase64 = (file: File) => {
return new Promise<string>((resolve, reject) => {
const reader = new FileReader();
reader.readAsDataURL(file);
reader.onload = () => resolve(reader.result as string);
reader.onerror = (error) => reject(error);
});
};
export const base64ToFile = (base64: string, filename: string) => {
const arr = base64.split(',');
const mime = arr[0].match(/:(.*?);/)?.[1];
const bstr = atob(arr[1]);
let n = bstr.length;
const u8arr = new Uint8Array(n);
while (n--) {
u8arr[n] = bstr.charCodeAt(n);
}
return new File([u8arr], filename, { type: mime });
};
...@@ -25,6 +25,7 @@ const LeftRadio = <T = any,>({ ...@@ -25,6 +25,7 @@ const LeftRadio = <T = any,>({
align = 'center', align = 'center',
px = 3.5, px = 3.5,
py = 4, py = 4,
gridGap = [3, 5],
defaultBg = 'myGray.50', defaultBg = 'myGray.50',
activeBg = 'primary.50', activeBg = 'primary.50',
onChange, onChange,
...@@ -75,7 +76,7 @@ const LeftRadio = <T = any,>({ ...@@ -75,7 +76,7 @@ const LeftRadio = <T = any,>({
); );
return ( return (
<Grid gridGap={[3, 5]} fontSize={['sm', 'md']} {...props}> <Grid gridGap={gridGap} fontSize={['sm', 'md']} {...props}>
{list.map((item) => { {list.map((item) => {
const isActive = value === item.value; const isActive = value === item.value;
return ( return (
...@@ -131,7 +132,7 @@ const LeftRadio = <T = any,>({ ...@@ -131,7 +132,7 @@ const LeftRadio = <T = any,>({
lineHeight={1} lineHeight={1}
color={'myGray.900'} color={'myGray.900'}
> >
<Box>{t(item.title as any)}</Box> <Box mb={1}>{t(item.title as any)}</Box>
{!!item.tooltip && <QuestionTip label={item.tooltip} color={'myGray.600'} />} {!!item.tooltip && <QuestionTip label={item.tooltip} color={'myGray.600'} />}
</HStack> </HStack>
) : ( ) : (
......
...@@ -9,6 +9,8 @@ ...@@ -9,6 +9,8 @@
"app_amount": "App amount", "app_amount": "App amount",
"avatar": "Avatar", "avatar": "Avatar",
"avatar_selection_exception": "Abnormal avatar selection", "avatar_selection_exception": "Abnormal avatar selection",
"avatar_can_only_select_one": "Avatar can only select one picture",
"avatar_can_only_select_jpg_png": "Avatar can only select jpg or png format",
"balance": "balance", "balance": "balance",
"billing_standard": "Standards", "billing_standard": "Standards",
"cancel": "Cancel", "cancel": "Cancel",
......
{ {
"Add_tool": "Add tool",
"AutoOptimize": "Automatic optimization", "AutoOptimize": "Automatic optimization",
"Click_to_delete_this_field": "Click to delete this field", "Click_to_delete_this_field": "Click to delete this field",
"Custom_params": "input parameters",
"Edit_tool": "Edit tool",
"Filed_is_deprecated": "This field is deprecated", "Filed_is_deprecated": "This field is deprecated",
"HTTPTools_Create_Type": "Create Type",
"HTTPTools_Create_Type_Tip": "Modification is not supported after selection",
"HTTP_tools_list_with_number": "Tool list: {{total}}", "HTTP_tools_list_with_number": "Tool list: {{total}}",
"Index": "Index", "Index": "Index",
"MCP_tools_debug": "debug", "MCP_tools_debug": "debug",
...@@ -30,6 +35,7 @@ ...@@ -30,6 +35,7 @@
"Selected": "Selected", "Selected": "Selected",
"Start_config": "Start configuration", "Start_config": "Start configuration",
"Team_Tags": "Team tags", "Team_Tags": "Team tags",
"Tool_name": "Tool name",
"ai_point_price": "Billing", "ai_point_price": "Billing",
"ai_settings": "AI Configuration", "ai_settings": "AI Configuration",
"all_apps": "All Applications", "all_apps": "All Applications",
...@@ -89,6 +95,7 @@ ...@@ -89,6 +95,7 @@
"document_upload": "Document Upload", "document_upload": "Document Upload",
"edit_app": "Application details", "edit_app": "Application details",
"edit_info": "Edit", "edit_info": "Edit",
"empty_tool_tips": "Please add tools on the left side",
"execute_time": "Execution Time", "execute_time": "Execution Time",
"export_config_successful": "Configuration copied, some sensitive information automatically filtered. Please check for any remaining sensitive data.", "export_config_successful": "Configuration copied, some sensitive information automatically filtered. Please check for any remaining sensitive data.",
"export_configs": "Export", "export_configs": "Export",
...@@ -99,12 +106,15 @@ ...@@ -99,12 +106,15 @@
"file_upload_tip": "Once enabled, documents/images can be uploaded. Documents are retained for 7 days, images for 15 days. Using this feature may incur additional costs. To ensure a good experience, please choose an AI model with a larger context length when using this feature.", "file_upload_tip": "Once enabled, documents/images can be uploaded. Documents are retained for 7 days, images for 15 days. Using this feature may incur additional costs. To ensure a good experience, please choose an AI model with a larger context length when using this feature.",
"go_to_chat": "Go to Conversation", "go_to_chat": "Go to Conversation",
"go_to_run": "Go to Execution", "go_to_run": "Go to Execution",
"http_toolset_add_tips": "Click the \"Add\" button to add tools",
"http_toolset_config_tips": "Click \"Start Configuration\" to add tools",
"image_upload": "Image Upload", "image_upload": "Image Upload",
"image_upload_tip": "How to activate model image recognition capabilities", "image_upload_tip": "How to activate model image recognition capabilities",
"import_configs": "Import", "import_configs": "Import",
"import_configs_failed": "Import configuration failed, please ensure the configuration is correct!", "import_configs_failed": "Import configuration failed, please ensure the configuration is correct!",
"import_configs_success": "Import Successful", "import_configs_success": "Import Successful",
"initial_form": "initial state", "initial_form": "initial state",
"input_params_tips": "Tool input parameters will not be passed directly to the request URL; they can be referenced on the right side using \"/\".",
"interval.12_hours": "Every 12 Hours", "interval.12_hours": "Every 12 Hours",
"interval.2_hours": "Every 2 Hours", "interval.2_hours": "Every 2 Hours",
"interval.3_hours": "Every 3 Hours", "interval.3_hours": "Every 3 Hours",
...@@ -283,6 +293,7 @@ ...@@ -283,6 +293,7 @@
"tool_detail": "Tool details", "tool_detail": "Tool details",
"tool_input_param_tip": "This plugin requires configuration of related information to run properly.", "tool_input_param_tip": "This plugin requires configuration of related information to run properly.",
"tool_not_active": "This tool has not been activated yet", "tool_not_active": "This tool has not been activated yet",
"tool_params_description_tips": "The description of parameter functions, if used as tool invocation parameters, affects the model tool invocation effect.",
"tool_run_free": "This tool runs without points consumption", "tool_run_free": "This tool runs without points consumption",
"tool_tip": "When executed as a tool, is this field used as a tool response result?", "tool_tip": "When executed as a tool, is this field used as a tool response result?",
"tool_type_tools": "tool", "tool_type_tools": "tool",
......
...@@ -9,6 +9,8 @@ ...@@ -9,6 +9,8 @@
"app_amount": "应用数量", "app_amount": "应用数量",
"avatar": "头像", "avatar": "头像",
"avatar_selection_exception": "头像选择异常", "avatar_selection_exception": "头像选择异常",
"avatar_can_only_select_one": "头像只能选择一张图片",
"avatar_can_only_select_jpg_png": "头像只能选择 jpg 或 png 格式",
"balance": "余额", "balance": "余额",
"billing_standard": "计费标准", "billing_standard": "计费标准",
"cancel": "取消", "cancel": "取消",
......
{ {
"Add_tool": "添加工具",
"AutoOptimize": "自动优化", "AutoOptimize": "自动优化",
"Click_to_delete_this_field": "点击删除该字段", "Click_to_delete_this_field": "点击删除该字段",
"Custom_params": "输入参数",
"Edit_tool": "编辑工具",
"Filed_is_deprecated": "该字段已弃用", "Filed_is_deprecated": "该字段已弃用",
"HTTPTools_Create_Type": "创建方式",
"HTTPTools_Create_Type_Tip": "选择后不支持修改",
"HTTP_tools_detail": "查看详情", "HTTP_tools_detail": "查看详情",
"HTTP_tools_list_with_number": "工具列表: {{total}}", "HTTP_tools_list_with_number": "工具列表: {{total}}",
"Index": "索引", "Index": "索引",
...@@ -31,6 +36,8 @@ ...@@ -31,6 +36,8 @@
"Selected": "已选择", "Selected": "已选择",
"Start_config": "开始配置", "Start_config": "开始配置",
"Team_Tags": "团队标签", "Team_Tags": "团队标签",
"Tool_description": "工具描述",
"Tool_name": "工具名称",
"ai_point_price": "AI积分计费", "ai_point_price": "AI积分计费",
"ai_settings": "AI 配置", "ai_settings": "AI 配置",
"all_apps": "全部应用", "all_apps": "全部应用",
...@@ -90,6 +97,8 @@ ...@@ -90,6 +97,8 @@
"document_upload": "文档上传", "document_upload": "文档上传",
"edit_app": "应用详情", "edit_app": "应用详情",
"edit_info": "编辑信息", "edit_info": "编辑信息",
"edit_param": "编辑参数",
"empty_tool_tips": "请在左侧添加工具",
"execute_time": "执行时间", "execute_time": "执行时间",
"export_config_successful": "已复制配置,自动过滤部分敏感信息,请注意检查是否仍有敏感数据", "export_config_successful": "已复制配置,自动过滤部分敏感信息,请注意检查是否仍有敏感数据",
"export_configs": "导出配置", "export_configs": "导出配置",
...@@ -100,12 +109,15 @@ ...@@ -100,12 +109,15 @@
"file_upload_tip": "开启后,可以上传文档/图片。文档保留7天,图片保留15天。使用该功能可能产生较多额外费用。为保证使用体验,使用该功能时,请选择上下文长度较大的AI模型。", "file_upload_tip": "开启后,可以上传文档/图片。文档保留7天,图片保留15天。使用该功能可能产生较多额外费用。为保证使用体验,使用该功能时,请选择上下文长度较大的AI模型。",
"go_to_chat": "去对话", "go_to_chat": "去对话",
"go_to_run": "去运行", "go_to_run": "去运行",
"http_toolset_add_tips": "点击添加按钮来添加工具",
"http_toolset_config_tips": "点击开始配置来添加工具",
"image_upload": "图片上传", "image_upload": "图片上传",
"image_upload_tip": "如何启动模型图片识别能力", "image_upload_tip": "如何启动模型图片识别能力",
"import_configs": "导入配置", "import_configs": "导入配置",
"import_configs_failed": "导入配置失败,请确保配置正常!", "import_configs_failed": "导入配置失败,请确保配置正常!",
"import_configs_success": "导入成功", "import_configs_success": "导入成功",
"initial_form": "初始状态", "initial_form": "初始状态",
"input_params_tips": "工具的输入参数,不会直接传递到请求地址,可在右侧通过 \"/\" 来引用变量",
"interval.12_hours": "每12小时", "interval.12_hours": "每12小时",
"interval.2_hours": "每2小时", "interval.2_hours": "每2小时",
"interval.3_hours": "每3小时", "interval.3_hours": "每3小时",
...@@ -297,6 +309,7 @@ ...@@ -297,6 +309,7 @@
"tool_detail": "工具详情", "tool_detail": "工具详情",
"tool_input_param_tip": "该插件正常运行需要配置相关信息", "tool_input_param_tip": "该插件正常运行需要配置相关信息",
"tool_not_active": "该工具尚未激活", "tool_not_active": "该工具尚未激活",
"tool_params_description_tips": "参数功能的描述,若作为工具调用参数,影响模型工具调用效果",
"tool_run_free": "该工具运行无积分消耗", "tool_run_free": "该工具运行无积分消耗",
"tool_tip": "作为工具执行时,该字段是否作为工具响应结果", "tool_tip": "作为工具执行时,该字段是否作为工具响应结果",
"tool_type_tools": "工具", "tool_type_tools": "工具",
......
...@@ -9,6 +9,8 @@ ...@@ -9,6 +9,8 @@
"app_amount": "應用數量", "app_amount": "應用數量",
"avatar": "頭像", "avatar": "頭像",
"avatar_selection_exception": "頭像選擇異常", "avatar_selection_exception": "頭像選擇異常",
"avatar_can_only_select_one": "頭像只能選擇一張圖片",
"avatar_can_only_select_jpg_png": "頭像只能選擇 jpg 或 png 格式",
"balance": "餘額", "balance": "餘額",
"billing_standard": "計費標準", "billing_standard": "計費標準",
"cancel": "取消", "cancel": "取消",
......
{ {
"Add_tool": "添加工具",
"AutoOptimize": "自動優化", "AutoOptimize": "自動優化",
"Click_to_delete_this_field": "點擊刪除該字段", "Click_to_delete_this_field": "點擊刪除該字段",
"Custom_params": "輸入參數",
"Filed_is_deprecated": "該字段已棄用", "Filed_is_deprecated": "該字段已棄用",
"HTTPTools_Create_Type": "創建方式",
"HTTPTools_Create_Type_Tip": "選擇後不支持修改",
"HTTP_tools_list_with_number": "工具列表: {{total}}", "HTTP_tools_list_with_number": "工具列表: {{total}}",
"Index": "索引", "Index": "索引",
"MCP_tools_debug": "偵錯", "MCP_tools_debug": "偵錯",
...@@ -30,6 +34,8 @@ ...@@ -30,6 +34,8 @@
"Selected": "已選擇", "Selected": "已選擇",
"Start_config": "開始配置", "Start_config": "開始配置",
"Team_Tags": "團隊標籤", "Team_Tags": "團隊標籤",
"Tool_description": "工具描述",
"Tool_name": "工具名稱",
"ai_point_price": "AI 積分計費", "ai_point_price": "AI 積分計費",
"ai_settings": "AI 設定", "ai_settings": "AI 設定",
"all_apps": "所有應用程式", "all_apps": "所有應用程式",
...@@ -89,6 +95,7 @@ ...@@ -89,6 +95,7 @@
"document_upload": "文件上傳", "document_upload": "文件上傳",
"edit_app": "應用詳情", "edit_app": "應用詳情",
"edit_info": "編輯資訊", "edit_info": "編輯資訊",
"empty_tool_tips": "請在左側添加工具",
"execute_time": "執行時間", "execute_time": "執行時間",
"export_config_successful": "已複製設定,自動過濾部分敏感資訊,請注意檢查是否仍有敏感資料", "export_config_successful": "已複製設定,自動過濾部分敏感資訊,請注意檢查是否仍有敏感資料",
"export_configs": "匯出設定", "export_configs": "匯出設定",
...@@ -99,12 +106,15 @@ ...@@ -99,12 +106,15 @@
"file_upload_tip": "開啟後,可以上傳文件/圖片。文件保留 7 天,圖片保留 15 天。使用這個功能可能產生較多額外費用。為了確保使用體驗,使用這個功能時,請選擇上下文長度較大的 AI 模型。", "file_upload_tip": "開啟後,可以上傳文件/圖片。文件保留 7 天,圖片保留 15 天。使用這個功能可能產生較多額外費用。為了確保使用體驗,使用這個功能時,請選擇上下文長度較大的 AI 模型。",
"go_to_chat": "前往對話", "go_to_chat": "前往對話",
"go_to_run": "前往執行", "go_to_run": "前往執行",
"http_toolset_add_tips": "點擊添加按鈕來添加工具",
"http_toolset_config_tips": "點擊開始配置來添加工具",
"image_upload": "圖片上傳", "image_upload": "圖片上傳",
"image_upload_tip": "如何啟用模型圖片辨識功能", "image_upload_tip": "如何啟用模型圖片辨識功能",
"import_configs": "匯入設定", "import_configs": "匯入設定",
"import_configs_failed": "匯入設定失敗,請確認設定是否正常!", "import_configs_failed": "匯入設定失敗,請確認設定是否正常!",
"import_configs_success": "匯入成功", "import_configs_success": "匯入成功",
"initial_form": "初始狀態", "initial_form": "初始狀態",
"input_params_tips": "工具的輸入參數,不會直接傳遞到請求地址,可在右側通過 \"/\" 來引用變數",
"interval.12_hours": "每 12 小時", "interval.12_hours": "每 12 小時",
"interval.2_hours": "每 2 小時", "interval.2_hours": "每 2 小時",
"interval.3_hours": "每 3 小時", "interval.3_hours": "每 3 小時",
...@@ -283,6 +293,7 @@ ...@@ -283,6 +293,7 @@
"tool_detail": "工具詳情", "tool_detail": "工具詳情",
"tool_input_param_tip": "這個外掛正常執行需要設定相關資訊", "tool_input_param_tip": "這個外掛正常執行需要設定相關資訊",
"tool_not_active": "該工具尚未激活", "tool_not_active": "該工具尚未激活",
"tool_params_description_tips": "參數功能的描述,若作為工具調用參數,影響模型工具調用效果",
"tool_run_free": "該工具運行無積分消耗", "tool_run_free": "該工具運行無積分消耗",
"tool_tip": "作為工具執行時,該字段是否作為工具響應結果", "tool_tip": "作為工具執行時,該字段是否作為工具響應結果",
"tool_type_tools": "工具", "tool_type_tools": "工具",
......
...@@ -22,7 +22,7 @@ ...@@ -22,7 +22,7 @@
"@lexical/utils": "0.12.6", "@lexical/utils": "0.12.6",
"@monaco-editor/react": "^4.6.0", "@monaco-editor/react": "^4.6.0",
"@tanstack/react-query": "^4.24.10", "@tanstack/react-query": "^4.24.10",
"ahooks": "^3.9.4", "ahooks": "^3.9.5",
"date-fns": "2.30.0", "date-fns": "2.30.0",
"dayjs": "^1.11.7", "dayjs": "^1.11.7",
"next": "14.2.32", "next": "14.2.32",
......
This source diff could not be displayed because it is too large. You can view the blob instead.
...@@ -41,7 +41,9 @@ S3_PORT=9000 ...@@ -41,7 +41,9 @@ S3_PORT=9000
S3_USE_SSL=false S3_USE_SSL=false
S3_ACCESS_KEY=minioadmin S3_ACCESS_KEY=minioadmin
S3_SECRET_KEY=minioadmin S3_SECRET_KEY=minioadmin
S3_PLUGIN_BUCKET=fastgpt-plugin # 插件文件存储bucket S3_PUBLIC_BUCKET=fastgpt-public # 插件文件存储公开桶
S3_PRIVATE_BUCKET=fastgpt-private # 插件文件存储公开桶
# Redis URL # Redis URL
REDIS_URL=redis://default:mypassword@127.0.0.1:6379 REDIS_URL=redis://default:mypassword@127.0.0.1:6379
# mongo 数据库连接参数,本地开发连接远程数据库时,可能需要增加 directConnection=true 参数,才能连接上。 # mongo 数据库连接参数,本地开发连接远程数据库时,可能需要增加 directConnection=true 参数,才能连接上。
......
...@@ -66,19 +66,17 @@ COPY --from=builder --chown=nextjs:nodejs /app/projects/app/.next/static /app/pr ...@@ -66,19 +66,17 @@ COPY --from=builder --chown=nextjs:nodejs /app/projects/app/.next/static /app/pr
# copy server chunks # copy server chunks
COPY --from=builder --chown=nextjs:nodejs /app/projects/app/.next/server/chunks /app/projects/app/.next/server/chunks COPY --from=builder --chown=nextjs:nodejs /app/projects/app/.next/server/chunks /app/projects/app/.next/server/chunks
# copy worker # copy worker
COPY --from=builder --chown=nextjs:nodejs /app/projects/app/.next/server/worker /app/projects/app/.next/server/worker COPY --from=builder --chown=nextjs:nodejs /app/projects/app/worker /app/projects/app/worker
# copy standload packages # copy standload packages
COPY --from=maindeps /app/node_modules/tiktoken ./node_modules/tiktoken COPY --from=maindeps /app/node_modules/tiktoken ./node_modules/tiktoken
RUN rm -rf ./node_modules/tiktoken/encoders RUN rm -rf ./node_modules/tiktoken/encoders
COPY --from=maindeps /app/node_modules/@zilliz/milvus2-sdk-node ./node_modules/@zilliz/milvus2-sdk-node COPY --from=maindeps /app/node_modules/@zilliz/milvus2-sdk-node ./node_modules/@zilliz/milvus2-sdk-node
# copy package.json to version file # copy package.json to version file
COPY --from=builder /app/projects/app/package.json ./package.json COPY --from=builder /app/projects/app/package.json ./package.json
# copy config # copy config
COPY ./projects/app/data /app/data COPY ./projects/app/data/config.json /app/data/config.json
RUN chown -R nextjs:nodejs /app/data RUN chown -R nextjs:nodejs /app/data
# Add tmp directory permission control # Add tmp directory permission control
......
...@@ -2,6 +2,10 @@ const { i18n } = require('./next-i18next.config.js'); ...@@ -2,6 +2,10 @@ const { i18n } = require('./next-i18next.config.js');
const path = require('path'); const path = require('path');
const fs = require('fs'); const fs = require('fs');
const withBundleAnalyzer = require('@next/bundle-analyzer')({
enabled: process.env.ANALYZE === 'true',
});
const isDev = process.env.NODE_ENV === 'development'; const isDev = process.env.NODE_ENV === 'development';
/** @type {import('next').NextConfig} */ /** @type {import('next').NextConfig} */
...@@ -11,6 +15,10 @@ const nextConfig = { ...@@ -11,6 +15,10 @@ const nextConfig = {
output: 'standalone', output: 'standalone',
reactStrictMode: isDev ? false : true, reactStrictMode: isDev ? false : true,
compress: true, compress: true,
// 禁用 source map(可选,根据需要)
productionBrowserSourceMaps: false,
// 优化编译性能
swcMinify: true, // 使用 SWC 压缩(生产环境已默认)
async headers() { async headers() {
return [ return [
{ {
...@@ -40,6 +48,7 @@ const nextConfig = { ...@@ -40,6 +48,7 @@ const nextConfig = {
} }
]; ];
}, },
webpack(config, { isServer, nextRuntime }) { webpack(config, { isServer, nextRuntime }) {
Object.assign(config.resolve.alias, { Object.assign(config.resolve.alias, {
'@mongodb-js/zstd': false, '@mongodb-js/zstd': false,
...@@ -73,17 +82,7 @@ const nextConfig = { ...@@ -73,17 +82,7 @@ const nextConfig = {
config.externals.push('@node-rs/jieba'); config.externals.push('@node-rs/jieba');
if (nextRuntime === 'nodejs') { if (nextRuntime === 'nodejs') {
const oldEntry = config.entry;
config = {
...config,
async entry(...args) {
const entries = await oldEntry(...args);
return {
...entries,
...getWorkerConfig()
};
}
};
} }
} else { } else {
config.resolve = { config.resolve = {
...@@ -100,6 +99,32 @@ const nextConfig = { ...@@ -100,6 +99,32 @@ const nextConfig = {
layers: true layers: true
}; };
if (isDev && !isServer) {
// 使用更快的 source map
config.devtool = 'eval-cheap-module-source-map';
// 减少文件监听范围
config.watchOptions = {
...config.watchOptions,
ignored: [
'**/node_modules',
'**/.git',
'**/dist',
'**/coverage'
],
};
// 启用持久化缓存
config.cache = {
type: 'filesystem',
name: isServer ? 'server' : 'client',
buildDependencies: {
config: [__filename]
},
cacheDirectory: path.resolve(__dirname, '.next/cache/webpack'),
maxMemoryGenerations: isDev ? 5 : Infinity,
maxAge: 7 * 24 * 60 * 60 * 1000, // 7 天
};
}
return config; return config;
}, },
// 需要转译的包 // 需要转译的包
...@@ -111,31 +136,14 @@ const nextConfig = { ...@@ -111,31 +136,14 @@ const nextConfig = {
'pg', 'pg',
'bullmq', 'bullmq',
'@zilliz/milvus2-sdk-node', '@zilliz/milvus2-sdk-node',
'tiktoken' 'tiktoken',
'@opentelemetry/api-logs'
], ],
outputFileTracingRoot: path.join(__dirname, '../../'), outputFileTracingRoot: path.join(__dirname, '../../'),
instrumentationHook: true instrumentationHook: true,
workerThreads: true
} }
}; };
module.exports = nextConfig; module.exports = nextConfig;
function getWorkerConfig() {
const result = fs.readdirSync(path.resolve(__dirname, '../../packages/service/worker'));
// 获取所有的目录名
const folderList = result.filter((item) => {
return fs
.statSync(path.resolve(__dirname, '../../packages/service/worker', item))
.isDirectory();
});
const workerConfig = folderList.reduce((acc, item) => {
acc[`worker/${item}`] = path.resolve(
process.cwd(),
`../../packages/service/worker/${item}/index.ts`
);
return acc;
}, {});
return workerConfig;
}
{ {
"name": "app", "name": "app",
"version": "4.13.1", "version": "4.13.2",
"private": false, "private": false,
"scripts": { "scripts": {
"dev": "next dev", "dev": "npm run build:workers && next dev",
"build": "next build", "build": "npm run build:workers && next build",
"start": "next start", "start": "next start",
"lint": "next lint" "lint": "next lint",
"build:workers": "npx tsx scripts/build-workers.ts",
"build:workers:watch": "npx tsx scripts/build-workers.ts --watch"
}, },
"dependencies": { "dependencies": {
"@chakra-ui/anatomy": "2.2.1", "@chakra-ui/anatomy": "2.2.1",
...@@ -24,13 +26,15 @@ ...@@ -24,13 +26,15 @@
"@fortaine/fetch-event-source": "^3.0.6", "@fortaine/fetch-event-source": "^3.0.6",
"@modelcontextprotocol/sdk": "^1.12.1", "@modelcontextprotocol/sdk": "^1.12.1",
"@node-rs/jieba": "2.0.1", "@node-rs/jieba": "2.0.1",
"@scalar/api-reference-react": "^0.8.1",
"@tanstack/react-query": "^4.24.10", "@tanstack/react-query": "^4.24.10",
"ahooks": "^3.7.11", "ahooks": "^3.9.5",
"axios": "^1.12.1", "axios": "^1.12.1",
"date-fns": "2.30.0", "date-fns": "2.30.0",
"dayjs": "^1.11.7", "dayjs": "^1.11.7",
"echarts": "5.4.1", "echarts": "5.4.1",
"echarts-gl": "2.0.9", "echarts-gl": "2.0.9",
"esbuild": "^0.25.11",
"framer-motion": "9.1.7", "framer-motion": "9.1.7",
"hyperdown": "^2.4.29", "hyperdown": "^2.4.29",
"i18next": "23.16.8", "i18next": "23.16.8",
...@@ -41,6 +45,7 @@ ...@@ -41,6 +45,7 @@
"jsonwebtoken": "^9.0.2", "jsonwebtoken": "^9.0.2",
"lodash": "^4.17.21", "lodash": "^4.17.21",
"mermaid": "^10.9.4", "mermaid": "^10.9.4",
"minio": "^8.0.5",
"nanoid": "^5.1.3", "nanoid": "^5.1.3",
"next": "14.2.32", "next": "14.2.32",
"next-i18next": "15.4.2", "next-i18next": "15.4.2",
...@@ -64,10 +69,10 @@ ...@@ -64,10 +69,10 @@
"request-ip": "^3.3.0", "request-ip": "^3.3.0",
"sass": "^1.58.3", "sass": "^1.58.3",
"use-context-selector": "^1.4.4", "use-context-selector": "^1.4.4",
"zod": "^3.24.2", "zod": "^3.24.2"
"minio": "^8.0.5"
}, },
"devDependencies": { "devDependencies": {
"@next/bundle-analyzer": "^15.5.6",
"@svgr/webpack": "^6.5.1", "@svgr/webpack": "^6.5.1",
"@types/js-yaml": "^4.0.9", "@types/js-yaml": "^4.0.9",
"@types/jsonwebtoken": "^9.0.3", "@types/jsonwebtoken": "^9.0.3",
...@@ -83,6 +88,7 @@ ...@@ -83,6 +88,7 @@
"@typescript-eslint/parser": "^6.21.0", "@typescript-eslint/parser": "^6.21.0",
"eslint": "8.56.0", "eslint": "8.56.0",
"eslint-config-next": "14.2.26", "eslint-config-next": "14.2.26",
"tsx": "^4.20.6",
"typescript": "^5.1.3", "typescript": "^5.1.3",
"vitest": "^3.0.2" "vitest": "^3.0.2"
} }
......
import { build, BuildOptions, context } from 'esbuild';
import fs from 'fs';
import path from 'path';
// 项目路径
const ROOT_DIR = path.resolve(__dirname, '../../..');
const WORKER_SOURCE_DIR = path.join(ROOT_DIR, 'packages/service/worker');
const WORKER_OUTPUT_DIR = path.join(__dirname, '../worker');
/**
* Worker 预编译脚本
* 用于在 Turbopack 开发环境下编译 Worker 文件
*/
async function buildWorkers(watch: boolean = false) {
console.log('🔨 开始编译 Worker 文件...\n');
// 确保输出目录存在
if (!fs.existsSync(WORKER_OUTPUT_DIR)) {
fs.mkdirSync(WORKER_OUTPUT_DIR, { recursive: true });
}
// 扫描 worker 目录
if (!fs.existsSync(WORKER_SOURCE_DIR)) {
console.error(`❌ Worker 源目录不存在: ${WORKER_SOURCE_DIR}`);
process.exit(1);
}
const workers = fs.readdirSync(WORKER_SOURCE_DIR).filter((item) => {
const fullPath = path.join(WORKER_SOURCE_DIR, item);
const isDir = fs.statSync(fullPath).isDirectory();
const hasIndexTs = fs.existsSync(path.join(fullPath, 'index.ts'));
return isDir && hasIndexTs;
});
if (workers.length === 0) {
return;
}
// esbuild 通用配置
const commonConfig: BuildOptions = {
bundle: true,
platform: 'node',
format: 'cjs',
target: 'node18',
sourcemap: false,
// Tree Shaking 和代码压缩优化
minify: true,
treeShaking: true,
keepNames: false,
// 移除调试代码
drop: process.env.NODE_ENV === 'production' ? ['console', 'debugger'] : []
};
if (watch) {
// Watch 模式:使用 esbuild context API
const contexts = await Promise.all(
workers.map(async (worker) => {
const entryPoint = path.join(WORKER_SOURCE_DIR, worker, 'index.ts');
const outfile = path.join(WORKER_OUTPUT_DIR, `${worker}.js`);
const config: BuildOptions = {
...commonConfig,
entryPoints: [entryPoint],
outfile,
logLevel: 'info'
};
try {
const ctx = await context(config);
await ctx.watch();
console.log(`👁️ ${worker} 正在监听中...`);
return ctx;
} catch (error: any) {
console.error(`❌ ${worker} Watch 启动失败:`, error.message);
return null;
}
})
);
// 过滤掉失败的 context
const validContexts = contexts.filter((ctx) => ctx !== null);
console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━');
console.log(`✅ ${validContexts.length}/${workers.length} 个 Worker 正在监听中`);
console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━');
console.log('\n💡 提示: 按 Ctrl+C 停止监听\n');
// 保持进程运行
process.on('SIGINT', async () => {
console.log('\n\n🛑 正在停止 Worker 监听...');
await Promise.all(validContexts.map((ctx) => ctx?.dispose()));
console.log('✅ 已停止');
process.exit(0);
});
} else {
// 单次编译模式
const buildPromises = workers.map(async (worker) => {
const entryPoint = path.join(WORKER_SOURCE_DIR, worker, 'index.ts');
const outfile = path.join(WORKER_OUTPUT_DIR, `${worker}.js`);
try {
const config: BuildOptions = {
...commonConfig,
entryPoints: [entryPoint],
outfile
};
await build(config);
console.log(`✅ ${worker} 编译成功 → ${path.relative(process.cwd(), outfile)}`);
return { success: true, worker };
} catch (error: any) {
console.error(`❌ ${worker} 编译失败:`, error.message);
return { success: false, worker, error };
}
});
// 等待所有编译完成
const results = await Promise.all(buildPromises);
// 统计结果
const successCount = results.filter((r) => r.success).length;
const failCount = results.filter((r) => !r.success).length;
console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━');
console.log(`✅ 编译成功: ${successCount}/${workers.length}`);
if (failCount > 0) {
console.log(`❌ 编译失败: ${failCount}/${workers.length}`);
const failedWorkers = results.filter((r) => !r.success).map((r) => r.worker);
console.log(`失败的 Worker: ${failedWorkers.join(', ')}`);
// 非监听模式下,如果有失败的编译,退出并返回错误码
process.exit(1);
}
console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━');
}
}
// 解析命令行参数
const args = process.argv.slice(2);
const watch = args.includes('--watch') || args.includes('-w');
// 显示启动信息
console.log('');
console.log('╔═══════════════════════════════════════╗');
console.log('║ FastGPT Worker 预编译工具 v1.0 ║');
console.log('╚═══════════════════════════════════════╝');
console.log('');
// 执行编译
buildWorkers(watch).catch((err) => {
console.error('\n❌ Worker 编译过程发生错误:', err);
process.exit(1);
});
...@@ -21,9 +21,8 @@ import MyBox from '@fastgpt/web/components/common/MyBox'; ...@@ -21,9 +21,8 @@ import MyBox from '@fastgpt/web/components/common/MyBox';
import { getCollectionSourceData } from '@fastgpt/global/core/dataset/collection/utils'; import { getCollectionSourceData } from '@fastgpt/global/core/dataset/collection/utils';
import Markdown from '.'; import Markdown from '.';
import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils'; import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils';
import { Types } from 'mongoose'; import { isObjectId } from '@fastgpt/global/common/string/utils';
import type { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat'; import type { OutLinkChatAuthProps } from '@fastgpt/global/support/permission/chat';
import { useCreation } from 'ahooks';
export type AProps = { export type AProps = {
chatAuthData?: { chatAuthData?: {
...@@ -67,7 +66,7 @@ const CiteLink = React.memo(function CiteLink({ ...@@ -67,7 +66,7 @@ const CiteLink = React.memo(function CiteLink({
const { isOpen, onOpen, onClose } = useDisclosure(); const { isOpen, onOpen, onClose } = useDisclosure();
if (!Types.ObjectId.isValid(id)) { if (!isObjectId(id)) {
return <></>; return <></>;
} }
......
import React, { useCallback, useEffect, useLayoutEffect, useRef, useState } from 'react'; import React, { useCallback, useEffect, useLayoutEffect, useRef, useState } from 'react';
import * as echarts from 'echarts';
import type { ECharts } from 'echarts'; import type { ECharts } from 'echarts';
import { Box, Skeleton } from '@chakra-ui/react'; import { Box, Skeleton } from '@chakra-ui/react';
import json5 from 'json5'; import json5 from 'json5';
...@@ -57,8 +56,10 @@ const EChartsCodeBlock = ({ code }: { code: string }) => { ...@@ -57,8 +56,10 @@ const EChartsCodeBlock = ({ code }: { code: string }) => {
if (chartRef.current) { if (chartRef.current) {
try { try {
eChart.current = echarts.init(chartRef.current); import('echarts').then((module) => {
eChart.current.setOption(option); eChart.current = module.init(chartRef.current!);
eChart.current.setOption(option);
});
} catch (error) { } catch (error) {
console.error('ECharts render failed:', error); console.error('ECharts render failed:', error);
} }
......
import React, { useEffect, useRef, useCallback, useState } from 'react'; import React, { useEffect, useRef, useCallback, useState } from 'react';
import { Box } from '@chakra-ui/react'; import { Box } from '@chakra-ui/react';
import mermaid from 'mermaid';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
const mermaidAPI = mermaid.mermaidAPI;
mermaidAPI.initialize({
startOnLoad: true,
theme: 'base',
flowchart: {
useMaxWidth: false
},
themeVariables: {
fontSize: '14px',
primaryColor: '#d6e8ff',
primaryTextColor: '#485058',
primaryBorderColor: '#fff',
lineColor: '#5A646E',
secondaryColor: '#B5E9E5',
tertiaryColor: '#485058'
}
});
const punctuationMap: Record<string, string> = { const punctuationMap: Record<string, string> = {
',': ',', ',': ',',
';': ';', ';': ';',
...@@ -44,10 +25,52 @@ const punctuationMap: Record<string, string> = { ...@@ -44,10 +25,52 @@ const punctuationMap: Record<string, string> = {
const MermaidBlock = ({ code }: { code: string }) => { const MermaidBlock = ({ code }: { code: string }) => {
const ref = useRef<HTMLDivElement>(null); const ref = useRef<HTMLDivElement>(null);
const [svg, setSvg] = useState(''); const [svg, setSvg] = useState('');
const [mermaid, setMermaid] = useState<any>(null);
const [isLoading, setIsLoading] = useState(true);
const [error, setError] = useState<string>('');
useEffect(() => {
let mounted = true;
import('mermaid')
.then((module) => {
if (!mounted) return;
const mermaidInstance = module.default;
mermaidInstance.mermaidAPI.initialize({
startOnLoad: true,
theme: 'base',
flowchart: {
useMaxWidth: false
},
themeVariables: {
fontSize: '14px',
primaryColor: '#d6e8ff',
primaryTextColor: '#485058',
primaryBorderColor: '#fff',
lineColor: '#5A646E',
secondaryColor: '#B5E9E5',
tertiaryColor: '#485058'
}
});
setMermaid(mermaidInstance);
setIsLoading(false);
})
.catch((error) => {
console.error('Failed to load mermaid:', error);
setIsLoading(false);
});
return () => {
mounted = false;
};
}, []);
useEffect(() => { useEffect(() => {
(async () => { (async () => {
if (!code) return; if (!code || !mermaid || isLoading) return;
try { try {
const formatCode = code.replace( const formatCode = code.replace(
new RegExp(`[${Object.keys(punctuationMap).join('')}]`, 'g'), new RegExp(`[${Object.keys(punctuationMap).join('')}]`, 'g'),
...@@ -56,16 +79,16 @@ const MermaidBlock = ({ code }: { code: string }) => { ...@@ -56,16 +79,16 @@ const MermaidBlock = ({ code }: { code: string }) => {
const { svg } = await mermaid.render(`mermaid-${Date.now()}`, formatCode); const { svg } = await mermaid.render(`mermaid-${Date.now()}`, formatCode);
setSvg(svg); setSvg(svg);
} catch (e: any) { } catch (e: any) {
// console.log('[Mermaid] ', e?.message); console.log('[Mermaid] ', e?.message);
} }
})(); })();
}, [code]); }, [code, isLoading, mermaid]);
const onclickExport = useCallback(() => { const onclickExport = useCallback(() => {
const svg = ref.current?.children[0]; const svgElement = ref.current?.children[0];
if (!svg) return; if (!svgElement) return;
const rate = svg.clientHeight / svg.clientWidth; const rate = svgElement.clientHeight / svgElement.clientWidth;
const w = 3000; const w = 3000;
const h = rate * w; const h = rate * w;
...@@ -74,12 +97,13 @@ const MermaidBlock = ({ code }: { code: string }) => { ...@@ -74,12 +97,13 @@ const MermaidBlock = ({ code }: { code: string }) => {
canvas.height = h; canvas.height = h;
const ctx = canvas.getContext('2d'); const ctx = canvas.getContext('2d');
if (!ctx) return; if (!ctx) return;
// 绘制白色背景
ctx.fillStyle = '#fff'; ctx.fillStyle = '#fff';
ctx.fillRect(0, 0, w, h); ctx.fillRect(0, 0, w, h);
const img = new Image(); const img = new Image();
img.src = `data:image/svg+xml;charset=utf-8,${encodeURIComponent(ref.current?.innerHTML)}`; const innerHTML = ref.current?.innerHTML || '';
img.src = 'data:image/svg+xml;charset=utf-8,' + encodeURIComponent(innerHTML);
img.onload = () => { img.onload = () => {
ctx.drawImage(img, 0, 0, w, h); ctx.drawImage(img, 0, 0, w, h);
...@@ -97,6 +121,31 @@ const MermaidBlock = ({ code }: { code: string }) => { ...@@ -97,6 +121,31 @@ const MermaidBlock = ({ code }: { code: string }) => {
}; };
}, []); }, []);
if (isLoading) {
return (
<Box
minW={'100px'}
minH={'50px'}
py={4}
bg={'gray.50'}
borderRadius={'md'}
textAlign={'center'}
>
Loading...
</Box>
);
}
if (error) {
return (
<Box minW={'100px'} minH={'50px'} py={4} bg={'red.50'} borderRadius={'md'} p={3}>
<Box color={'red.600'} fontSize={'sm'}>
{error}
</Box>
</Box>
);
}
return ( return (
<Box <Box
position={'relative'} position={'relative'}
......
import React from 'react'; import React, { useCallback } from 'react';
import { ModalFooter, ModalBody, Input, Button, Box, Textarea, HStack } from '@chakra-ui/react'; import { ModalFooter, ModalBody, Input, Button, Box, Textarea, HStack } from '@chakra-ui/react';
import MyModal from '@fastgpt/web/components/common/MyModal/index'; import MyModal from '@fastgpt/web/components/common/MyModal/index';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { useRequest2 } from '@fastgpt/web/hooks/useRequest'; import { useRequest2 } from '@fastgpt/web/hooks/useRequest';
import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel'; import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import { useSelectFile } from '@/web/common/file/hooks/useSelectFile';
import MyTooltip from '@fastgpt/web/components/common/MyTooltip'; import MyTooltip from '@fastgpt/web/components/common/MyTooltip';
import Avatar from '@fastgpt/web/components/common/Avatar'; import Avatar from '@fastgpt/web/components/common/Avatar';
import { useUploadAvatar } from '@fastgpt/web/common/file/hooks/useUploadAvatar';
import { getUploadAvatarPresignedUrl } from '@/web/common/file/api';
export type EditResourceInfoFormType = { export type EditResourceInfoFormType = {
id: string; id: string;
...@@ -41,14 +42,14 @@ const EditResourceModal = ({ ...@@ -41,14 +42,14 @@ const EditResourceModal = ({
} }
); );
const { const afterUploadAvatar = useCallback(
File, (avatar: string) => {
onOpen: onOpenSelectFile, setValue('avatar', avatar);
onSelectImage },
} = useSelectFile({ [setValue]
fileType: '.jpg,.png', );
multiple: false const { Component: AvatarUploader, handleFileSelectorOpen: handleAvatarSelectorOpen } =
}); useUploadAvatar(getUploadAvatarPresignedUrl, { onSuccess: afterUploadAvatar });
return ( return (
<MyModal isOpen onClose={onClose} iconSrc={avatar} title={title}> <MyModal isOpen onClose={onClose} iconSrc={avatar} title={title}>
...@@ -64,7 +65,7 @@ const EditResourceModal = ({ ...@@ -64,7 +65,7 @@ const EditResourceModal = ({
h={'2rem'} h={'2rem'}
cursor={'pointer'} cursor={'pointer'}
borderRadius={'sm'} borderRadius={'sm'}
onClick={onOpenSelectFile} onClick={handleAvatarSelectorOpen}
/> />
</MyTooltip> </MyTooltip>
<Input <Input
...@@ -86,15 +87,7 @@ const EditResourceModal = ({ ...@@ -86,15 +87,7 @@ const EditResourceModal = ({
</Button> </Button>
</ModalFooter> </ModalFooter>
<File <AvatarUploader />
onSelect={(e) =>
onSelectImage(e, {
maxH: 300,
maxW: 300,
callback: (e) => setValue('avatar', e)
})
}
/>
</MyModal> </MyModal>
); );
}; };
......
...@@ -62,7 +62,8 @@ const LabelAndFormRender = ({ ...@@ -62,7 +62,8 @@ const LabelAndFormRender = ({
name={props.fieldName} name={props.fieldName}
rules={{ rules={{
validate: (value) => { validate: (value) => {
if (!required || inputType === InputTypeEnum.switch) return true; if (!required) return true;
if (typeof value === 'number' || typeof value === 'boolean') return true;
return !!value; return !!value;
}, },
...(!!props?.minLength ...(!!props?.minLength
......
...@@ -24,7 +24,7 @@ import { ChatItemContext } from '@/web/core/chat/context/chatItemContext'; ...@@ -24,7 +24,7 @@ import { ChatItemContext } from '@/web/core/chat/context/chatItemContext';
import { ChatRecordContext } from '@/web/core/chat/context/chatRecordContext'; import { ChatRecordContext } from '@/web/core/chat/context/chatRecordContext';
import { useCreation } from 'ahooks'; import { useCreation } from 'ahooks';
import type { ChatTypeEnum } from './constants'; import type { ChatTypeEnum } from './constants';
import type { QuickAppType } from '@fastgpt/global/core/chat/setting/type'; import type { ChatQuickAppType } from '@fastgpt/global/core/chat/setting/type';
export type ChatProviderProps = { export type ChatProviderProps = {
appId: string; appId: string;
...@@ -39,7 +39,7 @@ export type ChatProviderProps = { ...@@ -39,7 +39,7 @@ export type ChatProviderProps = {
slogan?: string; slogan?: string;
currentQuickAppId?: string; currentQuickAppId?: string;
quickAppList?: QuickAppType[]; quickAppList?: ChatQuickAppType[];
onSwitchQuickApp?: (appId: string) => Promise<void>; onSwitchQuickApp?: (appId: string) => Promise<void>;
}; };
......
...@@ -21,7 +21,8 @@ export async function register() { ...@@ -21,7 +21,8 @@ export async function register() {
{ loadSystemModels }, { loadSystemModels },
{ connectSignoz }, { connectSignoz },
{ getSystemTools }, { getSystemTools },
{ trackTimerProcess } { trackTimerProcess },
{ initS3Buckets }
] = await Promise.all([ ] = await Promise.all([
import('@fastgpt/service/common/mongo/init'), import('@fastgpt/service/common/mongo/init'),
import('@fastgpt/service/common/mongo/index'), import('@fastgpt/service/common/mongo/index'),
...@@ -36,7 +37,8 @@ export async function register() { ...@@ -36,7 +37,8 @@ export async function register() {
import('@fastgpt/service/core/ai/config/utils'), import('@fastgpt/service/core/ai/config/utils'),
import('@fastgpt/service/common/otel/trace/register'), import('@fastgpt/service/common/otel/trace/register'),
import('@fastgpt/service/core/app/plugin/controller'), import('@fastgpt/service/core/app/plugin/controller'),
import('@fastgpt/service/common/middle/tracks/processor') import('@fastgpt/service/common/middle/tracks/processor'),
import('@fastgpt/service/common/s3')
]); ]);
// connect to signoz // connect to signoz
...@@ -46,9 +48,19 @@ export async function register() { ...@@ -46,9 +48,19 @@ export async function register() {
systemStartCb(); systemStartCb();
initGlobalVariables(); initGlobalVariables();
// init s3 buckets
initS3Buckets();
// Connect to MongoDB // Connect to MongoDB
await connectMongo(connectionMongo, MONGO_URL); await connectMongo({
connectMongo(connectionLogMongo, MONGO_LOG_URL); db: connectionMongo,
url: MONGO_URL,
connectedCb: () => startMongoWatch()
});
connectMongo({
db: connectionLogMongo,
url: MONGO_LOG_URL
});
//init system config;init vector database;init root user //init system config;init vector database;init root user
await Promise.all([getInitConfig(), initVectorStore(), initRootUser(), loadSystemModels()]); await Promise.all([getInitConfig(), initVectorStore(), initRootUser(), loadSystemModels()]);
...@@ -60,7 +72,6 @@ export async function register() { ...@@ -60,7 +72,6 @@ export async function register() {
initAppTemplateTypes() initAppTemplateTypes()
]); ]);
startMongoWatch();
startCron(); startCron();
startTrainingQueue(true); startTrainingQueue(true);
trackTimerProcess(); trackTimerProcess();
......
import React from 'react'; import React, { useCallback } from 'react';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { useSelectFile } from '@/web/common/file/hooks/useSelectFile';
import { useRequest } from '@fastgpt/web/hooks/useRequest'; import { useRequest } from '@fastgpt/web/hooks/useRequest';
import MyModal from '@fastgpt/web/components/common/MyModal'; import MyModal from '@fastgpt/web/components/common/MyModal';
import { import {
...@@ -21,6 +20,8 @@ import { type CreateTeamProps } from '@fastgpt/global/support/user/team/controll ...@@ -21,6 +20,8 @@ import { type CreateTeamProps } from '@fastgpt/global/support/user/team/controll
import { DEFAULT_TEAM_AVATAR } from '@fastgpt/global/common/system/constants'; import { DEFAULT_TEAM_AVATAR } from '@fastgpt/global/common/system/constants';
import Icon from '@fastgpt/web/components/common/Icon'; import Icon from '@fastgpt/web/components/common/Icon';
import dynamic from 'next/dynamic'; import dynamic from 'next/dynamic';
import { useUploadAvatar } from '@fastgpt/web/common/file/hooks/useUploadAvatar';
import { getUploadAvatarPresignedUrl } from '@/web/common/file/api';
const UpdateContact = dynamic(() => import('@/components/support/user/inform/UpdateContactModal')); const UpdateContact = dynamic(() => import('@/components/support/user/inform/UpdateContactModal'));
export type EditTeamFormDataType = CreateTeamProps & { export type EditTeamFormDataType = CreateTeamProps & {
...@@ -50,15 +51,6 @@ function EditModal({ ...@@ -50,15 +51,6 @@ function EditModal({
const avatar = watch('avatar'); const avatar = watch('avatar');
const notificationAccount = watch('notificationAccount'); const notificationAccount = watch('notificationAccount');
const {
File,
onOpen: onOpenSelectFile,
onSelectImage
} = useSelectFile({
fileType: '.jpg,.png,.svg',
multiple: false
});
const { mutate: onclickCreate, isLoading: creating } = useRequest({ const { mutate: onclickCreate, isLoading: creating } = useRequest({
mutationFn: async (data: CreateTeamProps) => { mutationFn: async (data: CreateTeamProps) => {
return postCreateTeam(data); return postCreateTeam(data);
...@@ -88,6 +80,19 @@ function EditModal({ ...@@ -88,6 +80,19 @@ function EditModal({
const { isOpen: isOpenContact, onClose: onCloseContact, onOpen: onOpenContact } = useDisclosure(); const { isOpen: isOpenContact, onClose: onCloseContact, onOpen: onOpenContact } = useDisclosure();
const afterUploadAvatar = useCallback(
(avatar: string) => {
setValue('avatar', avatar);
},
[setValue]
);
const { Component: AvatarUploader, handleFileSelectorOpen } = useUploadAvatar(
getUploadAvatarPresignedUrl,
{
onSuccess: afterUploadAvatar
}
);
return ( return (
<MyModal <MyModal
isOpen isOpen
...@@ -100,6 +105,7 @@ function EditModal({ ...@@ -100,6 +105,7 @@ function EditModal({
<Box color={'myGray.800'} fontWeight={'bold'}> <Box color={'myGray.800'} fontWeight={'bold'}>
{t('account_team:set_name_avatar')} {t('account_team:set_name_avatar')}
</Box> </Box>
<AvatarUploader />
<Flex mt={3} alignItems={'center'}> <Flex mt={3} alignItems={'center'}>
<MyTooltip label={t('common:set_avatar')}> <MyTooltip label={t('common:set_avatar')}>
<Avatar <Avatar
...@@ -109,7 +115,7 @@ function EditModal({ ...@@ -109,7 +115,7 @@ function EditModal({
h={['28px', '32px']} h={['28px', '32px']}
cursor={'pointer'} cursor={'pointer'}
borderRadius={'md'} borderRadius={'md'}
onClick={onOpenSelectFile} onClick={handleFileSelectorOpen}
/> />
</MyTooltip> </MyTooltip>
<Input <Input
...@@ -179,15 +185,6 @@ function EditModal({ ...@@ -179,15 +185,6 @@ function EditModal({
</Button> </Button>
)} )}
</ModalFooter> </ModalFooter>
<File
onSelect={(e) =>
onSelectImage(e, {
maxH: 300,
maxW: 300,
callback: (e) => setValue('avatar', e)
})
}
/>
{isOpenContact && ( {isOpenContact && (
<UpdateContact <UpdateContact
onClose={onCloseContact} onClose={onCloseContact}
......
...@@ -4,12 +4,13 @@ import Avatar from '@fastgpt/web/components/common/Avatar'; ...@@ -4,12 +4,13 @@ import Avatar from '@fastgpt/web/components/common/Avatar';
import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel'; import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import React from 'react'; import React from 'react';
import { useSelectFile } from '@/web/common/file/hooks/useSelectFile';
import { useRequest2 } from '@fastgpt/web/hooks/useRequest'; import { useRequest2 } from '@fastgpt/web/hooks/useRequest';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import { postCreateGroup, putUpdateGroup } from '@/web/support/user/team/group/api'; import { postCreateGroup, putUpdateGroup } from '@/web/support/user/team/group/api';
import { DEFAULT_TEAM_AVATAR } from '@fastgpt/global/common/system/constants'; import { DEFAULT_TEAM_AVATAR } from '@fastgpt/global/common/system/constants';
import { type MemberGroupListItemType } from '@fastgpt/global/support/permission/memberGroup/type'; import { type MemberGroupListItemType } from '@fastgpt/global/support/permission/memberGroup/type';
import { useUploadAvatar } from '@fastgpt/web/common/file/hooks/useUploadAvatar';
import { getUploadAvatarPresignedUrl } from '@/web/common/file/api';
export type GroupFormType = { export type GroupFormType = {
avatar: string; avatar: string;
...@@ -28,12 +29,13 @@ function GroupInfoModal({ ...@@ -28,12 +29,13 @@ function GroupInfoModal({
const { t } = useTranslation(); const { t } = useTranslation();
const { const {
File: AvatarSelect, Component: AvatarUploader,
onOpen: onOpenSelectAvatar, handleFileSelectorOpen: handleAvatarSelectorOpen,
onSelectImage uploading: uploadingAvatar
} = useSelectFile({ } = useUploadAvatar(getUploadAvatarPresignedUrl, {
fileType: '.jpg, .jpeg, .png', onSuccess: (avatar: string) => {
multiple: false setValue('avatar', avatar);
}
}); });
const { register, handleSubmit, getValues, setValue } = useForm<GroupFormType>({ const { register, handleSubmit, getValues, setValue } = useForm<GroupFormType>({
...@@ -43,20 +45,6 @@ function GroupInfoModal({ ...@@ -43,20 +45,6 @@ function GroupInfoModal({
} }
}); });
const { loading: uploadingAvatar, run: onSelectAvatar } = useRequest2(
async (file: File[]) => {
return onSelectImage(file, {
maxW: 300,
maxH: 300
});
},
{
onSuccess: (src: string) => {
setValue('avatar', src);
}
}
);
const { runAsync: onCreate, loading: isLoadingCreate } = useRequest2( const { runAsync: onCreate, loading: isLoadingCreate } = useRequest2(
(data: GroupFormType) => { (data: GroupFormType) => {
return postCreateGroup({ return postCreateGroup({
...@@ -96,7 +84,7 @@ function GroupInfoModal({ ...@@ -96,7 +84,7 @@ function GroupInfoModal({
<HStack> <HStack>
<Avatar <Avatar
src={getValues('avatar')} src={getValues('avatar')}
onClick={onOpenSelectAvatar} onClick={handleAvatarSelectorOpen}
cursor={'pointer'} cursor={'pointer'}
borderRadius={'md'} borderRadius={'md'}
/> />
...@@ -121,7 +109,8 @@ function GroupInfoModal({ ...@@ -121,7 +109,8 @@ function GroupInfoModal({
{editGroup ? t('common:Save') : t('common:new_create')} {editGroup ? t('common:Save') : t('common:new_create')}
</Button> </Button>
</ModalFooter> </ModalFooter>
<AvatarSelect onSelect={onSelectAvatar} /> {/* <AvatarSelect onSelect={onSelectAvatar} /> */}
<AvatarUploader />
</MyModal> </MyModal>
); );
} }
......
import { useSelectFile } from '@/web/common/file/hooks/useSelectFile'; import { getUploadAvatarPresignedUrl } from '@/web/common/file/api';
import { postCreateOrg, putUpdateOrg } from '@/web/support/user/team/org/api'; import { postCreateOrg, putUpdateOrg } from '@/web/support/user/team/org/api';
import { Button, HStack, Input, ModalBody, ModalFooter, Textarea } from '@chakra-ui/react'; import { Button, HStack, Input, ModalBody, ModalFooter, Textarea } from '@chakra-ui/react';
import { DEFAULT_ORG_AVATAR } from '@fastgpt/global/common/system/constants'; import { DEFAULT_ORG_AVATAR } from '@fastgpt/global/common/system/constants';
import { useUploadAvatar } from '@fastgpt/web/common/file/hooks/useUploadAvatar';
import Avatar from '@fastgpt/web/components/common/Avatar'; import Avatar from '@fastgpt/web/components/common/Avatar';
import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel'; import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel';
import MyModal from '@fastgpt/web/components/common/MyModal'; import MyModal from '@fastgpt/web/components/common/MyModal';
...@@ -90,26 +91,14 @@ function OrgInfoModal({ ...@@ -90,26 +91,14 @@ function OrgInfoModal({
); );
const { const {
File: AvatarSelect, Component: AvatarUploader,
onOpen: onOpenSelectAvatar, uploading: uploadingAvatar,
onSelectImage handleFileSelectorOpen: handleAvatarSelectorOpen
} = useSelectFile({ } = useUploadAvatar(getUploadAvatarPresignedUrl, {
fileType: '.jpg, .jpeg, .png', onSuccess: (avatar) => {
multiple: false setValue('avatar', avatar);
});
const { loading: uploadingAvatar, run: onSelectAvatar } = useRequest2(
async (file: File[]) => {
return onSelectImage(file, {
maxW: 300,
maxH: 300
});
},
{
onSuccess: (src: string) => {
setValue('avatar', src);
}
} }
); });
const isLoading = uploadingAvatar || isLoadingUpdate || isLoadingCreate; const isLoading = uploadingAvatar || isLoadingUpdate || isLoadingCreate;
...@@ -125,7 +114,7 @@ function OrgInfoModal({ ...@@ -125,7 +114,7 @@ function OrgInfoModal({
<HStack> <HStack>
<Avatar <Avatar
src={avatar || DEFAULT_ORG_AVATAR} src={avatar || DEFAULT_ORG_AVATAR}
onClick={onOpenSelectAvatar} onClick={handleAvatarSelectorOpen}
cursor={'pointer'} cursor={'pointer'}
borderRadius={'md'} borderRadius={'md'}
/> />
...@@ -158,7 +147,7 @@ function OrgInfoModal({ ...@@ -158,7 +147,7 @@ function OrgInfoModal({
{isEdit ? t('common:Save') : t('common:new_create')} {isEdit ? t('common:Save') : t('common:new_create')}
</Button> </Button>
</ModalFooter> </ModalFooter>
<AvatarSelect onSelect={onSelectAvatar} /> <AvatarUploader />
</MyModal> </MyModal>
); );
} }
......
import { getDashboardData } from '@/web/support/wallet/usage/api'; import { getDashboardData } from '@/web/support/wallet/usage/api';
import { Box, Flex } from '@chakra-ui/react'; import { Box } from '@chakra-ui/react';
import { formatNumber } from '@fastgpt/global/common/math/tools';
import MyBox from '@fastgpt/web/components/common/MyBox'; import MyBox from '@fastgpt/web/components/common/MyBox';
import { useRequest2 } from '@fastgpt/web/hooks/useRequest'; import { useRequest2 } from '@fastgpt/web/hooks/useRequest';
import { addDays } from 'date-fns'; import { addDays } from 'date-fns';
import { useTranslation } from 'next-i18next';
import React, { useMemo } from 'react'; import React, { useMemo } from 'react';
import { import { type UsageFilterParams } from './type';
ResponsiveContainer,
LineChart,
Line,
XAxis,
YAxis,
CartesianGrid,
Tooltip,
type TooltipProps
} from 'recharts';
import { type NameType, type ValueType } from 'recharts/types/component/DefaultTooltipContent';
import { type UnitType, type UsageFilterParams } from './type';
import dayjs from 'dayjs'; import dayjs from 'dayjs';
import dynamic from 'next/dynamic';
export type usageFormType = { const DashboardChart = dynamic(() => import('./DashboardChart'), {
date: string; ssr: false
totalPoints: number; });
};
const CustomTooltip = ({ active, payload }: TooltipProps<ValueType, NameType>) => {
const data = payload?.[0]?.payload as usageFormType;
const { t } = useTranslation();
if (active && data) {
return (
<Box
bg={'white'}
p={3}
borderRadius={'md'}
border={'0.5px solid'}
borderColor={'myGray.200'}
boxShadow={
'0px 24px 48px -12px rgba(19, 51, 107, 0.20), 0px 0px 1px 0px rgba(19, 51, 107, 0.20)'
}
>
<Box fontSize={'mini'} color={'myGray.600'} mb={3}>
{data.date}
</Box>
<Box fontSize={'14px'} color={'myGray.900'} fontWeight={'medium'}>
{`${formatNumber(data.totalPoints)} ${t('account_usage:points')}`}
</Box>
</Box>
);
}
return null;
};
const UsageDashboard = ({ const UsageDashboard = ({
filterParams, filterParams,
...@@ -61,8 +21,6 @@ const UsageDashboard = ({ ...@@ -61,8 +21,6 @@ const UsageDashboard = ({
Tabs: React.ReactNode; Tabs: React.ReactNode;
Selectors: React.ReactNode; Selectors: React.ReactNode;
}) => { }) => {
const { t } = useTranslation();
const { dateRange, selectTmbIds, usageSources, unit, isSelectAllSource, isSelectAllTmb } = const { dateRange, selectTmbIds, usageSources, unit, isSelectAllSource, isSelectAllTmb } =
filterParams; filterParams;
...@@ -99,41 +57,7 @@ const UsageDashboard = ({ ...@@ -99,41 +57,7 @@ const UsageDashboard = ({
<Box>{Tabs}</Box> <Box>{Tabs}</Box>
<Box mt={4}>{Selectors}</Box> <Box mt={4}>{Selectors}</Box>
<MyBox overflowY={'auto'} isLoading={totalPointsLoading}> <MyBox overflowY={'auto'} isLoading={totalPointsLoading}>
<Flex fontSize={'20px'} fontWeight={'medium'} my={6}> <DashboardChart totalPoints={totalPoints} totalUsage={totalUsage} />
<Box color={'black'}>{`${t('account_usage:total_usage')}:`}</Box>
<Box color={'primary.600'} ml={2}>
{`${formatNumber(totalUsage)} ${t('account_usage:points')}`}
</Box>
</Flex>
<Flex mb={4} fontSize={'mini'} color={'myGray.500'} fontWeight={'medium'}>
{t('account_usage:points')}
</Flex>
<ResponsiveContainer width="100%" height={424}>
<LineChart data={totalPoints} margin={{ top: 10, right: 30, left: -12, bottom: 0 }}>
<XAxis
dataKey="date"
padding={{ left: 40, right: 40 }}
tickMargin={10}
tickSize={0}
tick={{ fontSize: '12px', color: '#667085', fontWeight: '500' }}
/>
<YAxis
axisLine={false}
tickSize={0}
tickMargin={12}
tick={{ fontSize: '12px', color: '#667085', fontWeight: '500' }}
/>
<CartesianGrid strokeDasharray="3 3" horizontal={true} vertical={false} />
<Tooltip content={<CustomTooltip />} />
<Line
type="monotone"
dataKey="totalPoints"
stroke="#5E8FFF"
strokeWidth={2.5}
dot={false}
/>
</LineChart>
</ResponsiveContainer>
</MyBox> </MyBox>
</> </>
); );
......
import React, { useEffect, useState } from 'react';
import { Box, Flex, Skeleton } from '@chakra-ui/react';
import { formatNumber } from '@fastgpt/global/common/math/tools';
import { type NameType, type ValueType } from 'recharts/types/component/DefaultTooltipContent';
import type { TooltipProps } from 'recharts';
import { useTranslation } from 'next-i18next';
export type usageFormType = {
date: string;
totalPoints: number;
};
type RechartsComponents = {
ResponsiveContainer: any;
LineChart: any;
Line: any;
XAxis: any;
YAxis: any;
CartesianGrid: any;
Tooltip: any;
};
const CustomTooltip = ({ active, payload }: TooltipProps<ValueType, NameType>) => {
const data = payload?.[0]?.payload as usageFormType;
const { t } = useTranslation();
if (active && data) {
return (
<Box
bg={'white'}
p={3}
borderRadius={'md'}
border={'0.5px solid'}
borderColor={'myGray.200'}
boxShadow={
'0px 24px 48px -12px rgba(19, 51, 107, 0.20), 0px 0px 1px 0px rgba(19, 51, 107, 0.20)'
}
>
<Box fontSize={'mini'} color={'myGray.600'} mb={3}>
{data.date}
</Box>
<Box fontSize={'14px'} color={'myGray.900'} fontWeight={'medium'}>
{`${formatNumber(data.totalPoints)} ${t('account_usage:points')}`}
</Box>
</Box>
);
}
return null;
};
const DashboardChart = ({
totalPoints,
totalUsage
}: {
totalPoints: usageFormType[];
totalUsage: number;
}) => {
const { t } = useTranslation();
const [recharts, setRecharts] = useState<RechartsComponents | null>(null);
const [isLoading, setIsLoading] = useState(true);
const [error, setError] = useState<string>('');
// 动态导入 recharts
useEffect(() => {
let mounted = true;
import('recharts')
.then((module) => {
if (!mounted) return;
setRecharts({
ResponsiveContainer: module.ResponsiveContainer,
LineChart: module.LineChart,
Line: module.Line,
XAxis: module.XAxis,
YAxis: module.YAxis,
CartesianGrid: module.CartesianGrid,
Tooltip: module.Tooltip
});
setIsLoading(false);
})
.catch((error) => {
console.error('Failed to load recharts:', error);
setError('加载图表库失败');
setIsLoading(false);
});
return () => {
mounted = false;
};
}, []);
// 加载状态
if (isLoading) {
return (
<Box>
<Flex fontSize={'20px'} fontWeight={'medium'} my={6}>
<Box color={'black'}>{`${t('account_usage:total_usage')}:`}</Box>
<Box color={'primary.600'} ml={2}>
{`${formatNumber(totalUsage)} ${t('account_usage:points')}`}
</Box>
</Flex>
<Flex mb={4} fontSize={'mini'} color={'myGray.500'} fontWeight={'medium'}>
{t('account_usage:points')}
</Flex>
<Skeleton height="424px" width="100%" borderRadius={'md'} />
</Box>
);
}
// 错误状态
if (error || !recharts) {
return (
<Box>
<Flex fontSize={'20px'} fontWeight={'medium'} my={6}>
<Box color={'black'}>{`${t('account_usage:total_usage')}:`}</Box>
<Box color={'primary.600'} ml={2}>
{`${formatNumber(totalUsage)} ${t('account_usage:points')}`}
</Box>
</Flex>
<Box minH={'424px'} py={4} bg={'red.50'} borderRadius={'md'} p={3}>
<Box color={'red.600'} fontSize={'sm'}>
{error || '图表加载失败'}
</Box>
</Box>
</Box>
);
}
const { ResponsiveContainer, LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip } = recharts;
return (
<>
<Flex fontSize={'20px'} fontWeight={'medium'} my={6}>
<Box color={'black'}>{`${t('account_usage:total_usage')}:`}</Box>
<Box color={'primary.600'} ml={2}>
{`${formatNumber(totalUsage)} ${t('account_usage:points')}`}
</Box>
</Flex>
<Flex mb={4} fontSize={'mini'} color={'myGray.500'} fontWeight={'medium'}>
{t('account_usage:points')}
</Flex>
<ResponsiveContainer width="100%" height={424}>
<LineChart data={totalPoints} margin={{ top: 10, right: 30, left: -12, bottom: 0 }}>
<XAxis
dataKey="date"
padding={{ left: 40, right: 40 }}
tickMargin={10}
tickSize={0}
tick={{ fontSize: '12px', color: '#667085', fontWeight: '500' }}
/>
<YAxis
axisLine={false}
tickSize={0}
tickMargin={12}
tick={{ fontSize: '12px', color: '#667085', fontWeight: '500' }}
/>
<CartesianGrid strokeDasharray="3 3" horizontal={true} vertical={false} />
<Tooltip content={<CustomTooltip />} />
<Line
type="monotone"
dataKey="totalPoints"
stroke="#5E8FFF"
strokeWidth={2.5}
dot={false}
/>
</LineChart>
</ResponsiveContainer>
</>
);
};
export default DashboardChart;
...@@ -4,7 +4,7 @@ import { useContextSelector } from 'use-context-selector'; ...@@ -4,7 +4,7 @@ import { useContextSelector } from 'use-context-selector';
import { AppContext } from '../context'; import { AppContext } from '../context';
import ChatItemContextProvider from '@/web/core/chat/context/chatItemContext'; import ChatItemContextProvider from '@/web/core/chat/context/chatItemContext';
import ChatRecordContextProvider from '@/web/core/chat/context/chatRecordContext'; import ChatRecordContextProvider from '@/web/core/chat/context/chatRecordContext';
import { Box, Button, Flex, HStack } from '@chakra-ui/react'; import { Box, Button, Center, Flex, HStack } from '@chakra-ui/react';
import { cardStyles } from '../constants'; import { cardStyles } from '../constants';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { type HttpToolConfigType } from '@fastgpt/global/core/app/type'; import { type HttpToolConfigType } from '@fastgpt/global/core/app/type';
...@@ -19,6 +19,7 @@ import LabelAndFormRender from '@/components/core/app/formRender/LabelAndForm'; ...@@ -19,6 +19,7 @@ import LabelAndFormRender from '@/components/core/app/formRender/LabelAndForm';
import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel'; import FormLabel from '@fastgpt/web/components/common/MyBox/FormLabel';
import ValueTypeLabel from '../WorkflowComponents/Flow/nodes/render/ValueTypeLabel'; import ValueTypeLabel from '../WorkflowComponents/Flow/nodes/render/ValueTypeLabel';
import LightRowTabs from '@fastgpt/web/components/common/Tabs/LightRowTabs'; import LightRowTabs from '@fastgpt/web/components/common/Tabs/LightRowTabs';
import EmptyTip from '@fastgpt/web/components/common/EmptyTip';
const ChatTest = ({ const ChatTest = ({
currentTool, currentTool,
...@@ -52,14 +53,16 @@ const ChatTest = ({ ...@@ -52,14 +53,16 @@ const ChatTest = ({
const { runAsync: runTool, loading: isRunning } = useRequest2( const { runAsync: runTool, loading: isRunning } = useRequest2(
async (data: Record<string, any>) => { async (data: Record<string, any>) => {
if (!currentTool) return; if (!currentTool) return;
return postRunHTTPTool({
return await postRunHTTPTool({
baseUrl, baseUrl,
params: data, params: data,
headerSecret, headerSecret: currentTool.headerSecret || headerSecret,
toolPath: currentTool.path, toolPath: currentTool.path,
method: currentTool.method, method: currentTool.method,
customHeaders: customHeaders customHeaders: customHeaders,
staticParams: currentTool.staticParams,
staticHeaders: currentTool.staticHeaders,
staticBody: currentTool.staticBody
}); });
}, },
{ {
...@@ -74,6 +77,7 @@ const ChatTest = ({ ...@@ -74,6 +77,7 @@ const ChatTest = ({
} }
} }
); );
console.log(currentTool);
return ( return (
<Flex h={'full'} gap={2}> <Flex h={'full'} gap={2}>
...@@ -95,72 +99,81 @@ const ChatTest = ({ ...@@ -95,72 +99,81 @@ const ChatTest = ({
<Box flex={1} /> <Box flex={1} />
</Flex> </Flex>
<Box px={[2, 5]} mb={6}> {!currentTool ? (
<LightRowTabs <Center>
list={tabList} <EmptyTip text={t('app:empty_tool_tips')} />
value={activeTab} </Center>
onChange={(value) => {
setActiveTab(value);
}}
/>
</Box>
{activeTab === 'input' ? (
<Box flex={1} px={[2, 5]} overflow={'auto'}>
{Object.keys(currentTool?.inputSchema.properties || {}).length > 0 ? (
<>
<Box border={'1px solid'} borderColor={'myGray.200'} borderRadius={'8px'} p={3}>
{Object.entries(currentTool?.inputSchema.properties || {}).map(
([paramName, paramInfo]) => {
const inputType = valueTypeToInputType(
getNodeInputTypeFromSchemaInputType({ type: paramInfo.type })
);
const required = currentTool?.inputSchema.required?.includes(paramName);
return (
<LabelAndFormRender
label={
<HStack spacing={0} mr={2}>
<FormLabel required={required}>{paramName}</FormLabel>
<ValueTypeLabel
valueType={getNodeInputTypeFromSchemaInputType({
type: paramInfo.type,
arrayItems: paramInfo.items
})}
h={'auto'}
/>
</HStack>
}
required={required}
key={paramName}
inputType={inputType}
fieldName={paramName}
form={form}
placeholder={paramName}
/>
);
}
)}
</Box>
</>
) : (
<Box fontWeight={'medium'} pb={4} px={2}>
{t('app:this_tool_requires_no_input')}
</Box>
)}
<Button mt={3} isLoading={isRunning} onClick={handleSubmit(runTool)}>
{t('common:Run')}
</Button>
</Box>
) : ( ) : (
<Box flex={1} px={[2, 5]} overflow={'auto'}> <>
{output && ( <Box px={[2, 5]} mb={6}>
<Box> <LightRowTabs
<Markdown source={`~~~json\n${output}`} /> gap={4}
list={tabList}
value={activeTab}
onChange={(value) => {
setActiveTab(value);
}}
/>
</Box>
{activeTab === 'input' ? (
<Box flex={1} px={[2, 5]} overflow={'auto'}>
{Object.keys(currentTool?.inputSchema.properties || {}).length > 0 ? (
<>
<Box border={'1px solid'} borderColor={'myGray.200'} borderRadius={'8px'} p={3}>
{Object.entries(currentTool?.inputSchema.properties || {}).map(
([paramName, paramInfo]) => {
const inputType = valueTypeToInputType(
getNodeInputTypeFromSchemaInputType({ type: paramInfo.type })
);
const required = currentTool?.inputSchema.required?.includes(paramName);
return (
<LabelAndFormRender
label={
<HStack spacing={0} mr={2}>
<FormLabel required={required}>{paramName}</FormLabel>
<ValueTypeLabel
valueType={getNodeInputTypeFromSchemaInputType({
type: paramInfo.type,
arrayItems: paramInfo.items
})}
h={'auto'}
/>
</HStack>
}
required={required}
key={paramName}
inputType={inputType}
fieldName={paramName}
form={form}
placeholder={paramName}
/>
);
}
)}
</Box>
</>
) : (
<Box fontWeight={'medium'} pb={4} px={2}>
{t('app:this_tool_requires_no_input')}
</Box>
)}
<Button mt={3} isLoading={isRunning} onClick={handleSubmit(runTool)}>
{t('common:Run')}
</Button>
</Box>
) : (
<Box flex={1} px={[2, 5]} overflow={'auto'}>
{output && (
<Box>
<Markdown source={`~~~json\n${output}`} />
</Box>
)}
</Box> </Box>
)} )}
</Box> </>
)} )}
</Box> </Box>
</Flex> </Flex>
......
import MyModal from '@fastgpt/web/components/common/MyModal';
import React from 'react';
import { useTranslation } from 'next-i18next';
import { Button, ModalBody, ModalFooter, Textarea } from '@chakra-ui/react';
import { useForm } from 'react-hook-form';
import { parseCurl } from '@fastgpt/global/common/string/http';
import { useToast } from '@fastgpt/web/hooks/useToast';
import { type HttpMethod, ContentTypes } from '@fastgpt/global/core/workflow/constants';
import type { ParamItemType } from './ManualToolModal';
import type { StoreSecretValueType } from '@fastgpt/global/common/secret/type';
export type CurlImportResult = {
method: HttpMethod;
path: string;
params?: ParamItemType[];
headers?: ParamItemType[];
bodyType: string;
bodyContent?: string;
bodyFormData?: ParamItemType[];
headerSecret?: StoreSecretValueType;
};
type CurlImportModalProps = {
onClose: () => void;
onImport: (result: CurlImportResult) => void;
};
const CurlImportModal = ({ onClose, onImport }: CurlImportModalProps) => {
const { t } = useTranslation();
const { toast } = useToast();
const { register, handleSubmit } = useForm({
defaultValues: {
curlContent: ''
}
});
const handleCurlImport = (data: { curlContent: string }) => {
try {
const parsed = parseCurl(data.curlContent);
const convertToParamItemType = (
items: Array<{ key: string; value?: string; type?: string }>
): ParamItemType[] => {
return items.map((item) => ({
key: item.key,
value: item.value || ''
}));
};
const { headerSecret, filteredHeaders } = (() => {
let headerSecret: StoreSecretValueType | undefined;
const filteredHeaders = parsed.headers.filter((header) => {
if (header.key.toLowerCase() === 'authorization') {
const authValue = header.value || '';
if (authValue.startsWith('Bearer ')) {
const token = authValue.substring(7).trim();
headerSecret = {
Bearer: {
value: token,
secret: ''
}
};
return false;
}
if (authValue.startsWith('Basic ')) {
const credentials = authValue.substring(6).trim();
headerSecret = {
Basic: {
value: credentials,
secret: ''
}
};
return false;
}
}
return true;
});
return { headerSecret, filteredHeaders };
})();
const bodyType = (() => {
if (!parsed.body || parsed.body === '{}') {
return ContentTypes.none;
}
return ContentTypes.json;
})();
const result: CurlImportResult = {
method: parsed.method as HttpMethod,
path: parsed.url,
params: parsed.params.length > 0 ? convertToParamItemType(parsed.params) : undefined,
headers: filteredHeaders.length > 0 ? convertToParamItemType(filteredHeaders) : undefined,
bodyType,
bodyContent: bodyType === ContentTypes.json ? parsed.body : undefined,
...(headerSecret && { headerSecret })
};
onImport(result);
toast({
title: t('common:import_success'),
status: 'success'
});
} catch (error: any) {
toast({
title: t('common:import_failed'),
description: error.message,
status: 'error'
});
console.error('Curl import error:', error);
}
};
return (
<MyModal
isOpen
onClose={onClose}
iconSrc="modal/edit"
title={t('common:core.module.http.curl import')}
w={600}
>
<ModalBody>
<Textarea
rows={20}
mt={2}
autoFocus
{...register('curlContent')}
placeholder={t('common:core.module.http.curl import placeholder')}
/>
</ModalBody>
<ModalFooter>
<Button variant={'whiteBase'} mr={3} onClick={onClose}>
{t('common:Close')}
</Button>
<Button onClick={handleSubmit(handleCurlImport)}>{t('common:Confirm')}</Button>
</ModalFooter>
</MyModal>
);
};
export default React.memo(CurlImportModal);
import { Box, Flex } from '@chakra-ui/react'; import { Box, Flex } from '@chakra-ui/react';
import { useSystem } from '@fastgpt/web/hooks/useSystem'; import { useSystem } from '@fastgpt/web/hooks/useSystem';
import React, { useMemo, useState } from 'react'; import React, { useEffect, useMemo, useState } from 'react';
import styles from '../SimpleApp/styles.module.scss'; import styles from '../SimpleApp/styles.module.scss';
import { cardStyles } from '../constants'; import { cardStyles } from '../constants';
import AppCard from './AppCard'; import AppCard from './AppCard';
...@@ -27,7 +27,7 @@ const Edit = () => { ...@@ -27,7 +27,7 @@ const Edit = () => {
); );
const baseUrl = toolSetData?.baseUrl ?? ''; const baseUrl = toolSetData?.baseUrl ?? '';
const toolList = toolSetData?.toolList ?? []; const toolList = toolSetData?.toolList ?? [];
const apiSchemaStr = toolSetData?.apiSchemaStr ?? ''; const apiSchemaStr = toolSetData?.apiSchemaStr;
const headerSecret = toolSetData?.headerSecret ?? {}; const headerSecret = toolSetData?.headerSecret ?? {};
const customHeaders = useMemo(() => { const customHeaders = useMemo(() => {
try { try {
...@@ -37,6 +37,20 @@ const Edit = () => { ...@@ -37,6 +37,20 @@ const Edit = () => {
} }
}, [appDetail.pluginData?.customHeaders]); }, [appDetail.pluginData?.customHeaders]);
useEffect(() => {
if (!currentTool || toolList.length === 0) {
setCurrentTool(toolList[0]);
return;
}
const updatedTool = toolList.find((tool) => tool.name === currentTool.name);
if (updatedTool) {
setCurrentTool(updatedTool);
} else {
setCurrentTool(toolList[0]);
}
}, [toolSetData]);
return ( return (
<MyBox <MyBox
display={['block', 'flex']} display={['block', 'flex']}
......
import MyModal from '@fastgpt/web/components/common/MyModal'; import MyModal from '@fastgpt/web/components/common/MyModal';
import React, { useEffect, useMemo, useState } from 'react'; import React, { useMemo, useState } from 'react';
import { useTranslation } from 'next-i18next'; import { useTranslation } from 'next-i18next';
import { import {
Box, Box,
...@@ -26,7 +26,6 @@ import { useContextSelector } from 'use-context-selector'; ...@@ -26,7 +26,6 @@ import { useContextSelector } from 'use-context-selector';
import { AppContext } from '../context'; import { AppContext } from '../context';
import HttpInput from '@fastgpt/web/components/common/Input/HttpInput'; import HttpInput from '@fastgpt/web/components/common/Input/HttpInput';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
import type { OpenApiJsonSchema } from '@fastgpt/global/core/app/httpTools/type';
import { pathData2ToolList } from '@fastgpt/global/core/app/httpTools/utils'; import { pathData2ToolList } from '@fastgpt/global/core/app/httpTools/utils';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import { str2OpenApiSchema } from '@fastgpt/global/core/app/jsonschema'; import { str2OpenApiSchema } from '@fastgpt/global/core/app/jsonschema';
...@@ -36,7 +35,7 @@ import { ...@@ -36,7 +35,7 @@ import {
} from '@/components/common/secret/HeaderAuthConfig'; } from '@/components/common/secret/HeaderAuthConfig';
import HeaderAuthForm from '@/components/common/secret/HeaderAuthForm'; import HeaderAuthForm from '@/components/common/secret/HeaderAuthForm';
const ConfigModal = ({ onClose }: { onClose: () => void }) => { const SchemaConfigModal = ({ onClose }: { onClose: () => void }) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { toast } = useToast(); const { toast } = useToast();
...@@ -117,12 +116,6 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => { ...@@ -117,12 +116,6 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => {
}); });
onClose(); onClose();
reloadApp(); reloadApp();
},
onError: (err) => {
toast({
status: 'warning',
title: t('common:plugin.Invalid Schema')
});
} }
} }
); );
...@@ -181,6 +174,20 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => { ...@@ -181,6 +174,20 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => {
</Box> </Box>
<Box mt={6} mb={2} color={'myGray.900'} fontSize={'14px'} fontWeight={'medium'}> <Box mt={6} mb={2} color={'myGray.900'} fontSize={'14px'} fontWeight={'medium'}>
{t('common:auth_config')}
</Box>
<Box mt={2}>
<HeaderAuthForm
headerSecretValue={storeHeader2HeaderValue(headerSecret)}
onChange={(data) => {
const storeData = headerValue2StoreHeader(data);
setValue('headerSecret', storeData);
}}
fontWeight="normal"
/>
</Box>
<Box mt={6} mb={2} color={'myGray.900'} fontSize={'14px'} fontWeight={'medium'}>
{t('app:request_headers')} {t('app:request_headers')}
</Box> </Box>
<Box <Box
...@@ -296,20 +303,6 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => { ...@@ -296,20 +303,6 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => {
</Table> </Table>
</TableContainer> </TableContainer>
</Box> </Box>
<Box mt={6} mb={2} color={'myGray.900'} fontSize={'14px'} fontWeight={'medium'}>
{t('common:auth_config')}
</Box>
<Box mt={2}>
<HeaderAuthForm
headerSecretValue={storeHeader2HeaderValue(headerSecret)}
onChange={(data) => {
const storeData = headerValue2StoreHeader(data);
setValue('headerSecret', storeData);
}}
fontWeight="normal"
/>
</Box>
</ModalBody> </ModalBody>
<ModalFooter px={9} display={'flex'} flexDirection={'column'}> <ModalFooter px={9} display={'flex'} flexDirection={'column'}>
...@@ -330,4 +323,4 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => { ...@@ -330,4 +323,4 @@ const ConfigModal = ({ onClose }: { onClose: () => void }) => {
); );
}; };
export default React.memo(ConfigModal); export default React.memo(SchemaConfigModal);
import CollaboratorContextProvider from '@/components/support/permission/MemberManager/context'; import CollaboratorContextProvider from '@/components/support/permission/MemberManager/context';
import ResumeInherit from '@/components/support/permission/ResumeInheritText'; import ResumeInherit from '@/components/support/permission/ResumeInheritText';
import { AppContext } from './context'; import { AppContext } from './context';
import { useSelectFile } from '@/web/common/file/hooks/useSelectFile';
import { resumeInheritPer } from '@/web/core/app/api'; import { resumeInheritPer } from '@/web/core/app/api';
import { import {
deleteAppCollaborators, deleteAppCollaborators,
...@@ -21,7 +20,6 @@ import { ...@@ -21,7 +20,6 @@ import {
import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils'; import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
import type { AppSchema } from '@fastgpt/global/core/app/type.d'; import type { AppSchema } from '@fastgpt/global/core/app/type.d';
import { AppRoleList } from '@fastgpt/global/support/permission/app/constant'; import { AppRoleList } from '@fastgpt/global/support/permission/app/constant';
import type { PermissionValueType } from '@fastgpt/global/support/permission/type';
import Avatar from '@fastgpt/web/components/common/Avatar'; import Avatar from '@fastgpt/web/components/common/Avatar';
import MyIcon from '@fastgpt/web/components/common/Icon'; import MyIcon from '@fastgpt/web/components/common/Icon';
import MyModal from '@fastgpt/web/components/common/MyModal'; import MyModal from '@fastgpt/web/components/common/MyModal';
...@@ -32,20 +30,18 @@ import React, { useCallback } from 'react'; ...@@ -32,20 +30,18 @@ import React, { useCallback } from 'react';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import { useContextSelector } from 'use-context-selector'; import { useContextSelector } from 'use-context-selector';
import { ReadRoleVal } from '@fastgpt/global/support/permission/constant'; import { ReadRoleVal } from '@fastgpt/global/support/permission/constant';
import { useUploadAvatar } from '@fastgpt/web/common/file/hooks/useUploadAvatar';
import { getUploadAvatarPresignedUrl } from '@/web/common/file/api';
const InfoModal = ({ onClose }: { onClose: () => void }) => { const InfoModal = ({ onClose }: { onClose: () => void }) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { toast } = useToast(); const { toast } = useToast();
const { updateAppDetail, appDetail, reloadApp } = useContextSelector(AppContext, (v) => v); const { updateAppDetail, appDetail, reloadApp } = useContextSelector(AppContext, (v) => v);
const { const { Component: AvatarUploader, handleFileSelectorOpen: handleAvatarSelectorOpen } =
File, useUploadAvatar(getUploadAvatarPresignedUrl, {
onOpen: onOpenSelectFile, onSuccess: (avatar) => setValue('avatar', avatar)
onSelectImage });
} = useSelectFile({
fileType: '.jpg,.png',
multiple: false
});
const { const {
register, register,
...@@ -134,7 +130,7 @@ const InfoModal = ({ onClose }: { onClose: () => void }) => { ...@@ -134,7 +130,7 @@ const InfoModal = ({ onClose }: { onClose: () => void }) => {
borderRadius={'md'} borderRadius={'md'}
mr={4} mr={4}
title={t('common:set_avatar')} title={t('common:set_avatar')}
onClick={() => onOpenSelectFile()} onClick={handleAvatarSelectorOpen}
/> />
<FormControl> <FormControl>
<Input <Input
...@@ -220,15 +216,7 @@ const InfoModal = ({ onClose }: { onClose: () => void }) => { ...@@ -220,15 +216,7 @@ const InfoModal = ({ onClose }: { onClose: () => void }) => {
</Button> </Button>
</ModalFooter> </ModalFooter>
<File <AvatarUploader />
onSelect={(e) =>
onSelectImage(e, {
maxH: 300,
maxW: 300,
callback: (e) => setValue('avatar', e)
})
}
/>
</MyModal> </MyModal>
); );
}; };
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment