Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit ab57bfcc by Archer Committed by GitHub
Options

perf: completions api.fix: new chat question guide (#361)

parent 11848b8f
Showing with 555 additions and 538 deletions
README.md
......@@ -39,7 +39,7 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
- [x] 文本内容提取成结构化数据
- [x] HTTP 扩展
- [ ] 嵌入 Laf，实现在线编写 HTTP 模块
- [ ] 连续对话引导
- [x] 对话下一步指引
- [ ] 对话多路线选择
- [x] 源文件引用追踪
- [ ] 自定义文件阅读器
......@@ -55,7 +55,7 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
- [x] 知识库单点搜索测试
- [x] 对话时反馈引用并可修改与删除
- [x] 完整上下文呈现
- [ ] 完整模块中间值呈现
- [x] 完整模块中间值呈现
4. OpenAPI
- [x] completions 接口（对齐 GPT 接口）
- [ ] 知识库 CRUD
......
docSite/content/docs/development/openApi.md
......@@ -46,7 +46,7 @@ FastGPT çš„ API Key 有 2 类，一类是全局通用的 key；一类是æºå¸¦äº
**请求示例：**
```bash
curl --location --request POST 'https://fastgpt.run/api/openapi/v1/chat/completions' \
curl --location --request POST 'https://fastgpt.run/api/v1/chat/completions' \
--header 'Authorization: Bearer apikey' \
--header 'Content-Type: application/json' \
--data-raw '{
......
docSite/content/docs/use-cases/onwechat.md
......@@ -26,7 +26,7 @@ weight: 312
## 3. 创建 docker-compose.yml 文件
只需要修改 `OPEN_AI_API_KEY` 和 `OPEN_AI_API_BASE` 两个环境变量即可。其中 `OPEN_AI_API_KEY` 为第一步获取的秘钥，`OPEN_AI_API_BASE` 为 FastGPT 的 OpenAPI 地址，例如：`https://fastgpt.run/api/openapi/v1`。
只需要修改 `OPEN_AI_API_KEY` 和 `OPEN_AI_API_BASE` 两个环境变量即可。其中 `OPEN_AI_API_KEY` 为第一步获取的秘钥，`OPEN_AI_API_BASE` 为 FastGPT 的 OpenAPI 地址，例如：`https://fastgpt.run/api/v1`。
随便找一个目录，创建一个 docker-compose.yml 文件，将下面的代码复制进去。
......@@ -40,7 +40,7 @@ services:
- seccomp:unconfined
environment:
OPEN_AI_API_KEY: 'fastgpt-z51pkjqm9nrk03a1rx2funoy'
OPEN_AI_API_BASE: 'https://fastgpt.run/api/openapi/v1'
OPEN_AI_API_BASE: 'https://fastgpt.run/api/v1'
MODEL: 'gpt-3.5-turbo'
CHANNEL_TYPE: 'wx'
PROXY: ''
......
docSite/content/docs/use-cases/openapi.md
......@@ -25,7 +25,7 @@ Tips: 安全起见，你可以设置一个额度或者过期时间，放置 key
## 替换三方应用的变量
```bash
OPENAI_API_BASE_URL: https://fastgpt.run/api/openapi (改成自己部署的域名)
OPENAI_API_BASE_URL: https://fastgpt.run/api (改成自己部署的域名)
OPENAI_API_KEY = 上一步获取到的秘钥
```
......
projects/app/src/api/fetch.ts
......@@ -12,7 +12,7 @@ interface StreamFetchProps {
abortSignal: AbortController;
}
export const streamFetch = ({
url = '/api/openapi/v1/chat/completions',
url = '/api/v1/chat/completions',
data,
onMessage,
abortSignal
......
projects/app/src/components/ChatBox/index.tsx
......@@ -27,7 +27,7 @@ import { useMarkdown } from '@/hooks/useMarkdown';
import { AppModuleItemType, VariableItemType } from '@/types/app';
import { VariableInputEnum } from '@/constants/app';
import { useForm } from 'react-hook-form';
import { MessageItemType } from '@/pages/api/openapi/v1/chat/completions';
import type { MessageItemType } from '@/types/core/chat/type';
import { fileDownload } from '@/utils/web/file';
import { htmlTemplate } from '@/constants/common';
import { useRouter } from 'next/router';
......@@ -158,6 +158,7 @@ const ChatBox = (
onStartChat?: (e: StartChatFnProps) => Promise<{
responseText: string;
[TaskResponseKeyEnum.responseData]: ChatHistoryItemResType[];
isNewChat?: boolean;
}>;
onDelMessage?: (e: { contentId?: string; index: number }) => void;
},
......@@ -173,6 +174,7 @@ const ChatBox = (
const TextareaDom = useRef<HTMLTextAreaElement>(null);
const chatController = useRef(new AbortController());
const questionGuideController = useRef(new AbortController());
const isNewChatReplace = useRef(false);
const [refresh, setRefresh] = useState(false);
const [variables, setVariables] = useState<Record<string, any>>({}); // settings variable
......@@ -381,7 +383,11 @@ const ChatBox = (
const messages = adaptChat2GptMessages({ messages: newChatList, reserveId: true });
const { responseData, responseText } = await onStartChat({
const {
responseData,
responseText,
isNewChat = false
} = await onStartChat({
chatList: newChatList,
messages,
controller: abortSignal,
......@@ -389,6 +395,8 @@ const ChatBox = (
variables
});
isNewChatReplace.current = isNewChat;
// set finish status
setChatHistory((state) =>
state.map((item, index) => {
......@@ -542,9 +550,12 @@ const ChatBox = (
// page change and abort request
useEffect(() => {
isNewChatReplace.current = false;
return () => {
chatController.current?.abort('leave');
questionGuideController.current?.abort('leave');
if (!isNewChatReplace.current) {
questionGuideController.current?.abort('leave');
}
// close voice
cancelBroadcast();
};
......
projects/app/src/components/Markdown/index.module.scss
......@@ -171,11 +171,10 @@
.markdown ol li > *:first-child {
margin-top: 0;
}
.markdown ul ul,
.markdown ul ol,
.markdown ol ol,
.markdown ol ul {
.markdown ul,
.markdown ol {
margin-bottom: 0;
padding-left: 14px;
}
.markdown dl {
padding: 0;
......
projects/app/src/pages/api/chat/chatTest.ts
......@@ -5,7 +5,7 @@ import { sseErrRes } from '@/service/response';
import { sseResponseEventEnum } from '@/constants/chat';
import { sseResponse } from '@/service/utils/tools';
import { AppModuleItemType } from '@/types/app';
import { dispatchModules } from '../openapi/v1/chat/completions';
import { dispatchModules } from '@/pages/api/v1/chat/completions';
import { pushChatBill } from '@/service/common/bill/push';
import { BillSourceEnum } from '@/constants/user';
import { ChatItemType } from '@/types/chat';
......
projects/app/src/pages/api/openapi/v1/chat/completions.ts
import type { NextApiRequest, NextApiResponse } from 'next';
import { connectToDatabase } from '@/service/mongo';
import { authUser, authApp, AuthUserTypeEnum } from '@/service/utils/auth';
import { sseErrRes, jsonRes } from '@/service/response';
import { addLog, withNextCors } from '@/service/utils/tools';
import { ChatRoleEnum, ChatSourceEnum, sseResponseEventEnum } from '@/constants/chat';
import {
dispatchHistory,
dispatchChatInput,
dispatchChatCompletion,
dispatchKBSearch,
dispatchAnswer,
dispatchClassifyQuestion,
dispatchContentExtract,
dispatchHttpRequest
} from '@/service/moduleDispatch';
import type {
CreateChatCompletionRequest,
ChatCompletionRequestMessage
} from '@fastgpt/core/aiApi/type';
import { gptMessage2ChatType, textAdaptGptResponse } from '@/utils/adapt';
import { getChatHistory } from './getHistory';
import { saveChat } from '@/service/utils/chat/saveChat';
import { sseResponse } from '@/service/utils/tools';
import { TaskResponseKeyEnum } from '@/constants/chat';
import { FlowModuleTypeEnum, initModuleType } from '@/constants/flow';
import { AppModuleItemType, RunningModuleItemType } from '@/types/app';
import { pushChatBill } from '@/service/common/bill/push';
import { BillSourceEnum } from '@/constants/user';
import { ChatHistoryItemResType } from '@/types/chat';
import { UserModelSchema } from '@/types/mongoSchema';
import { SystemInputEnum } from '@/constants/app';
import { getSystemTime } from '@/utils/user';
import { authOutLinkChat } from '@/service/support/outLink/auth';
import requestIp from 'request-ip';
import { replaceVariable } from '@/utils/common/tools/text';
import { ModuleDispatchProps } from '@/types/core/modules';
import { selectShareResponse } from '@/utils/service/core/chat';
import { pushResult2Remote, updateOutLinkUsage } from '@/service/support/outLink';
import { updateApiKeyUsage } from '@/service/support/openapi';
export type MessageItemType = ChatCompletionRequestMessage & { dataId?: string };
type FastGptWebChatProps = {
chatId?: string; // undefined: nonuse history, '': new chat, 'xxxxx': use history
appId?: string;
};
type FastGptShareChatProps = {
shareId?: string;
authToken?: string;
};
export type Props = CreateChatCompletionRequest &
FastGptWebChatProps &
FastGptShareChatProps & {
messages: MessageItemType[];
stream?: boolean;
detail?: boolean;
variables: Record<string, any>;
};
export type ChatResponseType = {
newChatId: string;
quoteLen?: number;
};
import { withNextCors } from '@/service/utils/tools';
import ChatCompletion from '@/pages/api/v1/chat/completions';
export default withNextCors(async function handler(req: NextApiRequest, res: NextApiResponse) {
res.on('close', () => {
res.end();
});
res.on('error', () => {
console.log('error: ', 'request error');
res.end();
});
let {
chatId,
appId,
shareId,
authToken,
stream = false,
detail = false,
messages = [],
variables = {}
} = req.body as Props;
try {
// body data check
if (!messages) {
throw new Error('Prams Error');
}
if (!Array.isArray(messages)) {
throw new Error('messages is not array');
}
if (messages.length === 0) {
throw new Error('messages is empty');
}
await connectToDatabase();
let startTime = Date.now();
/* user auth */
const {
responseDetail: shareResponseDetail,
user,
userId,
appId: authAppid,
authType,
apikey
} = await (async (): Promise<{
user?: UserModelSchema;
responseDetail?: boolean;
userId: string;
appId: string;
authType: `${AuthUserTypeEnum}`;
apikey?: string;
}> => {
if (shareId) {
return authOutLinkChat({
shareId,
ip: requestIp.getClientIp(req),
authToken,
question:
(messages[messages.length - 2]?.role === 'user'
? messages[messages.length - 2].content
: messages[messages.length - 1]?.content) || ''
});
}
return authUser({ req, authToken: true, authApiKey: true, authBalance: true });
})();
if (!user) {
throw new Error('Account is error');
}
// must have a app
appId = appId ? appId : authAppid;
if (!appId) {
throw new Error('appId is empty');
}
// auth app, get history
const [{ app }, { history }] = await Promise.all([
authApp({
appId,
userId
}),
getChatHistory({ chatId, appId, userId })
]);
const isOwner = !shareId && userId === String(app.userId);
const responseDetail = isOwner || shareResponseDetail;
/* format prompts */
const prompts = history.concat(gptMessage2ChatType(messages));
if (prompts[prompts.length - 1]?.obj === 'AI') {
prompts.pop();
}
// user question
const prompt = prompts.pop();
if (!prompt) {
throw new Error('Question is empty');
}
// set sse response headers
if (stream) {
res.setHeader('Content-Type', 'text/event-stream;charset=utf-8');
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('X-Accel-Buffering', 'no');
res.setHeader('Cache-Control', 'no-cache, no-transform');
}
/* start flow controller */
const { responseData, answerText } = await dispatchModules({
res,
modules: app.modules,
user,
variables,
params: {
history: prompts,
userChatInput: prompt.value
},
stream,
detail
});
// save chat
if (chatId) {
await saveChat({
chatId,
appId,
userId,
variables,
isOwner, // owner update use time
shareId,
source: (() => {
if (shareId) {
return ChatSourceEnum.share;
}
if (authType === 'apikey') {
return ChatSourceEnum.api;
}
return ChatSourceEnum.online;
})(),
content: [
prompt,
{
dataId: messages[messages.length - 1].dataId,
obj: ChatRoleEnum.AI,
value: answerText,
responseData
}
]
});
}
addLog.info(`completions running time: ${(Date.now() - startTime) / 1000}s`);
/* select fe response field */
const feResponseData = isOwner ? responseData : selectShareResponse({ responseData });
if (stream) {
sseResponse({
res,
event: detail ? sseResponseEventEnum.answer : undefined,
data: textAdaptGptResponse({
text: null,
finish_reason: 'stop'
})
});
sseResponse({
res,
event: detail ? sseResponseEventEnum.answer : undefined,
data: '[DONE]'
});
if (responseDetail && detail) {
sseResponse({
res,
event: sseResponseEventEnum.appStreamResponse,
data: JSON.stringify(feResponseData)
});
}
res.end();
} else {
res.json({
...(detail ? { responseData: feResponseData } : {}),
id: chatId || '',
model: '',
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 1 },
choices: [
{
message: { role: 'assistant', content: answerText },
finish_reason: 'stop',
index: 0
}
]
});
}
// add record
const { total } = pushChatBill({
appName: app.name,
appId,
userId,
source: (() => {
if (authType === 'apikey') return BillSourceEnum.api;
if (shareId) return BillSourceEnum.shareLink;
return BillSourceEnum.fastgpt;
})(),
response: responseData
});
if (shareId) {
pushResult2Remote({ authToken, shareId, responseData });
updateOutLinkUsage({
shareId,
total
});
}
!!apikey &&
updateApiKeyUsage({
apikey,
usage: total
});
} catch (err: any) {
if (stream) {
sseErrRes(res, err);
res.end();
} else {
jsonRes(res, {
code: 500,
error: err
});
}
}
return ChatCompletion(req, res);
});
/* running */
export async function dispatchModules({
res,
modules,
user,
params = {},
variables = {},
stream = false,
detail = false
}: {
res: NextApiResponse;
modules: AppModuleItemType[];
user: UserModelSchema;
params?: Record<string, any>;
variables?: Record<string, any>;
stream?: boolean;
detail?: boolean;
}) {
variables = {
...getSystemVariable({ timezone: user.timezone }),
...variables
};
const runningModules = loadModules(modules, variables);
// let storeData: Record<string, any> = {}; // after module used
let chatResponse: ChatHistoryItemResType[] = []; // response request and save to database
let chatAnswerText = ''; // AI answer
let runningTime = Date.now();
function pushStore({
answerText = '',
responseData
}: {
answerText?: string;
responseData?: ChatHistoryItemResType;
}) {
const time = Date.now();
responseData &&
chatResponse.push({
...responseData,
runningTime: +((time - runningTime) / 1000).toFixed(2)
});
runningTime = time;
chatAnswerText += answerText;
}
function moduleInput(
module: RunningModuleItemType,
data: Record<string, any> = {}
): Promise<any> {
const checkInputFinish = () => {
return !module.inputs.find((item: any) => item.value === undefined);
};
const updateInputValue = (key: string, value: any) => {
const index = module.inputs.findIndex((item: any) => item.key === key);
if (index === -1) return;
module.inputs[index].value = value;
};
const set = new Set();
return Promise.all(
Object.entries(data).map(([key, val]: any) => {
updateInputValue(key, val);
if (!set.has(module.moduleId) && checkInputFinish()) {
set.add(module.moduleId);
// remove switch
updateInputValue(SystemInputEnum.switch, undefined);
return moduleRun(module);
}
})
);
}
function moduleOutput(
module: RunningModuleItemType,
result: Record<string, any> = {}
): Promise<any> {
pushStore(result);
return Promise.all(
module.outputs.map((outputItem) => {
if (result[outputItem.key] === undefined) return;
/* update output value */
outputItem.value = result[outputItem.key];
/* update target */
return Promise.all(
outputItem.targets.map((target: any) => {
// find module
const targetModule = runningModules.find((item) => item.moduleId === target.moduleId);
if (!targetModule) return;
return moduleInput(targetModule, { [target.key]: outputItem.value });
})
);
})
);
}
async function moduleRun(module: RunningModuleItemType): Promise<any> {
if (res.closed) return Promise.resolve();
if (stream && detail && module.showStatus) {
responseStatus({
res,
name: module.name,
status: 'running'
});
}
// get fetch params
const params: Record<string, any> = {};
module.inputs.forEach((item: any) => {
params[item.key] = item.value;
});
const props: ModuleDispatchProps<Record<string, any>> = {
res,
stream,
detail,
variables,
moduleName: module.name,
outputs: module.outputs,
userOpenaiAccount: user?.openaiAccount,
inputs: params
};
const dispatchRes = await (async () => {
const callbackMap: Record<string, Function> = {
[FlowModuleTypeEnum.historyNode]: dispatchHistory,
[FlowModuleTypeEnum.questionInput]: dispatchChatInput,
[FlowModuleTypeEnum.answerNode]: dispatchAnswer,
[FlowModuleTypeEnum.chatNode]: dispatchChatCompletion,
[FlowModuleTypeEnum.kbSearchNode]: dispatchKBSearch,
[FlowModuleTypeEnum.classifyQuestion]: dispatchClassifyQuestion,
[FlowModuleTypeEnum.contentExtract]: dispatchContentExtract,
[FlowModuleTypeEnum.httpRequest]: dispatchHttpRequest
};
if (callbackMap[module.flowType]) {
return callbackMap[module.flowType](props);
}
return {};
})();
return moduleOutput(module, dispatchRes);
}
// start process width initInput
const initModules = runningModules.filter((item) => initModuleType[item.flowType]);
await Promise.all(initModules.map((module) => moduleInput(module, params)));
return {
[TaskResponseKeyEnum.answerText]: chatAnswerText,
[TaskResponseKeyEnum.responseData]: chatResponse
};
}
/* init store modules to running modules */
function loadModules(
modules: AppModuleItemType[],
variables: Record<string, any>
): RunningModuleItemType[] {
return modules.map((module) => {
return {
moduleId: module.moduleId,
name: module.name,
flowType: module.flowType,
showStatus: module.showStatus,
inputs: module.inputs
.filter((item) => item.connected) // filter unconnected target input
.map((item) => {
if (typeof item.value !== 'string') {
return {
key: item.key,
value: item.value
};
}
// variables replace
const replacedVal = replaceVariable(item.value, variables);
return {
key: item.key,
value: replacedVal
};
}),
outputs: module.outputs.map((item) => ({
key: item.key,
answer: item.key === TaskResponseKeyEnum.answerText,
value: undefined,
targets: item.targets
}))
};
});
}
/* sse response modules staus */
export function responseStatus({
res,
status,
name
}: {
res: NextApiResponse;
status?: 'running' | 'finish';
name?: string;
}) {
if (!name) return;
sseResponse({
res,
event: sseResponseEventEnum.moduleStatus,
data: JSON.stringify({
status: 'running',
name
})
});
}
/* get system variable */
export function getSystemVariable({ timezone }: { timezone: string }) {
return {
cTime: getSystemTime(timezone)
};
}
export const config = {
api: {
responseLimit: '20mb'
}
};
projects/app/src/pages/api/v1/chat/completions.ts 0 → 100644
import type { NextApiRequest, NextApiResponse } from 'next';
import { connectToDatabase } from '@/service/mongo';
import { authUser, authApp, AuthUserTypeEnum } from '@/service/utils/auth';
import { sseErrRes, jsonRes } from '@/service/response';
import { addLog, withNextCors } from '@/service/utils/tools';
import { ChatRoleEnum, ChatSourceEnum, sseResponseEventEnum } from '@/constants/chat';
import {
dispatchHistory,
dispatchChatInput,
dispatchChatCompletion,
dispatchKBSearch,
dispatchAnswer,
dispatchClassifyQuestion,
dispatchContentExtract,
dispatchHttpRequest
} from '@/service/moduleDispatch';
import type { CreateChatCompletionRequest } from '@fastgpt/core/aiApi/type';
import type { MessageItemType } from '@/types/core/chat/type';
import { gptMessage2ChatType, textAdaptGptResponse } from '@/utils/adapt';
import { getChatHistory } from './getHistory';
import { saveChat } from '@/service/utils/chat/saveChat';
import { sseResponse } from '@/service/utils/tools';
import { TaskResponseKeyEnum } from '@/constants/chat';
import { FlowModuleTypeEnum, initModuleType } from '@/constants/flow';
import { AppModuleItemType, RunningModuleItemType } from '@/types/app';
import { pushChatBill } from '@/service/common/bill/push';
import { BillSourceEnum } from '@/constants/user';
import { ChatHistoryItemResType } from '@/types/chat';
import { UserModelSchema } from '@/types/mongoSchema';
import { SystemInputEnum } from '@/constants/app';
import { getSystemTime } from '@/utils/user';
import { authOutLinkChat } from '@/service/support/outLink/auth';
import requestIp from 'request-ip';
import { replaceVariable } from '@/utils/common/tools/text';
import { ModuleDispatchProps } from '@/types/core/modules';
import { selectShareResponse } from '@/utils/service/core/chat';
import { pushResult2Remote, updateOutLinkUsage } from '@/service/support/outLink';
import { updateApiKeyUsage } from '@/service/support/openapi';
type FastGptWebChatProps = {
chatId?: string; // undefined: nonuse history, '': new chat, 'xxxxx': use history
appId?: string;
};
type FastGptShareChatProps = {
shareId?: string;
authToken?: string;
};
export type Props = CreateChatCompletionRequest &
FastGptWebChatProps &
FastGptShareChatProps & {
messages: MessageItemType[];
stream?: boolean;
detail?: boolean;
variables: Record<string, any>;
};
export type ChatResponseType = {
newChatId: string;
quoteLen?: number;
};
export default withNextCors(async function handler(req: NextApiRequest, res: NextApiResponse) {
res.on('close', () => {
res.end();
});
res.on('error', () => {
console.log('error: ', 'request error');
res.end();
});
let {
chatId,
appId,
shareId,
authToken,
stream = false,
detail = false,
messages = [],
variables = {}
} = req.body as Props;
try {
// body data check
if (!messages) {
throw new Error('Prams Error');
}
if (!Array.isArray(messages)) {
throw new Error('messages is not array');
}
if (messages.length === 0) {
throw new Error('messages is empty');
}
await connectToDatabase();
let startTime = Date.now();
/* user auth */
const {
responseDetail: shareResponseDetail,
user,
userId,
appId: authAppid,
authType,
apikey
} = await (async (): Promise<{
user?: UserModelSchema;
responseDetail?: boolean;
userId: string;
appId: string;
authType: `${AuthUserTypeEnum}`;
apikey?: string;
}> => {
if (shareId) {
return authOutLinkChat({
shareId,
ip: requestIp.getClientIp(req),
authToken,
question:
(messages[messages.length - 2]?.role === 'user'
? messages[messages.length - 2].content
: messages[messages.length - 1]?.content) || ''
});
}
return authUser({ req, authToken: true, authApiKey: true, authBalance: true });
})();
if (!user) {
throw new Error('Account is error');
}
// must have a app
appId = appId ? appId : authAppid;
if (!appId) {
throw new Error('appId is empty');
}
// auth app, get history
const [{ app }, { history }] = await Promise.all([
authApp({
appId,
userId
}),
getChatHistory({ chatId, appId, userId })
]);
const isOwner = !shareId && userId === String(app.userId);
const responseDetail = isOwner || shareResponseDetail;
/* format prompts */
const prompts = history.concat(gptMessage2ChatType(messages));
if (prompts[prompts.length - 1]?.obj === 'AI') {
prompts.pop();
}
// user question
const prompt = prompts.pop();
if (!prompt) {
throw new Error('Question is empty');
}
// set sse response headers
if (stream) {
res.setHeader('Content-Type', 'text/event-stream;charset=utf-8');
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('X-Accel-Buffering', 'no');
res.setHeader('Cache-Control', 'no-cache, no-transform');
}
/* start flow controller */
const { responseData, answerText } = await dispatchModules({
res,
modules: app.modules,
user,
variables,
params: {
history: prompts,
userChatInput: prompt.value
},
stream,
detail
});
// save chat
if (chatId) {
await saveChat({
chatId,
appId,
userId,
variables,
isOwner, // owner update use time
shareId,
source: (() => {
if (shareId) {
return ChatSourceEnum.share;
}
if (authType === 'apikey') {
return ChatSourceEnum.api;
}
return ChatSourceEnum.online;
})(),
content: [
prompt,
{
dataId: messages[messages.length - 1].dataId,
obj: ChatRoleEnum.AI,
value: answerText,
responseData
}
]
});
}
addLog.info(`completions running time: ${(Date.now() - startTime) / 1000}s`);
/* select fe response field */
const feResponseData = isOwner ? responseData : selectShareResponse({ responseData });
if (stream) {
sseResponse({
res,
event: detail ? sseResponseEventEnum.answer : undefined,
data: textAdaptGptResponse({
text: null,
finish_reason: 'stop'
})
});
sseResponse({
res,
event: detail ? sseResponseEventEnum.answer : undefined,
data: '[DONE]'
});
if (responseDetail && detail) {
sseResponse({
res,
event: sseResponseEventEnum.appStreamResponse,
data: JSON.stringify(feResponseData)
});
}
res.end();
} else {
res.json({
...(detail ? { responseData: feResponseData } : {}),
id: chatId || '',
model: '',
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 1 },
choices: [
{
message: { role: 'assistant', content: answerText },
finish_reason: 'stop',
index: 0
}
]
});
}
// add record
const { total } = pushChatBill({
appName: app.name,
appId,
userId,
source: (() => {
if (authType === 'apikey') return BillSourceEnum.api;
if (shareId) return BillSourceEnum.shareLink;
return BillSourceEnum.fastgpt;
})(),
response: responseData
});
if (shareId) {
pushResult2Remote({ authToken, shareId, responseData });
updateOutLinkUsage({
shareId,
total
});
}
!!apikey &&
updateApiKeyUsage({
apikey,
usage: total
});
} catch (err: any) {
if (stream) {
sseErrRes(res, err);
res.end();
} else {
jsonRes(res, {
code: 500,
error: err
});
}
}
});
/* running */
export async function dispatchModules({
res,
modules,
user,
params = {},
variables = {},
stream = false,
detail = false
}: {
res: NextApiResponse;
modules: AppModuleItemType[];
user: UserModelSchema;
params?: Record<string, any>;
variables?: Record<string, any>;
stream?: boolean;
detail?: boolean;
}) {
variables = {
...getSystemVariable({ timezone: user.timezone }),
...variables
};
const runningModules = loadModules(modules, variables);
// let storeData: Record<string, any> = {}; // after module used
let chatResponse: ChatHistoryItemResType[] = []; // response request and save to database
let chatAnswerText = ''; // AI answer
let runningTime = Date.now();
function pushStore({
answerText = '',
responseData
}: {
answerText?: string;
responseData?: ChatHistoryItemResType;
}) {
const time = Date.now();
responseData &&
chatResponse.push({
...responseData,
runningTime: +((time - runningTime) / 1000).toFixed(2)
});
runningTime = time;
chatAnswerText += answerText;
}
function moduleInput(
module: RunningModuleItemType,
data: Record<string, any> = {}
): Promise<any> {
const checkInputFinish = () => {
return !module.inputs.find((item: any) => item.value === undefined);
};
const updateInputValue = (key: string, value: any) => {
const index = module.inputs.findIndex((item: any) => item.key === key);
if (index === -1) return;
module.inputs[index].value = value;
};
const set = new Set();
return Promise.all(
Object.entries(data).map(([key, val]: any) => {
updateInputValue(key, val);
if (!set.has(module.moduleId) && checkInputFinish()) {
set.add(module.moduleId);
// remove switch
updateInputValue(SystemInputEnum.switch, undefined);
return moduleRun(module);
}
})
);
}
function moduleOutput(
module: RunningModuleItemType,
result: Record<string, any> = {}
): Promise<any> {
pushStore(result);
return Promise.all(
module.outputs.map((outputItem) => {
if (result[outputItem.key] === undefined) return;
/* update output value */
outputItem.value = result[outputItem.key];
/* update target */
return Promise.all(
outputItem.targets.map((target: any) => {
// find module
const targetModule = runningModules.find((item) => item.moduleId === target.moduleId);
if (!targetModule) return;
return moduleInput(targetModule, { [target.key]: outputItem.value });
})
);
})
);
}
async function moduleRun(module: RunningModuleItemType): Promise<any> {
if (res.closed) return Promise.resolve();
if (stream && detail && module.showStatus) {
responseStatus({
res,
name: module.name,
status: 'running'
});
}
// get fetch params
const params: Record<string, any> = {};
module.inputs.forEach((item: any) => {
params[item.key] = item.value;
});
const props: ModuleDispatchProps<Record<string, any>> = {
res,
stream,
detail,
variables,
moduleName: module.name,
outputs: module.outputs,
userOpenaiAccount: user?.openaiAccount,
inputs: params
};
const dispatchRes = await (async () => {
const callbackMap: Record<string, Function> = {
[FlowModuleTypeEnum.historyNode]: dispatchHistory,
[FlowModuleTypeEnum.questionInput]: dispatchChatInput,
[FlowModuleTypeEnum.answerNode]: dispatchAnswer,
[FlowModuleTypeEnum.chatNode]: dispatchChatCompletion,
[FlowModuleTypeEnum.kbSearchNode]: dispatchKBSearch,
[FlowModuleTypeEnum.classifyQuestion]: dispatchClassifyQuestion,
[FlowModuleTypeEnum.contentExtract]: dispatchContentExtract,
[FlowModuleTypeEnum.httpRequest]: dispatchHttpRequest
};
if (callbackMap[module.flowType]) {
return callbackMap[module.flowType](props);
}
return {};
})();
return moduleOutput(module, dispatchRes);
}
// start process width initInput
const initModules = runningModules.filter((item) => initModuleType[item.flowType]);
await Promise.all(initModules.map((module) => moduleInput(module, params)));
return {
[TaskResponseKeyEnum.answerText]: chatAnswerText,
[TaskResponseKeyEnum.responseData]: chatResponse
};
}
/* init store modules to running modules */
function loadModules(
modules: AppModuleItemType[],
variables: Record<string, any>
): RunningModuleItemType[] {
return modules.map((module) => {
return {
moduleId: module.moduleId,
name: module.name,
flowType: module.flowType,
showStatus: module.showStatus,
inputs: module.inputs
.filter((item) => item.connected) // filter unconnected target input
.map((item) => {
if (typeof item.value !== 'string') {
return {
key: item.key,
value: item.value
};
}
// variables replace
const replacedVal = replaceVariable(item.value, variables);
return {
key: item.key,
value: replacedVal
};
}),
outputs: module.outputs.map((item) => ({
key: item.key,
answer: item.key === TaskResponseKeyEnum.answerText,
value: undefined,
targets: item.targets
}))
};
});
}
/* sse response modules staus */
export function responseStatus({
res,
status,
name
}: {
res: NextApiResponse;
status?: 'running' | 'finish';
name?: string;
}) {
if (!name) return;
sseResponse({
res,
event: sseResponseEventEnum.moduleStatus,
data: JSON.stringify({
status: 'running',
name
})
});
}
/* get system variable */
export function getSystemVariable({ timezone }: { timezone: string }) {
return {
cTime: getSystemTime(timezone)
};
}
export const config = {
api: {
responseLimit: '20mb'
}
};
projects/app/src/pages/chat/index.tsx
......@@ -113,7 +113,7 @@ const Chat = ({ appId, chatId }: { appId: string; chatId: string }) => {
history: ChatBoxRef.current?.getChatHistory() || state.history
}));
return { responseText, responseData };
return { responseText, responseData, isNewChat: forbidRefresh.current };
},
[appId, chatId, history, router, setChatData, updateHistory]
);
......
projects/app/src/pages/chat/share.tsx
......@@ -106,7 +106,7 @@ const OutLink = ({
'*'
);
return { responseText, responseData };
return { responseText, responseData, isNewChat: forbidRefresh.current };
},
[authToken, chatId, router, saveChatResponse, shareId]
);
......
projects/app/src/types/core/chat/type.d.ts 0 → 100644
import type { ChatCompletionRequestMessage } from '@fastgpt/core/aiApi/type';
export type MessageItemType = ChatCompletionRequestMessage & { dataId?: string };
projects/app/src/utils/adapt.ts
......@@ -4,7 +4,7 @@ import type { UserBillType } from '@/types/user';
import { ChatItemType } from '@/types/chat';
import { ChatCompletionRequestMessageRoleEnum } from '@fastgpt/core/aiApi/constant';
import { ChatRoleEnum } from '@/constants/chat';
import type { MessageItemType } from '@/pages/api/openapi/v1/chat/completions';
import type { MessageItemType } from '@/types/core/chat/type';
import type { AppModuleItemType } from '@/types/app';
import type { FlowModuleItemType } from '@/types/flow';
import type { Edge, Node } from 'reactflow';
......
projects/app/src/utils/common/adapt/message.ts
import type { ChatItemType } from '@/types/chat';
import { ChatRoleEnum } from '@/constants/chat';
import { ChatCompletionRequestMessageRoleEnum } from '@fastgpt/core/aiApi/constant';
import type { MessageItemType } from '@/pages/api/openapi/v1/chat/completions';
import type { MessageItemType } from '@/types/core/chat/type';
const chat2Message = {
[ChatRoleEnum.AI]: ChatCompletionRequestMessageRoleEnum.Assistant,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment