Skip to content

赵月辉 / fastgpt-migrated

Go to a project
Commit 6c16fa91 by 居里栈栈 Committed by GitHub
Options

Fix: Custom delimiter does not take effect when document type is link (#2565) 

Co-authored-by: 勤劳上班的卑微小张 <jiazhan.zhang@ggimage.com>
parent ac4854a4
Showing with 2 additions and 1 deletions
packages/service/core/dataset/collection/utils.ts
...@@ -159,7 +159,8 @@ export const reloadCollectionChunks = async ({ ...@@ -159,7 +159,8 @@ export const reloadCollectionChunks = async ({
// split data // split data
const { chunks } = splitText2Chunks({ const { chunks } = splitText2Chunks({
text: newRawText, text: newRawText,
chunkLen: col.chunkSize || 512 chunkLen: col.chunkSize || 512,
customReg: col.chunkSplitter ? [col.chunkSplitter] : [],
}); });
// insert to training queue // insert to training queue
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment