赵月辉 / fastgpt-migrated

* docs: update v4.16 release notes

* fix(dataset): support legacy CSV imports

* chore(i18n): refine dataset import prompts

* chore(i18n): show current dataset CSV format

* feat(dataset): support Excel template imports

* fix(dataset): reuse readFile worker for imports

* docs(dataset): document template imports

* fix(security): harden system and temporary tool secrets

Synchronize toolset secrets from parent records, clear temporary values when switching secret sources, and keep production system secrets out of debug tool flows.

Add regression coverage for parent-child authority, debug fail-closed behavior, and app secret normalization.

* fix(security): encrypt system tool secrets

Store plugin system secrets as AES-GCM wrappers, mask configured values in administrator responses, and keep legacy plaintext and inputListVal records readable during migration. Update the administrator form to edit write-only markers without sending stored ciphertext back to the server.

* fix(ui): use plain input for system secrets

Keep system secret values visible while editing and continue to replace saved values with the write-only configured marker after reload.

* refactor(workflow): simplify workflow configuration logic

* refactor: match ask ui with latest design

Signed-off-by: Nixieboluo <me@sagirii.me>

* refactor: disable transitions for ask ui

Signed-off-by: Nixieboluo <me@sagirii.me>

* refactor: tweaks to ask_user prompt

Signed-off-by: Nixieboluo <me@sagirii.me>

---------

Signed-off-by: Nixieboluo <me@sagirii.me>

Fix formatting and punctuation in README.md

* feat(agent): support helper tool loops and injected prompts

* prompt

* remove log

* feat: prepare support for ask_user v2 in aux gen

Signed-off-by: Nixieboluo <me@sagirii.me>

* fix: align ask composer UI

* feat: prepare support for ask_user v2 in agent

Signed-off-by: Nixieboluo <me@sagirii.me>

* feat: unified frontend logic for ask_user v2

Signed-off-by: Nixieboluo <me@sagirii.me>

* test: update tests for ask_user v2

Signed-off-by: Nixieboluo <me@sagirii.me>

* refactor: more strict ask_user schema validation

Signed-off-by: Nixieboluo <me@sagirii.me>

* refactor: add transitions and hide answer when retrying

Signed-off-by: Nixieboluo <me@sagirii.me>

* refactor: reorganize ask v2 composer ui

Signed-off-by: Nixieboluo <me@sagirii.me>

* test: update tests

Signed-off-by: Nixieboluo <me@sagirii.me>

* feat: format ask user answers

* refactor: remove compat for legacy agent_ask in aux gen

Signed-off-by: Nixieboluo <me@sagirii.me>

* fix: agent ask composer ui focus handling

Signed-off-by: Nixieboluo <me@sagirii.me>

* refactor: cleanup compatibility fallback for agent ask

Signed-off-by: Nixieboluo <me@sagirii.me>

* feat: legacy ask_user record compatibility

Signed-off-by: Nixieboluo <me@sagirii.me>

* test: fix test mismatches

Signed-off-by: Nixieboluo <me@sagirii.me>

* feat: convert legacy ask to the new one

Signed-off-by: Nixieboluo <me@sagirii.me>

* chore: trigger correct pro build

Signed-off-by: Nixieboluo <me@sagirii.me>

* refactor: remove legacy auxiliary ask fallback

* chore: update pro storage mock fix

Signed-off-by: Archer <545436317@qq.com>

* submodule

---------

Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Archer <545436317@qq.com>
Co-authored-by: Archer <545436317@qq.com>

* fix(workflow): preserve explicit tool input render types

* fix(tool-runtime): exclude referenced inputs from model schema

* perf: timeout and doc

* fix(docs): use valid dotenv code fence

* feat: optimize object storage multipart upload

* feat: unify browser object storage uploads

* chore: update storage package and pro submodule

* Bump deps

* fix: configure home chat custom PDF parsing

* fix: extend dataset parsing timeout

* fix: ocr comments

* fix: resolve pro submodule conflict

* chore: remove invalid multipart benchmark command

* refactor: select multipart uploads by file size

---------

Co-authored-by: Archer <545436317@qq.com>

The schema-detection aggregate in exportAll builds its $match from teamId
(returned by authDataset as a string). find() casts the string to ObjectId,
but aggregate() does not schema-cast $match values, so the aggregation
matched zero documents. exportSchema became undefined, indexCount/hasMetadata
fell back to 0/false, and the CSV header collapsed to q,a.

Wrap teamId and dataset _ids in Types.ObjectId so the aggregate $match works.
dataset _ids are already ObjectId from .lean(), so that wrap is defensive.

Co-authored-by: Claude <noreply@anthropic.com>

* Fix regex for S3 markdown and improve VLM model fallback

Update regex pattern for S3 markdown key and adjust fallback logic for VLM model retrieval.

[^)]+? → [^)>]+?：在排除列表中增加 > 字符
(${pattern}) → <?(${pattern})>?：URL 前后添加可选的 <> 包裹匹配

* Implement space replacement in S3 key handling

Add function to replace spaces with underscores in S3 key.

在 replaceParentheses 后新增 replaceSpaces，将所有空白字符（\s+）替换为下划线，避免新上传的文件产生含空格的 S3 key。

* Fix filename formatting tests in utils.test.ts

Updated test cases to ensure formatted filenames replace spaces with underscores and handle multiple spaces correctly.

更新 getFormatedFilename 的空格文件名测试期望
更新真实场景的测试期望
新增 5 个空格替换测试用例（单独替换、括号+空格组合、连续空格、首尾空格等场景）

* fix: handle S3 markdown key edge cases

---------

Co-authored-by: Xianquan <whoeverimf5@gmail.com>

* feat: add metadata field to dataset_data for custom metadata support\n\n- Zod schemas: add metadata to DatasetDataSchema, DatasetDataItemSchema, CreateDatasetDataPropsSchema, UpdateDatasetDataPropsSchema, PushDataChunkSchema, GetDataListItemSchema\n- MongoDB schemas: add metadata to dataset_datas and dataMetadata to dataset_trainings\n- CSV import: parse header row to identify non-q/a/indexes columns as metadata keys\n- Training pipeline: pass metadata through training queue (dataMetadata) to dataset_data (metadata)\n- Search output: include metadata in search result items via buildSearchResultItem\n- API: include metadata in v2/list responses\n- Template: update download template header to demo metadata column, relax header validation to check only first two columns are q,a\n\nCo-Authored-By: Claude <noreply@anthropic.com>

* fix: preserve dataset metadata across import and export

* chore: remove metadata design document

* fix: allow a single metadata template column

* fix: reject legacy dataset template headers

* docs: add dataset metadata release note

---------

Co-authored-by: hexiaoyan30010 <30010@sangfor.com>
Co-authored-by: Archer <545436317@qq.com>

* Add V4.15.6 documentation translations

* api

* fix(chat): restore share context and mobile citation details

* fix(chat): restore share context and mobile citation details

* submodule

* fix(chat): handle hydration failures and mobile quote actions

* chore(chat): keep quote review scope focused

---------

Co-authored-by: Archer <545436317@qq.com>

* refactor(tool): unify tool input runtime compilation

* fix(tool): hide schema annotations from models

* test(tool): expect filtered schema annotations

* fix(tool): resolve app typecheck errors

* fix(tool): drop sandbox type workaround

* fix(workflow): allow external values to override internal defaults

* fix(workflow): preserve plugin input types across workflow serialization

* fix(jsonschema): preserve enum constraints for agent-generated inputs

* test(workflow): allow parent input to override internal defaults

* feat(skill): optimize skill first-use UX and cross-tab associate flow

Add dashboard empty-state guidance, skill selector create redirect,
and cross-tab associate bridge for Agent skill binding after creation.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(skill): normalize root parentId to null and simplify modal/list flow

- Root directory is now uniformly `null`; collapse fetchParentId/parentId in
  useSkillSelectData and drop `?? ''` in skill context (server treats falsy
  parentId as root, so behavior is unchanged).
- Create/Import modals no longer swallow onSuccess errors with a toast;
  callers handle exceptions and list refresh becomes best-effort.
- Drop redundant redirectToDetail in CreateSkillModal; openDetailInNewTab
  fully controls new-tab vs current-page navigation.
- Extract passesSandboxGuard helper in skill List.
- Dedupe the shared glow-ellipse/filter geometry in SkillEmptyActionCard.

refactor(skill): remove unused skill association logic and clean up related components

chore(skill): update submodule and remove unused SVG assets

- Updated submodule reference in `pro`.
- Deleted unused SVG files related to skill selection and import functionality, including `createCardBgDefault.svg`, `createCardBgHover.svg`, `importCardBgDefault.svg`, `importCardBgHover.svg`, and `skillSelectEmptyIcon.svg`.
- Refactored `SkillSelectModal` and related components to streamline button actions and improve code clarity.

refactor(skill): simplify EmptyTip component and update SkillSelectModal logic

* feat(skill): implement skill import size validation and enhance user feedback

* refactor(skill): simplify empty card glow effect and import size check

---------

Co-authored-by: siigure <siiigureouo@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix http tool array json schema

* optimize http tool schema migration queries

* submodule

* document http tool schema migration

* fix: external

* refactor(skill): stream skill package import directly to S3

* refactor(sandbox): stream sandbox file upload directly via octet-stream

* refactor(sandbox): derive limits from storage size

* refactor(sandbox): rename storage size env

* refactor(sandbox): use Gi storage env name

* chore: keep v4.15 deployment templates unchanged

* Agent sandox user (#7383)

* feat(sandbox): implement user-level sandbox and legacy workspace migration (#7315)

* feat(sandbox): implement user-level sandbox and legacy workspace migration

* refactor(sandbox): implement durable-saga pattern and migrate lifecycle operations

* Revert "refactor(sandbox): implement durable-saga pattern and migrate lifecycle operations"

This reverts commit 921953d97819776645868038e44d75decd552e8a.

* refactor(sandbox): implement durable-saga runner and migrate lifecycle operations

* fix(sandbox): preserve legacy migration backups

* fix(sandbox): repair missing provider instances and support allowCreate option

* refactor(sandbox): cleanup legacy archive   endpoints and streamline migration workflow

* feat: add direct sandbox workspace preview (#7316)

* feat: add direct sandbox workspace preview

* refactor: align sandbox preview claims with upstream

refactor: fix IDE agent internal ports

fix: decode sandbox proxy claims generically

ci: install Rust toolchain without third-party actions

* chore: keep sandbox proxy on one port by default

feat: split sandbox proxy preview port
fix: require sandbox preview proxy URL

* Codex/agent sandbox preview (#7349)

* refactor: use sessions for sandbox preview links

* docs: update sandbox preview session design

* fix: align preview sessions with user sandboxes

* fix: align preview ids with sandbox runtime

* refactor(sandbox): unify App and Skill runtime image upgrade status (#7352)

* refactor(sandbox): simplify provider migration and runtime upgrade logic (#7357)

* feat(sandbox): improve app chat runtime resilience (#7361)

* refactor(sandbox): implement server-side silent runtime upgrade for App workflows

* feat(sandbox): implement graceful degradation for app chat when sandbox is unavailable

* refactor(sandbox): refactor ticket verification and rename preview session cache

* fix(sandbox): pause and reuse OpenSandbox resources (#7363)

* fix(sandbox): pause OpenSandbox resources on stop

* refactor(sandbox): upgrade OpenSandboxAdapter lifecycle and SDK integration

* refactor(sandbox): align adapter contract, structure and OpenSandbox lifecycle

* feat(sandbox): make suspend and archive thresholds configurable via env

* refactor(sandbox): unify workflow sandbox preparation and status streaming

* fix(sandbox): create parent directories before file writes (#7384)

* feat(sandbox): prepareSandboxFileParentDirectories utility and update sandbox file handling

* doc: summary

* feat(sandbox): improve resource controls and agent reliability

* doc

* doc

* fix(sandbox): move migration staging out of workspace (#7394)

* fix(sandbox): move migration staging out of workspace

* refactor(sandbox): replace batchRun waitForAll with batchRunSettled

* env

* fix(sandbox): restore legacy migration prerequisites (#7396)

* fix(sandbox): restore legacy migration prerequisites

* fix(sandbox): stop migrated targets

* fix(sandbox): harden preview sessions and document risks

* fix(sandbox): add preview-only proxy env validation

* docs(sandbox): clarify required preview proxy env

---------

Co-authored-by: DigHuang <114602213+DigHuang@users.noreply.github.com>

* feat(workflow): support agent generated tool parameters (#7267)

* feat(workflow): add agentGenerated input type for tool parameters

- Add `FlowNodeInputTypeEnum.agentGenerated` to distinguish parameters
  generated by the model during tool calls.
- Implement `initToolInputTypeByDefaultMode` to handle initialization
  based on tool descriptions and safe boundaries.
- Update ToolCall and Agent runtime schema generators to filter
  parameters based on final input type.
- Add security boundaries to prevent sensitive fields (e.g., secrets,
  files, model selection) from being marked as agent-generated.
- Update UI components to support configuration of parameter sources and
  improve state management for tool inputs.

* fix(workflow): surface agent generated tool params

* feat(workflow): align agent generated tool input UI

* refactor: simplify plugin runtime and cleanup workflow logic

* feat(workflow): introduce isToolParam to define agent-generated inputs

Add `isToolParam` to handle default input generation logic more
precisely
than `toolDescription`, ensuring better control over which fields are
exposed as model parameters in the tool call schema.

* fix(workflow): honor selected render type index

* Restrict agent tool params to selected generated fields

* Persist selected tool input type in workflow detail

* fix(formEdit): preserve saved input type when initializing defaults

* fix(workflow): avoid materializing unset selectedType

* refactor(formEdit): improve tool input selection

* refactor(formEdit): drop default tool input mode

Remove persisting default tool input mode (isToolParam) and centralize
selection via saved input by introducing stripToolInputDefaultMode and
updating call sites to pass savedInput only.

Propagate tool errors through the agent loop by carrying errorMessage
in tool results and dispatch, so errors do not produce visible node
responses.
Update tests, runtime types, and i18n descriptions accordingly.

* fix(form): normalize reference-only tool inputs to agent generated

* feat(global): preserve workflow tool node metadata in json schema

* refactor(config-tool-modal): unify input and output sections with shared

* fix(formEdit): default user chat inputs to agent generated in tools

* fix(tool): align runtime schemas with legacy tool inputs

* fix(jsonschema): preserve open union types and secret config state

* refactor: improve application workflows and supporting modules

* fix(form-edit): preserve textarea render type selection

* fix(workflow-tool): preserve hidden inputs and normalize legacy defaults

* fix(workflow): migrate legacy tool descriptions to AI inputs

* fix(workflow): scope legacy tool input migration to supported tools

--no-edit

* fix(workflow): restore legacy workflow tool input types at runtime

* fix(app): preserve legacy tool input compatibility

* fix(workflow): restrict legacy tool description fallback

* feat(agent): persist tool input modes independently

* refactor(tool): move agent tool schemas out of skill module

* fix(tool): filter agent-generated params from tool configs

* refactor: reorganize application modules and shared services

* fix(workflow): align tool input defaults and option handling

* fix(form-edit): preserve explicit input types and infer tool controls

* feat(agent-tools): support fixed and latest tool versions

* chore: submodule

* fix(workflow): preserve fixed tool versions and legacy input types

* submodule

* fix(sandbox): align v2 migration and Devbox resource limits (#7405)

* feat: clean dead code

* refactor(service): remove tmbId & legacy fields from skill runtime APIs

* feat(sandbox): pass resource limits to Sealos Devbox

* refactor(sandbox): share resource limit configuration (#7407)

* fix(workflow): preserve tool node input configuration (#7409)

* fix(workflow): align tool input defaults and option handling

* fix(workflow): preserve tool node input configuration

* doc

* doc

* fix(workflow): restore tool parameter label (#7410)

---------

Co-authored-by: DigHuang <114602213+DigHuang@users.noreply.github.com>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>

* feat: add SoMark PDF provider

* docs: document SoMark PDF provider

* docs: update environment and v4.15.5 notes

* fix:修复升级到 V4.15.x 后，部分早期创建的插件节点（节点级 version 为空）在工作流编辑器中输入框完全消失

* test(app): update workflow detail reference type expectation

---------

Co-authored-by: Finley Ge <finleyge@fastgpt.io>

* chore: refresh workspace dependencies

* submodule

* fix: complete OSS storage compatibility for v4.15.5

* fix: complete COS storage integration compatibility

* fix: align portable storage key limit

* test: expand cross-provider storage integration coverage

* feat: add Cloudflare R2 storage support

* fix: use supported docs code fence language