phsl / new-api

Merge pull request #1452 from QuantumNous/refactor/model-pricing

- Why:
  - Eliminate `gorm.io/datatypes` for a single field and fix scan errors when drivers return JSON as string.
  - Prevent JSONEditor manual mode from locking on invalid JSON and from appending stray characters after “Fill Template”.

- What:
  - Backend (`model/prefill_group.go`):
    - Replaced `datatypes.JSON` with `JSONValue` (based on `json.RawMessage`) for `PrefillGroup.Items`.
    - Implemented `sql.Scanner` and `driver.Valuer` to accept both `[]byte` and `string`.
    - Implemented `MarshalJSON`/`UnmarshalJSON` to preserve raw JSON in API without base64.
    - Converted comments to Chinese.
  - Frontend (`web/src/components/common/ui/JSONEditor.js`):
    - Added `manualText` buffer for manual mode to avoid input being overridden by external value.
    - Only propagate `onChange` when manual text is valid JSON; otherwise show error but do not block typing.
    - Safe manual-mode rendering: derive rows from `manualText` and avoid calling `split` on non-strings.
    - Improved mode toggle: populate `manualText` from visual data; validate before switching back to visual.
    - Fixed “Fill Template” to sync `manualText`, `jsonData`, and `onChange` to avoid stray trailing characters.

- Impact:
  - Resolves: “unsupported Scan, storing driver.Value type string into type *json.RawMessage”.
  - Resolves: `value.split is not a function` in manual mode.
  - Resolves: extra `s` appended after inserting template.
  - API shape and DB column type remain the same (`gorm:"type:json"`); no `go.mod` changes.
  - Lints pass for modified files.

Files changed:
- model/prefill_group.go
- web/src/components/common/ui/JSONEditor.js

♻️ refactor(model): replace gorm.io/datatypes with JSONValue for PrefillGroup.Items; fix JSON scan across drivers

- Why:
  - Avoid introducing `gorm.io/datatypes` for a single field.
  - Align with existing pattern (`ChannelInfo`, `Properties`) using `Scanner`/`Valuer`.
  - Fix runtime error when drivers return JSON as string.

- What:
  - Introduced `JSONValue` (based on `json.RawMessage`) implementing `sql.Scanner` and `driver.Valuer`, with `MarshalJSON`/`UnmarshalJSON` to preserve raw JSON in API.
  - Updated `PrefillGroup.Items` to use `JSONValue` with `gorm:"type:json"`.
  - Localized comments in `model/prefill_group.go` to Chinese.

- Impact:
  - Resolves “unsupported Scan, storing driver.Value type string into type *json.RawMessage”.
  - Works with MySQL/Postgres/SQLite whether JSON is returned as `[]byte` or `string`.
  - API and DB schema remain unchanged; no `go.mod` changes; lints pass.

Files changed:
- model/prefill_group.go

- Why: Avoid adding `gorm.io/datatypes` for a single field; the rest of the codebase does not use it, and using the standard library keeps dependencies lean.
- What:
  - Switched `PrefillGroup.Items` from `datatypes.JSON` to `json.RawMessage`.
  - Updated imports in `model/prefill_group.go` to use `encoding/json` and removed the unused `gorm.io/datatypes`.
  - Preserved `gorm:"type:json"` so DB column behavior remains the same.
- Impact:
  - API response/request shape for `items` remains unchanged (still JSON).
  - DB schema behavior is unchanged; GORM migration continues to handle the field as JSON.
  - No other references to `datatypes` exist; no `go.mod` changes needed.
  - Lints pass for the modified file.

Files changed:
- model/prefill_group.go

No breaking changes.

🎨 refactor(ui): merge “Content Configuration” into “Basic Information” card in EditPrefillGroupModal

- Move `items` field (`JSONEditor` for endpoint type, `Form.TagInput` otherwise) into the first “Basic Information” card
- Remove the second “Content Configuration” card and its header; consolidate to a single-card layout
- Preserve form initialization, validation, and submit logic; API payload structure remains unchanged
- Improves clarity and reduces visual clutter without altering behavior
- Lint passes

Affected file:
- `web/src/components/table/models/modals/EditPrefillGroupModal.jsx`

No breaking changes.

✨ feat: Improve models UX and robustness: add JSONEditor extraFooter, fix endpoints rendering, and clean up deps

- Why
  - Needed to separate help text from action buttons in JSONEditor for better layout and UX.
  - Models table should robustly render both new object-based endpoint mappings and legacy arrays.
  - Columns should re-render when vendor map changes.
  - Minor import cleanups for consistency.

- What
  - JSONEditor.js
    - Added optional prop extraFooter to render content below the extraText divider.
    - Kept extraText rendered via Divider; extraFooter appears on the next line for clear separation.
  - EditModelModal.jsx
    - Moved endpoint group buttons from extraText into extraFooter to display under the helper text.
    - Kept merge-logic: group items are merged into current endpoints JSON with key override semantics.
    - Consolidated lucide-react imports into a single line.
  - ModelsColumnDefs.js
    - Made endpoint renderer resilient:
      - Supports object-based JSON (keys as endpoint types) and legacy array format.
      - Displays keys/items as tags and limits the number shown; uses stringToColor for visual consistency.
    - Consolidated Semi UI imports into a single line.
  - ModelsTable.jsx
    - Fixed columns memoization dependency to include vendorMap, ensuring re-render when vendor data changes.

- Notes
  - Backward-compatible: extraFooter is additive; existing JSONEditor usage remains unchanged.
  - No API changes to backend.
  - No linter errors introduced.

- Files touched
  - web/src/components/common/ui/JSONEditor.js
  - web/src/components/table/models/modals/EditModelModal.jsx
  - web/src/components/table/models/ModelsColumnDefs.js
  - web/src/components/table/models/ModelsTable.jsx

- Impact
  - Clearer UI for endpoint editing (buttons now below helper text).
  - Correct endpoints display for object-based mappings in models list.
  - More reliable reactivity when vendor data updates.

✨ feat(pricing+endpoints+ui): wire custom endpoint mapping end‑to‑end and overhaul visual JSON editor

Backend (Go)
- Include custom endpoints in each model’s SupportedEndpointTypes by parsing Model.Endpoints (JSON) and appending keys alongside native endpoint types.
- Build a global supportedEndpointMap map[string]EndpointInfo{path, method} by:
  - Seeding with native defaults.
  - Overriding/adding from models.endpoints (accepts string path → default POST, or {path, method}).
- Expose supported_endpoint at the top level of /api/pricing (vendors-like), removing per-model duplication.
- Fix default path for EndpointTypeOpenAIResponse to /v1/responses.
- Keep concurrency/caching for pricing retrieval intact.

Frontend (React)
- Fetch supported_endpoint in useModelPricingData and propagate to PricingPage → ModelDetailSideSheet → ModelEndpoints.
- ModelEndpoints
  - Resolve path+method via endpointMap; replace {model} with actual model name.
  - Fix mobile visibility; always show path and HTTP method.
- JSONEditor
  - Wrap with Form.Slot to inherit form layout; simplify visual styles.
  - Use Tabs for “Visual” / “Manual” modes.
  - Unify editors: key-value editor now supports nested JSON:
    - “+” to convert a primitive into an object and add nested fields.
    - Add “Convert to value” for two‑way toggle back from object.
    - Stable key rename without reordering rows; new rows append at bottom.
    - Use Row/Col grid for clean alignment; region editor uses Form.Slot + grid.
- Editing flows
  - EditModelModal / EditPrefillGroupModal use JSONEditor (editorType='object') for endpoint mappings.
  - PrefillGroupManagement renders endpoint group items by JSON keys.

Data expectations / compatibility
- models.endpoints should be a JSON object mapping endpoint type → string path or {path, method}. Strings default to POST.
- No schema changes; existing TEXT field continues to store JSON.

QA
- /api/pricing now returns custom endpoint types and global supported_endpoint.
- UI shows both native and custom endpoints; paths/methods render on mobile; nested editing works and preserves order.

Fix/gemini-fetch-models

Fix/disable-gemini-ping

feat: support deepseek claude format (convert)

feat: support qwen claude format

feat: enable thinking mode on ali thinking model

This reverts commit 97b8d7de.

Add visual distinction for enabled/disabled models by applying different
background colors to table rows based on model status. This implementation
follows the same pattern used in ChannelsTable for consistent user experience.

Changes:
- Modified handleRow function in useModelsData.js to include row styling
- Disabled models (status !== 1) now display with gray background using
  --semi-color-disabled-border CSS variable
- Enabled models (status === 1) maintain normal background color
- Preserved existing row click selection functionality

This enhancement improves the visual feedback for users to quickly identify
which models are active vs inactive in the models management interface.

feat: support aws bedrock apikey

fix：修复添加模型倍率时的输入框锁定

feat: add support for claude-opus-4-1 model and update ratios

fix: responses cache token 未计费

This commit significantly refactors the `EditModelModal` component to streamline the user interface and enhance usability, aligning it with the interaction patterns found elsewhere in the application.

- **Consolidated Layout:** Merged the "Vendor Information" and "Feature Configuration" sections into a single "Basic Information" card. This simplifies the form, reduces clutter, and makes all settings accessible in one view.

- **Improved Prefill Groups:** Replaced the separate `Select` dropdowns for tag and endpoint groups with a more intuitive button-based system within the `extraText` of the `TagInput` components.

- **Additive Button Logic:** The prefill group buttons now operate in an additive mode. Users can click multiple group buttons to incrementally add tags or endpoints, with duplicates being automatically handled.

- **Clear Functionality:** Added "Clear" buttons for both tags and endpoints, allowing users to easily reset the fields.

- **Code Cleanup:** Removed the unused `endpointOptions` constant and unnecessary icon imports (`Building`, `Settings`) to keep the codebase clean.

Summary
• Backend
  – Moved duplicate-name validation and total vendor-count aggregation from controllers (`controller/model_meta.go`, `controller/vendor_meta.go`, `controller/prefill_group.go`) to model layer (`model/model_meta.go`, `model/vendor_meta.go`, `model/prefill_group.go`).
  – Added `GetVendorModelCounts()` and `Is*NameDuplicated()` helpers; controllers now call these instead of duplicating queries.
  – API response for `/api/models` now returns `vendor_counts` with per-vendor totals across all pages, plus `all` summary.
  – Removed redundant checks and unused imports, eliminating `go vet` warnings.

• Frontend
  – `useModelsData.js` updated to consume backend-supplied `vendor_counts`, calculate the `all` total once, and drop legacy client-side counting logic.
  – Simplified initial data flow: first render now triggers only one models request.
  – Deleted obsolete `updateVendorCounts` helper and related comments.
  – Ensured search flow also sets `vendorCounts`, keeping tab badges accurate.

Why
This refactor enforces single-responsibility (aggregation in model layer), delivers consistent totals irrespective of pagination, and removes redundant client queries, leading to cleaner code and better performance.