Three layered optimizations targeting Gemini-style 5MB base64 payloads where
RSS could balloon to tens of GB under concurrent load:
1. Byte-based param override (relay/common/override.go)
- Switch legacy/operations hot paths from common.Marshal round-trips and
map[string]any conversions to gjson/sjson on []byte directly.
- Avoids cloning 5MB strings during each Set/Delete operation.
2. strings.Builder for Gemini response markdown (relay/channel/gemini/relay-gemini.go)
- Replace string concatenation + strings.Join when assembling
"" content for inline image responses.
- Pre-allocates capacity from inline_data byte sizes.
3. Outbound BodyStorage + streaming Decoder (this commit's core)
- New relay/common/outbound_body.go helper wraps marshaled upstream bodies
in common.BodyStorage, allowing disk-cache mode to offload jsonData to
a temp file while waiting for upstream TTFB. The original []byte can
then be GC'd, removing ~5MB/req of heap residency during the longest
window of a request.
- All 7 relay handlers (gemini/claude/responses/embedding/image/compatible/
rerank) plus chat_completions_via_responses adopt the helper with
defer closer.Close() and explicit jsonData = nil.
- relay/common/relay_info.go: new UpstreamRequestBodySize so
relay/channel/api_request.go can populate req.ContentLength (lost when
body becomes a type-erased io.Reader).
- common/gin.go UnmarshalBodyReusable: when storage is disk-backed and
content-type is JSON, decode via DecodeJson(storage) instead of
storage.Bytes()+Unmarshal, removing one transient 5MB copy per request.
memory mode and form/multipart paths unchanged.
| Name |
Last commit
|
Last Update |
|---|---|---|
| .. | ||
| limiter | Loading commit data... | |
| api_type.go | Loading commit data... | |
| audio.go | Loading commit data... | |
| body_storage.go | Loading commit data... | |
| constants.go | Loading commit data... | |
| copy.go | Loading commit data... | |
| crypto.go | Loading commit data... | |
| custom-event.go | Loading commit data... | |
| database.go | Loading commit data... | |
| disk_cache.go | Loading commit data... | |
| disk_cache_config.go | Loading commit data... | |
| email-outlook-auth.go | Loading commit data... | |
| email.go | Loading commit data... | |
| embed-file-system.go | Loading commit data... | |
| endpoint_defaults.go | Loading commit data... | |
| endpoint_type.go | Loading commit data... | |
| env.go | Loading commit data... | |
| gin.go | Loading commit data... | |
| go-channel.go | Loading commit data... | |
| gopool.go | Loading commit data... | |
| hash.go | Loading commit data... | |
| init.go | Loading commit data... | |
| ip.go | Loading commit data... | |
| json.go | Loading commit data... | |
| json_test.go | Loading commit data... | |
| model.go | Loading commit data... | |
| page_info.go | Loading commit data... | |
| performance_config.go | Loading commit data... | |
| pprof.go | Loading commit data... | |
| pyro.go | Loading commit data... | |
| quota.go | Loading commit data... | |
| rate-limit.go | Loading commit data... | |
| redis.go | Loading commit data... | |
| ssrf_protection.go | Loading commit data... | |
| str.go | Loading commit data... | |
| sys_log.go | Loading commit data... | |
| system_monitor.go | Loading commit data... | |
| system_monitor_unix.go | Loading commit data... | |
| system_monitor_windows.go | Loading commit data... | |
| topup-ratio.go | Loading commit data... | |
| totp.go | Loading commit data... | |
| url_validator.go | Loading commit data... | |
| url_validator_test.go | Loading commit data... | |
| utils.go | Loading commit data... | |
| validate.go | Loading commit data... | |
| verification.go | Loading commit data... |