Skip to content

phsl / new-api

Go to a project
Commit 8d87d5fd by Seefs Committed by GitHub
Options

test: clean up reward-hacking backend tests (#5563)

parent b798e349
Showing with 56 additions and 485 deletions
AGENTS.md
......@@ -143,3 +143,20 @@ When creating a pull request:
- First compare the current git user (`git config user.name` / `git config user.email`) with the repository's historical core developers (for example, the recurring top authors in `git log`). Do not change git config.
- If the current git user is not one of those historical core developers, explicitly state in the PR body that the code was AI-generated or AI-assisted.
- Always use the repository PR template at `.github/PULL_REQUEST_TEMPLATE.md` when drafting the PR title/body. Preserve the template structure and fill in the relevant sections instead of replacing it with an ad hoc format.
### Rule 9: Backend Test Quality — No Reward-Hacking Tests
Backend tests must protect real behavior, API contracts, billing/accounting invariants, data compatibility, or regression paths. Do not add tests that only improve coverage numbers, prove that code happens to run, or lock in an implementation detail without a user-visible or cross-module contract.
Avoid these test shapes:
- Fake fuzz, stress, smoke, or performance tests built from random inputs, large loop counts, sleeps, timing comparisons, or log-only assertions.
- Duplicate tests that exercise the same branch with different names but no new invariant.
- Tests that force an incorrect provider or protocol semantic into production code.
- Tests that assert private constants, select-field lists, helper internals, or file layout when the observable behavior is already covered elsewhere.
- Hand-written replacements for standard library helpers inside tests.
Prefer deterministic table tests with explicit inputs and exact expected outputs. Merge overlapping tests, remove unclear or redundant cases, and keep file names aligned with the domain or module under test. When a test needs database, request context, user group, settings, or cache state, initialize that state explicitly inside the test fixture rather than relying on global leftovers from other tests.
New or substantially rewritten Go backend tests MUST use `github.com/stretchr/testify/require` for setup and fatal assertions, and `github.com/stretchr/testify/assert` for non-fatal value checks. Avoid hand-written assertion helpers unless they encode a reusable project-specific invariant.
When cleaning tests, preserve meaningful regression coverage. If a deleted test was covering a real contract indirectly, replace it with a smaller test that names and asserts that contract directly.
CLAUDE.md
......@@ -143,3 +143,20 @@ When creating a pull request:
- First compare the current git user (`git config user.name` / `git config user.email`) with the repository's historical core developers (for example, the recurring top authors in `git log`). Do not change git config.
- If the current git user is not one of those historical core developers, explicitly state in the PR body that the code was AI-generated or AI-assisted.
- Always use the repository PR template at `.github/PULL_REQUEST_TEMPLATE.md` when drafting the PR title/body. Preserve the template structure and fill in the relevant sections instead of replacing it with an ad hoc format.
### Rule 9: Backend Test Quality — No Reward-Hacking Tests
Backend tests must protect real behavior, API contracts, billing/accounting invariants, data compatibility, or regression paths. Do not add tests that only improve coverage numbers, prove that code happens to run, or lock in an implementation detail without a user-visible or cross-module contract.
Avoid these test shapes:
- Fake fuzz, stress, smoke, or performance tests built from random inputs, large loop counts, sleeps, timing comparisons, or log-only assertions.
- Duplicate tests that exercise the same branch with different names but no new invariant.
- Tests that force an incorrect provider or protocol semantic into production code.
- Tests that assert private constants, select-field lists, helper internals, or file layout when the observable behavior is already covered elsewhere.
- Hand-written replacements for standard library helpers inside tests.
Prefer deterministic table tests with explicit inputs and exact expected outputs. Merge overlapping tests, remove unclear or redundant cases, and keep file names aligned with the domain or module under test. When a test needs database, request context, user group, settings, or cache state, initialize that state explicitly inside the test fixture rather than relying on global leftovers from other tests.
New or substantially rewritten Go backend tests MUST use `github.com/stretchr/testify/require` for setup and fatal assertions, and `github.com/stretchr/testify/assert` for non-fatal value checks. Avoid hand-written assertion helpers unless they encode a reusable project-specific invariant.
When cleaning tests, preserve meaningful regression coverage. If a deleted test was covering a real contract indirectly, replace it with a smaller test that names and asserts that contract directly.
common/url_validator_test.go
package common
import (
"strings"
"testing"
"github.com/QuantumNous/new-api/constant"
......@@ -107,7 +108,7 @@ func TestValidateRedirectURL(t *testing.T) {
t.Errorf("ValidateRedirectURL(%q) expected error containing %q, got nil", tt.url, tt.errContains)
return
}
if tt.errContains != "" && !contains(err.Error(), tt.errContains) {
if tt.errContains != "" && !strings.Contains(err.Error(), tt.errContains) {
t.Errorf("ValidateRedirectURL(%q) error = %q, want error containing %q", tt.url, err.Error(), tt.errContains)
}
} else {
......@@ -118,17 +119,3 @@ func TestValidateRedirectURL(t *testing.T) {
})
}
}
func contains(s, substr string) bool {
return len(s) >= len(substr) && (s == substr || len(substr) == 0 ||
(len(s) > 0 && len(substr) > 0 && findSubstring(s, substr)))
}
func findSubstring(s, substr string) bool {
for i := 0; i <= len(s)-len(substr); i++ {
if s[i:i+len(substr)] == substr {
return true
}
}
return false
}
controller/channel_upstream_update_test.go
......@@ -81,10 +81,6 @@ func TestCollectPendingApplyUpstreamModelChanges(t *testing.T) {
require.Equal(t, []string{"old-model"}, pendingRemoveModels)
}
func TestChannelUpstreamModelUpdateSelectFieldsIncludeModelMapping(t *testing.T) {
require.Contains(t, channelUpstreamModelUpdateSelectFields, "model_mapping")
}
func TestNormalizeChannelModelMapping(t *testing.T) {
modelMapping := `{
" alias-model ": " upstream-model ",
......
controller/model_list_test.go
......@@ -220,10 +220,12 @@ func TestListModelsTokenLimitIncludesTieredBillingModel(t *testing.T) {
"zz-token-tiered-visible-model": `tier("base", p * 1 + c * 2)`,
"zz-token-tiered-empty-expr-model": "",
})
setupModelListControllerTestDB(t)
recorder := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(recorder)
ctx.Request = httptest.NewRequest(http.MethodGet, "/v1/models", nil)
common.SetContextKey(ctx, constant.ContextKeyUserGroup, "default")
common.SetContextKey(ctx, constant.ContextKeyTokenModelLimitEnabled, true)
common.SetContextKey(ctx, constant.ContextKeyTokenModelLimit, map[string]bool{
"zz-token-tiered-visible-model": true,
......
pkg/billingexpr/billingexpr_test.go
......@@ -2,7 +2,6 @@ package billingexpr_test
import (
"math"
"math/rand"
"testing"
"github.com/QuantumNous/new-api/pkg/billingexpr"
......@@ -594,91 +593,6 @@ func TestComputeTieredQuota_WithCacheCrossTier(t *testing.T) {
}
// ---------------------------------------------------------------------------
// Fuzz: random p/c/cache, verify non-negative result
// ---------------------------------------------------------------------------
func TestFuzz_NonNegativeResults(t *testing.T) {
exprs := []string{
claudeExpr,
claudeWithCacheExpr,
glmExpr,
"p * 0.5 + c * 1.0",
"max(p, c) * 0.1",
"p * 0.5 + cr * 0.1 + cc * 0.2",
}
rng := rand.New(rand.NewSource(42))
for _, exprStr := range exprs {
for i := 0; i < 500; i++ {
params := billingexpr.TokenParams{
P: math.Round(rng.Float64() * 1000000),
C: math.Round(rng.Float64() * 500000),
CR: math.Round(rng.Float64() * 200000),
CC: math.Round(rng.Float64() * 50000),
CC1h: math.Round(rng.Float64() * 10000),
}
cost, _, err := billingexpr.RunExpr(exprStr, params)
if err != nil {
t.Fatalf("expr=%q params=%+v: %v", exprStr, params, err)
}
if cost < 0 {
t.Errorf("expr=%q params=%+v: negative cost %f", exprStr, params, cost)
}
}
}
}
func TestFuzz_SettlementConsistency(t *testing.T) {
rng := rand.New(rand.NewSource(99))
for i := 0; i < 200; i++ {
estParams := billingexpr.TokenParams{
P: math.Round(rng.Float64() * 500000),
C: math.Round(rng.Float64() * 100000),
CR: math.Round(rng.Float64() * 100000),
CC: math.Round(rng.Float64() * 30000),
}
actParams := billingexpr.TokenParams{
P: math.Round(rng.Float64() * 500000),
C: math.Round(rng.Float64() * 100000),
CR: math.Round(rng.Float64() * 100000),
CC: math.Round(rng.Float64() * 30000),
}
groupRatio := 0.5 + rng.Float64()*2.0
estCost, estTrace, _ := billingexpr.RunExpr(claudeWithCacheExpr, estParams)
const qpu = 500_000.0
snap := &billingexpr.BillingSnapshot{
BillingMode: "tiered_expr",
ExprString: claudeWithCacheExpr,
ExprHash: billingexpr.ExprHashString(claudeWithCacheExpr),
GroupRatio: groupRatio,
EstimatedPromptTokens: int(estParams.P),
EstimatedCompletionTokens: int(estParams.C),
EstimatedQuotaBeforeGroup: estCost / 1_000_000 * qpu,
EstimatedQuotaAfterGroup: billingexpr.QuotaRound(estCost / 1_000_000 * qpu * groupRatio),
EstimatedTier: estTrace.MatchedTier,
QuotaPerUnit: qpu,
}
result, err := billingexpr.ComputeTieredQuota(snap, actParams)
if err != nil {
t.Fatalf("iter %d: %v", i, err)
}
directCost, _, _ := billingexpr.RunExpr(claudeWithCacheExpr, actParams)
directQuota := billingexpr.QuotaRound(directCost / 1_000_000 * qpu * groupRatio)
if result.ActualQuotaAfterGroup != directQuota {
t.Errorf("iter %d: settlement %d != direct %d", i, result.ActualQuotaAfterGroup, directQuota)
}
}
}
// ---------------------------------------------------------------------------
// Settlement-level tests for ComputeTieredQuota
// ---------------------------------------------------------------------------
......
relay/channel/claude/relay_claude_test.go
package claude
import (
"encoding/base64"
"strings"
"testing"
......@@ -279,41 +278,6 @@ func TestBuildOpenAIStyleUsageFromClaudeUsageDefaultsAggregateCacheCreationTo5m(
require.Equal(t, 0, openAIUsage.ClaudeCacheCreation1hTokens)
}
func TestRequestOpenAI2ClaudeMessage_IgnoresUnsupportedFileContent(t *testing.T) {
request := dto.GeneralOpenAIRequest{
Model: "claude-3-5-sonnet",
Messages: []dto.Message{
{
Role: "user",
Content: []any{
dto.MediaContent{
Type: dto.ContentTypeText,
Text: "see attachment",
},
dto.MediaContent{
Type: dto.ContentTypeFile,
File: &dto.MessageFile{
FileName: "blob.bin",
FileData: "JVBERi0xLjQK",
},
},
},
},
},
}
claudeRequest, err := RequestOpenAI2ClaudeMessage(nil, request)
require.NoError(t, err)
require.Len(t, claudeRequest.Messages, 1)
content, ok := claudeRequest.Messages[0].Content.([]dto.ClaudeMediaMessage)
require.True(t, ok)
require.Len(t, content, 1)
require.Equal(t, "text", content[0].Type)
require.NotNil(t, content[0].Text)
require.Equal(t, "see attachment", *content[0].Text)
}
func TestRequestOpenAI2ClaudeMessage_ClaudeOpus48HighUsesAdaptiveThinking(t *testing.T) {
request := dto.GeneralOpenAIRequest{
Model: "claude-opus-4-8-high",
......@@ -365,74 +329,3 @@ func TestRequestOpenAI2ClaudeMessage_ClaudeOpus48ThinkingUsesAdaptiveHighEffort(
require.Nil(t, claudeRequest.TopP)
require.Nil(t, claudeRequest.TopK)
}
func TestRequestOpenAI2ClaudeMessage_SupportsPDFFileContent(t *testing.T) {
request := dto.GeneralOpenAIRequest{
Model: "claude-3-5-sonnet",
Messages: []dto.Message{
{
Role: "user",
Content: []any{
dto.MediaContent{
Type: dto.ContentTypeFile,
File: &dto.MessageFile{
FileName: "spec.pdf",
FileData: "JVBERi0xLjQK",
},
},
dto.MediaContent{
Type: dto.ContentTypeText,
Text: "summarize it",
},
},
},
},
}
claudeRequest, err := RequestOpenAI2ClaudeMessage(nil, request)
require.NoError(t, err)
require.Len(t, claudeRequest.Messages, 1)
content, ok := claudeRequest.Messages[0].Content.([]dto.ClaudeMediaMessage)
require.True(t, ok)
require.Len(t, content, 2)
require.Equal(t, "document", content[0].Type)
require.NotNil(t, content[0].Source)
require.Equal(t, "base64", content[0].Source.Type)
require.Equal(t, "application/pdf", content[0].Source.MediaType)
require.Equal(t, "JVBERi0xLjQK", content[0].Source.Data)
require.Equal(t, "text", content[1].Type)
require.NotNil(t, content[1].Text)
require.Equal(t, "summarize it", *content[1].Text)
}
func TestRequestOpenAI2ClaudeMessage_ConvertsTextFileContentToText(t *testing.T) {
request := dto.GeneralOpenAIRequest{
Model: "claude-3-5-sonnet",
Messages: []dto.Message{
{
Role: "user",
Content: []any{
dto.MediaContent{
Type: dto.ContentTypeFile,
File: &dto.MessageFile{
FileName: "notes.txt",
FileData: base64.StdEncoding.EncodeToString([]byte("alpha\nbeta")),
},
},
},
},
},
}
claudeRequest, err := RequestOpenAI2ClaudeMessage(nil, request)
require.NoError(t, err)
require.Len(t, claudeRequest.Messages, 1)
content, ok := claudeRequest.Messages[0].Content.([]dto.ClaudeMediaMessage)
require.True(t, ok)
require.Len(t, content, 1)
require.Equal(t, "text", content[0].Type)
require.NotNil(t, content[0].Text)
require.Equal(t, "alpha\nbeta", *content[0].Text)
}
relay/helper/stream_scanner_test.go
......@@ -22,17 +22,14 @@ import (
func init() {
gin.SetMode(gin.TestMode)
if constant.StreamingTimeout == 0 {
constant.StreamingTimeout = 30
}
}
func setupStreamTest(t *testing.T, body io.Reader) (*gin.Context, *http.Response, *relaycommon.RelayInfo) {
t.Helper()
oldTimeout := constant.StreamingTimeout
constant.StreamingTimeout = 30
t.Cleanup(func() {
constant.StreamingTimeout = oldTimeout
})
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
......@@ -57,16 +54,6 @@ func buildSSEBody(n int) string {
return b.String()
}
type slowReader struct {
r io.Reader
delay time.Duration
}
func (s *slowReader) Read(p []byte) (int, error) {
time.Sleep(s.delay)
return s.r.Read(p)
}
// ---------- Basic correctness ----------
func TestStreamScannerHandler_NilInputs(t *testing.T) {
......@@ -127,26 +114,6 @@ func TestStreamScannerHandler_1000Chunks(t *testing.T) {
assert.Equal(t, numChunks, info.ReceivedResponseCount)
}
func TestStreamScannerHandler_10000Chunks(t *testing.T) {
t.Parallel()
const numChunks = 10000
body := buildSSEBody(numChunks)
c, resp, info := setupStreamTest(t, strings.NewReader(body))
var count atomic.Int64
start := time.Now()
StreamScannerHandler(c, resp, info, func(data string, sr *StreamResult) {
count.Add(1)
})
elapsed := time.Since(start)
assert.Equal(t, int64(numChunks), count.Load())
assert.Equal(t, numChunks, info.ReceivedResponseCount)
t.Logf("10000 chunks processed in %v", elapsed)
}
func TestStreamScannerHandler_OrderPreserved(t *testing.T) {
t.Parallel()
......@@ -243,98 +210,9 @@ func TestStreamScannerHandler_DataWithExtraSpaces(t *testing.T) {
assert.Equal(t, "{\"trimmed\":true}", got)
}
// ---------- Decoupling ----------
func TestStreamScannerHandler_ScannerDecoupledFromSlowHandler(t *testing.T) {
t.Parallel()
const numChunks = 50
const upstreamDelay = 10 * time.Millisecond
const handlerDelay = 20 * time.Millisecond
pr, pw := io.Pipe()
go func() {
defer pw.Close()
for i := 0; i < numChunks; i++ {
fmt.Fprintf(pw, "data: {\"id\":%d}\n", i)
time.Sleep(upstreamDelay)
}
fmt.Fprint(pw, "data: [DONE]\n")
}()
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
oldTimeout := constant.StreamingTimeout
constant.StreamingTimeout = 30
t.Cleanup(func() { constant.StreamingTimeout = oldTimeout })
resp := &http.Response{Body: pr}
info := &relaycommon.RelayInfo{ChannelMeta: &relaycommon.ChannelMeta{}}
var count atomic.Int64
start := time.Now()
done := make(chan struct{})
go func() {
StreamScannerHandler(c, resp, info, func(data string, sr *StreamResult) {
time.Sleep(handlerDelay)
count.Add(1)
})
close(done)
}()
select {
case <-done:
case <-time.After(15 * time.Second):
t.Fatal("StreamScannerHandler did not complete in time")
}
elapsed := time.Since(start)
assert.Equal(t, int64(numChunks), count.Load())
coupledTime := time.Duration(numChunks) * (upstreamDelay + handlerDelay)
t.Logf("elapsed=%v, coupled_estimate=%v", elapsed, coupledTime)
assert.Less(t, elapsed, coupledTime*85/100,
"decoupled elapsed time (%v) should be significantly less than coupled estimate (%v)", elapsed, coupledTime)
}
func TestStreamScannerHandler_SlowUpstreamFastHandler(t *testing.T) {
t.Parallel()
const numChunks = 50
body := buildSSEBody(numChunks)
reader := &slowReader{r: strings.NewReader(body), delay: 2 * time.Millisecond}
c, resp, info := setupStreamTest(t, reader)
var count atomic.Int64
start := time.Now()
done := make(chan struct{})
go func() {
StreamScannerHandler(c, resp, info, func(data string, sr *StreamResult) {
count.Add(1)
})
close(done)
}()
select {
case <-done:
case <-time.After(15 * time.Second):
t.Fatal("timed out with slow upstream")
}
elapsed := time.Since(start)
assert.Equal(t, int64(numChunks), count.Load())
t.Logf("slow upstream (%d chunks, 2ms/read): %v", numChunks, elapsed)
}
// ---------- Ping tests ----------
func TestStreamScannerHandler_PingSentDuringSlowUpstream(t *testing.T) {
t.Parallel()
setting := operation_setting.GetGeneralSetting()
oldEnabled := setting.PingIntervalEnabled
oldSeconds := setting.PingIntervalSeconds
......@@ -348,9 +226,9 @@ func TestStreamScannerHandler_PingSentDuringSlowUpstream(t *testing.T) {
pr, pw := io.Pipe()
go func() {
defer pw.Close()
for i := 0; i < 7; i++ {
for i := 0; i < 4; i++ {
fmt.Fprintf(pw, "data: chunk_%d\n", i)
time.Sleep(500 * time.Millisecond)
time.Sleep(400 * time.Millisecond)
}
fmt.Fprint(pw, "data: [DONE]\n")
}()
......@@ -359,12 +237,6 @@ func TestStreamScannerHandler_PingSentDuringSlowUpstream(t *testing.T) {
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
oldTimeout := constant.StreamingTimeout
constant.StreamingTimeout = 30
t.Cleanup(func() {
constant.StreamingTimeout = oldTimeout
})
resp := &http.Response{Body: pr}
info := &relaycommon.RelayInfo{ChannelMeta: &relaycommon.ChannelMeta{}}
......@@ -379,22 +251,19 @@ func TestStreamScannerHandler_PingSentDuringSlowUpstream(t *testing.T) {
select {
case <-done:
case <-time.After(15 * time.Second):
case <-time.After(5 * time.Second):
t.Fatal("timed out waiting for stream to finish")
}
assert.Equal(t, int64(7), count.Load())
assert.Equal(t, int64(4), count.Load())
body := recorder.Body.String()
pingCount := strings.Count(body, ": PING")
t.Logf("received %d pings in response body", pingCount)
assert.GreaterOrEqual(t, pingCount, 2,
"expected at least 2 pings during 3.5s stream with 1s interval; got %d", pingCount)
assert.GreaterOrEqual(t, pingCount, 1,
"expected at least 1 ping during slow stream with 1s interval; got %d", pingCount)
}
func TestStreamScannerHandler_PingDisabledByRelayInfo(t *testing.T) {
t.Parallel()
setting := operation_setting.GetGeneralSetting()
oldEnabled := setting.PingIntervalEnabled
oldSeconds := setting.PingIntervalSeconds
......@@ -405,27 +274,11 @@ func TestStreamScannerHandler_PingDisabledByRelayInfo(t *testing.T) {
setting.PingIntervalSeconds = oldSeconds
})
pr, pw := io.Pipe()
go func() {
defer pw.Close()
for i := 0; i < 5; i++ {
fmt.Fprintf(pw, "data: chunk_%d\n", i)
time.Sleep(500 * time.Millisecond)
}
fmt.Fprint(pw, "data: [DONE]\n")
}()
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
oldTimeout := constant.StreamingTimeout
constant.StreamingTimeout = 30
t.Cleanup(func() {
constant.StreamingTimeout = oldTimeout
})
resp := &http.Response{Body: pr}
resp := &http.Response{Body: io.NopCloser(strings.NewReader(buildSSEBody(5)))}
info := &relaycommon.RelayInfo{
DisablePing: true,
ChannelMeta: &relaycommon.ChannelMeta{},
......@@ -442,7 +295,7 @@ func TestStreamScannerHandler_PingDisabledByRelayInfo(t *testing.T) {
select {
case <-done:
case <-time.After(15 * time.Second):
case <-time.After(5 * time.Second):
t.Fatal("timed out")
}
......@@ -528,13 +381,13 @@ func TestStreamScannerHandler_StreamStatus_HandlerDone(t *testing.T) {
func TestStreamScannerHandler_StreamStatus_Timeout(t *testing.T) {
// Not parallel: modifies global constant.StreamingTimeout
oldTimeout := constant.StreamingTimeout
constant.StreamingTimeout = 2
constant.StreamingTimeout = 1
t.Cleanup(func() { constant.StreamingTimeout = oldTimeout })
pr, pw := io.Pipe()
go func() {
fmt.Fprint(pw, "data: {\"id\":1}\n")
time.Sleep(10 * time.Second)
time.Sleep(2 * time.Second)
pw.Close()
}()
......@@ -553,7 +406,7 @@ func TestStreamScannerHandler_StreamStatus_Timeout(t *testing.T) {
select {
case <-done:
case <-time.After(15 * time.Second):
case <-time.After(5 * time.Second):
t.Fatal("timed out waiting for stream timeout")
}
......@@ -645,63 +498,3 @@ func TestStreamScannerHandler_StreamStatus_ReplacesPreInitialized(t *testing.T)
assert.Equal(t, relaycommon.StreamEndReasonDone, info.StreamStatus.EndReason)
assert.Equal(t, 0, info.StreamStatus.TotalErrorCount())
}
func TestStreamScannerHandler_PingInterleavesWithSlowUpstream(t *testing.T) {
t.Parallel()
setting := operation_setting.GetGeneralSetting()
oldEnabled := setting.PingIntervalEnabled
oldSeconds := setting.PingIntervalSeconds
setting.PingIntervalEnabled = true
setting.PingIntervalSeconds = 1
t.Cleanup(func() {
setting.PingIntervalEnabled = oldEnabled
setting.PingIntervalSeconds = oldSeconds
})
pr, pw := io.Pipe()
go func() {
defer pw.Close()
for i := 0; i < 10; i++ {
fmt.Fprintf(pw, "data: chunk_%d\n", i)
time.Sleep(500 * time.Millisecond)
}
fmt.Fprint(pw, "data: [DONE]\n")
}()
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
oldTimeout := constant.StreamingTimeout
constant.StreamingTimeout = 30
t.Cleanup(func() {
constant.StreamingTimeout = oldTimeout
})
resp := &http.Response{Body: pr}
info := &relaycommon.RelayInfo{ChannelMeta: &relaycommon.ChannelMeta{}}
var count atomic.Int64
done := make(chan struct{})
go func() {
StreamScannerHandler(c, resp, info, func(data string, sr *StreamResult) {
count.Add(1)
})
close(done)
}()
select {
case <-done:
case <-time.After(15 * time.Second):
t.Fatal("timed out")
}
assert.Equal(t, int64(10), count.Load())
body := recorder.Body.String()
pingCount := strings.Count(body, ": PING")
t.Logf("received %d pings interleaved with 10 chunks over 5s", pingCount)
assert.GreaterOrEqual(t, pingCount, 3,
"expected at least 3 pings during 5s stream with 1s ping interval; got %d", pingCount)
}
service/task_billing_test.go
......@@ -684,7 +684,7 @@ func TestSettle_PerCallBilling_SkipsTotalTokens(t *testing.T) {
assert.Equal(t, int64(0), countLogs(t))
}
func TestSettle_NonPerCall_AdaptorAdjustWorks(t *testing.T) {
func TestSettle_NonPerCallBilling_AppliesAdaptorAdjustment(t *testing.T) {
truncate(t)
ctx := context.Background()
......
service/tiered_settle_test.go
......@@ -3,7 +3,6 @@ package service
import (
"math"
"math/rand"
"sync"
"testing"
"github.com/QuantumNous/new-api/dto"
......@@ -695,11 +694,6 @@ func TestBuildTieredTokenParams_Len_TierCondition(t *testing.T) {
}
}
// ---------------------------------------------------------------------------
// Stress test: 1000 concurrent goroutines, complex tiered expr vs ratio,
// random token counts, verify correctness and measure performance
// ---------------------------------------------------------------------------
const complexTieredExpr = `p <= 200000 ? tier("standard", p * 3 + c * 15 + cr * 0.3 + cc * 3.75 + cc1h * 6 + img * 3 + img_o * 30 + ai * 10 + ao * 40) : tier("long_context", p * 6 + c * 22.5 + cr * 0.6 + cc * 7.5 + cc1h * 12 + img * 6 + img_o * 60 + ai * 20 + ao * 80)`
func randomUsage(rng *rand.Rand) *dto.Usage {
......@@ -731,51 +725,6 @@ func randomUsage(rng *rand.Rand) *dto.Usage {
}
}
func TestStress_TieredBilling_1000Concurrent(t *testing.T) {
usedVars := billingexpr.UsedVars(complexTieredExpr)
var wg sync.WaitGroup
errCh := make(chan string, 1000)
for i := 0; i < 1000; i++ {
wg.Add(1)
go func(seed int64) {
defer wg.Done()
rng := rand.New(rand.NewSource(seed))
for j := 0; j < 100; j++ {
usage := randomUsage(rng)
groupRatio := 0.5 + rng.Float64()*2.0
params := BuildTieredTokenParams(usage, false, usedVars)
cost, trace, err := billingexpr.RunExpr(complexTieredExpr, params)
if err != nil {
errCh <- err.Error()
return
}
if cost < 0 {
errCh <- "negative cost"
return
}
quota := billingexpr.QuotaRound(cost / 1_000_000 * testQuotaPerUnit * groupRatio)
if quota < 0 {
errCh <- "negative quota"
return
}
_ = trace.MatchedTier
}
}(int64(i))
}
wg.Wait()
close(errCh)
for e := range errCh {
t.Fatal(e)
}
}
func BenchmarkTieredBilling_ComplexExpr(b *testing.B) {
rng := rand.New(rand.NewSource(42))
usedVars := billingexpr.UsedVars(complexTieredExpr)
......
web/default/AGENTS.md
......@@ -145,6 +145,9 @@
- 工具函数与纯逻辑优先单元测试（Vitest），测试文件 `*.test.ts`；组件用 React Testing Library 测交互与行为，避免测实现细节。
- 关键流程补充集成与 E2E（如 MSW 模拟 API、Playwright/Cypress）；核心功能目标覆盖率 80% 以上，关注业务路径与关键分支。
- 测试必须保护真实用户行为、稳定 API 契约或明确回归路径；禁止为了覆盖率添加 smoke、sleep/timing、随机输入、日志输出或只证明代码运行的测试。
- 新增或大幅重写测试时优先使用 Vitest 与 React Testing Library 的标准断言和查询方式，避免手写通用断言辅助函数；只有表达项目特定业务不变量时才抽取测试 helper。
- 清理测试时先合并重复场景、删除不明不白的实现细节断言；若旧测试间接覆盖了真实契约，需替换为更小、更直接的行为测试。
### 3.15 依赖管理
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment