-
Refactor: Optimize the request rate limiting for ModelRequestRateLimitCount. · 5aa2076e
Reason: The original steps 1 and 3 in the redisRateLimitHandler method were not atomic, leading to poor precision under high concurrent requests. For example, with a rate limit set to 60, sending 200 concurrent requests would result in none being blocked, whereas theoretically around 140 should be intercepted. Solution: I chose not to merge steps 1 and 3 into a single Lua script because a single atomic operation involving read, write, and delete operations could suffer from performance issues under high concurrency. Instead, I implemented a token bucket algorithm to optimize this, reducing the atomic operation to just read and write steps while significantly decreasing the memory footprint.
霍雨佳 committed
| Name |
Last commit
|
Last Update |
|---|---|---|
| .. | ||
| rate_limit.lua | Loading commit data... |