Tiered Token Bucket Rate Limiter
A robust Application-Layer (Layer 7) defense system designed to protect microservice architectures from DDoS attacks. Acting as a high-performance API Gateway, it utilizes the Token Bucket algorithm to enforce strict usage quotas with sub-millisecond overhead.
System Architecture
The system is architected as a centralized API Gateway acting as a reverse proxy. All external client traffic is routed through this enforcement point, allowing excess or malicious requests to be rejected before reaching backend microservices. Thread-safe lock primitives ensure atomic token consumption even under extreme concurrency.
Algorithmic Design: Token Bucket
The Token Bucket algorithm provides fine-grained control over traffic bursts while enforcing a strict long-term request rate. Unlike fixed-window counters, it allows short bursts without violating average throughput guarantees.
Each request consumes a token from the client’s bucket. If no tokens remain, the request is immediately rejected with an HTTP 429 response.
Tokens are replenished at a fixed refill rate until the maximum bucket capacity is reached, allowing sustained but controlled access.
The Challenge
DDoS attacks can quickly exhaust backend CPU and connection pools. In a microservice environment, a single unprotected endpoint can cause cascading failures across the entire system. The goal was to build a low-latency gateway that could differentiate between legitimate traffic and malicious floods in real-time.
The Solution
Developed a centralized API Gateway as a reverse proxy that implements lazy refill token logic for O(1) efficiency. The system uses a tiered policy model, allowing dynamic capacity upgrades from 10 to 50 tokens and faster refill rates for authenticated Gold-tier clients.
Experimental Load Testing
Load-Test-Results.png&w=3840&q=75)
-Load-Test-Results.png&w=3840&q=75)