High-Concurrency Redis Access in OpenResty: Stable Connection Counts, Lower P99 Latency

Peak traffic often exposes a frustrating pattern. Day to day, the service handles heavy traffic smoothly, and the Redis instance still has plenty of CPU and memory headroom. Then, at the critical moment — a flash sale that opens on the hour or a sudden surge — the number of connections from the gateway to Redis jumps tens or even hundreds of times, instantly exhausting the connection quota and sometimes triggering a cascading failure across the entire request path. Or the backend order service could take more requests, yet users can’t get their pages to load because session lookups on the gateway side have slowed down. For businesses that rely on Redis for authentication, high-frequency rate limiting, and real-time state checks, both bottlenecks translate directly into stability risk, cloud costs, and a worse user experience.

The question engineering teams often face: can they eliminate connection storms under sudden traffic without reflexively upgrading the Redis instance, while still letting more users complete their actions at peak? Answering it means looking at two things: how the client keeps its connection count under control during a surge, and what throughput and tail latency (P99) its queries deliver under high concurrency.

To address both, OpenResty offers two high-performance Redis private libraries, one for standalone deployments and one for clusters:

Smooth Day to Day, Down at Peak? Stop Connection Spikes and Quota Anxiety

Let’s start with the problem most likely to fail you at the worst possible moment: connection count.

Many teams have been bitten by this. Daily traffic is already heavy, and the gateway works smoothly with Redis. But as soon as a flash sale, a traffic burst, or even a minor network hiccup hits, the whole service can go down within seconds.

The root cause is the model traditional clients use: each request holds a connection exclusively and returns it to the connection pool when it’s done. Under normal conditions requests are quick, and a few dozen persistent connections, constantly reused, can sustain a very high QPS. But at the critical moment, if concurrency suddenly spikes or a response is even slightly delayed, in-flight requests don’t return their connections to the pool in time. The requests flooding in behind them then trigger a “new-connection storm” tens or hundreds of times the normal size, setting off a chain reaction that ends in a cascading failure:

1. Local ports run out, so the gateway can’t reach Redis. The sudden burst of new connections quickly exhausts the ephemeral ports on the gateway machines, so subsequent requests fail outright and can’t connect to Redis.
2. The server is busy accepting connections, so everything slows down. Redis is suddenly flooded with connection requests, and the overhead of accepting them climbs sharply, directly slowing the responses to regular read and write commands.
3. Without connection reuse, every request gets slower. Unlike reusing an existing persistent connection, a new connection must go through a full TCP handshake and authentication, which significantly increases the time each Redis request takes.
4. Timeouts and retries pile up until everything collapses. Because client requests aren’t answered in time, upstream callers or users time out, disconnect, and immediately retry. The retry traffic pushes concurrency and new connections even higher, creating a vicious cycle that ends in a full business outage.

Beyond causing outages, this uncontrolled surge in connections also creates serious quota and cost anxiety in architectures where several services share one Redis. To keep the gateway from suddenly using up the connection quota at peak and dragging down core transaction services on the same instance, teams either struggle to negotiate connection quotas across departments, or end up paying to upgrade the Redis instance before every big sale — even though Redis hasn’t used up its CPU or memory, and the only real constraint is the peak connection count.

For routine reads and writes such as authentication, rate limiting, and session validation, lua-resty-redis-fast breaks this chain at the source: its connection count does not grow as request concurrency grows. As long as the gateway’s own footprint (the number of machines and workers) stays the same, its connection count stays stable even if concurrent online requests suddenly double, and it won’t trigger a flood of new connections.

So when a sudden surge hits, the gateway won’t exhaust its local ports or drag down Redis responses by frantically opening connections, and it won’t blow through the connection quota at the critical moment. If your existing Redis instance still has CPU and memory headroom, switching to the standalone private library should let your team absorb peak traffic within the current connection quota, and save the budget you would otherwise spend upgrading the instance just to “buy a higher connection limit.” One caveat: request-concurrency growth is not the same as scaling out the gateway. If you later add gateway nodes or workers, recalculate the total connection requirement for the new footprint.

In addition, lua-resty-redis-fast keeps the commonly used Lua interfaces of the open-source lua-resty-redis, making it easy for existing services to evaluate and migrate. For the exact scope of supported features, see the standalone client documentation.

When Upfront Lookups Slow Down, Users Wait for Pages and Results

Keeping connection count under control solves two problems at the critical moment: no collapse from a connection storm and no blown quota. During everyday peaks, though, whether each lookup completes almost instantly is what directly determines how long users wait. This matters equally for standalone and cluster deployments.

Take login-state or token validation at the gateway layer. When a user opens the order page or submits an action, the gateway must first read the session or quota state from Redis, and only after the check passes can it forward the request to the backend service. Even if the backend order service is idle, the request just sits and waits as long as this gateway step is stuck.

Users can’t see what happens inside the request path. All they see is a page that keeps spinning, or a submit button that doesn’t respond. If the upfront Redis lookup takes too long and eats into the timeout budget for the entire path, the request simply times out and fails, forcing users to refresh and retry again and again. So optimizing Redis lookups on the gateway side isn’t just about getting good benchmark numbers for a single endpoint. It’s about making sure core business flows actually complete.

Here, high throughput and low tail latency (P99) map directly to two practical business benefits:

• Higher throughput means requests get in. When gateway lookups are the bottleneck while backend services still have capacity, handling more lookups per second clears the congestion at the entrance, so more valid requests flow smoothly to the backend at peak.
• Lower P99 latency means requests don’t time out. Cutting the time of slow lookups leaves a wider time window for core logic downstream, such as order processing. This reduces the risk of a slow upfront check dragging down the entire request, and spares users long waits and repeated attempts.

Two GET Benchmarks: Much Higher Throughput, P99 Under 1 ms in Both

When session or state data is stored as key-value pairs, the gateway’s most important operation is high-frequency GET. Below, we use concurrent GET benchmarks to look at the read performance of the two private libraries. End-to-end request time also includes the network and downstream application logic, so evaluate it against your actual request path.

We compared each private library head to head with its open-source counterpart. The tables below show the median of each set of test results:

Standalone: 3.47x the GET throughput of the open-source client, with P99 latency cut by more than half. (Baseline: open-source lua-resty-redis v0.34)

Cluster: 7.14x the GET throughput of the open-source client, with P99 latency down from 6.8 ms to under 1 ms. (Baseline: open-source lua-resty-redis-cluster, with the same Lua concurrency)

Both sets of data show that the private libraries substantially raise GET throughput while flattening tail latency. This lines up with the two business goals above: letting more peak traffic through the gateway, and leaving enough time for backend processing.

About the tests: the data above comes from pure GET benchmarks designed to reflect performance differences between the clients themselves. They do not directly measure how many connections are saved. The final benefit in production also depends on the specific Redis commands, the network environment, the read strategy, and bottlenecks elsewhere in the request path outside the gateway.

With the cluster client, you can also choose a read strategy that matches your application’s consistency requirements; see the cluster client documentation for configuration details.

If your team is considering a Redis upgrade because the connection quota is running out, or wants to reduce request waits and timeouts caused by slow upfront lookups at peak, contact us to request a trial and measure, under your real production traffic, how much resource cost they can save and how much they can improve response times.

Learn More: Other Costs Along the Same Request Path

A Redis lookup is only one step in how the gateway handles a request. If Redis lookups are already fast enough but overall API response times or server load are still not where you want them, check whether time and CPU are being spent elsewhere along the same path. The OpenResty private libraries overview covers several areas that are often optimized alongside Redis:

• Session data or API payloads that require high-frequency JSON parsing: after pulling JSON session or configuration data out of Redis, the gateway usually needs to decode it right away. If your flame graph shows JSON encoding and decoding eating a lot of CPU, read lua-cjson Too Slow? jit.cjson Speeds Up JSON Encoding in OpenResty 18x to see how faster parsing frees up gateway capacity for business traffic.
• A gateway that also ships logs or business events to Kafka asynchronously: if the gateway collects and reports risk-control events or access logs while forwarding requests, slow or stalled reporting must not hold up the main request path. lua-resty-kafka-fast is purpose-built for producing to Kafka from OpenResty, sustaining high-throughput data collection while tightly limiting the impact on user request latency.
• Sudden traffic that approaches the gateway’s own CPU limit: when internal lookups and parsing are already optimized but external traffic surges still exceed what the current machines can handle, take a look at lua-resty-limit-traffic-dynamic. It dynamically adjusts HTTP rate-limiting thresholds based on a target CPU utilization, preserving CPU headroom during traffic surges so that requests already admitted can complete.

These private libraries target serialization, message reporting, and dynamic traffic control respectively, so teams can pick what they need based on their own performance bottlenecks. For the complete component list and integration guides, see the private libraries documentation.

About The Author

Yichun Zhang (Github handle: agentzh), is the original creator of the OpenResty^® open-source project and the CEO of OpenResty Inc..

Yichun is one of the earliest advocates and leaders of “open-source technology”. He worked at many internationally renowned tech companies, such as Cloudflare, Yahoo!. He is a pioneer of “edge computing”, “dynamic tracing” and “machine coding”, with over 22 years of programming and 16 years of open source experience. Yichun is well-known in the open-source space as the project leader of OpenResty^®, adopted by more than 40 million global website domains.

OpenResty Inc., the enterprise software start-up founded by Yichun in 2017, has customers from some of the biggest companies in the world. Its flagship product, OpenResty XRay, is a non-invasive profiling and troubleshooting tool that significantly enhances and utilizes dynamic tracing technology. And its OpenResty Edge product is a powerful distributed traffic management and private CDN software product.

As an avid open-source contributor, Yichun has contributed more than a million lines of code to numerous open-source projects, including Linux kernel, Nginx, LuaJIT, GDB, SystemTap, LLVM, Perl, etc. He has also authored more than 60 open-source software libraries.