Limit Request Rate by Custom Keys in OpenResty Edge

OpenResty Edge rate limiting caps the request rate per custom key—client IP address, URI, URI argument, or cookie—using two thresholds: a soft “Shape at” rate that delays excess requests, and a hard “Reject at” rate that blocks them outright. This protects both your gateway and origin servers when clients send requests too fast, such as under a denial-of-service attack, so the servers never get overloaded. Below we add a rate-limiting page rule step by step, test it with wrk, and extend the same limits to SSL handshakes.

Add a request rate-limiting page rule in OpenResty Edge

Let’s go to a web console of OpenResty Edge. This is our sample deployment of the console. Every user would have their own deployment.

We can still use our continuing sample application, test-edge.com. Enter the application.

We already have a page rule defined.

This page rule sets up a reverse proxy to a pre-defined upstream, and there’s no request rate limiting defined yet.

Now let’s edit the existing page rule to add rate limiting. Add a new action and search for “Limit request rate”.

First we need to specify the key for rate limiting.

There are a lot of possible key types—the client IP address, the URI, the URI argument, a cookie, and many more. Choose the default key type, client IP address. Then the limit applies to the scope of each unique client IP address.

The “Shape at” rate is a soft limit. When a client tries to send requests faster than this rate, the gateway server delays those excessive requests to match this rate. So the faster the client sends requests, the longer the delay the gateway adds. Here we specify a rate of 50 requests per second.

Because we set the key type to client IP address, the limit is applied to individual client IP addresses.

The “Reject at” rate is a hard limit. When a client sends requests fast enough to exceed this hard limit, we can take more aggressive action like blocking them immediately without waiting. Specify a rate of 100 requests per second here.

We can choose from different interception actions—like closing the current connection right away, returning an error page, or returning a captcha page to block bots. Select the default “Error Page” action, which returns an error page over HTTP.

We use the default HTTP status code, 503, to mean service unavailable.

Save the changes to this page rule. Then we make a new release to push out the changes, as always.

Once released, the new page rule is synchronized to all our gateway servers.

Now the new page rule has been pushed to all the gateway clusters and servers. Our configuration changes do NOT require server reload, restart, or binary upgrade, so it’s very efficient and scalable.

Test the rate limit with wrk

Next, we’ll verify the effect of the new rate limits.

On the terminal, we can send a lot of requests very fast through the open-source utility named wrk.

wrk -c 50 -d 1s http://test-edge.com/

Here we use a concurrency level of 50 first. Note the -c option.

Run the command. The actual request rate is about 50 requests per second. This is our “Shape at” rate.

Now we increase the concurrency level to make wrk send requests much faster.

wrk -c 128 -d 1s http://test-edge.com/

Note the 128 concurrency level.

Run it! Note that there are a lot of rejected requests with erroneous responses.

The actual request rate is high this time, just because the server rejects those excessive requests very fast.

Limit the rate of SSL/TLS handshakes

In addition to limiting the rate of requests, OpenResty Edge can also limit the rate of SSL or TLS handshakes of HTTPS requests.

On this page we can configure SSL handshake rate limiting. Turn on the switch to reveal the configuration parameters. They are the same as the request rate limiting feature—choose a key, then set a “Shape at” and a “Reject at” rate.

Rate limiting handles application-layer request floods by shaping or rejecting the excess. It works alongside the other security layers in OpenResty Edge: the built-in WAF filters malicious requests — see the OpenResty Edge WAF performance benchmark against ModSecurity — while volumetric network-layer floods are dropped in the kernel by XDP-based DDoS mitigation.

OpenResty Edge rate limiting FAQ

How does OpenResty Edge rate limiting decide which requests to limit?

It limits by a key you choose. The default key is the client IP address, so the limit applies to each unique client IP address, but you can also key on the URI, a URI argument, a request cookie, the address in X-Forwarded-For, or a request header.

What is the difference between the “Shape at” and “Reject at” rates?

“Shape at” is a soft limit: the gateway delays requests that exceed the rate to match it, so the faster a client sends, the longer the delay it gets. “Reject at” is a hard limit: once a client exceeds it, the gateway takes a more aggressive interception action instead of waiting.

What happens to requests that exceed the rate limit?

You choose the interception action for the hard limit: close the connection immediately, return an error page, or return a captcha page to block bots. The default “Error Page” action returns HTTP status code 503 (service unavailable).

Can OpenResty Edge rate-limit SSL/TLS handshakes too?

Yes. Besides the request rate, OpenResty Edge can limit the rate of SSL or TLS handshakes for HTTPS requests, using the same key, “Shape at”, and “Reject at” parameters as request rate limiting.

Do rate-limit changes require a server reload or restart?

No. The configuration changes do not require a server reload, restart, or binary upgrade. You make a release and it is synchronized to all the gateway clusters and servers.

What is OpenResty Edge

OpenResty Edge is our all-in-one gateway software for microservices and distributed traffic architectures. It combines traffic management, private CDN construction, API gateway, security, and more to help you easily build, manage, and protect modern applications. OpenResty Edge delivers industry-leading performance and scalability to meet the demanding needs of high concurrency, high load scenarios. It supports scheduling containerized application traffic such as K8s and manages massive domains, making it easy to meet the needs of large websites and complex applications.

If you like this tutorial, please subscribe to this blog site and/or our YouTube channel. Thank you!

About The Author

Yichun Zhang (Github handle: agentzh), is the original creator of the OpenResty^® open-source project and the CEO of OpenResty Inc..

Yichun is one of the earliest advocates and leaders of “open-source technology”. He worked at many internationally renowned tech companies, such as Cloudflare, Yahoo!. He is a pioneer of “edge computing”, “dynamic tracing” and “machine coding”, with over 22 years of programming and 16 years of open source experience. Yichun is well-known in the open-source space as the project leader of OpenResty^®, adopted by more than 40 million global website domains.

OpenResty Inc., the enterprise software start-up founded by Yichun in 2017, has customers from some of the biggest companies in the world. Its flagship product, OpenResty XRay, is a non-invasive profiling and troubleshooting tool that significantly enhances and utilizes dynamic tracing technology. And its OpenResty Edge product is a powerful distributed traffic management and private CDN software product.

As an avid open-source contributor, Yichun has contributed more than a million lines of code to numerous open-source projects, including Linux kernel, Nginx, LuaJIT, GDB, SystemTap, LLVM, Perl, etc. He has also authored more than 60 open-source software libraries.