Today I’d demonstrate another feature in OpenResty Edge: to limit the request rate by some keys.

Embeded image

Sometimes the clients send requests too fast, like under a denial of service attack. In such cases, we should limit the request rate to protect both the gateway servers and origin servers. Otherwise the servers may get overloaded.

Add request rate limiting page rule for the sample application

Let’s go to a web console of OpenResty Edge. This is our sample deployment of the console. Every user would have their own deployment.

Screenshot

We can still use our continuing sample application, test-edge.com.

Screenshot

Enter the application.

Screenshot

We already have a page rule defined.

Screenshot

This page rule sets up a reverse proxy to a pre-defined upstream. And there’s no request rate limiting defined yet.

Screenshot

Now let’s edit the existing page rule to add rate limiting.

Screenshot

Add a new action.

Screenshot

You can search for the action you want to add here

Screenshot

Search for “Limit request rate”.

Screenshot

Select it.

Screenshot

First we need to specify the keys for rate limiting.

Screenshot

We can see that there are a lot of possible key types. Like the client IP address, the URI, the URI argument, cookie, and many more.

Choose the default key type, client IP address. Then the limit is for the scope of each unique client IP address.

Screenshot

The “Shape at” rate is a soft limit. When a client tries to send requests faster than this rate, the gateway server will delay those excessive requests to match this rate. So the faster the client sends requests, the longer delay the gateway will add to the requests.

Screenshot

Here we specify a rate of 50 requests per second.

Screenshot

Because we specified the key type to be client IP addresses, the limit will be applied to individual client IP addresses.

The “Reject at” rate is a hard limit. When the client sends the requests so fast that exceeds this hard limit, we can take more aggresive actions like blocking them immediately without waiting.

Screenshot

Specify a rate of 100 requests per second here.

Screenshot

We can choose from different interception actions.

Screenshot

Like closing the current connection right away, returning an error page, and returning a captcha page to block bots.

Select the default “Error Page” action, which will return an error page via HTTP.

Screenshot

We use the default HTTP status code, 503, to mean service unavailable.

Screenshot

Save our changes to to this page rule.

Screenshot

We need to make a new release to push out our new changes, as always.

Screenshot

Click on this button.

Screenshot

Ship it!

Screenshot

Our new release is now synchronized to all our gateway servers.

Screenshot

Now the new page rule has been pushed to all the gateway clusters and servers.

Our configuration changes do NOT require server reload, restart, or binary upgrade. So it’s very efficient and scalable.

Embeded image

Test

Next, we’ll verify the effect of the new rate limits.

On the terminal, we can send a lot of requests very fast through the open-source utility named wrk.

wrk -c 50 -d 1s http://test-edge.com/

Here we use a concurrency level of 50 first. Note the -c option.

screenshot 1

Run the command. The actual request rate is about 50 requests per second. This is our “Shape at” rate.

screenshot 3

Now we increase the concurrency level to make wrk send requests much faster.

wrk -c 128 -d 1s http://test-edge.com/

Note the 128 concurrency level.

screenshot 5

Run it! Note that there are a lot of rejected requests with erroneous responses.

screenshot 7

The actual request rate is high this time, just because the server rejects those excessive requests very fast.

screenshot 8

Limit the rate of SSL handshakes

In addition to limiting the rate of requests, OpenResty Edge can also limit the rate of SSL or TLS handshakes of HTTPS requests.

Screenshot

On this page, we can configure to limit the rate of SSL handshakes.

Screenshot

Let’s turn on the switch to see the configuration parameters.

Screenshot

These parameters are the same as the request rate limiting feature.

Screenshot

What is OpenResty Edge

OpenResty Edge is our all-in-one gateway software for microservices and distributed traffic architectures. It combines traffic management, private CDN construction, API gateway, security, and more to help you easily build, manage, and protect modern applications. OpenResty Edge delivers industry-leading performance and scalability to meet the demanding needs of high concurrency, high load scenarios. It supports scheduling containerized application traffic such as K8s and manages massive domains, making it easy to meet the needs of large websites and complex applications.

If you like this tutorial, please subscribe to this blog site and/or our YouTube channel. Thank you!

About The Author

Yichun Zhang (Github handle: agentzh), is the original creator of the OpenResty® open-source project and the CEO of OpenResty Inc..

Yichun is one of the earliest advocates and leaders of “open-source technology”. He worked at many internationally renowned tech companies, such as Cloudflare, Yahoo!. He is a pioneer of “edge computing”, “dynamic tracing” and “machine coding”, with over 22 years of programming and 16 years of open source experience. Yichun is well-known in the open-source space as the project leader of OpenResty®, adopted by more than 40 million global website domains.

OpenResty Inc., the enterprise software start-up founded by Yichun in 2017, has customers from some of the biggest companies in the world. Its flagship product, OpenResty XRay, is a non-invasive profiling and troubleshooting tool that significantly enhances and utilizes dynamic tracing technology. And its OpenResty Edge product is a powerful distributed traffic management and private CDN software product.

As an avid open-source contributor, Yichun has contributed more than a million lines of code to numerous open-source projects, including Linux kernel, Nginx, LuaJIT, GDB, SystemTap, LLVM, Perl, etc. He has also authored more than 60 open-source software libraries.