How Sticky Cookies Keep Session Affinity in OpenResty Edge

Sticky cookies in OpenResty Edge pin each client to the same backend upstream server, so every request in a session lands on the same host. OpenResty Edge issues a sticky cookie so that subsequent requests from the same client return to the same server — giving you session affinity (session persistence) without touching any Nginx configuration or reloading the gateway. You turn it on with a single switch in a page rule, at either the upstream-cluster or the individual-server level. This tutorial shows how to enable sticky cookies, verify that requests stick to one backend, then disable them and watch traffic return to round robin.

Sticky cookie concept: two requests from the same client routed to the same backend server

We’ll send two requests with Sticky Cookie enabled and will be able to see that the two requests are sent to the same backend server.

Demo with sticky cookies enabled: the first request reaches a backend server

Demo with sticky cookies enabled: the second request reaches the same backend server

Demo with sticky cookies enabled: both requests recorded on the same upstream server

Then send two requests without Sticky cookies enabled, and you will see that the two requests were sent to two different backend servers.

Demo without sticky cookies: the first request reaches one backend server

Demo without sticky cookies: the second request reaches a different backend server

Demo without sticky cookies: the two requests recorded on two different upstream servers

Enable Sticky Cookies for an Upstream (Server-Level Affinity)

Let’s go to the web console of OpenResty Edge. This is our sample deployment of the console. Every user would have their deployment.

OpenResty Edge web console dashboard

We use our continuing sample application for the test-edge.com domain.

Application list showing the test-edge.com sample application

Let’s enter this application.

Opening the test-edge.com application in OpenResty Edge

Go to the Upstreams page.

Navigating to the Upstreams page in the console

We already have an upstream defined.

An existing upstream defined in the application

This upstream currently has an upstream server.

The upstream currently has a single upstream server

We need two upstream servers. let’s add another one.

Preparing to add a second upstream server

Click the “Add a new upstream server” button.

Clicking the Add a new upstream server button

Enter the upstream host.

Entering the host for the new upstream server

Verify upstream server availability.

Verifying the new upstream server returns the OpenResty index page

The default index page of the open-source OpenResty was returned, which was as expected.

Save the upstream.

Saving the upstream now that it has two servers

Go to the Page Rules page.

Opening the Page Rules page

Page Rules list for the application

We already have a reverse proxy page rule set up in an earlier tutorial in this application.

Now let’s edit the page rule to enable the Sticky Cookie.

Editing the existing reverse proxy page rule to enable Sticky Cookie

Turn on the Sticky Cookie switch.

Turning on the Sticky Cookie switch in the page rule

Our upstream servers are within an upstream cluster, so we use server-level Sticky Cookies.

Choosing between cluster-level and server-level sticky cookies

Select the server level.

Selecting server-level sticky cookies

Here we can set the expiry time of the Sticky Cookie.

Setting the expiry time of the sticky cookie

We use the default value of 0, which never expires.

Save the page rules.

Saving the page rule with sticky cookies enabled

Next, we add a dynamic metric to see the distribution of requests. This is not required. The sticky cookie feature does not rely on any dynamic metrics.

Adding a dynamic metric to observe request distribution across upstream servers

Click the New Metric button.

Clicking the New Metric button

We name it “Upstream Server Request Count”.

Naming the metric Upstream Server Request Count

Add a description.

Adding a description for the metric

Set the Report Interval to 10 seconds.

Setting the metric report interval to 10 seconds

We type in the Metric SQL statement for this metric. It will select the upstream server address.

Entering the metric SQL to select the upstream server address

from all requests.

Metric SQL selecting from all requests

group by upstream server address.

Metric SQL grouping by upstream server address

Save the metric.

Saving the dynamic metric

We need to make a new release to push out our new changes, as always.

Preparing a new release to push the configuration changes

Click on this button.

Starting the release

Ship it!

Confirming and shipping the release

Our new release is now synchronized to all our gateway servers.

The release synchronized to all gateway servers

Now the change has been pushed to all the gateway clusters and servers.

Configuration changes synchronizing to all gateway clusters and servers

Configuration changes propagating to every gateway node

Configuration changes fully synchronized across the gateway fleet

Our configuration changes do NOT require server reload, restart, or binary upgrade. So it’s very efficient and scalable.

Configuration pushed to all gateway servers without reload or restart

Test Sticky Cookies: All Requests Hit the Same Backend

Then send two requests. Send the first request.

Sending the first test request with sticky cookies enabled

Send the second request.

Sending the second test request with sticky cookies enabled

All requests completed.

we now look at the data recorded by the dynamic metrics.

Opening the dynamic metric data

Select the bar chart.

Selecting the bar chart view for the metric

Bar chart showing all requests on the same upstream server

All requests fall on the same upstream server.

As shown below.

With sticky cookies enabled, the first request falls on one upstream server

With sticky cookies enabled, the second request falls on the same upstream server

With sticky cookies enabled, both requests recorded on a single upstream server

Disable Sticky Cookies and Return to Round Robin

Next, we demonstrate the case without Sticky Cookies enabled.

Preparing to disable sticky cookies

Let’s clean up the metric data.

Cleaning up the recorded metric data

Confirm.

Confirming the metric data cleanup

Now that the cleanup is complete let’s disable the Sticky Cookie.

Opening the page rule to disable sticky cookies

Edit this page rule again.

Editing the page rule again

Turn off the Sticky Cookie switch.

Turning off the Sticky Cookie switch

Note that the balancing policy is “Round Robin”.

The balancing policy is set to Round Robin

Save the page rules.

Saving the page rule with sticky cookies disabled

We need to make a new release to push out our new changes, as always.

Preparing a new release for the disabled sticky cookie change

Click on this button.

Starting the release

Ship it!

Confirming and shipping the release

Our new release is now synchronized to all our gateway servers.

The release synchronized to all gateway servers

Test Again: Requests Spread Across Backends

Next, we’ll send two requests again.

Send the first request.

Sending the first test request with sticky cookies disabled

Send the second request.

Sending the second test request with sticky cookies disabled

All requests completed.

Recheck the dynamic metrics.

Rechecking the dynamic metric data

Click on the bar chart.

Selecting the bar chart view

both requests visit the different upstream server this time.

Bar chart showing the two requests on different upstream servers

As shown below.

With sticky cookies disabled, the first request goes to one upstream server

With sticky cookies disabled, the second request goes to a different upstream server

With sticky cookies disabled, the two requests are distributed to different upstream servers

With sticky cookies, OpenResty Edge keeps client affinity to an upstream cluster or an individual upstream server — session persistence without any Nginx configuration or reload. Sticky cookies pin traffic within a single upstream; to route users across regions or data centers, see how global server load balancing works, and for non-HTTP services, see TCP load balancing in OpenResty Edge.

Frequently Asked Questions

What is the difference between sticky sessions and session affinity?

They describe the same behavior: keeping a client pinned to one backend for the length of a session. “Session affinity” (also called session persistence) is the goal; the sticky cookie is the mechanism that achieves it. In this demo, OpenResty Edge issues a sticky cookie so that every request from the same client goes back to the same upstream server.

When should I use sticky cookies?

Use sticky cookies when requests from one client need to keep reaching the same backend server. The demo shows the effect directly: with sticky cookies enabled, two requests land on the same upstream server; with them disabled, the same two requests are spread across different servers by round robin.

What is the difference between server-level and cluster-level sticky cookies?

Server-level sticky cookies pin a client to an individual upstream server, while cluster-level sticky cookies pin a client to an upstream cluster. In this tutorial the upstream servers sit within one upstream cluster, so we choose the server level to keep each client on the same individual server.

The expiry time controls how long the sticky cookie stays valid. The default value of 0 means the cookie never expires, so the client stays pinned to the same backend server indefinitely. Set a non-zero value if you want the affinity to reset after a given period.

What is OpenResty Edge

OpenResty Edge is our all-in-one gateway software for microservices and distributed traffic architectures. It combines traffic management, private CDN construction, API gateway, security, and more to help you easily build, manage, and protect modern applications. OpenResty Edge delivers industry-leading performance and scalability to meet the demanding needs of high concurrency, high load scenarios. It supports scheduling containerized application traffic such as K8s and manages massive domains, making it easy to meet the needs of large websites and complex applications.

If you like this tutorial, please subscribe to this blog site and/or our YouTube channel. Thank you!

About The Author

Yichun Zhang (Github handle: agentzh), is the original creator of the OpenResty® open-source project and the CEO of OpenResty Inc..

Yichun is one of the earliest advocates and leaders of “open-source technology”. He worked at many internationally renowned tech companies, such as Cloudflare, Yahoo!. He is a pioneer of “edge computing”, “dynamic tracing” and “machine coding”, with over 22 years of programming and 16 years of open source experience. Yichun is well-known in the open-source space as the project leader of OpenResty®, adopted by more than 40 million global website domains.

OpenResty Inc., the enterprise software start-up founded by Yichun in 2017, has customers from some of the biggest companies in the world. Its flagship product, OpenResty XRay, is a non-invasive profiling and troubleshooting tool that significantly enhances and utilizes dynamic tracing technology. And its OpenResty Edge product is a powerful distributed traffic management and private CDN software product.

As an avid open-source contributor, Yichun has contributed more than a million lines of code to numerous open-source projects, including Linux kernel, Nginx, LuaJIT, GDB, SystemTap, LLVM, Perl, etc. He has also authored more than 60 open-source software libraries.