Are you curious about how to dynamically measure the number of requests handled by Nginx or OpenResty? Do you want to use a simple and powerful tool that can directly query running processes as read-only databases without relying on logs or APIs? If so, you will love OpenResty XRay and its
run-ysql command line tool.
OpenResty XRay is a revolutionary product that allows you to perform dynamic tracing and analysis on any process using YSQL, a SQL-like language that supports various data sources and operations. One of the data sources that YSQL can access is
ngx.reqs, which contains information about nginx requests. Under the hood, OpenResty XRay is powered by our Y language targeting various runtimes like Stap+, eBPF+, GDB, and ODB, depending on the contexts.
In this article, we will show you how to use the
run-ysql command line tool to run YSQL queries on nginx processes and get insights into their performance and behavior. We will use some simple examples to demonstrate the power and simplicity of YSQL and OpenResty XRay.
Before you can use the
run-ysql tool, you need to install the
openresty-xray-cli package on your Linux system. The
openresty-xray-cli package contains the
run-ysql command and other utilities for interacting with OpenResty XRay.
The installation method depends on the Linux distribution that you are using. However, in all cases, you need to first install the Agent software of OpenResty XRay, which will enable the correct package repository for your system automatically. You can download the Agent software from the OpenResty website and follow the installation instructions.
Here are some common examples of installing the
openresty-xray-cli package after installing the Agent software:
If you are using Fedora, CentOS, or RHEL, you can use the dnf or yum command to install the
openresty-xray-cli package. For example:
$ sudo dnf install openresty-xray-cli
Or on older versions of these distros:
$ sudo yum install openresty-xray-cli
If you are using Debian or Ubuntu, you can use the apt command to install the
openresty-xray-cli package. For example:
$ sudo apt-get update
$ sudo apt-get install openresty-xray-cli
After installing the
openresty-xray-cli package, you can verify that the
run-ysql tool is working by running:
$ which run-ysql
You should see something like this:
Now you are ready to use the
run-ysql tool to run YSQL queries on any process.
How to count the total number of requests handled by nginx? One of the most basic metrics that you might want to know about your nginx server is how many requests it has processed in a given period of time. To get this information, you can use the following YSQL query:
select count(*) from ngx.reqs
This query will return the number of rows in the
ngx.reqs table, which corresponds to the number of requests handled by nginx. You can run this query using the
run-ysql command line tool, which takes the query as an argument and the process ID (PID) of the nginx worker process as another argument. For example, if the PID of the nginx worker process is 9708, you can run:
$ run-ysql -e 'select count(*) from ngx.reqs' -p 9708
Goto https://8pu4z6.xray.openresty.com/targets/1355/history/4378702043 for charts
run-ysql command will execute the query on the target process and output a URL that contains the results in a chart format. You can open the URL in your browser and see something like this:
The chart shows that in 3 seconds (the default duration of
run-ysql), the nginx worker process handled 106512 requests. That’s impressive!
How to count the number of requests with a specific URI prefix? Another useful metric that you might want to know about your nginx server is how many requests have a certain URI prefix. For example, you might want to know how many requests are for static files under the “/css/” directory. To get this information, you can use the following YSQL query:
select count(*) from ngx.reqs where uri prefix '/css/'
This query will filter the rows in the
ngx.reqs table by checking if their
uri column starts with
/css/, and then count the number of matching rows. You can run this query using the same
run-ysql command as before, but with a different query argument. For example:
$ run-ysql -e "select count(*) from ngx.reqs where uri prefix '/css/'" -p 9708
Goto https://8pu4z6.xray.openresty.com/targets/1355/history/4378707156 for charts
run-ysql command will execute the query and output a URL that contains the results in a chart format. You can open the URL in your browser and see something like this:
The chart shows that in 3 seconds, the nginx worker process handled 47026 requests with URI prefix
/css/. That’s a lot of CSS files!
Another way to use YSQL to query and analyze data from any process is to use the web console of OpenResty XRay. The web console is a graphical user interface that allows you to create and manage targets, analyzers, charts, dashboards, and alerts.
- Sign into the web console of OpenResty XRay.
- Click on the “Analyzers” tab and click on the “Create Analyzer” button. An analyzer is a YSQL query that runs on a target and produces data for charts.
- Enter a name and a description for your analyzer. Then, enter your YSQL query in the “Query” box. You can use any syntax and data source that YSQL supports.
- Click on the “Save” button to save your analyzer. You can also click on the “Run Now” button to run your analyzer immediately.
One of the most amazing features of OpenResty XRay and its YSQL tool chain is that they can work without any collaboration from the target Nginx or OpenResty processes. This means that you don’t need to do any of the following things to use OpenResty XRay and YSQL:
- You don’t need to install any extra modules or plugins in your target processes. OpenResty XRay and YSQL can access and analyze data from any process regardless of its configuration or functionality.
- You don’t need to use any special build options when compiling your target processes. OpenResty XRay and YSQL can work with any binary executable file without requiring any source code modification or recompilation.
- You don’t need to restart your target processes when using OpenResty XRay and YSQL. OpenResty XRay and YSQL can attach and detach to any running process dynamically without interrupting its normal operation or causing any downtime.
- You don’t need to worry about the performance impact of using OpenResty XRay and YSQL. OpenResty XRay and YSQL use advanced techniques such as dynamic tracing, sampling, aggregation, and filtering to minimize the overhead of data collection and analysis. The overhead is usually not measurable and negligible compared to the benefits of using OpenResty XRay and YSQL. And when you are not tracing and sampling, your target processes run at absolute 100% full speed.
We have shown you how to use YSQL statements and the
run-ysql command line tool to dynamically measure nginx request numbers. You have seen how easy and convenient it is to use YSQL and OpenResty XRay to access and analyze data from any running process without needing logs or APIs.
YSQL supports many more syntaxes and data sources than what we have covered here. For example, you can use YSQL to access data from Lua variables, system calls, network packets, MySQL queries, Redis commands, and more. You can also use YSQL to perform various operations on data such as aggregation, grouping, sorting, filtering, joining, etc.
We also have plans to support more target application types as “virtual tables” for YSQL queries, more server processes such as Envoy, Apache Traffic Server (ATS), various frameworks such as Python, Go, Java, etc., and more database processes such as MySQL, Pg. The possibilities are endless.
Yichun is one of the earliest advocates and leaders of “open-source technology”. He worked at many internationally renowned tech companies, such as Cloudflare, Yahoo!. He is a pioneer of “edge computing”, “dynamic tracing” and “machine coding”, with over 22 years of programming and 16 years of open source experience. Yichun is well-known in the open-source space as the project leader of OpenResty®, adopted by more than 40 million global website domains.
OpenResty Inc., the enterprise software start-up founded by Yichun in 2017, has customers from some of the biggest companies in the world. Its flagship product, OpenResty XRay, is a non-invasive profiling and troubleshooting tool that significantly enhances and utilizes dynamic tracing technology. And its OpenResty Edge product is a powerful distributed traffic management and private CDN software product.
As an avid open-source contributor, Yichun has contributed more than a million lines of code to numerous open-source projects, including Linux kernel, Nginx, LuaJIT, GDB, SystemTap, LLVM, Perl, etc. He has also authored more than 60 open-source software libraries.