Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup

This tutorial demonstrates how to pre-compile Lua modules into LuaJIT bytecode. This can help reduce the startup time of an OpenResty application.

export PATH=/usr/local/openresty/bin:$PATH
cd ~
mkdir -p precomp
cd precomp/

Here we will use a big Lua module file named pkg-stap.lua. It was generated by our opslang compiler.

cp ~/git/opslang/pkg-stap.lua ./
ls -lh *.lua

We can see that this Lua module is 1.6MB.

Let’s try loading this Lua module with the resty utility.

time resty -I. -e 'require "pkg-stap"'

It takes 23 milliseconds in total.

Let’s check the original overhead of running an empty Lua program.

time resty -e ''

It is about 11 milliseconds. So loading the module itself takes about 12 milliseconds.

Let’s try precompiling the Lua module into LuaJIT bytecode.

time /usr/local/openresty/luajit/bin/luajit -bg pkg-stap.lua pkg-stap.ljbc

Here we use OpenResty’s luajit program.

It generates a LuaJIT bytecode file with the .ljbc file extension.

ls -lh *.ljbc

It’s fun to see the bytecode file is also more than 50% smaller!

Then try loading it with resty again.

time resty -I. -e 'require "pkg-stap"'

Just 13 milliseconds in total!

It’s now almost like loading an empty Lua program! Only about 2 milliseconds extra.

time resty -e ''

The resty utility always tries to load an .ljbc file before a .lua file when loading a module.

Let’s see how to make it work for an OpenResty server.

mkdir conf logs lua

Copy our Lua module files to the lua/ subdirectory.

mv *.lua *.ljbc lua/
tree .

Write a simple nginx configuration file, conf/nginx.conf. We make the following edits:

1. Use a single worker process.
2. Use 1024 worker connections.
3. In the lua_package_path directive, it is important to try .ljbc files before .lua files. You can also try keeping the .ljbc files only to be sure.
4. Inside init_by_lua_block, we preload our modules so that any module loading failures can be caught upon server startup. This also leads to faster first requests, and smaller memory footprint due to the COW optimization.

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    lua_package_path "$prefix/lua/?.ljbc;$prefix/lua/?.lua;;";

    init_by_lua_block {
        require "pkg-stap"
    }

    server {
        listen 8080;
        location / { return 200 "ok\n"; }
    }
}

Check the directory tree.

tree .

Looking good!

Try testing the server configuration using the -t option.

time openresty -p $PWD/ -t

Start it up.

time openresty -p $PWD/

About 7 milliseconds.

Try removing the LuaJIT bytecode file.

rm lua/*.ljbc

Stop the server.

kill -QUIT `cat logs/nginx.pid`

Start the server again.

time openresty -p $PWD/

This time it is 17 milliseconds! We can see it is 10 milliseconds slower too by loading the Lua source file.

Finally, try removing the Lua source files too.

rm lua/*.lua

Stop the server and start it again.

time openresty -p $PWD/

This time, we get an error as expected since both versions of the module are gone.

For small Lua module files, it is already very fast to load their source code.

echo 'local _M = {} function _M.foo() end return _M' > a.lua
ls -l a.lua
time resty -I. -e 'require "a"'

Precompiling a small Lua module file won’t help much.

time /usr/local/openresty/luajit/bin/luajit -bg a.lua a.ljbc
time resty -I. -e 'require "a"'

Indeed not much. But the time saving can accumulate quickly if you have many small modules to load up.

If you like this tutorial, please subscribe to this blog site and our YouTube channel. Thank you!

About This Article and Associated Video

This article and its associated video are both generated automatically from a simple screenplay file.

Boost your application's performance by OpenResty XRay

About The Author

Yichun Zhang is the creator of the OpenResty^® open source project. He is also the founder and CEO of the OpenResty Inc. company. He contributed a dozen open source Nginx 3rd-party modules, quite some Nginx and LuaJIT core patches, and designed the OpenResty XRay platform.

Translations

We provide the Chinese translation for this article on blog.openresty.com.cn. We also welcome interested readers to contribute translations in other natural languages as long as the full article is translated without any omissions. We thank them in advance.

We are hiring

We always welcome talented and enthusiastic engineers to join our team at OpenResty Inc. to explore various open source software’s internals and build powerful analyzers and visualizers for real world applications built atop the open source software. If you are interested, please send your resume to talents@openresty.com . Thank you!

Try out OpenResty XRay yourself