Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup
This tutorial demonstrates how to pre-compile Lua modules into LuaJIT bytecode. This can help reduce the startup time of an OpenResty application.
1 | export PATH=/usr/local/openresty/bin:$PATH |
Here we will use a big Lua module file named pkg-stap.lua
. It was generated by our opslang compiler.
1 | cp ~/git/opslang/pkg-stap.lua ./ |
We can see that this Lua module is 1.6MB.
Let’s try loading this Lua module with the resty
utility.
1 | time resty -I. -e 'require "pkg-stap"' |
It takes 23 milliseconds in total.
Let’s check the original overhead of running an empty Lua program.
1 | time resty -e '' |
It is about 11 milliseconds. So loading the module itself takes about 12 milliseconds.
Let’s try precompiling the Lua module into LuaJIT bytecode.
1 | time /usr/local/openresty/luajit/bin/luajit -bg pkg-stap.lua pkg-stap.ljbc |
Here we use OpenResty’s luajit
program.
It generates a LuaJIT bytecode file with the .ljbc
file extension.
1 | ls -lh *.ljbc |
It’s fun to see the bytecode file is also more than 50% smaller!
Then try loading it with resty
again.
1 | time resty -I. -e 'require "pkg-stap"' |
Just 13 milliseconds in total!
It’s now almost like loading an empty Lua program! Only about 2 milliseconds extra.
1 | time resty -e '' |
The resty
utility always tries to load an .ljbc
file before a .lua
file when loading a module.
Let’s see how to make it work for an OpenResty server.
1 | mkdir conf logs lua |
Copy our Lua module files to the lua/
subdirectory.
1 | mv *.lua *.ljbc lua/ |
Write a simple nginx configuration file, conf/nginx.conf
. We make the following edits:
- Use a single worker process.
- Use 1024 worker connections.
- In the
lua_package_path
directive, it is important to try.ljbc
files before.lua
files. You can also try keeping the.ljbc
files only to be sure. - Inside
init_by_lua_block
, we preload our modules so that any module loading failures can be caught upon server startup. This also leads to faster first requests, and smaller memory footprint due to theCOW
optimization.
1 | worker_processes 1; |
Check the directory tree.
1 | tree . |
Looking good!
Try testing the server configuration using the -t
option.
1 | time openresty -p $PWD/ -t |
Start it up.
1 | time openresty -p $PWD/ |
About 7 milliseconds.
Try removing the LuaJIT bytecode file.
1 | rm lua/*.ljbc |
Stop the server.
1 | kill -QUIT `cat logs/nginx.pid` |
Start the server again.
1 | time openresty -p $PWD/ |
This time it is 17 milliseconds! We can see it is 10 milliseconds slower too by loading the Lua source file.
Finally, try removing the Lua source files too.
1 | rm lua/*.lua |
Stop the server and start it again.
1 | time openresty -p $PWD/ |
This time, we get an error as expected since both versions of the module are gone.
For small Lua module files, it is already very fast to load their source code.
1 | echo 'local _M = {} function _M.foo() end return _M' > a.lua |
Precompiling a small Lua module file won’t help much.
1 | time /usr/local/openresty/luajit/bin/luajit -bg a.lua a.ljbc |
Indeed not much. But the time saving can accumulate quickly if you have many small modules to load up.
If you like this tutorial, please subscribe to this blog site and our YouTube channel. Thank you!
About This Article and Associated Video
This article and its associated video are both generated automatically from a simple screenplay file.
About The Author
Yichun Zhang is the creator of the OpenResty® open source project. He is also the founder and CEO of the OpenResty Inc. company. He contributed a dozen open source Nginx 3rd-party modules, quite some Nginx and LuaJIT core patches, and designed the OpenResty XRay platform.
Translations
We provide the Chinese translation for this article on blog.openresty.com.cn. We also welcome interested readers to contribute translations in other natural languages as long as the full article is translated without any omissions. We thank them in advance.
We are hiring
We always welcome talented and enthusiastic engineers to join our team at OpenResty Inc.
to explore various open source software’s internals and build powerful analyzers and
visualizers for real world applications built atop the open source software. If you are
interested, please send your resume to talents@openresty.com
. Thank you!