In complex Java application development, file operation issues often cause developers significant headaches. When applications encounter file access exceptions or resource leaks, traditional debugging methods frequently fall short: log analysis is time-consuming, reproducing issues requires precise environmental conditions, and you can typically only see the aftermath of problems without tracing back to their root causes.

Now, there’s a revolutionary debugging approach that works like installing a “black box” in your running program—recording and reconstructing the entire system execution process, allowing you to travel back to the exact moment when problems occur and analyze every detail in depth.

Today, we’re introducing the tool that makes this capability possible: the UDB time-travel debugger. When paired with OpenResty XRay, it provides unprecedented observability for your applications—from system calls to business logic stacks, from performance bottlenecks to security vulnerabilities, offering full-chain, multi-dimensional insights that make debugging and optimization truly efficient and controllable.

What is UDB?

UDB is a Time-Travel Debugger developed by Undo, enabling developers to move backward and forward through program execution to precisely examine program states and variable changes. This capability allows developers to more efficiently locate and fix complex bugs, especially those hard-to-reproduce issues in multi-threaded environments.

• Complete Visibility into File Operations: No more guesswork about when file operations fail. UDB’s time-travel functionality lets you freely navigate through your program’s execution history, observing the complete call stack and context for every file open, read/write, and close operation.

• Capture Fleeting Exceptions: For those elusive file access exceptions that are difficult to reproduce, UDB allows you to set intelligent breakpoints that automatically capture the state when exceptions occur, letting you trace back to identify the root cause.

• Optimize File I/O Performance: When used in conjunction with OpenResty XRay, you can analyze Java application file I/O patterns and execution times, helping you identify issues like frequent small-chunk operations, unoptimized buffer usage, or resources that aren’t released in a timely manner.

Power Duo: UDB Meets OpenResty XRay

If you thought UDB was impressive on its own, wait until you see what happens when it joins forces with OpenResty XRay. This combination is nothing short of game-changing.

OpenResty XRay is our team’s dynamic tracing powerhouse - think of it as your application’s personal detective. It automatically analyzes your systems to pinpoint performance bottlenecks, identify anomalous behaviors, and even uncover potential security vulnerabilities before they become problems.

The dynamic tracing technology in OpenResty XRay monitors Java application file system interactions in real-time, automatically detecting unusual patterns and delivering in-depth analysis. When combined with UDB’s time-travel capabilities, engineering teams gain a comprehensive view of application behavior from macro to micro levels, transforming complex file operation issues into manageable challenges.

By integrating UDB’s time-travel debugging with OpenResty XRay’s deep performance analysis capabilities, developers access a more complete picture of runtime information. This creates a full analytical perspective from high-level performance metrics down to granular call stacks, dramatically improving diagnostic efficiency and problem resolution. The result? You get the best of both worlds - the convenience of time-travel debugging paired with the insights of deep performance analysis, covering everything from the big picture down to the smallest details.

Hands-On Tutorial: Analyzing Java Application File Operations with OpenResty XRay and UDB

In this practical case study, we’ll demonstrate how to leverage OpenResty XRay and UDB to precisely analyze complete call stacks for file read/write operations in Java applications:

Step 1: Recording Application Execution Traces

First, we’ll use UDB’s Live Record tool to capture the execution process of a Java application:

1. Utilize the Live Record tool to record a sample of a running Java application.

2. Load the recorded sample with the UDB tool and set up the debugging environment:

udb -ex "set pagination off" -ex "set python print-stack full" java.rec

Step 2: Locating File Operation Breakpoints

In the UDB environment, set breakpoints to capture file write operations:

start 1> break write
start 1> c
Continuing.
[Switching to Thread 3923049.3923068]

Thread 20 "Thread-0" hit Breakpoint 1.768, 0x00007ffff7cfdb40 in write () from /tmp/undodb.3976686.1747987405.935444.61ae9aec99c4629b/debuggee-1-bus1z2h5/symbol-files/lib64/libc.so.6

Step 3: Analyzing the Underlying C Call Stack

Use the bt command to examine the current C-level call stack:

4% 7,955> bt
#0  0x00007ffff7cfdb40 in write () from /tmp/undodb.3976686.1747987405.935444.61ae9aec99c4629b/debuggee-1-bus1z2h5/symbol-files/lib64/libc.so.6
#1  0x00007ffff7e32c5f in Java_sun_nio_ch_FileDispatcherImpl_write0 (env=0x7ffff0181290, clazz=<optimized out>, fdo=<optimized out>, address=140733730265536, len=81)
    at src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:118
#2  0x00007fffe0baacf6 in ?? ()
#3  0x000000062a39ed70 in ?? ()

From the C call stack, we can see that the system’s low-level write system call has been triggered. However, this only shows information at the JNI layer and doesn’t provide visibility into the complete call path of the Java business code.

Step 4: Analyzing the Complete Java Call Stack

1. Load the Java call stack analysis tool provided by OpenResty XRay:

4% 7,955> source java-udb.y.py

2. Use the java_bt command to obtain the complete Java call stack:

4% 7,955> java_bt
Start tracing...
C:__libc_write
sun.nio.ch.FileDispatcherImpl:write0(Native Method)
@FileDispatcherImpl.java:62
sun.nio.ch.FileDispatcherImpl:write
@IOUtil.java:114
sun.nio.ch.IOUtil:writeFromNativeBuffer
@IOUtil.java:75
sun.nio.ch.IOUtil:write
@IOUtil.java:67
sun.nio.ch.IOUtil:write
@FileChannelImpl.java:288
sun.nio.ch.FileChannelImpl:write
@Channels.java:89
java.nio.channels.Channels:writeFully
@Channels.java:158
java.nio.channels.Channels$1:write
@StreamEncoder.java:223
sun.nio.cs.StreamEncoder:writeBytes
@StreamEncoder.java:325
sun.nio.cs.StreamEncoder:implClose
@StreamEncoder.java:165
sun.nio.cs.StreamEncoder:close
@OutputStreamWriter.java:252
java.io.OutputStreamWriter:close
@BufferedWriter.java:263
java.io.BufferedWriter:close
@Files.java:3579
java.nio.file.Files:write
@Processor.java:42
FileWriterTask:run
@Thread.java:840
java.lang.Thread:run

Through this complete Java call stack, we can clearly visualize the entire path from the business code FileWriterTask:run method, through Java’s standard library Files.write method, all the way down to the underlying system calls. This visibility provides critical value for understanding application file operation behaviors, identifying performance bottlenecks, or troubleshooting file-related issues.

Unlocking Next-Gen Debugging with Time Travel

Time-travel debugging capability in UDB represents one of its most powerful features. This functionality allows developers to freely move forward or backward through recorded execution traces, precisely pinpointing different file operation timestamps, and comprehensively analyzing the complete context and call stack for each write operation. This capability becomes particularly crucial when troubleshooting complex file operation issues.

To demonstrate this capability, we can continue executing the program to capture the next file write operation:

8% 16,337> c
Continuing.

Thread 20 "Thread-0" hit Breakpoint 1.768, 0x00007ffff7cfdb40 in write () from /tmp/undodb.3976686.1747987405.935444.61ae9aec99c4629b/debuggee-1-bus1z2h5/symbol-files/lib64/libc.so.6

8% 16,337> java_bt
Start tracing...
C:__libc_write
sun.nio.ch.FileDispatcherImpl:write0(Native Method)
@FileDispatcherImpl.java:62
sun.nio.ch.FileDispatcherImpl:write
@IOUtil.java:114
sun.nio.ch.IOUtil:writeFromNativeBuffer
@IOUtil.java:75
sun.nio.ch.IOUtil:write
@IOUtil.java:67
sun.nio.ch.IOUtil:write
@FileChannelImpl.java:288
sun.nio.ch.FileChannelImpl:write
@Channels.java:89
java.nio.channels.Channels:writeFully
@Channels.java:158
java.nio.channels.Channels$1:write
@StreamEncoder.java:223
sun.nio.cs.StreamEncoder:writeBytes
@StreamEncoder.java:325
sun.nio.cs.StreamEncoder:implClose
@StreamEncoder.java:165
sun.nio.cs.StreamEncoder:close
@OutputStreamWriter.java:252
java.io.OutputStreamWriter:close
@BufferedWriter.java:263
java.io.BufferedWriter:close
@Files.java:3579
java.nio.file.Files:write
@Processor.java:203
CoreWriterTask:run
@Thread.java:840
java.lang.Thread:run

By comparing two captured call stacks, we can clearly see:

1. The first write operation originates from the FileWriterTask:run method (located at Processor.java:42)
2. The second write operation comes from the CoreWriterTask:run method (located at Processor.java:203)

This analytical approach isn’t limited to file write operations—it’s equally applicable for debugging file reads, network communications, and various other I/O operations, providing a powerful toolkit for comprehensively understanding application behavior.

The dynamic analysis capabilities of OpenResty XRay combined with UDB enable us to gain complete visibility into file operation patterns within applications, breaking through the limitations of traditional debugging methods that only capture state at a single moment. Particularly in scenarios where processes have terminated or no longer exist, UDB’s retrospective analysis capabilities demonstrate unique advantages over traditional GDB coredump analysis.

A coredump only provides a static snapshot at the moment of a program crash, unable to recreate the complete execution path leading up to the failure. In contrast, UDB’s recording functionality empowers OpenResty XRay to navigate freely through the entire program execution history, even when the original process is no longer running. This allows for precise examination of program state and behavioral details at any point in time, truly achieving comprehensive debugging across the time dimension.

Conclusion

The Java call stack analysis capabilities provided by UDB in conjunction with OpenResty XRay offer developers unprecedented insight into application behavior. Through time-travel debugging and comprehensive call stack analysis, developers can:

1. Precisely pinpoint the source and context of file operations
2. Perform deep analysis of I/O performance bottlenecks
3. Efficiently troubleshoot file operation-related issues
4. Thoroughly validate the security and correctness of file operations

This depth of analysis is particularly crucial for Java application development, especially in today’s microservices-dominated landscape, where such deep analytical capabilities can be a genuine lifesaver.

We hope this practical demonstration helps you better understand and appreciate the use cases and power of UDB and OpenResty XRay. If you’re struggling with complex debugging challenges, this powerful combination might be exactly what you need.

What is OpenResty XRay

OpenResty XRay is a dynamic-tracing product that automatically analyzes your running applications to troubleshoot performance problems, behavioral issues, and security vulnerabilities with actionable suggestions. Under the hood, OpenResty XRay is powered by our Y language targeting various runtimes like Stap+, eBPF+, GDB, and ODB, depending on the contexts.

If you like this tutorial, please subscribe to this blog site and/or our YouTube channel. Thank you!

About The Author

Yichun Zhang (Github handle: agentzh), is the original creator of the OpenResty^® open-source project and the CEO of OpenResty Inc..

Yichun is one of the earliest advocates and leaders of “open-source technology”. He worked at many internationally renowned tech companies, such as Cloudflare, Yahoo!. He is a pioneer of “edge computing”, “dynamic tracing” and “machine coding”, with over 22 years of programming and 16 years of open source experience. Yichun is well-known in the open-source space as the project leader of OpenResty^®, adopted by more than 40 million global website domains.

OpenResty Inc., the enterprise software start-up founded by Yichun in 2017, has customers from some of the biggest companies in the world. Its flagship product, OpenResty XRay, is a non-invasive profiling and troubleshooting tool that significantly enhances and utilizes dynamic tracing technology. And its OpenResty Edge product is a powerful distributed traffic management and private CDN software product.

As an avid open-source contributor, Yichun has contributed more than a million lines of code to numerous open-source projects, including Linux kernel, Nginx, LuaJIT, GDB, SystemTap, LLVM, Perl, etc. He has also authored more than 60 open-source software libraries.