Featured Articles

Photo by Yichun Zhang

One String Value Crashed Asia's Biggest Streaming Site — We Found It with OpenResty XRay

One String Value Crashed Asia's Biggest Streaming Site — We Found It with OpenResty XRay

Binary Vulnerability Scanning: Catch Backported Patches, Cut False Positives

Binary Vulnerability Scanning: Catch Backported Patches, Cut False Positives

Binary Vulnerability Scanning: Catch Backported Patches, Cut False Positives

Introduction to OpenResty XRay

Introduction to OpenResty XRay

Introduction to OpenResty XRay

Nginx Reverse Proxy QPS Dropped 15x: A Missing upstream keepalive and an -O0 Debug Build

Nginx Reverse Proxy QPS Dropped 15x: A Missing upstream keepalive and an -O0 Debug Build

Nginx Reverse Proxy QPS Dropped 15x: A Missing upstream keepalive and an -O0 Debug Build

Non-Invasive Java Profiler: Capture Method Arguments in Production Without a Java Agent

Non-Invasive Java Profiler: Capture Method Arguments in Production Without a Java Agent

Non-Invasive Java Profiler: Capture Method Arguments in Production Without a Java Agent

Photo by Philip Brown

The Wonderland of Dynamic Tracing (Part 1 of 3)

The Wonderland of Dynamic Tracing (Part 1 of 3)

Photo by Jose G. Ortega Castro

How OpenResty and Nginx Shared Memory Zones Consume RAM

How OpenResty and Nginx Shared Memory Zones Consume RAM

Photo by Harrison Broadbent

Nginx Memory Usage Explained: Where the RAM Actually Goes

Nginx Memory Usage Explained: Where the RAM Actually Goes

OpenResty XRay reduce your CPU usage up to  90%
OpenResty XRay reduce your CPU usage up to 90%
Request free trial

Latest Articles

Binary Vulnerability Scanning: Catch Backported Patches, Cut False Positives

  • Why Version-Matching Vulnerability Scanning Fails: Three Compounding Costs
  • How OpenResty XRay’s Binary Vulnerability Scanning Works: The Evidence-First Model
  • Case Study: Binary Vulnerability Scanning on Rocky Linux with Dual OpenSSL Versions
  • From Alert Fatigue to Precision: A Sustainable Vulnerability Scanning Practice
  • Why Version-Matching Vulnerability Scanning Fails: Three Compounding Costs
  • How OpenResty XRay’s Binary Vulnerability Scanning Works: The Evidence-First Model
  • Case Study: Binary Vulnerability Scanning on Rocky Linux with Dual OpenSSL Versions
  • From Alert Fatigue to Precision: A Sustainable Vulnerability Scanning Practice

How OpenResty XRay Enables Full-Stack Dynamic Tracing in Production

  • Where Do Existing Dynamic Tracing Frameworks Fall Short in Production?
  • How Did OpenResty XRay Rethink Dynamic Tracing at the Architecture Level?
  • What Is a Full-Stack Flame Graph and How Do You Use It to Diagnose Performance Bottlenecks?
  • How Is OpenResty XRay Expanding Its Observability Coverage Over Time?
  • Further Reading and Real-World Case Studie
  • Where Do Existing Dynamic Tracing Frameworks Fall Short in Production?
  • How Did OpenResty XRay Rethink Dynamic Tracing at the Architecture Level?
  • What Is a Full-Stack Flame Graph and How Do You Use It to Diagnose Performance Bottlenecks?
  • How Is OpenResty XRay Expanding Its Observability Coverage Over Time?
  • Further Reading and Real-World Case Studie

Introduction to OpenResty XRay

  • Core Philosophy: A Tracing Engine Built for the Unknown
  • Turning Noise into Signal: How XRay Demystifies Production Issues
  • Empowering Engineering Teams in Cloud-Native Reality
  • Built on a Robust, Open Technological Foundatione
  • Real-World Success Stories
  • Core Philosophy: A Tracing Engine Built for the Unknown
  • Turning Noise into Signal: How XRay Demystifies Production Issues
  • Empowering Engineering Teams in Cloud-Native Reality
  • Built on a Robust, Open Technological Foundatione
  • Real-World Success Stories

Hunting Down a Silent Memory Leak in OpenResty Without a Reboot

  • A Complete Dissection of a Production Memory Leak
  • Why Are These Problems Nearly Unsolvable?
  • A System Transformation
  • When Performance Issues Go Beyond Just Tools
  • A Complete Dissection of a Production Memory Leak
  • Why Are These Problems Nearly Unsolvable?
  • A System Transformation
  • When Performance Issues Go Beyond Just Tools

Nginx Reverse Proxy QPS Dropped 15x: A Missing upstream keepalive and an -O0 Debug Build

  • A 15x Slower Nginx Reverse Proxy — and No Errors in the Logs
  • Flame Graphs Expose a Connection Storm: The Missing upstream keepalive
  • The Remaining 10% Gap: Baseline Comparison Points to the Compiler
  • How an -O0 Debug Build Cost Us 10% of Performance
  • A 15x Slower Nginx Reverse Proxy — and No Errors in the Logs
  • Flame Graphs Expose a Connection Storm: The Missing upstream keepalive
  • The Remaining 10% Gap: Baseline Comparison Points to the Compiler
  • How an -O0 Debug Build Cost Us 10% of Performance

Fix LuaJIT RSS Memory Leak in OpenResty: Stop OOM Kills Without Restarts

  • What Is the LuaJIT Pseudo Memory Leak?
  • Diagnosing the Problem with Real Production Data
  • Why Standard Fixes Don’t Work
  • How LuaJIT-plus Fixes This at the Allocator Level
  • Production Impact After Deployment
  • What Is the LuaJIT Pseudo Memory Leak?
  • Diagnosing the Problem with Real Production Data
  • Why Standard Fixes Don’t Work
  • How LuaJIT-plus Fixes This at the Allocator Level
  • Production Impact After Deployment

How We Pinpointed a 244ms Latency Spike in a 500k QPS OpenResty Gateway

  • When Traditional Monitoring Fails to Pinpoint the Root Cause
  • From Heuristics to the Power of Dynamic Observability
  • Quantifiable Engineering Efficiency and Resource Optimization Results
  • Building Continuous Performance Observability
  • When Traditional Monitoring Fails to Pinpoint the Root Cause
  • From Heuristics to the Power of Dynamic Observability
  • Quantifiable Engineering Efficiency and Resource Optimization Results
  • Building Continuous Performance Observability

How to Diagnose Java Memory Leaks in Production — Without Heap Dumps or Restarts

  • Why Full GCs Didn’t Reclaim the Heap: A Production Java Memory Leak
  • How to Diagnose a Java Memory Leak in Production Without a Heap Dump
  • How to Find a Java Memory Leak: A Reusable 3-Step Method
  • Why Use OpenResty XRay to Find Java Memory Leaks on a Live JVM?
  • FAQ
  • Why Full GCs Didn’t Reclaim the Heap: A Production Java Memory Leak
  • How to Diagnose a Java Memory Leak in Production Without a Heap Dump
  • How to Find a Java Memory Leak: A Reusable 3-Step Method
  • Why Use OpenResty XRay to Find Java Memory Leaks on a Live JVM?
  • FAQ

OpenResty XRay Version 25.10.11 Now Available

  • Three Key Highlights
  • Enhanced Diagnostic Accuracy and Stability
  • Usability and Security Enhancements
  • Three Key Highlights
  • Enhanced Diagnostic Accuracy and Stability
  • Usability and Security Enhancements

Nginx C++ Module Memory Leak: Root Cause in Production, No Restart

  • The Production Challenge: A C++ Module Memory Leak You Can’t Debug With GDB
  • How the Flame Graph Pinpointed the Leaking Function in ngx_dubbo_module
  • From Reactive to Proactive: XRay’s New Approach to Troubleshooting
  • The OpenResty XRay Closed-Loop Approach
  • The Production Challenge: A C++ Module Memory Leak You Can’t Debug With GDB
  • How the Flame Graph Pinpointed the Leaking Function in ngx_dubbo_module
  • From Reactive to Proactive: XRay’s New Approach to Troubleshooting
  • The OpenResty XRay Closed-Loop Approach

Non-Invasive Java Profiler: Capture Method Arguments in Production Without a Java Agent

  • Why Non-Invasive Java Profiling Matters in Production
  • Hands-on Demo: Capturing Java Method Arguments in Production
  • How OpenResty XRay Differs from Common Java Profilers
  • Frequently Asked Questions
  • Why Non-Invasive Java Profiling Matters in Production
  • Hands-on Demo: Capturing Java Method Arguments in Production
  • How OpenResty XRay Differs from Common Java Profilers
  • Frequently Asked Questions

How OpenResty XRay Enables Full-Stack Dynamic Tracing in Production

  • Where Do Existing Dynamic Tracing Frameworks Fall Short in Production?
  • How Did OpenResty XRay Rethink Dynamic Tracing at the Architecture Level?
  • What Is a Full-Stack Flame Graph and How Do You Use It to Diagnose Performance Bottlenecks?
  • How Is OpenResty XRay Expanding Its Observability Coverage Over Time?
  • Further Reading and Real-World Case Studie
  • Where Do Existing Dynamic Tracing Frameworks Fall Short in Production?
  • How Did OpenResty XRay Rethink Dynamic Tracing at the Architecture Level?
  • What Is a Full-Stack Flame Graph and How Do You Use It to Diagnose Performance Bottlenecks?
  • How Is OpenResty XRay Expanding Its Observability Coverage Over Time?
  • Further Reading and Real-World Case Studie

Introduction to OpenResty XRay

  • Core Philosophy: A Tracing Engine Built for the Unknown
  • Turning Noise into Signal: How XRay Demystifies Production Issues
  • Empowering Engineering Teams in Cloud-Native Reality
  • Built on a Robust, Open Technological Foundatione
  • Real-World Success Stories
  • Core Philosophy: A Tracing Engine Built for the Unknown
  • Turning Noise into Signal: How XRay Demystifies Production Issues
  • Empowering Engineering Teams in Cloud-Native Reality
  • Built on a Robust, Open Technological Foundatione
  • Real-World Success Stories

Hunting Down a Silent Memory Leak in OpenResty Without a Reboot

  • A Complete Dissection of a Production Memory Leak
  • Why Are These Problems Nearly Unsolvable?
  • A System Transformation
  • When Performance Issues Go Beyond Just Tools
  • A Complete Dissection of a Production Memory Leak
  • Why Are These Problems Nearly Unsolvable?
  • A System Transformation
  • When Performance Issues Go Beyond Just Tools

Nginx Reverse Proxy QPS Dropped 15x: A Missing upstream keepalive and an -O0 Debug Build

  • A 15x Slower Nginx Reverse Proxy — and No Errors in the Logs
  • Flame Graphs Expose a Connection Storm: The Missing upstream keepalive
  • The Remaining 10% Gap: Baseline Comparison Points to the Compiler
  • How an -O0 Debug Build Cost Us 10% of Performance
  • A 15x Slower Nginx Reverse Proxy — and No Errors in the Logs
  • Flame Graphs Expose a Connection Storm: The Missing upstream keepalive
  • The Remaining 10% Gap: Baseline Comparison Points to the Compiler
  • How an -O0 Debug Build Cost Us 10% of Performance

How We Pinpointed a 244ms Latency Spike in a 500k QPS OpenResty Gateway

  • When Traditional Monitoring Fails to Pinpoint the Root Cause
  • From Heuristics to the Power of Dynamic Observability
  • Quantifiable Engineering Efficiency and Resource Optimization Results
  • Building Continuous Performance Observability
  • When Traditional Monitoring Fails to Pinpoint the Root Cause
  • From Heuristics to the Power of Dynamic Observability
  • Quantifiable Engineering Efficiency and Resource Optimization Results
  • Building Continuous Performance Observability

How to Diagnose Java Memory Leaks in Production — Without Heap Dumps or Restarts

  • Why Full GCs Didn’t Reclaim the Heap: A Production Java Memory Leak
  • How to Diagnose a Java Memory Leak in Production Without a Heap Dump
  • How to Find a Java Memory Leak: A Reusable 3-Step Method
  • Why Use OpenResty XRay to Find Java Memory Leaks on a Live JVM?
  • FAQ
  • Why Full GCs Didn’t Reclaim the Heap: A Production Java Memory Leak
  • How to Diagnose a Java Memory Leak in Production Without a Heap Dump
  • How to Find a Java Memory Leak: A Reusable 3-Step Method
  • Why Use OpenResty XRay to Find Java Memory Leaks on a Live JVM?
  • FAQ

OpenResty XRay Version 25.10.11 Now Available

  • Three Key Highlights
  • Enhanced Diagnostic Accuracy and Stability
  • Usability and Security Enhancements
  • Three Key Highlights
  • Enhanced Diagnostic Accuracy and Stability
  • Usability and Security Enhancements

Nginx C++ Module Memory Leak: Root Cause in Production, No Restart

  • The Production Challenge: A C++ Module Memory Leak You Can’t Debug With GDB
  • How the Flame Graph Pinpointed the Leaking Function in ngx_dubbo_module
  • From Reactive to Proactive: XRay’s New Approach to Troubleshooting
  • The OpenResty XRay Closed-Loop Approach
  • The Production Challenge: A C++ Module Memory Leak You Can’t Debug With GDB
  • How the Flame Graph Pinpointed the Leaking Function in ngx_dubbo_module
  • From Reactive to Proactive: XRay’s New Approach to Troubleshooting
  • The OpenResty XRay Closed-Loop Approach

Non-Invasive Java Profiler: Capture Method Arguments in Production Without a Java Agent

  • Why Non-Invasive Java Profiling Matters in Production
  • Hands-on Demo: Capturing Java Method Arguments in Production
  • How OpenResty XRay Differs from Common Java Profilers
  • Frequently Asked Questions
  • Why Non-Invasive Java Profiling Matters in Production
  • Hands-on Demo: Capturing Java Method Arguments in Production
  • How OpenResty XRay Differs from Common Java Profilers
  • Frequently Asked Questions

How OpenResty XRay's Node.js Function Probes Enable a New Monitoring Paradigm

  • Why You Need a New Paradigm for Non-Invasive Function Probes
  • Hands-on Demo: Monitoring Function Parameters
  • How Top Teams Leverage Non-Intrusive Probes to Boost Engineering Efficiency
  • Why You Need a New Paradigm for Non-Invasive Function Probes
  • Hands-on Demo: Monitoring Function Parameters
  • How Top Teams Leverage Non-Intrusive Probes to Boost Engineering Efficiency

The Nginx Worker That Refused to Serve More Than 300 RPS: Traced to a 75ms Event-Loop Block

  • High Load but No Useful Work: CPU Contention from a Missing worker_cpu_affinity
  • How Blocking Lua I/O Calls Halted the Nginx Event Loop
  • Measuring File I/O Latency: Up to 1.5ms per Call
  • How Bad Was the Nginx Event-Loop Blocking? 43,952 Samples, a 75ms Worst Case
  • Frequently Asked Questions
  • High Load but No Useful Work: CPU Contention from a Missing worker_cpu_affinity
  • How Blocking Lua I/O Calls Halted the Nginx Event Loop
  • Measuring File I/O Latency: Up to 1.5ms per Call
  • How Bad Was the Nginx Event-Loop Blocking? 43,952 Samples, a 75ms Worst Case
  • Frequently Asked Questions

Analyzing CPU Usage in Online Node.js Applications (Using OpenResty XRay)

  • High CPU Usage Issues in Node.js Applications
  • CPU Blocking Issues in Node.js Applications
  • Automated Sampling and Expert Analysis
  • Performance Impact and Overhead
  • Future Plans
  • High CPU Usage Issues in Node.js Applications
  • CPU Blocking Issues in Node.js Applications
  • Automated Sampling and Expert Analysis
  • Performance Impact and Overhead
  • Future Plans

Nginx Core Dump Analysis: Catch a Worker Process Crash at the First Scene

  • The Case: Frequent Nginx Worker Process Crashes No Log Could Explain
  • Analyzing the Recording: Replaying the Crash Back to the First Scene
  • Root Cause: A Use-After-Free in Nginx’s Connection Lifecycle
  • Frequently Asked Questions
  • The Case: Frequent Nginx Worker Process Crashes No Log Could Explain
  • Analyzing the Recording: Replaying the Crash Back to the First Scene
  • Root Cause: A Use-After-Free in Nginx’s Connection Lifecycle
  • Frequently Asked Questions

UDB + OpenResty XRay: Unlocking the Mystery of OpenResty Application Performance

  • How OpenResty XRay Enhances Efficiency with UDB
  • Hands-On: Analyzing OpenResty Lua Call Stacks with UDB and OpenResty XRay
  • Advantages of Time-Travel Debugging
  • Summary
  • How OpenResty XRay Enhances Efficiency with UDB
  • Hands-On: Analyzing OpenResty Lua Call Stacks with UDB and OpenResty XRay
  • Advantages of Time-Travel Debugging
  • Summary

How to Debug File I/O in Java with Time-Travel Call Stacks

  • Where IDE Breakpoints and Coredumps Fall Short in Java File I/O
  • Reconstructing the Full Java Call Stack from a Native write() Syscall
  • Debug Java File I/O Step by Step with Time-Travel Replay
  • Frequently Asked Questions
  • Where IDE Breakpoints and Coredumps Fall Short in Java File I/O
  • Reconstructing the Full Java Call Stack from a Native write() Syscall
  • Debug Java File I/O Step by Step with Time-Travel Replay
  • Frequently Asked Questions

Unveiling Python Code Execution with UDB

  • The Unique Value of UDB in Python Debugging
  • Enhancing Debugging Capabilities with OpenResty XRay
  • Real-World Example: Analyzing Python Network Request Call Stacks
  • Summary
  • The Unique Value of UDB in Python Debugging
  • Enhancing Debugging Capabilities with OpenResty XRay
  • Real-World Example: Analyzing Python Network Request Call Stacks
  • Summary

Tracing an LRU Cache Memory Leak in OpenResty — Down to the Cached SSL Certificates

  • Why a Low-Traffic OpenResty App Kept Leaking Memory
  • How We Traced the LRU Cache Memory Leak, Step by Step
  • The Result: From Unbounded Growth to Stable Memory
  • Why a Low-Traffic OpenResty App Kept Leaking Memory
  • How We Traced the LRU Cache Memory Leak, Step by Step
  • The Result: From Unbounded Growth to Stable Memory

Uneven Nginx Worker CPU Usage: Tracing a Missing reuseport Flag and a cjson Hotspot

  • The Symptom: Some Nginx Workers Maxed Out While Others Sat Idle
  • How OpenResty XRay Pinpointed the Root Causes
  • The Fix: Three Optimizations and Their Measured Gains
  • Summary: What OpenResty XRay Found in Minutes
  • Frequently Asked Questions
  • What is OpenResty XRay
  • The Symptom: Some Nginx Workers Maxed Out While Others Sat Idle
  • How OpenResty XRay Pinpointed the Root Causes
  • The Fix: Three Optimizations and Their Measured Gains
  • Summary: What OpenResty XRay Found in Minutes
  • Frequently Asked Questions
  • What is OpenResty XRay

The Swiss Army Knife for Dynamic Tracing, How Ylang Tames Complexity

  • Industry Pain Point: Why We Desperately Need a New Tracing Language
  • The Core Tech: What Makes Y Language a Game-Changer?
  • Ylang Syntax: The Familiarity of C, The Power of Advanced Tracing
  • Killer Applications: Solving Problems Beyond the Reach of Traditional Tools
  • Universal Platform Support: Y Lang Goes Everywhere
  • Compiler Wizardry: Forging a Superior Language with Superior Tools
  • Industry Pain Point: Why We Desperately Need a New Tracing Language
  • The Core Tech: What Makes Y Language a Game-Changer?
  • Ylang Syntax: The Familiarity of C, The Power of Advanced Tracing
  • Killer Applications: Solving Problems Beyond the Reach of Traditional Tools
  • Universal Platform Support: Y Lang Goes Everywhere
  • Compiler Wizardry: Forging a Superior Language with Superior Tools

Why Dynamic Tracing is the Future of Production Troubleshooting

  • The Unseen Crisis in Modern Software
  • Why Traditional Tools Are Failing
  • Dynamic Tracing: The New Paradigm for Troubleshooting
  • The OpenResty XRay Difference
  • From Firefighting to True Observability
  • The Unseen Crisis in Modern Software
  • Why Traditional Tools Are Failing
  • Dynamic Tracing: The New Paradigm for Troubleshooting
  • The OpenResty XRay Difference
  • From Firefighting to True Observability

Deep Dive into Dynamic Tracing Technology: How OpenResty XRay Revolutionizes Problem Diagnosis

  • Dynamic Tracing: A Powerful Tool for Solving Complex System Issues
  • Why Choose OpenResty XRay?
  • Product Focus: XRay Applications in Real-World Scenarios
  • Common Misconceptions and FAQs: What You Might Want to Know About Dynamic Tracing
  • Dynamic Tracing: A Powerful Tool for Solving Complex System Issues
  • Why Choose OpenResty XRay?
  • Product Focus: XRay Applications in Real-World Scenarios
  • Common Misconceptions and FAQs: What You Might Want to Know About Dynamic Tracing

The Next Stop for Observability: How OpenResty XRay Redefines Problem Troubleshooting

  • Why is Problem Troubleshooting Becoming Increasingly Difficult in Modern Software Architecture?
  • Trend Insights: The Future of Observability Technology
  • How Does Dynamic Tracing Technology Break Through Abstraction Layers?
  • Use Case: Real-time Statistics of Plugin Resource Consumption in Production Kong Services
  • Conclusion
  • Why is Problem Troubleshooting Becoming Increasingly Difficult in Modern Software Architecture?
  • Trend Insights: The Future of Observability Technology
  • How Does Dynamic Tracing Technology Break Through Abstraction Layers?
  • Use Case: Real-time Statistics of Plugin Resource Consumption in Production Kong Services
  • Conclusion

Traditional Methods vs. Dynamic Tracing: A Revolution in Efficiency and Control

  • A Revolution in Efficiency and Control
  • Dynamic Tracing: X-Ray Vision for Your Systems
  • From Flame Graphs to System Panorama
  • Why Choose OpenResty XRay?
  • The Future of Observability: New Frontiers in Dynamic Tracing
  • A Revolution in Efficiency and Control
  • Dynamic Tracing: X-Ray Vision for Your Systems
  • From Flame Graphs to System Panorama
  • Why Choose OpenResty XRay?
  • The Future of Observability: New Frontiers in Dynamic Tracing

Why is Dynamic Tracing Difficult to Implement? How Does OpenResty XRay Break Through?

  • Why is Problem Troubleshooting Becoming Increasingly Difficult in Modern Software Architecture?
  • Trend Insights: The Future of Observability Technology
  • How Does Dynamic Tracing Technology Break Through Abstraction Layers?
  • Use Case: Real-time Statistics of Plugin Resource Consumption in Production Kong Services
  • Conclusion
  • Why is Problem Troubleshooting Becoming Increasingly Difficult in Modern Software Architecture?
  • Trend Insights: The Future of Observability Technology
  • How Does Dynamic Tracing Technology Break Through Abstraction Layers?
  • Use Case: Real-time Statistics of Plugin Resource Consumption in Production Kong Services
  • Conclusion

Self-Optimization: The Performance Evolution of OpenResty XRay

  • Challenges and Pain Points
  • OpenResty XRay’s Self-Analysis Process
  • Remarkable Optimization Results
  • Technical Insights and Business Value
  • Conclusion
  • Challenges and Pain Points
  • OpenResty XRay’s Self-Analysis Process
  • Remarkable Optimization Results
  • Technical Insights and Business Value
  • Conclusion

Case Study: OpenResty XRay Doubles Ylang Compiler Performance

  • Case Study: OpenResty XRay Doubles Ylang Compiler Performance
  • Significant Optimization Results
  • Ylang Language: The Ideal Tool for Dynamic Tracing
  • A Model of Technical Synergy
  • Further Reading
  • Case Study: OpenResty XRay Doubles Ylang Compiler Performance
  • Significant Optimization Results
  • Ylang Language: The Ideal Tool for Dynamic Tracing
  • A Model of Technical Synergy
  • Further Reading

Java Application Profiling: CPU, Off-CPU & Disk I/O Without Safepoint Bias

  • Profiling Java CPU hotspots with flame graphs
  • Off-CPU Java profiling: lock contention and blocking analysis
  • Java disk I/O profiling: tracing file and socket writes
  • OpenJDK 8/11/17/21 application profiling support
  • Automated sampling and expert analysis
  • Performance impact and overhead
  • FAQ
Photo by Yichun Zhang
  • Profiling Java CPU hotspots with flame graphs
  • Off-CPU Java profiling: lock contention and blocking analysis
  • Java disk I/O profiling: tracing file and socket writes
  • OpenJDK 8/11/17/21 application profiling support
  • Automated sampling and expert analysis
  • Performance impact and overhead
  • FAQ

Introduction to OpenResty XRay Mobile Apps

  • Download and install OpenResty XRay for Android
  • Sign into OpenResty XRay
  • Automatic analysis report
  • View the performance chart data on the dashboard page
  • Guided analysis feature
  • What is OpenResty XRay
  • Download and install OpenResty XRay for Android
  • Sign into OpenResty XRay
  • Automatic analysis report
  • View the performance chart data on the dashboard page
  • Guided analysis feature
  • What is OpenResty XRay

The Wonderland of Dynamic Tracing (Part 1 of 3)

  • Dynamic Tracing
  • What It Is
  • Too Good to be True?
  • How it Works
  • Still Having Doubts?
  • The Advantages of Dynamic Tracing
  • DTrace
  • SystemTap
  • DTrace and SystemTap
  • Applications of SystemTap in Production
Photo by Philip Brown
  • Dynamic Tracing
  • What It Is
  • Too Good to be True?
  • How it Works
  • Still Having Doubts?
  • The Advantages of Dynamic Tracing
  • DTrace
  • SystemTap
  • DTrace and SystemTap
  • Applications of SystemTap in Production

The Wonderland of Dynamic Tracing (Part 3 of 3)

  • Dynamic Tracing Support in Linux Kernels
  • Hardware Tracing
  • Analyze Remains of Dead Process
  • Traditional Debugging Technologies
  • A Messy World of Debugging
  • OpenResty XRay
Photo by L N
  • Dynamic Tracing Support in Linux Kernels
  • Hardware Tracing
  • Analyze Remains of Dead Process
  • Traditional Debugging Technologies
  • A Messy World of Debugging
  • OpenResty XRay

One String Value Crashed Asia's Biggest Streaming Site — We Found It with OpenResty XRay

  • The Platform Behind Hundreds of Millions of Streams
  • When Rebooting Doesn’t Help
  • Finding the Root Cause Without Touching a Single Line of Code
  • One Type Mismatch, Entire System Down
Photo by Yichun Zhang
  • The Platform Behind Hundreds of Millions of Streams
  • When Rebooting Doesn’t Help
  • Finding the Root Cause Without Touching a Single Line of Code
  • One Type Mismatch, Entire System Down

Install OpenResty XRay’s Agents on Kubernetes cluster

  • Logging in to the Console
  • Installing the Agent on Kubernetes cluster
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay
  • Logging in to the Console
  • Installing the Agent on Kubernetes cluster
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay

Install OpenResty XRay’s Agents on Amazon Linux via Package Bundles

  • Logging in to the Console
  • Installing the Agent via Bundle Packages
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay
  • Logging in to the Console
  • Installing the Agent via Bundle Packages
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay

Install OpenResty XRay’s Agents on Ubuntu via Deb Packages

  • Logging in to the Console
  • Installing the Agent via Deb Packages
  • Checking the Agent Status and Logs
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay
  • Logging in to the Console
  • Installing the Agent via Deb Packages
  • Checking the Agent Status and Logs
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay

Install OpenResty XRay’s Agents on CentOS via RPM Packages

  • Logging in to the Console
  • Installing the Agent via RPM Packages
  • Checking the Agent Status and Logs
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay
  • Logging in to the Console
  • Installing the Agent via RPM Packages
  • Checking the Agent Status and Logs
  • Configuring and Detecting Applications
  • Initiating Analyzers and Viewing Insights
  • Install Another Agent
  • What is OpenResty XRay

How to deploy the on-premises version of OpenResty XRay on Microsoft's Azure cloud

  • Create a subscription and a resource group
  • Create the Azure Kubernetes service
  • Prepare the environment variables
  • Log in to Azure from the command line
  • Create the namespace and secret for Kubernetes
  • Create the Azure Disk
  • Update the configuration file for the persistent volume
  • Update the Kubernetes configuration file
  • Create the persistent volume
  • Deploy the Kubernetes service
  • Configure the application gateway
  • Create a subscription and a resource group
  • Create the Azure Kubernetes service
  • Prepare the environment variables
  • Log in to Azure from the command line
  • Create the namespace and secret for Kubernetes
  • Create the Azure Disk
  • Update the configuration file for the persistent volume
  • Update the Kubernetes configuration file
  • Create the persistent volume
  • Deploy the Kubernetes service
  • Configure the application gateway

Debugging High Disk I/O in Go Services on Linux

  • Identifying the Go process with high disk I/O
  • Tracing Go disk read and write code paths with flame graphs
  • Automatic disk I/O analysis and reports
  • What is OpenResty XRay
  • Frequently Asked Questions
  • Identifying the Go process with high disk I/O
  • Tracing Go disk read and write code paths with flame graphs
  • Automatic disk I/O analysis and reports
  • What is OpenResty XRay
  • Frequently Asked Questions

Pinpointing CPU-Hottest Go Code Paths Inside Prometheus (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the CPU-hottest Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the CPU-hottest Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay

How CPU Time is Spent inside Go’s CockroachDB (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the most CPU-intensive Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the most CPU-intensive Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay

Measuring the Performance Overhead of OpenResty XRay When Profiling Go Applications

  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay
  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay

Dynamic-Tracing Custom Go Programs with Custom Ylang Analyzers (using OpenResty XRay)

  • Insert two key-value Pairs into a Go Variable
  • Write a Customize Ylang Analyzer to Dynamically Trace the Go program
  • Test the Results
  • What is OpenResty XRay
  • Insert two key-value Pairs into a Go Variable
  • Write a Customize Ylang Analyzer to Dynamically Trace the Go program
  • Test the Results
  • What is OpenResty XRay

How CPU time is spent inside Go's etcd server (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay

Go Off-CPU Analysis: Tracing a Running Process Stuck at 2% CPU to a Blocking Shell Call

  • The symptom: Go service CPU usage stays low under load
  • Running off-CPU analysis on the live Go process with OpenResty XRay
  • Root cause: a blocking shell command at a single source line
  • Automatic analysis and reports
  • The symptom: Go service CPU usage stays low under load
  • Running off-CPU analysis on the live Go process with OpenResty XRay
  • Root cause: a blocking shell command at a single source line
  • Automatic analysis and reports

Pinpointing the hottest Go code paths with high CPU usage (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest Go code paths
  • Automatic analysis and reports
  • What is OpenResty XRay

Diagnosing High Disk I/O in Python Applications with XRay

  • Identifying the Python process with high disk I/O
  • Tracing Python disk read and write code paths with flame graphs
  • Automatic disk I/O analysis and reports
  • What is OpenResty XRay
  • Frequently Asked Questions
  • Identifying the Python process with high disk I/O
  • Tracing Python disk read and write code paths with flame graphs
  • Automatic disk I/O analysis and reports
  • What is OpenResty XRay
  • Frequently Asked Questions

Performance Impact of OpenResty XRay when Tracing Python Apps

  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay
  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay

Python Memory Profiler: Find the Largest Objects Eating RAM Without Code Changes

  • Find which Python process is using the most memory
  • Profile Python memory with guided analysis: find the largest objects and their modules
  • Automatically monitor Python memory usage with scheduled reports
  • What is OpenResty XRay
  • Find which Python process is using the most memory
  • Profile Python memory with guided analysis: find the largest objects and their modules
  • Automatically monitor Python memory usage with scheduled reports
  • What is OpenResty XRay

Python Process Stuck at 8% CPU: Off-CPU Analysis Traces a Blocking subprocess.run Call

  • The symptom: Python process CPU usage stays low under heavy requests
  • Find the blocking Python code paths with OpenResty XRay’s Guided Analysis
  • Automatic analysis and reports in the Insights page
  • FAQ
  • What is OpenResty XRay
  • About The Author
  • The symptom: Python process CPU usage stays low under heavy requests
  • Find the blocking Python code paths with OpenResty XRay’s Guided Analysis
  • Automatic analysis and reports in the Insights page
  • FAQ
  • What is OpenResty XRay
  • About The Author

Java Tomcat Slow Response Time: Pinpoint the Exact Slow Request with PCAP

  • Diagnose Tomcat slow response time by catching the exact slow request
  • Selective packet capture: filter by duration, status, and path
  • Per-request PCAP download for one specific slow request
  • Tell server-side latency from network delay
  • Works for Spring Boot apps with embedded Tomcat
Photo by Yichun Zhang
  • Diagnose Tomcat slow response time by catching the exact slow request
  • Selective packet capture: filter by duration, status, and path
  • Per-request PCAP download for one specific slow request
  • Tell server-side latency from network delay
  • Works for Spring Boot apps with embedded Tomcat

Perl Process Stuck at 13% CPU: Off-CPU Flame Graphs Pinpoint a Blocking HTTP Call

  • The symptom: Perl process CPU usage stays low under heavy requests
  • Find the blocking Perl code paths with OpenResty XRay’s off-CPU flame graphs
  • Automatic analysis and reports in the Insights page
  • FAQ
  • What is OpenResty XRay
  • About The Author
  • The symptom: Perl process CPU usage stays low under heavy requests
  • Find the blocking Perl code paths with OpenResty XRay’s off-CPU flame graphs
  • Automatic analysis and reports in the Insights page
  • FAQ
  • What is OpenResty XRay
  • About The Author

Performance Overhead of OpenResty XRay when Tracing Perl Apps

  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay
  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay

Locate the Largest Memory Objects in a Live Perl Process

  • Symptom: a Perl process using high memory in top
  • Profile the process with OpenResty XRay guided analysis to find the largest Perl objects
  • Trace the largest object back to its source file
  • Monitor Perl process memory automatically with Insights reports
  • FAQ
  • What is OpenResty XRay
  • Symptom: a Perl process using high memory in top
  • Profile the process with OpenResty XRay guided analysis to find the largest Perl objects
  • Trace the largest object back to its source file
  • Monitor Perl process memory automatically with Insights reports
  • FAQ
  • What is OpenResty XRay

Pinpointing hottest Perl code paths (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest Perl code paths
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest Perl code paths
  • Automatic analysis and reports
  • What is OpenResty XRay

Why Your Rust Process Has High Disk I/O (and How to Find the Cause)

  • Identifying the Rust process with high disk I/O
  • Tracing the hottest Rust disk write code paths with flame graphs
  • Automatic disk I/O analysis and reports
  • What is OpenResty XRay
  • Frequently Asked Questions
  • Identifying the Rust process with high disk I/O
  • Tracing the hottest Rust disk write code paths with flame graphs
  • Automatic disk I/O analysis and reports
  • What is OpenResty XRay
  • Frequently Asked Questions

Tested in Production: The Performance Impact of OpenResty XRay When Tracing Rust Applications

  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay
  • Process Performance Before Running Analysis
  • Performance Impact When the Analyzer is Running
  • Impact Measurement on Throughput and Latency
  • What is OpenResty XRay

How CPU time is spent inside Rust’s Sled library (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the most CPU-intensive Rust code paths
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the most CPU-intensive Rust code paths
  • Automatic analysis and reports
  • What is OpenResty XRay

Use C++ to Dynamic-Trace C++ Applications

  • Setting Up the Target C++ Program
  • Crafting the C++ (or Y++) Analyzer
  • Operationalizing the Target and Analyzer
  • Advancing Support for Complex C++ Applications
  • About the Debug Symbols
Photo by Yichun Zhang
  • Setting Up the Target C++ Program
  • Crafting the C++ (or Y++) Analyzer
  • Operationalizing the Target and Analyzer
  • Advancing Support for Complex C++ Applications
  • About the Debug Symbols

Nginx 'No Debugging Symbols Found'? Rebuild Them Automatically with OpenResty XRay

  • Confirming Nginx Is Stripped and Debug Symbols Are Missing
  • Rebuild Missing Nginx Debug Symbols Automatically — No Recompile Needed
  • Continuous Analysis: Daily and Weekly Insights Reports
  • FAQ: nginx-dbg, recompiling, and profiling stripped binaries
  • Confirming Nginx Is Stripped and Debug Symbols Are Missing
  • Rebuild Missing Nginx Debug Symbols Automatically — No Recompile Needed
  • Continuous Analysis: Daily and Weekly Insights Reports
  • FAQ: nginx-dbg, recompiling, and profiling stripped binaries

Automatic Core Dump Analysis (using OpenResty XRay)

  • Locate the core dump file from a crashed OpenResty/Nginx worker
  • Start automatic (guided) core dump analysis in OpenResty XRay
  • Read the signal, CPU registers, and C backtrace of the crash
  • Follow the Lua backtrace to the exact crashing line
  • Automatic Insights reports for online core dumps
  • Locate the core dump file from a crashed OpenResty/Nginx worker
  • Start automatic (guided) core dump analysis in OpenResty XRay
  • Read the signal, CPU registers, and C backtrace of the crash
  • Follow the Lua backtrace to the exact crashing line
  • Automatic Insights reports for online core dumps

How CPU Time is Spent inside Envoy Servers (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the most CPU-intensive C++ code paths inside Envoy Servers
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the most CPU-intensive C++ code paths inside Envoy Servers
  • Automatic analysis and reports
  • What is OpenResty XRay

How to Diagnose 504 Gateway Timeout Errors in OpenResty & Nginx

  • Confirm HTTP 504 gateway timeout errors in the OpenResty/Nginx access log
  • Diagnose the 504 root cause with OpenResty XRay Guided Analysis
  • Slow upstream, slow network link, or a too-short timeout setting
  • Automatic root-cause reports on the Insights page
  • Confirm HTTP 504 gateway timeout errors in the OpenResty/Nginx access log
  • Diagnose the 504 root cause with OpenResty XRay Guided Analysis
  • Slow upstream, slow network link, or a too-short timeout setting
  • Automatic root-cause reports on the Insights page

Django Memory Profiler: Analyze Object-Level Memory Usage in a Live App

  • Spot a high-memory Django process with ps
  • Profile the Django app’s memory usage with Guided Analysis
  • Read the flame graph: which Python module uses the most memory
  • Monitor Django memory usage automatically with Insights
  • What is OpenResty XRay
  • Spot a high-memory Django process with ps
  • Profile the Django app’s memory usage with Guided Analysis
  • Read the flame graph: which Python module uses the most memory
  • Monitor Django memory usage automatically with Insights
  • What is OpenResty XRay

How CPU time is spent inside llama.cpp + LLaMA2 (using OpenResty XRay)

  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest C++ code paths
  • Automatic analysis and reports
  • What is OpenResty XRay
  • Problem: high CPU usage
  • Use the guidede analysis feature of OpenResty XRay to spot the hottest C++ code paths
  • Automatic analysis and reports
  • What is OpenResty XRay

Find Leaked Lua Tables: the registry / _LOADED (package.loaded) Reference Path in OpenResty XRay

  • How LuaJIT manages memory
  • OpenResty XRay’s command-line tools
  • A leaking Lua table example
  • Breaking down memory by LuaJIT GC object type
  • Reading the lj-gco-ref flame graph: registry, _LOADED, package.loaded
  • Fully-automated Lua memory leak analysis
  • How LuaJIT manages memory
  • OpenResty XRay’s command-line tools
  • A leaking Lua table example
  • Breaking down memory by LuaJIT GC object type
  • Reading the lj-gco-ref flame graph: registry, _LOADED, package.loaded
  • Fully-automated Lua memory leak analysis

Real-Time Request Counting against Nginx Processes via YSQL

  • How to install the run-ysql tool
  • Count total requests served in real time
  • Filter out specific requests
  • Using YSQL in the Web Console
  • True non-invasive tracing
Photo by Yichun Zhang
  • How to install the run-ysql tool
  • Count total requests served in real time
  • Filter out specific requests
  • Using YSQL in the Web Console
  • True non-invasive tracing

Tracing the Slowest PCRE Regular Expressions in OpenResty or Nginx Processes

  • System Environment
  • Narrowing Down the Culprit without Guessing
  • Limiting the execution overhead of PCRE
  • Non-Backtracking Regular Expression Engines
  • Lua’s Builtin Patterns
  • Tracing Applications inside Containers
  • How The Tools are Implemented
  • The Overhead of the Tools
  • System Environment
  • Narrowing Down the Culprit without Guessing
  • Limiting the execution overhead of PCRE
  • Non-Backtracking Regular Expression Engines
  • Lua’s Builtin Patterns
  • Tracing Applications inside Containers
  • How The Tools are Implemented
  • The Overhead of the Tools

Listing Loaded Lua Modules in OpenResty or Nginx Processes

  • System Environment
  • Names of Loaded Lua Modules
  • Running Directly in the Web Console
  • Tracing Applications inside Containers
  • How The Tools are Implemented
  • The Overhead of the Tools
Photo by Yichun Zhang
  • System Environment
  • Names of Loaded Lua Modules
  • Running Directly in the Web Console
  • Tracing Applications inside Containers
  • How The Tools are Implemented
  • The Overhead of the Tools

Analyzing the Most CPU-Consuming Requests in OpenResty or Nginx

  • System Environment
  • CPU-Hottest Request Hostnames
  • CPU-Hottest Request URIs
  • Digging Deeper
  • Running Directly in the Web Console
  • Tracing Applications inside Containers
  • How The Tools are Implemented
  • The Overhead of the Tools
Photo by Yichun Zhang
  • System Environment
  • CPU-Hottest Request Hostnames
  • CPU-Hottest Request URIs
  • Digging Deeper
  • Running Directly in the Web Console
  • Tracing Applications inside Containers
  • How The Tools are Implemented
  • The Overhead of the Tools

Ylang in Production: Container Tracing, Stack Unwinding & Core Dumps (Series Part 4)

  • Transparent Container Tracing
  • Efficient Stack Unwinding
  • Analyzing Dead Processes (Core Dumps)
  • Very Low Tracing Overhead
  • Standard Ylang Libraries & Tools
  • Network Filtering & Control
  • The Ylang Compiler’s Implementation
  • Operating System Support
  • Contributions to Open-Source Communities
Photo by Yichun Zhang
  • Transparent Container Tracing
  • Efficient Stack Unwinding
  • Analyzing Dead Processes (Core Dumps)
  • Very Low Tracing Overhead
  • Standard Ylang Libraries & Tools
  • Network Filtering & Control
  • The Ylang Compiler’s Implementation
  • Operating System Support
  • Contributions to Open-Source Communities

Ylang: Regex, Full Control Flow & Transparent Debug Symbol Handling (Series Part 3)

  • The Language Syntax (Continued)
    • String Literals
    • Built-in Regex Support
    • Full Control-Flow Support
    • Floating-Point Number Support
    • Compared to Open-Source Toolchains
  • Transparent Debug Symbol Handling
    • Debug Symbols: No Runtime Overhead
    • Centralized Package Database
    • Fuzzy Matching Debug Symbols
Photo by Yichun Zhang
  • The Language Syntax (Continued)
    • String Literals
    • Built-in Regex Support
    • Full Control-Flow Support
    • Floating-Point Number Support
    • Compared to Open-Source Toolchains
  • Transparent Debug Symbol Handling
    • Debug Symbols: No Runtime Overhead
    • Centralized Package Database
    • Fuzzy Matching Debug Symbols

Ylang: A Dynamic Tracing Language for eBPF, SystemTap, GDB, and More (Part 1 of 4)

  • What is Dynamic Tracing
  • Why the “Y” Name
  • Getting Started with Ylang
  • Dynamic Tracing Backends: eBPF+, Stap+, GDB, and ODB
  • Why a Unified Frontend Language for Dynamic Tracing
  • The Ylang Syntax: Genuinely C-Compatible
  • Frequently Asked Questions
Photo by Yichun Zhang
  • What is Dynamic Tracing
  • Why the “Y” Name
  • Getting Started with Ylang
  • Dynamic Tracing Backends: eBPF+, Stap+, GDB, and ODB
  • Why a Unified Frontend Language for Dynamic Tracing
  • The Ylang Syntax: Genuinely C-Compatible
  • Frequently Asked Questions

LuaJIT GC64 Mode: Removing the 2GB Memory Limit

  • The Old Memory Limit
    • When Hitting The Memory Limit
    • The Memory Limit Is Per Process
    • GC-Managed Memory
    • Memory Allocated Outside GC
    • Extending The x64 Mode to The 4 GB Limit
  • The New GC64 Mode
    • How to Enable The GC64 Mode
    • Performance Impact
    • Debugging and Profiling Tool Chains
  • The Old Memory Limit
    • When Hitting The Memory Limit
    • The Memory Limit Is Per Process
    • GC-Managed Memory
    • Memory Allocated Outside GC
    • Extending The x64 Mode to The 4 GB Limit
  • The New GC64 Mode
    • How to Enable The GC64 Mode
    • Performance Impact
    • Debugging and Profiling Tool Chains
Contact us

Love to hear from you, Get in touch 👋

Message was sent successfully!

Our Team will get back to you within 24 hours.
x