Nginx C++ Module Memory Leak: Root Cause in Production, No Restart

A memory leak in an Nginx C++ module is one of the hardest bugs to catch in production: tools like GDB can’t safely run against a live service, and auditing the code by hand is a needle-in-a-haystack effort. Using OpenResty XRay’s dynamic tracing, we captured a memory-leak flame graph directly in production in minutes — with no restart — and traced a runaway ngx_dubbo_module allocation down to the exact line, where a memory block was mistakenly marked as externally managed and silently skipped by the reclamation mechanism.

Below, we walk through the full diagnosis — from the first flame graph to the root-cause line — and how OpenResty XRay turns it into a repeatable diagnose → plan → verify loop you can reuse on any C/C++ process in production.

The Production Challenge: A C++ Module Memory Leak You Can’t Debug With GDB

One of our clients’ core Nginx services experienced continuously rising memory usage, akin to a “memory black hole.” Although temporary restarts offered brief relief, the problem quickly resurfaced.

Challenges and Risks:

• Technical Perspective: A C/C++ process memory leak in a production environment is notoriously one of the most challenging issues to resolve. Traditional debugging tools (like GDB) cannot be directly deployed on live systems, and code auditing is akin to finding a needle in a haystack—time-consuming and inefficient.
• Business Impact: If left unaddressed, memory consumption would continue to escalate, leading to slow responses, frequent crashes, and ultimately jeopardizing core business continuity. This would result in: degraded user experience, gradual user churn, damage to brand reputation, and direct economic losses.

Relying on scheduled restarts to “keep the system alive” is merely a stopgap measure, akin to drinking poison to quench thirst, leaving the entire business perpetually exposed to risk. Upon receiving the request for assistance, we immediately utilized our dynamic tracing product, OpenResty XRay, to directly analyze memory allocation in the production environment. Within minutes, the tool automatically generated a memory leak flame graph, providing a clear breakthrough for troubleshooting the issue.

The flame graph clearly illustrated that memory allocation was primarily concentrated in three areas:

1. Nginx memory pool
2. TLS-related memory
3. ngx_dubbo_module function

Both the Nginx memory pool and TLS are well-established, mature components with an extremely low probability of memory leaks. Therefore, we swiftly narrowed our analysis to the most suspect component: the ngx_dubbo_module function.

(When a leak instead traces back to the Nginx core memory pool itself rather than a third-party module, the diagnosis differs — see our Nginx production memory leak case.)

How the Flame Graph Pinpointed the Leaking Function in ngx_dubbo_module

Step One: Identify the Culprit Function.

Leveraging the powerful dynamic tracing capabilities of OpenResty XRay, we can efficiently dissect problems with surgical precision, eliminating the need to sift through massive logs or conduct exhaustive code audits. Our focus then narrows down to the most suspect ngx_dubbo_module function.

The memory leak flame graph clearly indicates that the peak of memory allocation points directly to the ngx_dubbo_hessian2_encode_payload_map function, marking it as the primary hotspot for the memory leak.

Step 2: Delving into the Code to Uncover Anomalies.

Leveraging the flame graph, we directly pinpointed the most suspect function. Upon deeper examination of its source code, we quickly identified the core issue: during data processing, the code would specifically allocate a new block of memory to store a portion of the content.

However, after reviewing the entire processing flow, we found no operation to release this memory block. Our intuition strongly suggested this was the source of the problem. This newly allocated memory, once encapsulated, was then passed to subsequent modules for processing. The critical question arose: who was truly responsible for managing its lifecycle?

Step 3: Tracing the Clues to Unveil the Truth.

To fully understand the situation, we traced the propagation path of this memory block within the subsequent modules. We discovered that the module receiving this memory block possessed its own memory reclamation mechanism, designed to automatically clean up objects no longer in use.

The crux of the problem lay in how an object’s management status was communicated to this mechanism. In the customer’s specific scenario, when this memory block was passed in, due to its unique creation method, it was mistakenly marked as “not to be managed by this module.” This effectively instructed the reclamation mechanism: “This memory block is handled externally; no action is required from you.” Yet, in reality, no other entity was designated to release it.

Consequently, due to this erroneous signal, the automatic reclamation mechanism completely bypassed this memory block. As requests continued to flood in, countless such memory blocks were allocated but never released, resulting in a significant memory leak. We used the flame graph to directly pinpoint the exact code line within the memory allocation function and clearly articulated why this block failed to be automatically reclaimed. Based on this precise insight, the customer quickly confirmed the fundamental cause of the memory leak through analysis of the relevant source code.

From Reactive to Proactive: XRay’s New Approach to Troubleshooting

In the pursuit of system stability, many teams find themselves caught in a vicious cycle: to solve problems, they collect more data for more clues; to collect more data, they add more monitoring metrics, build more complex dashboards, and invest huge budgets in additional monitoring tools. The result is an overwhelming amount of fragmented information, with data proliferating while the signal-to-noise ratio plummets. Engineers are left sifting through a “data graveyard,” searching for valuable insights like prospectors panning for gold.

The fundamental flaw of this model is its failure to adequately connect technical problems, such as memory leaks, with their true business costs:

• A memory leak alert, seemingly just a fluctuation in technical metrics, actually represents a significant drain on engineers’ valuable time.
• A frequent production incident not only threatens system stability but also carries the potential for business disruption.
• Continuous, high monitoring investment often fails to yield commensurate actionable insights.

Ultimately, the issue isn’t “too little data,” but “too few useful answers.” A truly efficient observability tool should act like a “scalpel,” directly pinpointing the root cause of a problem, rather than handing teams another “shovel” and forcing them to endlessly dig through data day and night. This is precisely the core challenge that the next-generation observability platform aims to solve:

• Moving from surface-level metrics to causal chains
• Shifting from passive alerts to active analysis and insights
• Evolving from consuming human resources to truly unlocking R&D productivity

The OpenResty XRay Closed-Loop Approach

In this context, OpenResty XRay introduces a transformative methodology: converting complex, time-consuming processes into a repeatable, self-sustaining cycle. We define this as “Diagnose → Plan → Verify”.

• Pinpoint Accuracy: Traditional methods often involve “sifting through more data, hoping to find a clue.” XRay’s approach is fundamentally different. Utilizing advanced visualization tools like flame graphs, it directly identifies the problem’s location, enabling teams to uncover the root cause within minutes. It delivers not just more monitoring metrics, but critical, actionable insights that drive immediate resolution.
• Evidence-Based Solutioning: When a problem is precisely traced to a specific line of code or a particular C/C++ process, the remediation plan becomes direct and targeted. Teams can bypass iterative trial and error, instead formulating clear, effective action steps based on irrefutable evidence.
• Instant Validation for a Complete Cycle: After a fix is deployed, XRay can re-analyze the system to confirm the issue has been thoroughly resolved. This rapid validation mechanism ensures a truly closed-loop process: diagnosis, action, and verification are seamlessly integrated.

The value derived from this closed-loop capability extends far beyond resolving individual bugs:

• Technical Empowerment: It empowers teams to non-invasively gain deep insights into C/C++ processes within production environments, simplifying and visualizing inherently complex issues.
• Strategic Business Impact:
  • Enhanced Business Stability: Proactively eliminates potential failures, ensuring robust service high availability.
  • Accelerated R&D Efficiency: Frees engineers from tedious, inefficient debugging cycles, allowing them to focus on higher-value business innovation.
  • Optimized Resource Utilization: Prevents over-provisioning of resources due to unidentified performance bottlenecks, leading to genuine cost control.

In an era of data overload, technical teams no longer need just more monitoring dashboards. What they truly require is a smarter, more dependable approach to quickly identify and resolve problems. This, fundamentally, is the true power of observability.

FAQ

How do you find a memory leak in an Nginx C/C++ module without restarting the service?

OpenResty XRay analyzes memory allocation directly in the running production process using dynamic tracing, so there is no need to restart Nginx or recompile it. Within minutes it generates a memory-leak flame graph that shows exactly where allocations concentrate — in this case, the ngx_dubbo_module code path.

Can you debug a C/C++ memory leak in production without GDB?

Yes. Traditional tools like GDB cannot be safely deployed on a live system, and code auditing is slow and error-prone. OpenResty XRay uses non-invasive dynamic tracing to pinpoint the leaking function — and even the exact line of code — without attaching a debugger or stopping the service.

What does a memory leak flame graph show?

It shows where memory allocation concentrates across the call stack. Here the flame graph concentrated allocation in three areas — the Nginx memory pool, TLS-related memory, and ngx_dubbo_module — and its peak pointed directly to the ngx_dubbo_hessian2_encode_payload_map function.

Why is allocated memory never freed even when a reclamation mechanism exists?

In this case the leaked block was mistakenly marked as “not to be managed by this module,” so the downstream module’s automatic reclamation mechanism skipped it — while no other owner was ever assigned to release it. Every incoming request allocated another such block that was never freed, producing the leak.

What is OpenResty XRay

OpenResty XRay is a dynamic-tracing product that automatically analyzes your running applications to troubleshoot performance problems, behavioral issues, and security vulnerabilities with actionable suggestions. Under the hood, OpenResty XRay is powered by our Y language targeting various runtimes like Stap+, eBPF+, GDB, and ODB, depending on the contexts.

About The Author

Yichun Zhang (Github handle: agentzh), is the original creator of the OpenResty^® open-source project and the CEO of OpenResty Inc..

Yichun is one of the earliest advocates and leaders of “open-source technology”. He worked at many internationally renowned tech companies, such as Cloudflare, Yahoo!. He is a pioneer of “edge computing”, “dynamic tracing” and “machine coding”, with over 22 years of programming and 16 years of open source experience. Yichun is well-known in the open-source space as the project leader of OpenResty^®, adopted by more than 40 million global website domains.

OpenResty Inc., the enterprise software start-up founded by Yichun in 2017, has customers from some of the biggest companies in the world. Its flagship product, OpenResty XRay, is a non-invasive profiling and troubleshooting tool that significantly enhances and utilizes dynamic tracing technology. And its OpenResty Edge product is a powerful distributed traffic management and private CDN software product.

As an avid open-source contributor, Yichun has contributed more than a million lines of code to numerous open-source projects, including Linux kernel, Nginx, LuaJIT, GDB, SystemTap, LLVM, Perl, etc. He has also authored more than 60 open-source software libraries.