llama.cpp

OpenResty XRay Nov 1, 2023

OpenResty XRay Updated Nov 1, 2023 6 mins read

How CPU time is spent inside llama.cpp + LLaMA2 (using OpenResty XRay)

Problem: high CPU usage
Use the guidede analysis feature of OpenResty XRay to spot the hottest C++ code paths
Automatic analysis and reports
What is OpenResty XRay

Problem: high CPU usage
Use the guidede analysis feature of OpenResty XRay to spot the hottest C++ code paths
Automatic analysis and reports
What is OpenResty XRay

OpenResty XRay Jan 14, 2023

OpenResty XRay Jan 14, 2023

Automatic Analysis Reports in OpenResty XRay

Photo by Yichun Zhang

The Past
The Present
The Future

OpenResty XRay Nov 12, 2021

OpenResty XRay Nov 12, 2021

Ylang: Universal Language for eBPF, Stap+, GDB, and More (Part 1 of 4)

Photo by Yichun Zhang

What is Dynamic Tracing
Why the “Y” Name
Getting Started
Various Backends and Runtimes
Why a Unified Frontend Language
The Language Syntax

OpenResty XRay Aug 31, 2020

OpenResty XRay Aug 31, 2020

Introduction to Lua-Land CPU Flame Graphs

Photo by Guido Jansen

What is a Flame Graph
Simple Lua samples
Complicated Lua applications
Sampling overhead
Safety
Compatibility
Other types of Lua-land Flame Graphs

OpenResty XRay Aug 10, 2020

OpenResty XRay Aug 10, 2020

Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones

Photo by Elena Mozhvilo

An empty zone
Filling entries of similar sizes
Deleting odd-numbered keys
Deleting the keys in the first half
Mitigating Fragmentation

OpenResty XRay Aug 4, 2020

OpenResty XRay Aug 4, 2020

How OpenResty and Nginx Shared Memory Zones Consume RAM

Photo by Jose G. Ortega Castro

Slabs and pages
What is allocated is not what is paid for
Fake Memory Leaks
HUP reload

OpenResty XRay Jan 21, 2020

OpenResty XRay Jan 21, 2020

How OpenResty and Nginx Allocate and Manage Memory

Photo by Harrison Broadbent

On The System Level
On The Application Level
For Traditional Nginx Servers

OpenResety Edge May 8, 2024

OpenResety Edge May 8, 2024

Enable Automatic Healthcheck for Gateway Servers in OpenResty Edge

Photo by Yichun Zhang

Create a new page rule
Enable the healthcheck for gateway server
Test the result
What is OpenResty Edge

OpenResety Edge Apr 28, 2024

OpenResety Edge Apr 28, 2024

How to Use Gateway Partitions in OpenResty Edge

Photo by Yichun Zhang

Create a new partition
Create a new application in the new partition
Create a new page rule
Test the new partition
What is OpenResty Edge

OpenResety Edge Apr 10, 2024

OpenResety Edge Apr 10, 2024

Accurately Restore the Real Client IP Address in OpenResty Edge

Photo by Yichun Zhang

Set “Trusted hosts to set real IP” and “Real IP from” Globally
Configure the Application to Output the Client Address
Check the Client Address Received on OpenResty Edge
What is OpenResty Edge

OpenResty XRay Apr 4, 2024

OpenResty XRay Apr 4, 2024

Use C++ to Dynamic-Trace C++ Applications

Photo by Yichun Zhang

Setting Up the Target C++ Program
Crafting the C++ (or Y++) Analyzer
Operationalizing the Target and Analyzer
Advancing Support for Complex C++ Applications
About the Debug Symbols

OpenResty XRay Apr 1, 2024

OpenResty XRay Apr 1, 2024

Pinpointing the hottest Erlang code paths with high CPU usage (using OpenResty XRay)

Pinpointing the hottest Erlang code paths with high CPU usage (using OpenResty XRay)

Problem: high CPU usage
Spot the CPU-hottest Erlang code paths
Automatic analysis and reports
What is OpenResty XRay

Contact us