Tracing & Performance

How to Read a Flame Graph (Width Is What Matters)

Flame graphs look intimidating and read simply: width is time, the y-axis is stack depth, and colors mean nothing. A practical guide to spotting hot paths.

Flame graphs are one of the most powerful tools for understanding CPU performance, yet they're widely misread. The most common mistake is treating the x-axis as time—it isn't. Learning how to read a flame graph correctly takes minutes, but the payoff is enormous: you'll stop chasing phantom bottlenecks and focus optimization effort where it actually matters. This guide walks you through the counter-intuitive rules that make flame graphs work, shows you what patterns mean, and teaches you how to distinguish flame graphs from similar-looking tools that work entirely differently.

The Counter-Intuitive Rules

Flame graphs look deceptively simple—stacked rectangles that visualize CPU time. But the axes work nothing like you'd expect, and misunderstanding them is the root of most flame graph confusion.

The x-axis is NOT time. Instead, it's an alphabetically-sorted merge of all the stacks in your sample. The profiler samples which function is running (say, 10,000 times per second) and records the full call stack. Then it merges stacks that look identical and lays them out alphabetically. Position left-to-right tells you nothing about call order or timing. It's purely a layout choice to keep similar stacks together.

Width IS time spent. The wider a bar, the more samples landed in that function—meaning the more CPU time that function consumed, relative to your total sampling duration. If a function's bar spans half the graph, that function was executing for half your profiling window (in aggregate).

The y-axis is stack depth. Functions that call other functions stack vertically. A frame sits directly on top of its caller. If you see function b on top of function a, then b was called by a.

Colors are almost always meaningless. Most flame graphs assign random warm colors (reds, oranges, yellows) to functions to make the visualization less monotonous. Don't infer meaning from color. Some profilers do encode information in color—red for regex, orange for garbage collection—but this is tool-specific. Check the legend.

What the Shapes Tell You

Once you know the axes, patterns jump out.

Wide leaf frames (frames with nothing stacked on top) are your real hotspots. They're functions spending CPU time directly, not just calling other functions. If you see a bar spanning 20% of the graph at a leaf level, that's 20% of your samples—optimize there first.

A wide frame with narrow children means the parent function is spending time in its own code, not delegated to children. For example, if JSON.stringify() is wide and its children (allocations, recursive calls) are narrow, the JSON serialization engine itself is your bottleneck.

Unexpected wide frames (JSON parsing, regex matching, logging, garbage collection) often signal that the obvious code isn't your problem. If you're wondering why an API is slow and the flame graph shows half its time in Object.stringify() or Pattern.match(), you've found the real culprit.

Everything is 1-pixel wide means your sampling is too sparse, the work is spread thinly across many functions, or you didn't profile long enough. Increase your sample duration or sampling rate.

Reading Rules in Pictures

Here's what a typical flame graph structure looks like. Imagine a 20ms profiling window:

                             ┌─────────────────────────────────────────┐
                             │         main (request handler)          │ ← caller
                             └─────────────────────────────────────────┘
            ┌──────────────────────────┬───────────────────┐
            │                          │                   │
      ┌─────────────┐         ┌────────────────┐    ┌──────────────┐
      │ getUserData │         │ formatResponse │    │   logging    │ ← children (callees)
      │ (8ms)       │         │ (7ms)          │    │ (2ms)        │
      └─────────────┘         └────────────────┘    └──────────────┘
            │                         │
      ┌─────────────┐         ┌──────────────┐
      │ db.query    │         │ JSON.stringify │
      │ (8ms)       │         │ (6ms)          │
      └─────────────┘         └──────────────┘

In this snapshot, main called three functions. getUserData spent 8ms in db.query, formatResponse spent 6ms in JSON.stringify, and logging was its own work. The widths reflect the time spent: getUserData and formatResponse are roughly equal; logging is 25% of their width.

If you can interact with the flame graph (in tools like Go's pprof or async-profiler's web UI), click on wide frames to zoom. This reveals the internal call structure of hot functions—often showing surprising inefficiencies in code you thought was fast.

Flame Graphs, Icicle Graphs, Differential Graphs, and Flame Charts: Know the Difference

The term "flame graph" has spawned variants that look similar but work differently. Conflating them is a recipe for misinterpretation.

Flame graphs are what we've discussed: alphabetically-sorted, merged stacks with x-axis as layout (not time) and width as CPU time. Classic format, invented by Brendan Gregg.

Icicle graphs invert the visualization—the root function is at the top, and callees point downward. They're functionally identical to flame graphs, just flipped. Some teams prefer icicles because they're used to top-down call trees. The reading rules are the same: width = time, y-axis = depth.

Differential flame graphs compare two profiles (baseline vs. regression, before-and-after optimization) by encoding the difference in color. Red frames mean regression (got slower), blue means improvement (got faster), neutral color means no change. They're powerful for tracking optimization impact, but only if your baseline profile and optimized profile are captured under identical load.

Flame charts (common in Chrome DevTools, Firefox, and browser profilers) ARE time-ordered on the x-axis. This is the critical difference. In a flame chart, the x-axis truly is time; left-to-right is chronological. Width still represents duration, but because time is real, a function's position matters. If you see a function on the far right, it happened late in the profiling window.

The #1 mistake: confusing a flame chart with a flame graph. Flame charts (in Chrome DevTools) ARE time-ordered on the x-axis. Flame graphs are NOT. This changes how you read them. In a flame chart, you can see temporal patterns—a function that only runs after a certain point, or pauses. In a flame graph, that information is lost. Know which tool you're using.

How Flame Graphs Are Generated

Different ecosystems have different profilers, but the workflow is always the same: sample the call stack, merge identical stacks, render.

Linux: perf collects samples; the flamegraph.pl and stackcollapse.pl scripts process them into the visualization. Example:

perf record -F 99 -p <pid> -g -- sleep 30
perf script > out.perf
stackcollapse-perf.pl out.perf > out.folded
flamegraph.pl out.folded > flame.svg

Java: async-profiler is the industry standard. It attaches to a running JVM with near-zero overhead and produces HTML flame graphs:

./profiler.sh -d 30 -f /tmp/flame.html <pid>

Go: pprof is built-in. Visit http://localhost:6060/debug/pprof/profile?seconds=30 on any instrumented service, then visualize:

go tool pprof -http=:8080 <profile_file>

Node.js: Use the native --prof flag or clinic.js for a polished experience. clinic.js also provides flame graphs alongside other profiling output.

Python: py-spy attaches to running processes with minimal overhead:

py-spy record -o profile.svg -- python myapp.py

All these tools produce similar-looking flame graphs, so a flame graph from Java async-profiler and one from Go pprof are read the same way.

The Honest Caveat: Sampling Profilers Miss Off-CPU Time

Flame graphs show CPU time, but they show nothing about time your code spends waiting—blocked on I/O, waiting for a lock, sleeping. A service that's I/O-bound (waiting on database queries, network requests, disk) will have a tiny, flat flame graph even if it's slow from the user's perspective.

If your application is making 100 database queries sequentially, each taking 10ms, the flame graph sees only the query library functions (maybe 1% CPU overhead). The real bottleneck—the 1000ms in I/O—is invisible.

Off-CPU flame graphs exist to address this (they sample when your thread is blocked, not running), but they're less common and require special profiler support. For most production debugging, combine flame graphs with distributed tracing: flame graphs tell you if CPU is the bottleneck; traces tell you if the bottleneck is I/O, lock contention, or garbage collection.

Using Flame Graphs in Your Performance Workflow

Flame graphs excel at answering one question: "Where is my CPU time going?" But they're part of a larger system.

Distributed tracing (tracked with span waterfalls) shows the end-to-end flow of a request and where time is spent across services. If a trace shows a slow transaction, you see which service or external call is responsible.

Once you've identified a slow service, flame graphs drill in: is it CPU-bound? If so, which functions? This is the critical handoff. You see a slow database query span in a trace? Flame graphs won't help—the database is a black box. But you see a slow custom function in a trace? Flame profile that service to find the hot code.

The workflow looks like this:

  1. Check transaction latency percentiles in your monitoring (p50, p95, p99).
  2. If latency is high, inspect a slow transaction's span waterfall.
  3. Identify the slowest span (might be a database query, a cache miss, or a custom function).
  4. If it's a custom function, debug the slow API endpoint by profiling that service's CPU.
  5. Open the flame graph and find the widest frames—those are your optimization targets.

Flame graphs bypass guesswork. Instead of hypothesizing about where time goes, you see evidence.

LightTrace doesn't do CPU profiling—it captures distributed traces and performance metrics. But when a span waterfall points you to a suspect service, you'll profile that service separately with your language's native profiler and read the flame graph to drill into the CPU-bound functions.


Start tracking errors in minutes

Start with distributed tracing to find slow transactions, then use flame graphs to drill into the CPU-bound functions. LightTrace captures your span waterfalls and transaction latency; profile your services with your language's native profiler to generate flame graphs. Join teams debugging performance systematically—start your free trial today.

Fix your next production error faster

Point any Sentry SDK at LightTrace — free up to 5,000 events/month.