Explain why a metric changed

A dashboard tells you that a metric moved. A useful investigation explains which population changed, how much each segment contributed, and whether the movement represents customer behavior, system behavior, or broken data.

GraphJSON does not provide an automatic root-cause engine. Use its SQL notebook to build a reproducible decomposition, then validate the leading explanation against source systems and operational evidence.

This guide starts after you have detected a meaningful change. For detection methods, use Time-series baselines and anomaly detection.

Freeze the investigation contract#

Write the comparison before slicing the data:

metric: completed checkouts
grain: successful checkout event
current window: 2026-07-20 through 2026-07-26 UTC
baseline window: previous seven complete UTC days
entity: checkout
required maturity: 24 hours
exclusions: test accounts and internal traffic
candidate dimensions: plan, region, platform, release
owner: Growth Engineering

Use complete half-open intervals:

[current_start, current_end)
[baseline_start, baseline_end)

Do not compare a partial Monday with a complete prior Monday, a seven-day window with an eight-day window, or one period before its late events have had time to arrive.

Reproduce the top-line change#

Start with the smallest query that reproduces the dashboard:

WITH periods AS (
  SELECT
    if(
      timestamp >= toUnixTimestamp(toDateTime('2026-07-20 00:00:00')),
      'current',
      'baseline'
    ) AS period,
    count() AS value
  FROM product_events
  WHERE
    JSONExtractString(json, 'event') = 'checkout_completed'
    AND timestamp >= toUnixTimestamp(toDateTime('2026-07-13 00:00:00'))
    AND timestamp < toUnixTimestamp(toDateTime('2026-07-27 00:00:00'))
    AND JSONExtractBool(json, 'is_internal') = false
  GROUP BY period
)
SELECT period, value
FROM periods
ORDER BY period

Confirm the absolute and relative movement:

absolute change = current - baseline
relative change = (current - baseline) / baseline

If this query does not match the chart, stop. Check the time zone, filters, aggregation, deduplication rule, comparison range, and event definition before investigating dimensions.

Rule out a data-pipeline explanation#

A product metric can move because the product changed or because measurement changed. Check:

ingestion request success and retry volume
expected event volume by producer
required-field completeness
schema-version distribution
producer release and deployment times
duplicate rate
event occurrence time versus arrival time
source-system control totals

If all events from one platform disappear, do not describe that as customer churn until the producer is verified. Use Monitor instrumentation health and Reconcile analytics with source systems.

Build a segment contribution table#

The first useful decomposition asks how much each segment added to the total change:

WITH base AS (
  SELECT
    JSONExtractString(json, 'plan') AS segment,
    if(
      timestamp >= toUnixTimestamp(toDateTime('2026-07-20 00:00:00')),
      'current',
      'baseline'
    ) AS period
  FROM product_events
  WHERE
    JSONExtractString(json, 'event') = 'checkout_completed'
    AND timestamp >= toUnixTimestamp(toDateTime('2026-07-13 00:00:00'))
    AND timestamp < toUnixTimestamp(toDateTime('2026-07-27 00:00:00'))
),
by_segment AS (
  SELECT
    if(segment = '', '(missing)', segment) AS segment,
    countIf(period = 'baseline') AS baseline_value,
    countIf(period = 'current') AS current_value
  FROM base
  GROUP BY segment
)
SELECT
  segment,
  baseline_value,
  current_value,
  current_value - baseline_value AS absolute_change,
  round(
    100 * (current_value - baseline_value) /
      nullIf(baseline_value, 0),
    2
  ) AS relative_change_pct
FROM by_segment
ORDER BY abs(absolute_change) DESC

The segments' absolute changes should sum to the top-line absolute change. That reconciliation is more important than a visually impressive percentage. A new segment with no baseline can have an undefined relative change while still contributing a clear number of events.

Separate mix shift from within-segment movement#

For rates and averages, the total can move even when every segment is stable. Suppose enterprise accounts convert at 40% and starter accounts at 10%. If the traffic mix shifts toward starter accounts, the overall rate falls without either segment getting worse.

For each segment, calculate:

baseline weight
current weight
baseline segment rate
current segment rate

Then compare two counterfactual totals:

mix effect:
  current weights × baseline segment rates

within-segment effect:
  baseline weights × current segment rates

The exact decomposition method depends on the metric. Document the method and ensure contributions reconcile to the total within an understood rounding difference. Do not add percentage-point contributions from incompatible denominators.

Use compatible denominators#

For conversion, failure, or retention rates, decompose the numerator and denominator separately:

SELECT
  period,
  plan,
  uniqExactIf(account_id, attempted = 1) AS eligible_accounts,
  uniqExactIf(account_id, completed = 1) AS completed_accounts,
  round(
    100 * completed_accounts / nullIf(eligible_accounts, 0),
    2
  ) AS conversion_rate_pct
FROM prepared_account_outcomes
GROUP BY period, plan

A falling conversion rate can mean:

completions fell
eligible attempts grew faster than completions
the population mix changed
the denominator definition changed
recent entities have not had time to convert

Always display the counts beside the rate.

Drill down as a tree, not a fishing expedition#

Choose dimensions from a causal or operational hypothesis:

total checkout change
├─ platform
│  ├─ web
│  ├─ iOS
│  └─ Android
├─ release
├─ plan
└─ region

At each level:

calculate contributions
confirm they reconcile
select the largest decision-relevant branch
inspect one deeper dimension
stop when evidence is actionable

Do not search hundreds of fields and present the largest accidental difference as a cause. High-cardinality identifiers, URLs, error messages, and customer names are usually investigation evidence, not first-line dimensions.

Connect the change to evidence#

A segment explains where the movement occurred, not automatically why. Corroborate it with:

release and feature-flag exposure
provider incidents
acquisition or pricing changes
customer lifecycle transitions
support contacts
application logs and trace identifiers
a known change in eligibility or business policy

Use Release-impact analytics for deployment context and Behavioral driver and outcome analysis for associations between earlier behavior and later outcomes.

Unless an experiment or another valid identification strategy supports it, write:

Most of the decline occurred among Android accounts on release 8.14.

not:

Release 8.14 caused the decline.

Publish an investigation record#

Keep a compact decision record beside the alert, incident, or metric review:

observed movement:
comparison windows:
definition version:
top contributing segments:
pipeline checks:
supporting evidence:
excluded explanations:
remaining uncertainty:
decision and owner:
follow-up validation date:

Save the final SQL with a description that names the time zone, windows, denominator, and maturity rule. A screenshot without its query is not a reproducible explanation.

Investigation checklist#

Current and baseline windows are complete and comparable.
The SQL reproduces the dashboard's top-line value.
Instrumentation, duplicates, and late delivery were checked first.
Segment contributions reconcile to the total change.
Rates show numerator and denominator counts.
Mix effects are separated from within-segment movement.
Dimensions were chosen from hypotheses, not unlimited field scanning.
Small or privacy-sensitive segments are suppressed appropriately.
Operational evidence supports the leading explanation.
Descriptive evidence is not presented as causal proof.
The query, definition, conclusion, and owner are recorded.