Behavioral driver and outcome analysis

Driver analysis asks which earlier behaviors are associated with a later outcome. Examples include whether connecting a second data source is associated with retention or whether inviting a teammate is associated with paid conversion.

The goal is to generate and test product hypotheses. It is not to turn correlation into a causal claim.

GraphJSON does not provide a native correlation scanner, causal-inference engine, matching service, or predictive model. Use governed SQL for bounded, predefined comparisons; use a statistical environment for more advanced estimation.

Write the question as a temporal contract#

Population: eligible new business accounts
Index time: account creation
Behavior window: days 0–14
Candidate behavior: connected at least 2 production data sources
Outcome window: active in days 45–74
Outcome: completed at least 1 decision-ready report
Exclusions: internal, test, migrated, or incomplete-data accounts
Maturity: at least 74 days observed plus allowed lateness
Definition version: 2

The behavior must occur before the outcome window. Otherwise the analysis can mistake consequences of retention for causes of retention.

Choose one entity#

Use account-level behavior for account retention and user-level behavior for user outcomes. Aggregate member events before assigning an account-level exposure:

account exposed
  = at least one qualifying member behavior

or:

account exposed
  = at least two distinct members completed complementary actions

Document the rule. Do not count member events as independent accounts.

Define behavior before viewing outcomes#

Candidate behaviors should be stable, interpretable, and actionable:

• completed a meaningful setup step
• adopted a specific workflow
• reached a frequency threshold
• collaborated with another role
• used a capability successfully

Avoid high-cardinality interface details, accidental telemetry, and behaviors selected only because they produced the strongest retrospective result.

Version thresholds such as “three uses” or “two teammates.” Trying twenty thresholds is twenty comparisons, not one.

Create a mature cohort#

If the behavior window ends on day 14 and the outcome window ends on day 74, include only entities whose day 74 is complete plus allowed event lateness.

Do not label newer accounts as non-retained. Keep:

• eligible and mature
• eligible but immature
• excluded
• missing required identity or source state

The mature population should be fixed before comparing exposed and unexposed entities.

Build one feature row per entity#

For a bounded analysis, create an entity feature table in SQL:

WITH accounts AS (
  SELECT
    JSONExtractString(json, 'account_id') AS account_id,
    min(toDateTime(timestamp)) AS created_at
  FROM product_events
  WHERE JSONExtractString(json, 'event') = 'account_created'
  GROUP BY account_id
),
behavior AS (
  SELECT
    JSONExtractString(json, 'account_id') AS account_id,
    uniqExactIf(
      JSONExtractString(json, 'source_id'),
      JSONExtractString(json, 'event') = 'data_source_connected'
      AND JSONExtractString(json, 'result') = 'success'
    ) AS connected_sources
  FROM product_events
  INNER JOIN accounts USING account_id
  WHERE
    toDateTime(timestamp) >= accounts.created_at
    AND toDateTime(timestamp) < accounts.created_at + INTERVAL 14 DAY
  GROUP BY account_id
),
outcome AS (
  SELECT
    JSONExtractString(json, 'account_id') AS account_id,
    countIf(
      JSONExtractString(json, 'event') = 'report_completed'
      AND JSONExtractString(json, 'result') = 'success'
    ) > 0 AS retained_value
  FROM product_events
  INNER JOIN accounts USING account_id
  WHERE
    toDateTime(timestamp) >= accounts.created_at + INTERVAL 45 DAY
    AND toDateTime(timestamp) < accounts.created_at + INTERVAL 75 DAY
  GROUP BY account_id
)
SELECT
  accounts.account_id,
  coalesce(behavior.connected_sources, 0) >= 2 AS exposed,
  coalesce(outcome.retained_value, false) AS retained_value
FROM accounts
LEFT JOIN behavior USING account_id
LEFT JOIN outcome USING account_id
WHERE accounts.created_at < now() - INTERVAL 75 DAY

Each row is one eligible mature account.

Compare outcomes with full denominators#

WITH feature_rows AS (
  /* Use the one-row-per-account query above. */
  SELECT
    JSONExtractString(json, 'account_id') AS account_id,
    JSONExtractBool(json, 'exposed') AS exposed,
    JSONExtractBool(json, 'retained_value') AS retained_value
  FROM analysis_features
  WHERE JSONExtractString(json, 'event') = 'driver_analysis_feature'
)
SELECT
  exposed,
  count() AS accounts,
  countIf(retained_value) AS retained_accounts,
  retained_accounts / nullIf(accounts, 0) AS retention_rate
FROM feature_rows
GROUP BY exposed

Show group sizes, base rates, absolute difference, and relative difference. Relative lift can look dramatic when the base rate is tiny.

The example assumes a derived feature collection for clarity. GraphJSON does not create or schedule it; publish it with a governed external worker when the feature table is reused.

Control obvious confounding#

Exposed and unexposed accounts can differ before the behavior:

• company or team size
• acquisition channel
• plan or sales motion
• geography
• starting intent
• prior experience
• account creation period
• platform and data availability

Stratify or match on pre-behavior variables. Compare exposed and unexposed accounts within similar plan, size, age, and acquisition groups before aggregating.

Do not control for a variable caused by the candidate behavior. That can remove part of the effect under investigation.

Inspect dose and timing#

If the behavior is plausibly meaningful, check:

• zero, one, two, and several completions
• early versus late occurrence within the behavior window
• successful versus failed attempts
• breadth across distinct members or objects

A monotonic dose-response pattern can strengthen a hypothesis, but heavy users may simply have higher intent. Do not convert the pattern into causal proof.

Guard against leakage#

Common leakage:

• using current plan to explain an earlier conversion
• including activity that occurred after the outcome
• defining “adopted” using a field available only after renewal
• selecting accounts based on a future support or billing event
• using a score trained on the same outcomes being reported

Freeze state as of the index or behavior window. Review every feature’s availability time.

Limit multiple comparisons#

Scanning hundreds of events, thresholds, segments, and outcomes will produce apparently strong associations by chance.

Use:

1. a short predeclared candidate set
2. one primary outcome and maturity rule
3. a discovery cohort
4. a separate confirmation cohort or later time period
5. correction or false-discovery controls when many tests remain

GraphJSON does not calculate significance or correct comparisons automatically.

Use negative and falsification checks#

Test patterns that should not occur:

• behavior after the outcome should not “predict” the earlier outcome
• a random stable bucket should not show a large lift
• unrelated diagnostic events should not dominate
• the result should not disappear entirely in every major stratum

These checks cannot prove causality, but they catch leakage and fragile analysis.

Promote the finding to an experiment#

A strong association should produce a specific intervention:

If we help eligible accounts connect a second valid source by day 14,
then mature retained report value will increase,
without worsening setup failure or support burden.

Randomize the intervention, preserve assignment and exposure, and measure the same governed outcome. Follow A/B testing.

Do not experiment by withholding a safety, accessibility, or contractual requirement.

Communicate the result#

Report:

• question and definition version
• population and maturity
• behavior and outcome windows
• exposed and unexposed counts
• absolute and relative differences
• pre-existing group differences
• stratified results
• missing-data and sampling limits
• confirmation status
• causal status: exploratory, observational, or experimental

Avoid titles such as “Feature X drives retention” for observational work. Use “Accounts adopting X were associated with higher mature retention.”

Review checklist#

• Behavior occurs before the outcome window.
• Entity, eligibility, and maturity are explicit.
• One row represents one independent entity.
• Candidate behaviors and thresholds were declared before scanning.
• Denominators and absolute differences are visible.
• Pre-behavior confounders are compared or controlled.
• Event-time state prevents future leakage.
• Multiple comparisons have a discovery and confirmation plan.
• Negative checks challenge the analysis.
• Observational results are not labeled causal.
• Actionable findings progress to a controlled test where appropriate.