Custom sources and webhook adapters

GraphJSON has managed integrations for Segment, Stripe, and Vercel. For any other provider, place a small adapter you control between the provider and the GraphJSON logging API.

provider
  │ signed webhook
  ▼
your adapter
  ├─ verify signature
  ├─ allowlist event types
  ├─ remove unsafe fields
  ├─ map timestamp and event ID
  ├─ make delivery durable
  └─ send to GraphJSON
        ▼
dedicated collection

Do not expose the GraphJSON workspace key as a provider webhook URL unless GraphJSON supplies a managed integration designed for that provider.

Choose direct logging or an adapter#

Use the logging API directly when your own trusted server already owns the event.

Use an adapter when:

• a third-party provider signs webhooks
• the provider’s payload needs filtering or normalization
• retries can create duplicates
• the source timestamp differs from receipt time
• the provider expects a quick acknowledgement
• several source event types should map to different collections

The adapter becomes a security and reliability boundary. Keep it narrow.

Define the source contract#

Before writing code, record:

Do not begin with “store the entire payload forever.” Begin with the questions the data must answer.

Verify the signature before parsing#

Many providers sign the exact raw request body. Reading and reserializing JSON before verification can change whitespace or key ordering and invalidate the signature.

A simplified Node.js handler:

import crypto from "node:crypto";

function safeEqual(left: string, right: string) {
  const a = Buffer.from(left, "hex");
  const b = Buffer.from(right, "hex");
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

function verifyWebhook(rawBody: Buffer, signature: string) {
  const expected = crypto
    .createHmac("sha256", process.env.PROVIDER_WEBHOOK_SECRET!)
    .update(rawBody)
    .digest("hex");

  return safeEqual(expected, signature);
}

Use the provider’s official verification algorithm, timestamp tolerance, header format, and replay guidance. The example above is a pattern, not a universal provider protocol.

Reject an invalid signature before logging any payload detail.

Acknowledge after durable acceptance#

Providers often expect a response quickly. Do not wait for analysis or a large downstream workflow.

verify
  → validate
    → write normalized job to durable queue
      → return success to provider

queue worker
  → send GraphJSON batch
    → acknowledge job after 2xx

If the adapter returns success before the event is durable, a process crash can lose the event. If it waits on every downstream dependency, provider retries can amplify an outage.

Normalize the event#

A provider payload may be hundreds of fields. Send the subset needed for analysis:

{
  "event": "build_completed",
  "event_id": "provider_evt_01J...",
  "source": "deployment_provider",
  "source_event_type": "build.finished",
  "project_id": "prj_42",
  "environment": "production",
  "status": "failed",
  "duration_ms": 84231,
  "schema_version": 1
}

Recommended common fields:

Keep raw payloads in an operational system only when a reviewed workflow requires them. GraphJSON should receive an analytical representation.

Preserve occurrence time#

Use the provider’s event occurrence time as the GraphJSON timestamp.

const occurredAt = Math.floor(
  new Date(providerEvent.created_at).getTime() / 1000
);

Do not substitute:

• adapter receipt time
• queue processing time
• retry time
• import time

If both occurrence and receipt matter, use occurrence as the request timestamp and include received_at or delivery_delay_ms as controlled properties.

Reject timestamps that are invalid or outside a source-specific safety range. A malformed provider time should not silently become “now.”

Deliver to GraphJSON#

async function deliver(events: Array<{
  timestamp: number;
  body: Record<string, unknown>;
}>) {
  const response = await fetch(
    "https://api.graphjson.com/api/bulk-log",
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        api_key: process.env.GRAPHJSON_API_KEY,
        collection: "provider_events",
        timestamps: events.map((event) => event.timestamp),
        jsons: events.map((event) => JSON.stringify(event.body))
      }),
      signal: AbortSignal.timeout(10_000)
    }
  );

  const text = await response.text();

  if (!response.ok) {
    throw new Error(
      `GraphJSON ${response.status}: ${text.slice(0, 300)}`
    );
  }
}

Batch at most 50 events for one collection. Keep concurrency bounded and retry 429, network failures, and 5xx with exponential backoff and jitter.

Deduplicate provider retries#

Providers can redeliver the same event. Preserve the source event ID across every adapter retry:

WITH unique_source_events AS (
  SELECT
    JSONExtractString(json, 'event_id') AS event_id,
    any(JSONExtractString(json, 'event')) AS event,
    any(JSONExtractString(json, 'status')) AS status
  FROM provider_events
  WHERE toDateTime(timestamp) >= subtractDays(now(), 30)
  GROUP BY event_id
)
SELECT event, status, count() AS events
FROM unique_source_events
GROUP BY event, status
ORDER BY events DESC

any is safe only when the same source event ID always maps to the same immutable payload. For mutable objects, emit explicit state-change events or use argMax with a source sequence.

Handle schema changes#

Provider payloads evolve. Keep a deterministic mapping layer and version it:

{
  "event": "invoice_paid",
  "source_event_type": "invoice.paid",
  "schema_version": 2
}

When a provider field moves:

1. save representative old and new fixtures
2. update the deterministic transform
3. test required fields and forbidden fields
4. dual-read old and new shapes where necessary
5. update GraphJSON queries
6. deploy before the provider removes the old form

Do not let provider-specific nested paths become an undocumented business metric.

Quarantine invalid events#

An invalid source event belongs in a dead-letter or quarantine store with:

• provider event ID
• received time
• event type
• mapping version
• machine-readable failure reason
• redacted diagnostics

Do not log the GraphJSON key, signature secret, authorization headers, or full unsafe payload in the quarantine message.

Reconcile#

At a fixed interval, compare:

• provider events emitted
• valid adapter events accepted
• invalid or quarantined events
• GraphJSON unique event IDs
• minimum and maximum occurrence time
• critical totals by day

provider emitted
  = accepted by adapter
  + rejected as invalid

accepted by adapter
  ≈ unique events visible in GraphJSON

Timing and retries can cause temporary differences. Write the acceptable delay and count tolerance.

Disconnect safely#

1. disable the provider destination
2. wait through the provider’s retry window
3. drain the adapter queue
4. reconcile final counts
5. remove the webhook secret
6. remove the GraphJSON key from the adapter
7. archive or delete quarantined payloads according to policy
8. remove unused dashboards and alerts
9. shorten or delete the destination collection only after sign-off

Deleting the GraphJSON collection does not stop the provider from sending to your adapter.

Launch checklist#

• Provider signature verification follows official documentation.
• Verification uses the raw request body.
• Event types and properties are allowlisted.
• Provider event ID becomes a stable event_id.
• Original occurrence time becomes the GraphJSON timestamp.
• Adapter acknowledges only after durable acceptance.
• GraphJSON delivery uses bounded batches and retries.
• Invalid events are quarantined safely.
• Reconciliation has an owner and tolerance.
• Disconnect and secret-rotation steps are documented.

Continue with Reliable event delivery, Deduplicate events, and Monitor instrumentation health.