Using Webhooks

Webhooks let DataRecs push events to your systems in real time. The model has two parts:

• Endpoints — an HTTPS URL where deliveries are sent. Each endpoint has its own signing secret for payload verification.
• Subscriptions — bind one or more event types to an endpoint. You can have multiple subscriptions per endpoint, each filtering for different events.

This separation means you can point several subscriptions (e.g. reconciliation failures, connection tests, API key changes) at the same endpoint URL, or spread them across dedicated endpoints.

How deliveries work

• Every delivery is a CloudEvent JSON document matching the schema used on our internal event bus (see the event catalog). Need a different shape — Slack, Teams, or a bespoke body and headers? See Customising Webhook Payloads.
• Deliveries are signed with the Standard Webhooks scheme, so you can verify them with an off-the-shelf library. Headers include:
  • Content-Type: application/cloudevents+json
  • webhook-id: a unique delivery id that is stable across retries — use it as your idempotency key.
  • webhook-timestamp: unix-seconds send time (for replay protection).
  • webhook-signature: one or more space-delimited v1,<base64-signature> entries.
• Delivery is at-least-once: you may receive the same webhook-id more than once (retries, redelivery). Deduplicate on webhook-id and make your handler idempotent. Events are not ordered — don’t assume reconciliation.run.queued arrives before reconciliation.run.completed.
• The webhook service respects the per-endpoint rate limit (requests per second) and retries failed deliveries (429/5xx/timeout) with exponential backoff: 30s → 2m → 10m → 1h → 4h → 12h → 24h — 8 attempts in total (the first delivery plus 7 retries, ~41 hours before dead-letter). Return any 2xx quickly to acknowledge.

Step 1 — Create an endpoint

An endpoint is the HTTPS URL that will receive deliveries.

Console

1. Go to Automation → Webhooks and click New Endpoint.
2. Enter the URL (must start with https://).
3. Add an optional Description (e.g. “Slack #data-alerts”).
4. Set the Rate Limit (default 50 req/s).
5. Click Create Endpoint.
6. A signing secret is generated and copied to your clipboard. Store it securely — you will not see it again.

REST API

curl -X POST https://api.datarecs.io/webhook-endpoints \
  -H "Authorization: Bearer $DATARECS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://hooks.example.com/datarecs",
    "description": "Slack failure alerts",
    "rate_limit_rps": 10
  }'

The response includes the endpoint object and a one-time signing_secret (a whsec_… value). Store it now — it is never returned again.

CLI

datarecs webhooks endpoints create \
  --url https://hooks.example.com/datarecs \
  --description "Slack failure alerts" \
  --rate-limit 10

Terraform

resource "datarecs_webhook_endpoint" "slack" {
  url            = "https://hooks.example.com/datarecs"
  description    = "Slack failure alerts"
  rate_limit_rps = 10
}

Step 2 — Add subscriptions

A subscription tells DataRecs which event types to deliver to an endpoint.

Console

1. Open the endpoint you just created (click its row in the Webhooks list).
2. In the Subscriptions section, click Add Subscription.
3. Select one or more Event Types from the catalog (or choose * for all events).
4. Optionally add Filters as a JSON object (e.g. {"workspace_id": "ws-aurora"}).
5. Click Create Subscription.

REST API

curl -X POST https://api.datarecs.io/webhook-subscriptions \
  -H "Authorization: Bearer $DATARECS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint_id": "ep_abc123",
    "event_types": [
      "reconciliation.run.completed",
      "reconciliation.run.errored"
    ],
    "filters": { "workspace_id": "ws-aurora" }
  }'

CLI

datarecs webhooks subscriptions create \
  --endpoint-id ep_abc123 \
  --event-types "reconciliation.run.completed,reconciliation.run.errored"

Terraform

resource "datarecs_webhook_subscription" "run_outcomes" {
  endpoint_id = datarecs_webhook_endpoint.slack.id
  event_types = [
    "reconciliation.run.completed",
    "reconciliation.run.errored"
  ]
  filters = {
    workspace_id = datarecs_workspace.ops.id
  }
}

Available event types

Use GET /webhook-event-types (or the Console’s event type picker) to see the full catalog grouped by domain. Common examples:

• Reconciliation: reconciliation.run.triggered, reconciliation.run.queued, reconciliation.run.completed, reconciliation.run.errored, reconciliation.run.cancelled, reconciliation.run.stage.completed
• Connections: connection.connection.created, connection.connection.tested, connection.connection.deleted
• Platform: platform.api_key.created, platform.api_key.updated, platform.api_key.deleted
• Webhooks: webhook.endpoint.created, webhook.delivery.failed

Use ["*"] as the event types array to subscribe to all events.

Verifying deliveries

Every delivery is signed with the Standard Webhooks scheme. Verify the signature on every request before trusting the body — an unverified webhook endpoint is an open, unauthenticated write into your system.

You have two options:

• Use an official Standard Webhooks library (available for Node, Python, Go, Rust, PHP, and more). Point it at the whsec_… signing secret and the request headers and it does everything below — HMAC, the multi-signature v1,… parsing, the timestamp/replay check, and the constant-time compare — for you. This is the recommended path.
• Verify manually if you’d rather not add a dependency. The HMAC is small, but you are responsible for the constant-time compare and the replay-window check.

The verification algorithm

Whichever route you take, this is what is being checked:

1. Read the raw request body — the exact bytes received. Do not parse and re-serialize the JSON first; any whitespace or key-order change breaks the HMAC.
2. Build the signed content by joining three values with dots: `${webhook-id}.${webhook-timestamp}.${rawBody}`.
3. Derive the key: take the signing secret, strip the whsec_ prefix, and base64-decode the remainder to raw bytes.
4. Compute base64( HMAC_SHA256(key, signedContent) ).
5. The webhook-signature header is a space-delimited list of v1,<base64sig> entries — there are two during a secret rotation (current + previous). Accept the request if any entry’s signature matches yours, using a constant-time comparison.
6. Reject the request if webhook-timestamp (unix seconds) is more than 5 minutes from now, in either direction — this is replay protection.

Verify against the raw body

Most signature-mismatch bugs come from verifying against a re-serialized body. Capture the raw bytes in your framework (e.g. express.raw(), Flask’s request.get_data(), Go’s io.ReadAll(r.Body)) and verify those — then parse the JSON only after verification passes.

Verify with the official library

Node / TypeScript

// npm install standardwebhooks
import { Webhook } from "standardwebhooks";

const wh = new Webhook(process.env.DATARECS_WEBHOOK_SECRET!); // the whsec_… value

// rawBody MUST be the exact string/bytes received (not re-serialized JSON).
function verify(rawBody: string, headers: Record<string, string>) {
  // Throws if the signature is invalid or the timestamp is outside the 5-min window.
  // Returns the parsed payload on success.
  return wh.verify(rawBody, {
    "webhook-id": headers["webhook-id"],
    "webhook-timestamp": headers["webhook-timestamp"],
    "webhook-signature": headers["webhook-signature"],
  });
}

Python

# pip install standardwebhooks
from standardwebhooks import Webhook

wh = Webhook(os.environ["DATARECS_WEBHOOK_SECRET"])  # the whsec_… value

# raw_body MUST be the exact bytes received (not re-serialized JSON).
def verify(raw_body: bytes, headers: dict) -> dict:
    # Raises WebhookVerificationError on bad signature or stale timestamp.
    return wh.verify(raw_body, {
        "webhook-id": headers["webhook-id"],
        "webhook-timestamp": headers["webhook-timestamp"],
        "webhook-signature": headers["webhook-signature"],
    })

Go

// go get github.com/standard-webhooks/standard-webhooks/libraries/go
import standardwebhooks "github.com/standard-webhooks/standard-webhooks/libraries/go"

wh, err := standardwebhooks.NewWebhook(os.Getenv("DATARECS_WEBHOOK_SECRET")) // whsec_…
if err != nil {
    log.Fatal(err)
}

// rawBody MUST be the exact bytes received (not re-serialized JSON).
func verify(rawBody []byte, headers http.Header) error {
    // Returns an error on bad signature or a timestamp outside the 5-min window.
    return wh.Verify(rawBody, headers)
}

Verify manually (no dependency)

These implement the algorithm above directly. Each reads the raw body, checks the timestamp, and accepts the request if any v1,… signature matches (constant-time) — so they keep working through a secret rotation.

Node / TypeScript

import { createHmac, timingSafeEqual } from "node:crypto";

export function verify(rawBody: string, headers: Record<string, string>, secret: string): boolean {
  const id = headers["webhook-id"];
  const ts = headers["webhook-timestamp"];
  const sigHeader = headers["webhook-signature"] ?? "";

  // 5-minute replay window.
  if (Math.abs(Date.now() / 1000 - Number(ts)) > 300) return false;

  // Strip the whsec_ prefix and base64-decode the remainder to raw key bytes.
  const key = Buffer.from(secret.replace(/^whsec_/, ""), "base64");
  const expected = createHmac("sha256", key).update(`${id}.${ts}.${rawBody}`).digest("base64");
  const expectedBuf = Buffer.from(expected);

  // The header may carry several space-delimited "v1,<sig>" entries (rotation). Accept any match.
  return sigHeader.split(" ").some((part) => {
    const sig = part.split(",")[1] ?? "";
    const sigBuf = Buffer.from(sig);
    return sigBuf.length === expectedBuf.length && timingSafeEqual(sigBuf, expectedBuf);
  });
}

Python

import base64, hashlib, hmac, time

def verify(raw_body: bytes, headers: dict, secret: str) -> bool:
    msg_id = headers["webhook-id"]
    ts = headers["webhook-timestamp"]
    sig_header = headers.get("webhook-signature", "")

    # 5-minute replay window.
    if abs(time.time() - int(ts)) > 300:
        return False

    # Strip the whsec_ prefix and base64-decode the remainder to raw key bytes.
    key = base64.b64decode(secret.removeprefix("whsec_"))
    signed_content = f"{msg_id}.{ts}.{raw_body.decode()}".encode()
    expected = base64.b64encode(hmac.new(key, signed_content, hashlib.sha256).digest())

    # The header may carry several space-delimited "v1,<sig>" entries (rotation). Accept any match.
    for part in sig_header.split(" "):
        _, _, sig = part.partition(",")
        if hmac.compare_digest(sig.encode(), expected):
            return True
    return False

Go

import (
    "crypto/hmac"
    "crypto/sha256"
    "encoding/base64"
    "strconv"
    "strings"
    "time"
)

func verify(rawBody []byte, headers map[string]string, secret string) bool {
    id := headers["webhook-id"]
    ts := headers["webhook-timestamp"]

    // 5-minute replay window.
    sent, err := strconv.ParseInt(ts, 10, 64)
    if err != nil || abs(time.Now().Unix()-sent) > 300 {
        return false
    }

    // Strip the whsec_ prefix and base64-decode the remainder to raw key bytes.
    key, err := base64.StdEncoding.DecodeString(strings.TrimPrefix(secret, "whsec_"))
    if err != nil {
        return false
    }
    mac := hmac.New(sha256.New, key)
    mac.Write([]byte(id + "." + ts + "." + string(rawBody)))
    expected := []byte(base64.StdEncoding.EncodeToString(mac.Sum(nil)))

    // The header may carry several space-delimited "v1,<sig>" entries (rotation). Accept any match.
    for _, part := range strings.Split(headers["webhook-signature"], " ") {
        if _, sig, ok := strings.Cut(part, ","); ok {
            if hmac.Equal([]byte(sig), expected) {
                return true
            }
        }
    }
    return false
}

func abs(n int64) int64 {
    if n < 0 {
        return -n
    }
    return n
}

Building a reliable consumer

Signature verification proves a delivery is authentic. These three properties govern how deliveries arrive — design your handler around them or you will drop or double-process events.

Delivery is at-least-once — be idempotent

DataRecs guarantees at-least-once delivery, never exactly-once. The same event can be POSTed to your endpoint more than once — this is by design and unavoidable. A crash on our side between your endpoint returning 2xx and us recording that success will redeliver: you already processed it, but we never saw the acknowledgement, so we try again. Network retries and internal recovery sweeps can also redeliver.

Your handler must be idempotent — processing the same event twice must be safe and have the same effect as processing it once.

Deduplicate on webhook-id

The webhook-id header is the stable delivery id: it is identical across every retry and every redelivery of the same event. (The separate webhook-attempt-id header changes on every attempt and is for support/debugging only — never dedupe on it.)

1. On receipt, verify the signature, then read webhook-id.
2. Check whether you have already processed that id. If so, return 2xx immediately and do nothing else.
3. Otherwise process the event, then record the id as processed.
4. Keep the processed-id record for longer than the full retry window (~41 hours) — a redelivery can arrive at the very tail of that window. A store with a TTL of 48 hours is a safe choice.

// Sketch — dedupe before doing any work.
async function handle(rawBody: string, headers: Record<string, string>) {
  const payload = wh.verify(rawBody, headers);      // verify first (throws on failure)
  const deliveryId = headers["webhook-id"];

  if (await seen.has(deliveryId)) return;            // already handled — 2xx and stop
  await process(payload);                            // your business logic
  await seen.add(deliveryId, { ttlSeconds: 48 * 3600 });
}

Ordering is NOT guaranteed

Deliveries are not ordered. Two events for the same endpoint can arrive in any order, because a failed earlier event is retried later (and can land after a successful later event), and recovery sweeps re-enqueue by retry schedule, not emission order, while multiple workers deliver in parallel.

Never infer order from arrival. If sequence matters to your application, order by a field in the event payload — the CloudEvent time (the event timestamp), or a domain field like run_id / a monotonic sequence — or by the webhook-timestamp header. For example, when you receive a reconciliation.run.completed, don’t assume you’ve already seen that run’s reconciliation.run.queued; reconcile state using the payload, not order of receipt.

Acknowledge fast, work async

Return a 2xx as soon as you have verified and durably accepted the delivery (e.g. enqueued it) — ideally within a couple of seconds. Anything non-2xx, or a timeout, is treated as a failure and retried on the backoff schedule (30s → 2m → 10m → 1h → 4h → 12h → 24h — 8 attempts in total, the first delivery plus 7 retries), after which the delivery is dead-lettered. Do your heavy processing after acknowledging, off the request path.

Rotating the signing secret

You can rotate an endpoint’s signing secret at any time. Rotation is zero-downtime: for a 24-hour grace period the old secret stays valid and DataRecs signs each delivery with both secrets (the webhook-signature header carries two v1,<sig> entries). Update your receiver to the new secret any time within that window. After 24 hours the old secret stops being sent.

Console

Open the endpoint detail page and click Rotate Secret. The new secret is copied to your clipboard.

REST API

curl -X POST https://api.datarecs.io/webhook-endpoints/ep_abc123/rotate-secret \
  -H "Authorization: Bearer $DATARECS_API_KEY"

CLI

datarecs webhooks endpoints rotate-secret ep_abc123

If your receiver accepts any of the v1,<sig> entries (as the verification snippet above does), rotation needs no coordinated deploy — you have the full grace window to cut over.

Send a test event

From an endpoint’s detail page in the Console, click Send test event to emit a webhook.test delivery to your URL and confirm your signature verification works end-to-end before relying on live events.

Managing endpoints and subscriptions

Disable without deleting

Both endpoints and subscriptions have an enabled flag. Disabling an endpoint pauses all deliveries to that URL. Disabling a subscription stops deliveries for those specific event types while keeping other subscriptions on the same endpoint active.

Editing

• Endpoints: you can update the URL, description, and rate limit.
• Subscriptions: you can change the event types, filters, and enabled state.

Deleting

Deleting an endpoint removes the endpoint and all of its subscriptions. Deleting a subscription only removes that subscription — the endpoint and other subscriptions remain.

Event schemas & examples

Sample payload: reconciliation run errored

{
  "specversion": "1.0",
  "id": "a3f1c9e0-7b2d-4e5a-9c8b-1f2e3d4a5b6c",
  "type": "reconciliation.run.errored",
  "source": "/datarecs/reconciliation-worker",
  "subject": "tenants/e6d82e9a36e2724f/jobs/job_a1b2c3/runs/run_7f3a9c2e",
  "time": "2026-06-09T14:32:10.000Z",
  "datacontenttype": "application/json",
  "tenantid": "e6d82e9a36e2724f",
  "data": {
    "run_id": "run_7f3a9c2e",
    "job_id": "job_a1b2c3",
    "tenant_id": "e6d82e9a36e2724f",
    "workflow_name": "recon-run-7f3a9c2e",
    "error": {
      "code": "QUERY_FAILED",
      "message": "Extractor SQL error on source_1"
    }
  }
}

Sample payload: stage completed

{
  "specversion": "1.0",
  "id": "b7c2d4e1-3a9f-4c6b-8d2e-5f1a2b3c4d5e",
  "type": "reconciliation.run.stage.completed",
  "source": "/datarecs/reconciliation-worker",
  "subject": "tenants/e6d82e9a36e2724f/jobs/job_a1b2c3/runs/run_7f3a9c2e",
  "time": "2026-06-09T14:32:10.000Z",
  "datacontenttype": "application/json",
  "tenantid": "e6d82e9a36e2724f",
  "data": {
    "run_id": "run_7f3a9c2e",
    "job_id": "job_a1b2c3",
    "tenant_id": "e6d82e9a36e2724f",
    "stage_name": "compare-ledgers",
    "result": "MATCHED",
    "tolerances": [
      {
        "measure_name": "net_amount",
        "tolerance_type": "ABSOLUTE",
        "tolerance_value": 0.01,
        "within_tolerance_count": 998,
        "outside_tolerance_count": 2,
        "passed": true
      }
    ]
  }
}

Security considerations

Note: Webhooks are only as secure as the endpoint receiving them. Treat the signing secret as a credential.

• Your endpoint must be a public https:// URL. DataRecs verifies your TLS certificate and refuses to deliver to non-HTTPS URLs or to private/internal/metadata addresses.
• Verify the webhook-signature on every request, and reject deliveries whose webhook-timestamp is more than 5 minutes old to prevent replay.
• Deduplicate on webhook-id (stable across retries) so a redelivered event isn’t processed twice.
• Rotate the signing secret periodically via Console, CLI, or API (24-hour dual-signature grace — see above).
• Store secrets in your own secret manager; never log them.

Troubleshooting

Symptom	Resolution
Repeated delivery failures	Check your endpoint logs. Ensure it returns 2xx within the timeout window.
Signature mismatch	Sign the exact string `${webhook-id}.${webhook-timestamp}.${rawBody}` over the raw body (no JSON reformatting), base64-encode, and compare against each space-delimited v1,<sig> in webhook-signature. During rotation, two signatures are sent — accept either. Confirm you base64-decode the secret after stripping whsec_.
Endpoint disabled unexpectedly	Admins can disable via Console/CLI. Check audit logs.
Not receiving expected events	Verify the subscription’s event types and filters. A filter like workspace_id narrows which events match.
Deleted endpoint still receiving events	Delivery may have been enqueued before deletion. New events will not be delivered.