Running a Reconciliation

A reconciliation job compares data across two connections. This guide shows you how to create and run one. For the full job structure — dimensions, measures, stages, and tolerances — see Configuring a reconciliation job.

Prerequisites

• At least two connections configured in your workspace.
• For each connection, the schema/table and the column mapping you want to compare.

Create a reconciliation job

A job is defined declaratively: you choose the connections to compare, the dimensions (the columns that identify a row group), the measures (the aggregated values to check), and one or more stages (each stage is the unit of pass/fail, with its own tolerances). There is no “source query / target query / key columns” model — see Configuring a reconciliation job for every field.

Console

1. Go to Jobs in the sidebar and click New Job.
2. Add the connections to compare and map their columns.
3. Choose the dimensions (grouping columns) and measures (aggregations).
4. Add one or more stages with their tolerances.
5. Click Save.

DataRecs CLI

The CLI reads the job definition from a JSON file:

datarecs jobs create --file ./daily-balance-check.json --workspace-id "$WORKSPACE_ID"

A minimal daily-balance-check.json:

{
  "name": "daily-balance-check",
  "workspace_id": "<workspace-id>",
  "dimensions": ["account_id"],
  "measures": [
    { "name": "balance", "aggregation": "SUM" }
  ],
  "connections": [
    {
      "connection_id": "<source-connection-id>",
      "source_schema": "public",
      "source_table": "accounts",
      "column_mapping": { "account_id": "id", "balance": "balance" }
    },
    {
      "connection_id": "<target-connection-id>",
      "source_schema": "public",
      "source_table": "accounts_snapshot",
      "column_mapping": { "account_id": "id", "balance": "balance" }
    }
  ],
  "stages": [
    {
      "dimensions": ["account_id"],
      "tolerances": [
        { "measure_name": "balance", "type": "ABSOLUTE", "value": 0.01 }
      ]
    }
  ]
}

REST API

curl -X POST https://api.datarecs.io/jobs \
  -H "Authorization: Bearer $DATARECS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "daily-balance-check",
    "workspace_id": "<workspace-id>",
    "dimensions": ["account_id"],
    "measures": [{ "name": "balance", "aggregation": "SUM" }],
    "connections": [
      {
        "connection_id": "<source-connection-id>",
        "source_schema": "public",
        "source_table": "accounts",
        "column_mapping": { "account_id": "id", "balance": "balance" }
      },
      {
        "connection_id": "<target-connection-id>",
        "source_schema": "public",
        "source_table": "accounts_snapshot",
        "column_mapping": { "account_id": "id", "balance": "balance" }
      }
    ],
    "stages": [
      {
        "dimensions": ["account_id"],
        "tolerances": [{ "measure_name": "balance", "type": "ABSOLUTE", "value": 0.01 }]
      }
    ]
  }'

Run the job

Triggering a run requires the job ID and the workspace ID.

Console

Open the job and click Run Now. Progress is shown in real time.

DataRecs CLI

datarecs jobs runs trigger "$JOB_ID" --workspace-id "$WORKSPACE_ID"

Optional flags: --label (a label for the run), --mode IMMEDIATE|DRY_RUN, and --param key=value (repeatable) to supply runtime parameters referenced by filter placeholders.

REST API

curl -X POST "https://api.datarecs.io/jobs/$JOB_ID/runs/trigger?workspaceId=$WORKSPACE_ID" \
  -H "Authorization: Bearer $DATARECS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "job_id": "<job-id>", "mode": "IMMEDIATE" }'

The response contains the new run_id, job_id, status, an optional tracking_url, and created_at.

Your plan limits how many runs can be in progress at once. If you’re already at the limit, the trigger returns 403 QUOTA_EXCEEDED — wait for a run to finish and retry. See Limits & Quotas.

Watch the run

• Poll GET /jobs/{jobId}/runs/{runId}?workspaceId={id} for the run’s status — one of QUEUED, RUNNING, COMPLETED, ERRORED, or CANCELLED.
• Subscribe to webhooks for the run lifecycle events to be notified the moment a run finishes.

Understanding results

After a run completes, each stage produces a result. At the row-group level the comparison yields:

Category	Meaning
Matched	The group exists across all sources and every measure is within tolerance.
Unmatched	The group is missing in one or more sources, or a measure is outside tolerance.

A run’s overall result is MATCHED only when every group joins across all sources and all tolerances pass; any unmatched group or tolerance failure makes it UNMATCHED. See Data types & comparison for the comparison semantics.