Incantory
Sign in

Webhooks

Get signed HTTP callbacks when prompts change, labels move, evals finish, makes link your prompts or someone comments.

A webhook is an HTTPS endpoint of yours that Incantory POSTs to when something happens to your prompts. Use one to redeploy when the production label moves, to post release notes to chat, or to run your own checks on every new version.

Create a webhook

Webhooks belong to your account. Create them in Settings → Webhooks, with the CLI, or through the API:

bash
incantory webhook add https://hooks.example.com/incantory --events prompt.label.moved,prompt.version.created
bash
curl -X POST https://incantory.ai/api/v1/webhooks \
  -H "Authorization: Bearer $INCANTORY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://hooks.example.com/incantory","events":["prompt.label.moved"],"prompts":["alice/haiku"]}'

The response contains the signing secret (whsec_…). It is shown once; store it where your endpoint can read it. If you lose it, rotate it (PATCH /api/v1/webhooks/{id} with {"rotateSecret": true}, or the Rotate button in settings); the old secret stops working immediately.

Rules for the URL:

  • It must use https and resolve to a public internet address. Private, loopback, link-local and cloud-metadata addresses are refused, both when you save the URL and again, after DNS resolution, on every delivery.
  • No credentials in the URL. Authenticate us with the signature instead.
  • Redirects are not followed; a 3xx answer counts as a failed delivery.

Optional settings:

  • prompts: a list of owner/slug refs. When set, only events about those prompts are delivered. When omitted, you get events for every prompt you own, plus public prompts you watch at a level that includes the event.
  • description: a note for yourself.
  • active: turn delivery off without deleting the webhook.

You can have up to 20 webhooks.

Events

EventWhen
prompt.version.createdA new version was committed (including v1 when a draft is published).
prompt.label.movedA label was created or moved to another version.
prompt.publishedA draft prompt was published.
prompt.deletedA prompt was deleted.
eval.run.completedAn eval run finished, passed or not.
eval.run.failedAn eval run errored before producing a result.
change_request.openedSomeone opened a change request against your prompt.
change_request.mergedA change request was merged into a new version.
make.linkedA published make linked a version of your prompt.
make.approvedA make's link to the prompt was approved (sent to the prompt owner and the maker).
comment.createdA visible comment was posted on your prompt, one of its versions, your make, post or change request.

Events about a public prompt also reach the webhooks of people who watch it: releases watchers get prompt.version.created and prompt.label.moved; all watchers also get make.linked, comment.created and change_request.opened. A private or draft prompt only notifies its owner.

GET /api/v1/webhooks/events returns this list with descriptions. The Send test button (and incantory webhook test) delivers a ping event, which you cannot subscribe to.

The request

Every delivery is a POST with a JSON body:

http
POST /incantory HTTP/1.1
Host: hooks.example.com
Content-Type: application/json
User-Agent: Incantory-Webhooks/1.0 (+https://incantory.ai/docs/webhooks)
Incantory-Event: prompt.label.moved
Incantory-Delivery: cm1x9s0zq0001abcd1234efgh
Incantory-Signature: t=1790000000,v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd

{"id":"evt_0f6c…","event":"prompt.label.moved","createdAt":"2026-09-28T12:00:00.000Z","data":{"label":{"name":"production","versionNumber":4,"reason":null},"prompt":{"id":"…","ref":"alice/haiku","url":"https://incantory.ai/alice/haiku"},"actor":{"id":"…"}}}

The envelope:

FieldMeaning
idThe event id. The same on a redelivery, so use it to de-duplicate.
eventThe event name, also in the Incantory-Event header.
createdAtWhen the event happened (UTC, ISO 8601).
dataEvent details. Prompt events carry data.prompt (id, ref, url); events with a person behind them carry data.actor.id.

Incantory-Delivery identifies one delivery attempt series; a redelivery gets a new one.

Reply with any 2xx status within 10 seconds. Do slow work after replying (queue it). Only the first 2 KB of your response body is kept for the delivery log.

Verify the signature

Incantory-Signature is t=<unix seconds>,v1=<signature>, where the signature is the hex HMAC-SHA256 of "<t>." + <raw request body>, keyed with your secret. Always:

  1. Verify against the raw body bytes, before parsing the JSON.
  2. Reject timestamps more than 5 minutes from your clock. This stops replayed requests.
  3. Compare in constant time.

The SDK helpers do all three.

TypeScript

ts
import express from 'express';
import { verifyWebhookSignature } from '@incantory/sdk';

const app = express();
app.post('/incantory', express.raw({ type: 'application/json' }), async (req, res) => {
  const ok = await verifyWebhookSignature(process.env.INCANTORY_WEBHOOK_SECRET!, req.header('Incantory-Signature'), req.body);
  if (!ok) return res.status(400).send('bad signature');
  const event = JSON.parse(req.body.toString('utf8'));
  res.sendStatus(204);
  // handle event.event / event.data after replying
});

verifyWebhookSignature(secret, header, body, toleranceSec = 300) returns a promise of a boolean and never throws. body may be a string, a Uint8Array (a Node Buffer is one) or an ArrayBuffer. It uses Web Crypto, so it also runs in edge runtimes and Cloudflare Workers.

Python

python
from flask import Flask, request, abort
from incantory import verify_webhook_signature

app = Flask(__name__)

@app.post("/incantory")
def incantory_hook():
    if not verify_webhook_signature(SECRET, request.headers.get("Incantory-Signature"), request.get_data()):
        abort(400)
    event = request.get_json()
    return "", 204

verify_webhook_signature(secret, header, body, tolerance_sec=300) accepts bytes or str and never raises.

Any language

bash
# Recompute the signature for a saved request body (body.json) and header values.
t=1790000000
{ printf '%s.' "$t"; cat body.json; } | openssl dgst -sha256 -hmac "$INCANTORY_WEBHOOK_SECRET" -hex

Compare the output with the v1 value. If the header carries several v1 entries, accept the request when any of them matches.

Retries and auto-disable

A delivery fails when your endpoint answers anything but 2xx, redirects, times out after 10 seconds, or cannot be reached. Failed deliveries are retried up to 8 attempts in total, with growing gaps: 1 minute, 5 minutes, 30 minutes, 2 hours, 5 hours, 8 hours and 8 hours (about 24 hours in all).

Delivery is at least once. A delivery can arrive more than once, and deliveries can arrive out of order, so de-duplicate on the envelope id and use createdAt when order matters.

After 15 failed attempts in a row across deliveries, the webhook is disabled. You get an email and a notification. Pending retries for a disabled webhook are not sent. A successful delivery resets the count. To turn it back on, fix the endpoint, press Send test, then Enable in settings (or PATCH it with {"active": true}). Enabling also clears the failure count.

Test pings are sent once, even to a disabled webhook, and never count towards disabling it.

Delivery log and redelivery

Settings lists each webhook's recent deliveries with the status, response code, response time, response excerpt and error. The log is kept for 30 days.

bash
incantory webhook deliveries <webhook-id>
incantory webhook redeliver <webhook-id> <delivery-id>

A redelivery sends the same envelope (same id) as a new delivery. The webhook must be enabled.

API reference

MethodPath
GET/api/v1/webhooksList your webhooks
POST/api/v1/webhooksCreate; returns the secret once
GET/api/v1/webhooks/eventsThe event catalogue
GET/api/v1/webhooks/{id}One webhook
PATCH/api/v1/webhooks/{id}Update, enable, rotate the secret
DELETE/api/v1/webhooks/{id}Delete with its log
POST/api/v1/webhooks/{id}/testQueue a ping
GET/api/v1/webhooks/{id}/deliveriesThe delivery log (keyset cursor)
GET/api/v1/webhooks/{id}/deliveries/{deliveryId}One delivery with its payload
POST/api/v1/webhooks/{id}/deliveries/{deliveryId}/redeliverSend it again

Reading needs a token with the read scope; changes need write. Full schemas are in the REST API reference.