Webhooks

Troubleshooting & Best Practices

Build a webhook receiver that never loses or double-counts an event: acknowledge fast, verify first, deduplicate by event id, cope with ordering, keep secrets safe, and fix the common failures: signature mismatches, clock skew, redirects, timeouts and refused URLs.

Best practices

Acknowledge fast, process later

DevStride waits at most 10 seconds for an answer, and sends deliveries to one endpoint one at a time, so the time your receiver takes to answer is the speed of its queue. Do the minimum in the request:

  1. verify the signature;
  2. store the event (a job queue, a database row, a message on SQS, Pub/Sub or Kafka);
  3. answer 2xx with an empty body.

Then do the real work (calling other APIs, updating records, posting to chat) in a separate worker. Answering before the event is stored safely risks losing it if your process crashes in between; answering after storing it is the safe order.

Verify before you trust anything

Check X-DevStride-Signature on every request, against the raw body, before you parse it or act on it. Reject failures with 401. A request that fails verification is either not from DevStride or was altered on the way; nothing in it should reach your systems. The Verifying Signatures page has tested code for six languages.

Deduplicate by event id

Delivery is at least once, so the same event can arrive twice. Record every event id you have processed and skip repeats:

CREATE TABLE devstride_webhook_events (
  event_id    uuid        PRIMARY KEY,
  received_at timestamptz NOT NULL DEFAULT now()
);

-- In the same transaction as the work the event causes:
INSERT INTO devstride_webhook_events (event_id) VALUES ($1)
ON CONFLICT (event_id) DO NOTHING;
-- 0 rows inserted: you have already handled this event; skip it.

Keep ids for at least 30 days, the period over which a delivery can be replayed, then delete old rows. The id is the same on every retry and replay, and the same for every endpoint that receives the event.

Answer 2xx to everything you verified

Any status other than 2xx is retried for about a day and, while it is, holds up every newer delivery to that endpoint. So:

  • answer 2xx to event types you do not handle, including ping and types added in future;
  • answer 2xx to an event your code cannot process (a bug, an unexpected value), store it, and alert yourself, rather than letting it block the queue;
  • reserve error statuses for "try again later" situations you expect to clear, such as your database being briefly unavailable.

Be ready for any order

Within one endpoint deliveries arrive in the order DevStride queued them, but across endpoints there is no order at all, and two changes made almost at once can be queued the other way round. When order matters:

  • compare occurredAt and ignore an event older than the state you already hold;
  • for status changes, check that previousStatusId matches the status you have on record;
  • when in doubt, treat the event as a signal and read the current state from the API by the ids it carries.

Ignore what you do not recognize

New fields and event types are added without notice. Read the fields you need, ignore unknown keys, and never fail on an unknown type. See Compatibility.

Filter internal notes and drop your own echoes

comment.created includes internal notes: check visibility is public before showing a comment to anyone outside your team. If your integration also writes to DevStride (for example syncing comments back), run it as a dedicated member and skip events whose origin.actorId is that member's username, so you do not process your own writes as new ones.

Protect the secret

Keep each endpoint's secret in a secret manager or environment variable, never in source code or logs. Use a different endpoint, and so a different secret, for each system and environment (production and staging should never share one). Rotate on a schedule and whenever someone who had access leaves; if a secret may have leaked, rotate twice to retire it immediately (why).

Keep responses small

DevStride reads at most 64 KB of your response; a larger one counts as a failure even with a 2xx status. An empty 204 No Content is ideal.

Watch the health signals

The failing and recovered emails go to everyone with Manage Webhooks: make sure that includes someone who will act on them. The red banner and the delivery log on the Webhooks page show what is failing and why. See Failing and recovered alerts.

Network and firewall notes

  • Allow-list by signature, not by IP address. Deliveries do not come from a fixed, published set of IP addresses, so source-IP filtering is not a reliable way to accept DevStride and refuse everyone else. Authenticate each request by its signature.
  • Your endpoint must be reachable from the public internet on HTTPS, with a certificate a standard client trusts (not expired, not self-signed, matching the host name).
  • DevStride will not call private addresses. URLs whose host resolves to a private network, loopback, link-local or cloud-metadata address are refused when you save the endpoint and on every attempt. To deliver into a private network, expose a small public relay that verifies the signature and forwards inward.
  • User-Agent: DevStride-Webhooks/1 identifies the requests in your logs and WAF rules. Anyone can send that header, so never treat it as proof.

Common problems

SymptomLikely causeFix
Every delivery gets 401 from your receiverThe body was parsed and re-serialized before verifying, or the wrong secret is configured.Verify the raw bytes (pitfalls by framework); check you copied the whole secret, whsec_ prefix included, for this endpoint. Run your code against the test vector.
Signatures fail only on some eventsCharacters outside plain ASCII (accented names, emoji) are being re-encoded, or a size limit truncates large comment events.Hash the bytes as received, not a decoded and re-encoded string; allow request bodies of at least 1 MB.
Signatures fail on every event after a while, or intermittentlyYour server's clock has drifted more than 5 minutes.Synchronize the clock with NTP.
Signatures start failing about a day after a rotationThe receiver still has the previous secret, which stopped being sent 24 hours after the rotation.Deploy the current secret, or rotate again and deploy that one.
Deliveries fail with 413Your server or proxy limits request size (Express defaults to 100 KB).Raise the limit to at least 1 MB on the webhook route.
Last response 301, 302 or 303The URL redirects (for example a trailing slash, or www to the apex), and DevStride does not follow redirects that would turn the POST into a GET.Register the final URL.
No response: a timeout or Fetch deadline exceededThe receiver took longer than 10 seconds to answer, or the whole request took more than 15 seconds.Answer first and process in the background.
No response: Blocked url scheme 'http:'A redirect sent the request to plain http://.Serve the endpoint, and any redirect target, over HTTPS only.
No response: Blocked request to non-public addressThe host now resolves to a private or internal address.Point the name at a public address, or use a public relay.
No response: Host could not be resolvedThe DNS name does not exist (or no longer does).Fix the DNS record or the endpoint URL.
No response: a certificate errorThe certificate is expired, self-signed or for another name.Install a certificate from a public certificate authority.
Creating or saving an endpoint is refusedThe URL is not https://, does not resolve, resolves to a refused address, or the organization already has 20 endpoints.Read the message under Endpoint URL; fix the URL or delete an unused endpoint.
A 2xx response still counts as failedThe response body is over 64 KB.Answer with an empty or short body.
Events stopped arriving and nothing shows as failingThe endpoint is disabled (changes are not queued while it is), or it is no longer subscribed to that event type.Check Enabled and Events in its row. Changes made while it was disabled are not sent; catch up through the API.
New events arrive hours lateAn older delivery to the same endpoint is failing and waiting for retries, which holds back the ones behind it.Find the oldest pending row in the delivery log, fix whatever makes it fail, and the queue drains in order.
The same event processed twiceA retry or replay after your receiver had already handled it.Deduplicate by event id.

Privacy: erasure and redaction

Webhook deliveries are copies of your data, so DevStride's privacy tools reach them too:

  • Erasing a requester (Anonymize…) deletes every stored item.created and comment.created delivery that carries that requester's tickets or words, including those of requesters merged into them, from the delivery log. Deliveries still waiting to be sent are deleted too, so they are never sent.
  • Redacting a comment deletes that comment's stored comment.created deliveries the same way.

Once deleted, those deliveries can no longer be viewed or replayed. Status and SLA events carry only ids, and are not affected.