Webhooks: 8 Checks for Reliable Business Event Notifications

Webhooks send a message from one app to another when an event occurs. For example, a payment service can notify your CRM when a payment succeeds. Your app can then update the invoice or start the next workflow without repeatedly asking whether something changed.

Yet receiving a message does not guarantee that the business action happened correctly. A reliable webhook integration must handle duplicate messages, delays, failures, and security checks. This article explains how webhooks work and eight checks to use before putting them into daily operation.

How webhooks work

The receiving app provides a URL, often called an endpoint. The sending service calls that URL when a selected event occurs. Its message usually contains an event type, an identifier, and data about the event.

Consider an online order. The payment service sends a payment event to your endpoint. Your system verifies the message, records it, and updates the correct order. Then it can notify staff or schedule fulfillment according to your rules.

The sender and receiver have separate duties. The sender documents delivery behavior and message formats. Meanwhile, the receiver must process the event safely and record what happened. Neither side should assume that every network request will succeed at once.

Webhooks versus polling

Polling means asking a service for updates at intervals. A webhook sends an update when an event happens. Both methods can be useful, and a business may use them together.

For example, webhooks can support prompt updates, while a periodic comparison can detect missed changes. In contrast, polling alone may introduce delay or unnecessary requests when events are rare. The right choice depends on the service, volume, and acceptable delay.

Webhooks also do not replace every API call. A message may find a changed object without including all the data you need. Your app may then call the provider’s API to retrieve the current record.

Oxen’s guide to backend development services explains where this processing logic fits. The webhook endpoint is an entry point, while the backend decides how the event affects business records.

8 checks for a reliable webhook integration

1. Subscribe only to events you need

Choose the event types that support a defined workflow. A payment integration may need payment success, refund, and dispute events. It probably does not need every event the provider offers.

Next, map each event to an action. State which record it affects and what conditions must hold. For example, a payment event should not mark an unrelated invoice as paid simply because the email address matches.

Record the provider’s event schema and version. Also plan what the endpoint should do with an unsupported event type. Ignoring an irrelevant event can be valid, but the team should distinguish that decision from a processing failure.

2. Verify that the message came from the sender

A public endpoint can receive requests from anyone. So, check the provider’s documented signature or verification method before trusting the message. Use the official library where it fits the platform and deployment.

GitHub’s webhook best practices recommends using a webhook secret to validate deliveries. Providers differ in their methods, so implement the exact process for the service you use rather than copying assumptions from another integration.

Signature checks may depend on the original request body. If middleware changes that body before verification, a valid message may fail the check. Include this path in testing and keep secrets out of source code and logs.

Also follow the provider’s guidance on timestamps and replay protection. A valid signature alone may not address every replay case. The receiver needs both message verification and safe business processing.

3. Match the event to the right business record

Use stable identifiers to connect the event to your customer, order, invoice, or subscription. Store the provider’s object IDs when creating those records. Also, verify the account or tenant context before changing data.

A shared SaaS app may receive events for several customers. An event for one account must not update another account’s record. Oxen’s article on multi-tenant SaaS platforms explains the wider importance of tenant boundaries.

Define how the system handles an unknown object. It may need to fetch more details, delay processing, or raise a review task. Do not silently create a financial record from an event whose relationship is unclear.

4. Prevent duplicate business actions

A sender may deliver the same event more than once. If every delivery creates another action, a retry could send two receipts or create two fulfillment jobs. So, record the provider’s event ID and track processing status.

Stripe’s webhook documentation describes duplicate deliveries and notes that separate event objects can sometimes relate to the same underlying change. The receiver may need both event-level tracking and a business rule that prevents repeating the action.

This property is often called idempotency: repeating the request does not repeat its intended effect. In practice, the system might allow only one fulfillment job for an order and a particular payment state. The exact key depends on the workflow.

Test concurrent duplicate deliveries too. Two workers may inspect the same event at nearly the same time. A simple check in app memory may not stop both from proceeding. The build needs a reliable shared control.

5. Handle delayed and out-of-order events

Do not assume messages arrive in the same order as the underlying actions. A receiver might see a later event before an earlier one. Likewise, a delayed message may describe a state that has already changed.

For example, an order may have a refund event by the time an earlier payment event finishes processing. The app should not blindly overwrite the latest state with an older interpretation. Instead, define valid state changes and retrieve current data when needed.

Use the provider’s documented ordering and retrieval behavior. A timestamp alone may not provide a safe sequence because different events can share a time value. Test the particular transitions that matter to your business.

6. Acknowledge promptly and process safely

Long work inside the receiving request can cause timeouts. A queue can help: the endpoint verifies the event, durably stores or queues it, and returns the expected response. A worker then performs the longer business action.

Yet return success only after the receiver has safely accepted the event according to its design. If the system responds before storing anything, a crash can lose the event while the sender believes delivery succeeded.

Track separate states for receipt and processing. A delivered message may still fail during the business update. This distinction helps support teams avoid treating a successful HTTP response as proof that the invoice or order changed.

7. Define retries and recovery

Read the provider’s retry policy and failure responses. Some services retry failed deliveries, while others require manual redelivery. Never assume all providers behave alike.

Your own processing jobs also need a retry plan. Temporary connection failures may justify another attempt after a delay. In contrast, invalid data or an unknown account may need review rather than endless retries.

Keep a record of events that exhaust their retries. Provide a controlled way to inspect and reprocess them after fixing the cause. Also preserve the original event identity so recovery does not bypass duplicate safeguards.

For key workflows, add a reconciliation process. For instance, compare recent provider payments with local invoice records. This creates another path to detect a missed event without relying only on delivery logs.

8. Monitor the result that matters

Monitor delivery failures, processing failures, queue age, and repeated events. Then connect those signals to business outcomes. A healthy endpoint can still have a growing queue of unprocessed payments.

Log event IDs and local record references so support can trace a case. Yet exclude secrets and unnecessary personal data. Keep access to event payloads limited to people who need it.

Set an owner and response process for alerts. A notification that no one reviews will not restore the workflow. Test the response procedure as well as the code.

A hypothetical payment-to-order workflow

Imagine a seller whose custom portal creates an order and a payment request. The portal stores the payment provider’s object ID against the order. Later, the webhook endpoint receives a payment success event.

First, the endpoint verifies the signature and stores the event. Next, a worker finds the order by the saved ID and checks the expected amount and currency. It updates the payment state and creates a fulfillment task only if that task does not already exist.

A duplicate delivery then arrives. The system recognizes the event and avoids repeating the task. Meanwhile, a later refund follows a separate rule that updates the order and asks staff to review fulfillment if needed.

This example separates receipt, validation, and action. It also shows why the workflow needs explicit rules for more than the first success case. A single status update can affect several parts of daily work.

Test cases to include before launch

• A valid event with the expected record and values.
• An invalid signature and an unexpected event type.
• The same event twice, including simultaneous delivery.
• A delayed event after the record has changed.
• A temporary database or queue failure.
• An unknown object or incorrect tenant context.
• A processing failure after successful receipt.
• A controlled replay after the underlying issue is fixed.

Run these checks in a safe environment using provider-supported tools. Also ask business users to confirm the resulting records and notifications. Oxen’s guide to user acceptance testing helps connect technical event tests to real outcomes.

What to ask your integration partner

Ask how the system verifies messages, prevents repeat actions, and recovers missed events. Request a demo of those cases rather than only a successful live notification. Also, ask where support staff can see receipt and processing status.

Clarify ongoing ownership. Provider versions, secrets, and business rules can change after launch. The handover should include event mappings, test instructions, alert ownership, and recovery steps so the integration remains maintainable.

Also discuss data retention. Keeping every full payload forever may create avoidable cost and privacy exposure. Keep the evidence the workflow needs, and apply the business’s approved access and retention rules.

Design for the second delivery and the failed attempt

Webhooks work well when the receiver expects imperfect delivery. Verify messages, preserve their identity, and make business actions safe to repeat. Then add monitoring and recovery that prove the workflow completed.