Implementing a Webhook Handler

Upvest's Webhooks are HTTP requests made from our servers to the endpoints you define. We'll talk about how you inform us of the addresses of those endpoints in a later step of the "Implementing Webhooks" tutorial.

Because we don't know what technology you're building your application with, we can't yet give you a step-by-step guide to the implementation. Instead, this sub-task will guide you through the things you have to consider when implementing an API endpoint to handle incoming Webhook requests from the Upvest Investment API. Given this knowledge you should be able to successfully create the Webhook handlers required.

 1    Delivery and Retry Logic

This section describes how Upvest delivers its requests, and how failures in that process are handled via retries. You'll need to make sure your handlers take these behaviours into account:

• Upvest's Webhook service sends multiple events bundled into a single request. This keeps the overhead of establishing a TCP / HTTPS connection per Webhook as low as possible.
• Events within a request are ordered by the created_at field, in ascending order (from oldest to newest). There are no ordering guarantees across requests.
• If a Webhook delivery is successful, your handler should return the HTTP 200 status code. If that code is not received, the Webhook service retries the delivery of the entire request using an exponential back off mechanism. This means the service will back off and retry within an increasingly large interval (1,2,3,5,8...600 sec) with no limit on the number or retry attempts.
• An unsuccessful Webhook delivery blocks all subsequent deliveries, which will be queued indefinitely.
• A successful delivery retry unblocks all queued deliveries.

Implementation recommendations

To deal robustly with the Webhook delivery logic, we highly recommend

1. Idempotent Webhook handlers based on the event_id as an idempotent id.
2. Keeping an event log.

The reasons for those recommendations:

• The Webhook service follows "at least once (or more)" logic. That means you're guaranteed to get every event at least once, but you may also see the same event multiple times.

• If multiple Webhook subscriptions are set up for the same event type, the same event will be delivered to each one of them, causing duplication.

• If a Webhook calls fails, and an HTTP status other than 200 returned has been returned from your endpoint, the entire Webhook call, which may contain multiple events, will be retried.

• Once a Webhook is delivered and a HTTP status of 200 is returned from your endpoint, the Webhook service will never resend the same request.

 2    Structure of Webhook requests

The payload of a Webhook request is a JSON structure with a single top-level key, payload. This key contains a list of JSON objects, each containing the details of an event.

Each event contains the following fields:

Example batch payload

Including a single event related to a user object and emitted when a USER.CREATED event occurs:

{
    "payload": [
        {
            "id": "fbecea50-2f35-4969-96af-342271da9eca",
            "created_at": "2021-07-21T14:10:00.00Z",
            "event_type": "USER.CREATED",
            "object": {
                "id": "83d83ec2-d2ca-49ff-bbea-b92b5c3be202",
                "type": "USER"
            },
            "webhook_id": "091915d8-7b4b-4d25-8208-f8e2d4b30070"
        }
    ]
}

 3    Validating Webhook Signatures

To ensure that Webhook requests originate from us and have not been modified without authorisation, you should validate the HTTP message signatures we include in these requests.

They follow the same standard as the API request signatures we ask you to use.

Example of signature validation

A typical set of headers sent as part of a Webhook request might looks like this:

Example headers

Accept-Encoding: gzip
Content-Length: 1177
Digest: SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=
Host: tenant.tld
Signature: sig1=:MIGIAkIB2MuP6GGvrOxZhvSROnBjMlofArjjGaazcyusqyGoiHiGVUuH2+qmwQR0GI6NrvkYUW0bRRuxO3f9Ng1Suh6SR4cCQgCQsyQvoNfeLs/PImskOvvjU9dhmRc0z4YshLKxB3krYYY9Ae6nP4Vq/4hJSrGHhTT6V46edzOen/9nxuCPtJ6SoQ==:
Signature-Input: sig1=("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333

Given these headers, the validation steps are as follows.

3.1   Parse Signature-Input Header

Parse the Signature-Input header. It has metadata for one signature called sig1. It lists content-length, @method, @path, digest as signature components, identifies the signing key, provides two timestamps and one nonce value.

3.2   Check Webhook time frame

Check that the time you received the Webhook request lies between the created and expires timestamps. If it does not lie between these bounds, discard the request.

3.3   Check body length

Check whether the length of the Webhook request body matches the number specified in the content-length header. If they do not match, discard the request.

Note: the length is counted in bytes, not UTF-8 characters.

3.4   Check the content digest

Check if the digest of the Webhook request body matches the hash in the Digest header. If they do not match, discard the request.

The calculation of the digest is described here.

3.5   Obtain the public key

Check if you have cached a public key with ID 9f030355-3da5-4417-b3fe-4726f462b4b7. If not, you can retrieve it with GET /auth/verify_keys.

3.6   Calculate component values

For the four listed components, calculate their values. (Details can be found here).

• For content-length, use the value of the Content-Length header, which in this example is 1177.

• For @method, use POST as a constant value. We send all Webhooks as POST.

• For @path, use the path part of your Webhook URL.

• For digest, use the value of the Digest header, which in this example is SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=.

3.7   Calculate @signature-params

For the mandatory @signature-params component, use the metadata for signature sig1 as the value.

For example:

("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333

3.8   Calculate signature-input

Create the signature input string by concatenating the signature components (in the same order as listed, plus @signature-params) as follows:

content-length: 1177
@method: POST
@path: /webhooks/users
digest: SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=
@signature-params: ("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333

3.9   Base64 decode the signature

Base64-decode the value between the colon (:) characters in the Signature header to get the binary value of the signature.

3.10   Validate the signature

Validate the signature with the signature input string and the public key 9f030355-3da5-4417-b3fe-4726f462b4b7.

3.11   Accept or Discard

If the validation outcome is negative, discard the Webhook request.

 ✔    You're ready to implement Webhooks!

Congratulations! If you've read and understood this content, you should be ready to implement a handler endpoint for Upvest Webhooks.

Next Steps

We suggest you continue by returning to the "Implementing Webhooks" tutorial and reading the "Register your Webhook" section.