Understanding Webhooks

Webhooks provide a way for our API to notify your application when certain events occur, such as completion of batch processing jobs. Instead of polling the API repeatedly to check job status, you can register a webhook URL and receive a POST request when the operation completes. This enables efficient asynchronous workflows for large-scale data processing.

Setting Up Webhooks

Configure your webhook endpoint in your dashboard or include it in API requests that support webhook callbacks:

                    
// Include webhook URL in batch request

POST /v1/batch/categorize

{

  "domains": ["example1.com", "example2.com", "..."],

  "webhook_url": "https://your-app.com/webhooks/categorization",

  "webhook_secret": "your_webhook_secret"

}

Webhook Events

We send webhooks for the following event types:

batch.completed

Triggered when a batch categorization job finishes processing all domains. Includes results summary and download link for full results.

batch.failed

Triggered if a batch job fails due to errors. Includes error details and partial results if available.

quota.warning

Triggered when your API usage reaches 80% of your monthly quota. Helps you anticipate and manage usage limits.

quota.exceeded

Triggered when your monthly quota is exhausted. Prompts you to upgrade your plan to continue service.

Webhook Payload Structure

Webhook payloads follow a consistent structure with event metadata and event-specific data:

                    
{

  "event": "batch.completed",

  "timestamp": "2024-01-15T10:30:00Z",

  "webhook_id": "wh_abc123xyz",

  "data": {

    "batch_id": "batch_def456",

    "total_domains": 1000,

    "successful": 985,

    "failed": 15,

    "processing_time_seconds": 45,

    "results_url": "https://api.websitecategorizationapi.com/v1/batch/batch_def456/results"

  }

}

Verifying Webhook Signatures

All webhooks include a signature header for verification. Always verify signatures to ensure webhooks originated from our servers:

                    
// Webhook headers

X-Webhook-Signature: sha256=abcd1234...

X-Webhook-Timestamp: 1705315800

// Verification in Node.js

const crypto = require('crypto');

function verifyWebhook(payload, signature, secret) {

  const timestamp = req.headers['x-webhook-timestamp'];

  const signedPayload = `${timestamp}.${JSON.stringify(payload)}`;

  const expectedSignature = crypto

    .createHmac('sha256', secret)

    .update(signedPayload)

    .digest('hex');

  return `sha256=${expectedSignature}` === signature;

}

Security Best Practice

Always verify webhook signatures before processing. Reject webhooks with invalid signatures to prevent spoofed requests. Store your webhook secret securely using the same practices as your API key.

Handling Webhook Delivery

Your webhook endpoint should respond quickly with a 2xx status code to acknowledge receipt. We retry failed deliveries with exponential backoff:

Initial delivery attempt immediately after event
First retry after 30 seconds
Second retry after 2 minutes
Third retry after 10 minutes
Final retry after 1 hour

                    
// Example webhook handler (Express.js)

app.post('/webhooks/categorization', (req, res) => {

  // Verify signature first

  if (!verifyWebhook(req.body, req.headers['x-webhook-signature'], WEBHOOK_SECRET)) {

    return res.status(401).send('Invalid signature');

  }

  // Acknowledge receipt immediately

  res.status(200).send('OK');

  // Process webhook asynchronously

  processWebhookAsync(req.body);

});

Webhooks Guide