【Smart Mode】【Flowchart Mode】How to use Webhook | Web Scraping Tool | ScrapeStorm

ScrapeStorm supports Webhook. By using this function, ScrapeStorm can export the scraped data to an HTTP/HTTPS URL.
This function can be set up in the run settings, as shown below:

After the Webhook function is enabled, the scraped data will be sent in JSON format. A notification will also be sent through webhook when the task has been finished.

The HTTP Header is "Content-Type: application/json; charset=utf-8".

Example of sending data:

{
    "task_id": 3920415, // Scraping task ID, find it by "View All Tasks"
    "task_name": "Scraping task name", // Scraping task name
    "type": "data", //This is a webhook of scraped data
    "urls": [ //The entries of the scraping task
        "http://www.*****.com/list",
        "http://www.*****.com/list2"
    ],
    "sign": "**********************", // md5(webhook_token+timestamp)
    "timestamp": 1555326657, // current timestamp
    "data_list": [ // Scrape data list
        {
            "_id": "0000000000001", // data ID
            "data": {
                "title": "Landscape",
                "url": "http://www.*****.com/scenery/"
            } // Scrape data field content
        }
    ]
}

Example of finish notification:

{
    "task_id": 3920415, // Scraping task ID, find it by "View All Tasks"
    "task_name": "Scraping task name", // Scraping task name
    "type": "finish", //This is a webhook of finish notification
    "urls": [ //The entries of the scraping task
        "http://www.*****.com/list",
        "http://www.*****.com/list2"
    ],
    "sign": "**********************", // md5(webhook_token+timestamp)
    "timestamp": 1555326657 // current timestamp
}