【Smart Mode】【Flowchart Mode】How to use Webhook | Web Scraping Tool | ScrapeStorm
Abstract:This tutorial will show you how to use Webhook ScrapeStormFree Download
ScrapeStorm supports Webhook. By using this function, ScrapeStorm can export the scraped data to an HTTP/HTTPS URL.
This function can be set up in the run settings, as shown below:
After the Webhook function is enabled, the scraped data will be sent in JSON format. A notification will also be sent through webhook when the task has been finished.
The HTTP Header is "Content-Type: application/json; charset=utf-8"
.
Example of sending data:
{ "task_id": 3920415, // Scraping task ID, find it by "View All Tasks" "task_name": "Scraping task name", // Scraping task name "type": "data", //This is a webhook of scraped data "urls": [ //The entries of the scraping task "http://www.*****.com/list", "http://www.*****.com/list2" ], "sign": "**********************", // md5(webhook_token+timestamp) "timestamp": 1555326657, // current timestamp "data_list": [ // Scrape data list { "_id": "0000000000001", // data ID "data": { "title": "Landscape", "url": "http://www.*****.com/scenery/" } // Scrape data field content } ] }
Example of finish notification:
{ "task_id": 3920415, // Scraping task ID, find it by "View All Tasks" "task_name": "Scraping task name", // Scraping task name "type": "finish", //This is a webhook of finish notification "urls": [ //The entries of the scraping task "http://www.*****.com/list", "http://www.*****.com/list2" ], "sign": "**********************", // md5(webhook_token+timestamp) "timestamp": 1555326657 // current timestamp }