Getting Started Main Features Examples

【Flowchart mode】Basic operational procedures.

2018-09-07 17:22:50

Abstract:This tutorial demonstrates the basic operational procedures of flowchart mode.

(1) Enter the correct URL
Copy the URL you want to scrape in the browser and open the ScrapeStorm flowchart mode to paste the URL to create a new scraping task.
Click here to learn more about how to enter the correct URL.

(2) Scrape web pages that need to be logged in to view

In the process of data scraping, we sometimes encounter webpages that need to log in to view the content. At this time, we need to use the pre-login function to log in to the webpage and then perform normal data scraping.
Click here to learn more about how to log in to the web page.

(3) How to use the components

The ScrapeStorm team turns the development scraping rules into components by visually encapsulating the complex scraping coding process. In  flowchart mode, components are divided into behavior components and flow components. Components are the most basic elements that make up a flowchart scraping task.

Click here to learn more about “Behavior Components” and “Flow Components”.

(4) Set the scraping task component
The user can set the scraping task component by means of system-assisted tapping, or it can be set by manual dragging. Different scraping tasks require different components to be set up.

Click here for more application scenarios for scraping tasks.

(5) Set the extraction field
After setting up the scraping task component, the user can set the fields to be extracted on the Extract Data component.
Click here to learn more about setting up the extracted fields.

(6) Configure the scraping task

After the extraction field is set, the user can set the scraping task. The user can use the system default setting or set the scraping task by himself.

Click here to learn more about how to configure the scraping task.

(7) Scheduled job
Ordinary users can choose to start scraping data at a fixed point in time. In addition to allowing users to select data at a fixed time, Professional Plan and above users can also continuously scrape data in a fixed period.
Click here for more information on scheduled job.

(8) Sync to the database
Professional Plan and above users can use the Sync to Database function to export data while running data. It is not necessary to wait until the end of the task to export the data, and synchronize to the database with the timing collection function, which can greatly save time and improve efficiency. Suitable for users who need to continuously query data or monitor grievances.
Click here to learn more about syncing to the database.

(9) Download images
If the user needs to scrape the image on the web page to the local, you can use the download image feature to complete this requirement.
Click here to learn more about how to download images to the local.

(10) View the extraction results and export data
After the task is set, the user can view the extraction result and export the data.
ScrapeStorm export without any restrictions, completely free to export, you can rest assured to use.
Click here for more ways to view the results of the extraction and export the data.