Download and Sign Up
Get a $5 Coupon For Free
Getting Started Main Features

【Flowchart Mode】How to turn pages by entering page numbers in batches | Web Scraping Tool | ScrapeStorm

2022-04-22 15:17:18

Abstract:This article will introduce how to turn pages by entering page numbers in batches in the flowchart mode of ScrapeStorm. ScrapeStormFree Download

In the setting of the scraping task, it is often encountered that the webpage does not have a page turning button or the website cannot turn the page by clicking the next page button. At this time, we can use flowchart mode of ScrapeStorm to turn pages by entering page numbers in batches.

1. Create a new task

1) Copy the URL

Note: Please copy the URL of the search result page, not the home page.

How to enter a correct URL


2) Create a new flowchart mode task

You can enter the URL from the software home page or import the task you have.

How to import and export scraping task


2. Configure the task

1) Set the component to batch input page numbers

Click the page number input box and enter the page number in the operation tips.

Note: Since multiple page numbers need to be entered, please click “Input text list” in the operation tips.

Then select “Single text list“.

Then enter the page number to be set in the popup. For example “1”, “2” and “3”.

Click “OK” and the software will automatically generate a loop component that populates the page number.

Then click the “Go” button on the page, select “Click the element” in the operation tips, jump to the corresponding web page and generate the click component.


2) Extract the list data

After the page number loop is set, we set to extract the list data, click the field on the web page, and select “Extract all elements in the list” in the operation tips.

The software will automatically detect the paging. This task does not need to set automatic paging, so select “No, extract only the current webpage”.

The scraping field can then be set on this basis.

For more details, please refer to the following tutorial:

How to set the fields


3. Set and start the task

1) Start the task

Click the “Start” button to make some settings in the Run Settings, including functions such as “Schedule, IP Rotation & Delay, Automatic Export, Download Files”. The above functions are not used in this operation, just click the “Start” button to start scraping.

Please refer to the tutorial below for how to set each function.

What is Schedule function

How to set up Automatic Export

How to Download Files

Note: Schedule, Automatic Export and Download Images are available for users of Premium plan and above, and Download Other Files is available for users of Business plan and above.


2) Run the task to extract data

After the task is started, the data will be automatically scraped. We can intuitively see the running process and scraping results from the interface, and there will be a reminder after the scraping stops.


4. Export and view data

After the data scraping is completed, we can export and view the data. ScrapeStorm supports a variety of export methods and export file formats (EXCEL, CSV, HTML, and TXT), and also supports exporting specific numbers. You can select the data you want to export. , and then click “Export“.


python crawler Data scraping with python Download images in batches Download web page as word Automatically organize data into excel Generate URLs in batches Keyword extraction from web content Match emails with Regex Download videos in batches php crawler