Download and Sign Up
Get a $5 Coupon For Free
English
Getting Started Main Features Examples

【Smart Mode】How to set up Paging

2019-09-05 15:07:21
1345 views

Abstract:This article will show you how to set up paging. ScrapeStormFree Download

In Smart Mode, ScrapeStorm automatically recognizes paging. The paging types usually include the following:

(1) Next Page Button

(2) Scroll to Load

(3) Scroll to Load + Next Page Button

(4) None

However, the recognition results are sometimes incorrect. The reasons usually include the following:

(1) The page loading speed is too slow. During the automatic recognition process, the next page button has not appeared.

(2) There are multiple next page buttons in the page, and the software will only choose one of them.

(3) In the case where both Scroll to Load and Next Page Button exist, the next page button does not appear after the software automatically scrolls multiple times.

(4) The next page button in the current page is temporarily incompatible.

The settings menu for “Paging” is shown in the following figure.

(1) Next Page Button

ⅰ: Auto Detect

Click on the “Auto Detect” option.

The software will automatically recognize the next page button. After the recognition is successful, the page will automatically scroll to the position of the next page button and identify it with a red background box.

ⅱ: Select Button

If the software does not automatically identify the next page button, you will need to manually “Select button“.

Step 1: Click on the “Select button” option

Step 2: Click the next page button on the page

ⅲ: Edit XPath

If the above two cases cannot correctly identify the next page button, you need to edit the XPath to identify the next page button.

(2) Sroll to Load

Suitable for web pages that do not have a next page button and need to be scrolled to load content.

(3) Scroll to Load + Next Page Button

Suitable for pages that do not have a next page button at the beginning, and that require multiple scrolling of the page before the next page button can be displayed.

Or the next page button has been displayed, but some content of the current webpage is not displayed, and the page needs to be scrolled multiple times to display the entire content.

This type of paging is more difficult to identify. Although the software will automatically scroll when it is automatically identified, the number of scrolls may not match the number of scrolls required for the current page, so this type of paging usually requires some manual operations.

There are several situations:

ⅰ:Scroll to Load” was identified, but the next page button was not identified.

Step 1: Scroll the page manually until the next page button appears on the page

Step 2:  Select “Auto Detect“.

If the auto detect fails, select “Select Button” and click the next page button on the page.

ⅱ: The next page button was identified, but “Scroll to Load” was not identified.

In this case, just select “Scroll to Load“.

P.S. If the current web page is not needed, but the software Identifies “Scroll to Load”, it will not affect the scraping result, but canceling the “Scroll to Load” option can improve the scraping speed.

(4) None

If you do not need paging, select “None“.

P.S. Whether to set up paging or not has nothing to do with whether there is next page button in the current web page. It is only related to your scraping requirements. Not setting pages can narrow the scraping scope and improve the scraping speed.