Getting Started Main Features Examples

【Flowchart Mode】How to scrape data from multiple URLs

2018-10-15 19:26:26

Abstract:This tutorial focuses on how to scrape data from multiple URLs in flowchart mode.

In the “How to enter the URL correctly in Flowchart Mode” tutorial, we have a detailed description of the format of the input URL in the Flowchart Mode, this article is mainly for a detailed description of the settings of multiple URLs scraping.

In the Flowchart Mode, scraping multiple URLs can be divided into the following steps:

1. Create a new multi-site scraping task

The first method is to enter multiple URLs directly on the new task interface. Each URL is divided by the Enter key. At this time, the system will automatically generate multiple URL components, including the “Loop” component and the “Open URL” component.

Another method is to manually set the loop configuration of multiple URLs, drag the “Loop” component and the “Open URL” component from the component window, the connection relationship is as shown below.

Then set the Loop Condition parameter of the “Loop” component to the “URL List” and enter the URL in the URL list parameter.

Set the URL parameter in the “Open URL” component to “Use the URL in the current loop“.

For detailed settings of components, click Behavior Components and Flow Components.