【Smart Mode】【Flowchart Mode】How to Download Files | Web Scraping Tool | ScrapeStorm

ScrapeStorm supports downloading files from webpages while scraping data. The supported file formats include image, audio, video, document and other formats.
This function can be set up in the run settings, as shown below:

The setting here is only for enabling the file download function. All file downloads need to set the corresponding download link in the scraping field. Files that cannot get the download link do not support downloading.

When downloading files, the software supports to modify the file download path, you can set it according to the following figure:

When downloading files, the software supports the creation of independent folders to store files in accordance with certain rules. The rules include naming folders according to File type, Date, Task name or Field value, as shown in the following figure:

When downloading files, the software supports renaming the downloaded files according to certain rules. The rules include naming the downloaded files according to the File name, File md5, Task name_Index, Date_Index, Index, Customize and Field value, as follows As shown in the figure:

When choosing a custom rule, you can combine different fixed rules to create the required naming rule, as shown in the following figure:

When encountering a file with the same name, you can choose from the following three ways:

 

File downloads can be divided into three types, the first is to directly click the download link, the second is to click the download button, and the third is to modify the data to generate the download link.

Type 1: Download link

1. Image

The supported image formats include jpg, png, gif, jpeg, bmp, eps, psd and other mainstream image formats. When you need to download an image, please check the image option, as shown in the figure below:

The tutorials for downloading images:

【Smart Mode】How to download images

【Flowchart Mode】How to download images

P.S. There is no essential difference between smart mode and flowchart mode in downloading images. Just because the interface is different, the two are described separately.

 

2. Audio

Supported audio formats include mp3, amr, wav, m4a and other mainstream audio formats. When you need to download audio, please check the audio option, as shown in the figure below.

If you can click to get the audio download link, please follow the steps below:

Step 1: Add a field, and then click the audio download link

Step 2: Right-click to set the field, and set the extract type to “Image/Video/Audio URL”

If the above methods cannot solve your needs, please follow the steps below:

Step 1: Add a field, then right-click the field, select “Edit Xpath”, and select the audio download link by manually editing Xpath

Step 2: Right-click to set the field, and set the extract type to “Image/Video/Audio URL”

 

3. Video

Supported video formats include mp4, avi, mov, 3gp, flv and other mainstream video formats. When you need to download a video, please check the video option, as shown in the figure below.

Step 1: Add a field, and then click the video download link

Step 2: Right-click to set the field, and set the extract type to “Image/Video/Audio URL”

If the above methods cannot solve your needs, please follow the steps below:

Step 1: Add a field, then right-click the field, select “Edit Xpath”, and select the video download link by manually editing Xpath

Step 2: Right-click to set the field, and set the extract type to “Image/Video/Audio URL”

 

4. Document

Supported document formats include doc(x), ppt(x), xls(x), pdf, snb, txt and other mainstream document formats. When you need to download a document, please check the document option, as shown in the figure below.

 

5. Other format

Other file formats supported include apk, exe, rar, zip, gzip, dmg, torrent, epub and other mainstream file formats. When you need to download these files, please check other file options, as shown in the figure below.

 

Type2: download button

For the second type, all file types are set in the same way. The specific steps are as follows:

Step 1: Add a field, and then click the download button

Step 2: Right-click the field and set the value attribute to “Download button”

In the setting section of the download file, check the corresponding file option. For the specific method, please refer to the first type of download setting above.

 

Type 3: modify data to generate download link

For the third type, after modifying the data of the download link, one more step needs to be added, as shown in the following figure: