【Smart Mode】How to filter data
Abstract：This tutorial introduces you to data filtering in smart mode.
In the process of setting up the scraping task, sometimes we encounter some unneeded data, such as data with empty values or data containing characters, which affects the speed and effect of the scraping.
The ScrapeStorm team developed data filtering capabilities that can be filtered before starting a task.
The Data Filter button is located in the upper right corner of the task action bar. You can click the button directly and then filter the data.
On the “Data Filter” interface, the user can choose whether to save data or end scraping when the lower condition is satisfied.
We can add new filters by clicking the Add button.
2. Logic Condition
There are two alternative conditions, you can choose the “AND” or “OR” option, when setting multiple filter conditions, you can set multiple “OR” side-by-side conditions, or multiple “AND” intersection conditions. Generally, the conditions of “OR” + “AND” are not set.
Users can select the fields that need data filtering in the options. If you want to set the filter conditions for different fields, you can click the “Add” button to add multiple conditions.
In the condition option box, there are 10 conditional options including “not null, contains, not contains, equal, not equal, more than, less than, time equal, time before, time after”, and the user can select conditions according to requirements.
You can enter text, numbers, time, etc. in the input box.
If you don’t need this filter, you can delete it by clicking “Delete”.