Getting Started Main Features Examples

How to Scrape S&P 500 INDEX from CNN Business

2018-10-30 13:41:09
92 views

Abstract:In this article, we will tell you how to scrape S&P 500 INDEX from CNN Business using ScrapeStorm’s "Smart mode".

In this article, we will tell you how to scrape S&P 500 INDEX from CNN Business using ScrapeStorm’s “Smart mode“.

Introduction to the scraping tool

ScrapeStorm is a new generation of software for Web Scraping Tool based on artificial intelligence technology. It is the first scraper to support both Windows, Mac and Linux operating systems.

Introduction of scraping objects

Cable News Network is an American news-based pay television channel owned by Turner Broadcasting System, a division of AT&T’s WarnerMedia. CNN was founded in 1980 by American media proprietor Ted Turner as a 24-hour cable news channel.

Official website: edition.cnn.com

Scraping fields

title, title_link, % Change, Change, Company, P/E, Price, Volume, YTDchange, Scraping Time

Function point directory

How to manually set the page

What is Scheduled Job

Preview of the scraped result

Export to Excel2007:

Let’s take a closer look at how to scrape S&P 500 INDEX from CNN Business. The specific steps are as follows:

1. Download and install ScrapeStorm, then register and log in

(1) Open the ScrapeStorm official website, download and install the latest version.

(2) Click Register/Login to register a new account and then log in to ScrapeStorm.

2. Create a task

(1) Copy the URL of CNN Business 

Click here to learn more about how to enter the URL correctly.

(2) Create a new smart mode task

You can create a new scraping task directly on the software, or you can create a task by importing rules.

Click here to learn how to import and export scraping rule.

3. Configure the scraping rules

(1) Manually select

If you are not satisfied with the automatically recognized data or the effect of recognition is not good, you can manually select the list on the page.

(2) Set the fields

Intelligent mode automatically recognizes the fields on the page. You can right-click the field to rename the name, add or delete fields, modify data, and so on.

Click here to learn how to how to configure the extracted field.

Add or remove fields as needed, and rename the fields. The results of the field settings are as follows:

(3) Add special fields

Since we need to scrape the data in real time, we can add a special field of “Scraping Time” to the field.

(4) Manually set the page

Some web pages have special buttons on the next page, and the system may not recognize them. In this case, you need to manually set the page to “Select Pge Button”, then click the symbol for next page.

4. Set up and start the scraping task

(1) Running and Anti-block settings

Click “Setting”, set waiting time based on web page open speed. You can check “Block Images” and “Block Ads”. The anti-block settings follow the system default settings. Then click “Save”.

Click here to learn more about how to configure the scraping task.

P.S. “Block Images” will reduce the load time and speed up the scraping process. And this operation does not affect the scraping and downloading of images.

(2) Start scraping data

Premium Plan and above users can use “Scheduled job” and “Sync to Database”. If you want to download images, you can check “Download images while running”. Then click “Start”.

Click here to learn about scheduled job.

Click here to learn about sync to database.

Click here to learn about download images.

(3) Wait a moment, you will see the data being scraped. 

5. Export and view the data

(1) Click “Export” to download your data.

(2) Choose the format to export according to your needs.

ScrapeStorm provides a variety of export methods to export locally, such as excel, csv, html, txt or database. Professional Plan and above users can also post directly to wordpress.

Click here to learn more about how to view the extraction results and clear the extracted data.

Click here to learn more about how to export the result of extraction.

Here are some other tutorials for news websites:

How to Scrape News from HuffPost

How to Scrape World News from Newsweek

How to Scrape World News from Fox News