Google Multi-Page

Scrape Google Search Results (Multiple Pages)

Goal:

Use Excel VBA to automate the task of (1) searching for a single search term and (2) retrieving the search results for multiple pages into an Excel worksheet

Example:

In this example, we want to search for the term “Roger Federer” and get all the search results from page 1 to page 5

Worksheet Prerequisites:

Cell A1: Input your search term (“Roger Federer”)

Cell A2: Number of pages to gather data from (5 = scrape all data from Page 1 to Page 5)

Breakdown:

Open Internet Explorer

Browse to Google.com

Input search value from Excel (cell A1) into Google’s search bar

Click the “Google Search” button

Loop Step 1: For each search result, print the (1) Title and (2) Link* into worksheet (Cell B1 & B2)

Loop Step 2: Add a page counter, print the page# the data was retrieved from in the last column

Loop Step 3: Click “Next Page”

*Note: When VBA retrieve the url, it is stored as text, to change it to a clickable hyperlink, add the following:

Set Hlinks = Application.Selection
For Link In Hlinks
Application.ActiveSheet.Hyperlinks.Add Link, Link.Value
Next

Repeat Loop until the Google result page is 1 page pass the inputted value in cell A1

Close browser (ie.quit; Optional)

Input appropriate headers for each column in worksheet row 1

Result: