Twitter

Scrape Tweets from Twitter

For educational & demonstrative purpose / No political affiliation

Goal:

Use Excel VBA to scrape tweets on Twitter.com, retrieving¬†each tweet’s data into an Excel worksheet

Sample Project:

Build Tableau Dashboard analyzing Donald Trump’s Tweets
– Graph Timeline of Trump’s number of Tweets from Jan. 2016 to May 2018
– Dynamic table to load Tweet content when Timeline in clicked
– Breakdown Tweet content into words & count which word is used most often

Tableau Dashboard:

Scraping Twitter:

For each tweet, scrape the following:

Time Posted

# of Replies

# of Retweets

# of Likes

Tweet Message

Link

Example:

In this example, we will retrieve all of the tweets from Donald J Trump’s twitter account to analyze which ones have the most number of replies, retweet and likes.

Educational & demonstrative purpose / No political affiliation.

Worksheet Prerequisites:

None – Twitter does not require an input/search parameter.

Code Breakdown:

Open browser window

Browse to Twitter.com/[username]

Employ a Do While Loop on the element “stream-footer”

  • “Stream-footer” is an div element used by Twitter on the bottom of the page
  • The “.ScrollIntoView” functions mimics the ‘PageDown’ function and scrolls to the footer element at bottom as Twitter loads more tweets dynamically when it detects the browser window at the bottom of the page
  • Add a counter parameter to continue this loop until it detects there are at least 100 tweets loaded

Do While ff.document.getelementsbyclassname(“js-stream-item stream-item stream-item”).Length < 100
ff.document.getelementsbyclassname(“stream-footer”)(0).ScrollIntoView
Loop

For each tweet, retreieve the Time Posted, # of Replies, # of Retweets, # of Likes, Tweet Message and Link

Close browser (ie.quit; Optional)

Input appropriate headers for each column in worksheet row 1

Result: