Automate Google Trends download using R

Updated 13.10.2014

If you're using Google Trends for research or analysis, downloading the data can be time consuming. This tool uses R to automate the downloading process. The function URL_GT creates the URL for the Google Trends file export. It takes the following parameters:

keyword
A single query or a vector of queries. Google Trends does not accept more than five queries at a time.

country
If you are interested in the search volume for a specific country, add the country code here. I.e. country="US" for searches in the US.

region
If you have specified a country, you may also add a particular region in that country. Region="AL" gives you the searches in Alaska, if you have specified "US" as country.

year
If you want to get the searches for a specific year, enter year=2012 here.

month
Select the starting month by adding the months number here.

length
The parameter length specifies how many months of data to retrieve. The default is three months, as this is the maximum length for daily data. Queries for over three months will result in weekly or monthly data.

The function returns the URL for the csv download.

In order to automate the file download, you can use the downloadGT function. Add the URL to the file (as returned by the URL_GT function) and your browser's download path. The function uses your default web browser to open the link and returns the file path for the csv file.

#This script automates the downloading of Google Trends.
#It works best with firefox in combination with the Tab Mix Plus add-on that is used to automate tab closing.
#Ask firefox not to prompt for new downloads and this script should run automatically.
#Google Trends restricts the number of download to roughly 400 at a time.

URL_GT=function(keyword="", country=NA, region=NA, year=NA, month=1, length=3){

  start="http://www.google.com/trends/trendsReport?hl=en-US&q="
  end="&cmpt=q&content=1&export=1"
  geo=""
  date=""

#Geographic restrictions
  if(!is.na(country)) {
    geo="&geo="
    geo=paste(geo, country, sep="")
    if(!is.na(region)) geo=paste(geo, "-", region, sep="")
  }

  queries=keyword[1]
  if(length(keyword)>1) {
    for(i in 2:length(keyword)){
    queries=paste(queries, "%2C ", keyword[i], sep="")
    }
  }

#Dates
  if(!is.na(year)){
    date="&date="
    date=paste(date, month, "%2F", year, " ", month+length-1, "m", sep="")
  }

  URL=paste(start, queries, geo, date, end, sep="")
  return(URL)
}


downloadGT=function(URL, downloadDir){

#Determine if download has been completed by comparing the number of files in the download directory to the starting number
  startingFiles=list.files(downloadDir)
  browseURL(URL)
  endingFiles=list.files(downloadDir)

  while(length(setdiff(endingFiles,startingFiles))==0) {
    Sys.sleep(3)
    endingFiles=list.files(downloadDir)
  }
  filePath=setdiff(endingFiles,startingFiles)
  return(filePath)
}