Intro to Text Analysis with R

By Tal Galili

Guest post by Christopher Johnson from www.codeitmagazine.com

One of the most powerful aspects of using R is that you can download free packages for so many tools and types of analysis. Text analysis is still somewhat in its infancy, but is very promising. It is estimated that as much as 80% of the world’s data is unstructured, while most types of analysis only work with structured data. In this paper, we will explore the potential of R packages to analyze unstructured text.

R provides two packages for working with unstructured text – TM and Sentiment. TM can be installed in the usual way. Unfortunately, Sentiment has been archived in 2012, and is therefore more difficult to install. However, it can still be installed using the following method, according to Frank Wang (Wang).

install.packages("devtools")

require(devtools)

install_url("http://cran.r-project.org/src/contrib/Archive/sentiment/sentiment_0.1.tar.gz")

install_url("http://cran.r-project.org/src/contrib/Archive/sentiment/sentiment_0.2.tar.gz")

The remaining required packaged can be installed as follows.

install.packages("plyr")

install.packages("ggplot2")

install.packages("wordcloud")

install.packages("RColorBrewer")

install.packages("tm")

install.packages("SnowballC")

Once initially installed, each can be loaded later as library(name).

The next step is to load the data. I chose to download comments from a newspaper vent line (Charleston Gazette-Mail ). This data was saved to a text file and loaded and processed as follows.

###Get the data

data 
df 
textdata 
textdata = gsub("[[:punct:]]", "", textdata)
Next, we remove nonessential characters such as punctuation, numbers, web addresses, etc from the text, before we begin processing the actual words themselves.  The code that follows was partially adapted from Gaston Sanchez in his work with sentiment analysis of Twitter data (Sanchez).
textdata = gsub("[[:punct:]]", "", textdata)
textdata = gsub("[[:digit:]]", "", textdata)
textdata = gsub("httpw+", "", textdata)
textdata = gsub("[ t]{2,}", "", textdata)
textdata = gsub("^s+|s+$", "", textdata)
try.error = function(x)
{
  y = NA
  try_error = tryCatch(tolower(x), error=function(e) e)
  if (!inherits(try_error, "error"))
    y = tolower(x)
  return(y)
}
textdata = sapply(textdata, try.error)

textdata = textdata[!is.na(textdata)]
names(textdata) = NULL
Next, we perform the sentiment analysis, classifying comments  ...read more
Source:: r-bloggers.com