By The R Trader
(This article was originally published at The R Trader, and syndicated at StatsBlogs.)
Doing quantitative research implies a lot of data crunching and one needs clean and reliable data to achieve this. What is really needed is clean data that is easily accessible (even without an internet connection). The most efficient way to do this for me has been to maintain a set of csv files. Obviously this process can be handled in many ways but I found very efficient and simple overtime to maintain a directory where I store and update csv files. I have one csv file per instrument and each file is named after the instrument it contains. The reason I do so is twofold: First, I don’t want to download (price) data from Yahoo, Google etc… every time I want to test a new idea but more importantly once I identified and fixed a problem, I don’t want to have to do it again the next time I need the same instrument. Simple yet very efficient so far. The process is summarized in the chart below.
In everything that follows, I assume that data is coming from Yahoo. The code will have to be amended for data from Google, Quandl etc… In addition I present the process of updating daily price data. The setup will be different for higher frequency data and other type of dataset (i.e. different from prices).
1 – Initial data downloading (listOfInstruments.R & historicalData.R)
The file listOfInstruments.R is a file containing only the list of all instruments.
##########################################
## List of securities (Yahoo tickers)
## thertrader@gmail.com - Nov. 2015
##########################################
theInstruments = c("^GSPC",
"SPY",
...read moreSource:: statsblogs.com