Getting Started with Data Science – R
A great introductory post from DataRobot on getting started with data science in R, including cleaning data and performing predictive modeling.
By Dallin Akagi and Mark Steadman, DataRobot.
This short tutorial will not only guide you through some basic data analysis methods but it will also show you how to implement some of the more sophisticated techniques available today. We will look into traffic accident data from the National Highway Traffic Safety Administration and try to predict fatal accidents using state-of-the-art statistical learning techniques. If you are interested, download the code at the bottom and follow along as we work through a real world data set. This post is in R while a companion post covers the same techniques in Python.
Getting started in R
swirl package is designed to teach people R. You can visit their website to see links to YouTube videos on installing R on both Mac and Windows.
You can download R directly using these links for the Windows installer or the Mac package. The R install packages are hosted by the CRAN project. Just bear in mind that these are large files (60-70MB) and will take some time to download and install.
We suggest you install and view this file in RStudio. After you have installed R, download and run the installer for Rstudio. We recommend using RStudio as the IDE, but you can also use the R console directly if you choose.
If you open this file in RStudio, you can see the code is stored in “cells”“ or “chunks” like this:
You can also enter the code in the cells directly at the R command prompt.
If you click in a cell, you can run the code in that cell by selecting “Run Current Chunk” under the Chunks menu.
Try running the code in the following cell, which will load the libraries needed for the rest of this tutorial.
If you get an error, delete the
# and run the following code chunk to install the needed packages.
If you prefer, you can download a version with just the R code here, which you can load into R via the
Now that you have R installed, we can start with our analysis. Inside RStudio, select “Run Next Chunk” under the “Chunks” menu to run the examples one at a time.
Get some data
Being able to play with data requires having the data available, so let’s take care of that right now. The National Highway Traffic Safety Administration (NHTSA) has some really cool data that they make public. The following code snippet will take care of downloading the data to a temporary file, and extract the file we are interested in, “PERSON.TXT”, from the zipfile. Finally, it loads the data into R. The zip is 14.9 MB so it might take some time to run – it is worth the wait! This is really cool data.
With our data downloaded and readily accessible, we can start to play around and see what we can learn from the data. Many of the columns have an encoding that you will need to read the manual in order to understand; it might be useful to download the PDF so you can easily refer to it. Again, we will be looking at
PERSON.TXT, which contains information about individuals involved in road accidents.
Clean up the data
One prediction task you might find interesting is predicting whether or not a crash was fatal. The column
INJSEV_IM contains imputed values for the severity of the injury, but there is one value that might complicate analysis – level 6 indicates that the person died prior to the crash.
Fortunately, there are only four of those cases within the dataset, so it is not unreasonable to ignore them during our analysis. However, we will find that a few of these columns have missing values:
For this analysis, we will just drop these rows (they are all the same rows) – but you certainly don’t have to do that. In fact, maybe there is a systematic data entry error that is causing them to be interpreted incorrectly. Regardless of the way you cleanup this data, we will most assuredly want to drop the column INJ_SEV, as it is the non-imputed version of INJSEV_IM and is a pretty severe data leak – there are others as well.
One more preprocessing step we’ll do is to transform the response. If you flip to the manual it shows that category
4 is a fatal injury – so we will encode our target as such.
|Top Stories Past 30 Days|