useR: Recipes for data processing
These are my notes for the tutorial given by Max Kuhn on the afternoon of the first day of the UseR 2018 conference.
Full confession here: I was having trouble deciding between this tutorial and another one, and eventually decided on the other one. But then I accidentally came to the wrong room and I took it as a sign that it was time to learn more about preprocessing.
Also, the recipes package is adorable.
The recipes package is adorable. You "prep" your data with a recipe and then you "bake". Also, this hex logo is just precious. #useR2018 pic.twitter.com/nPMxCnM9Jx
— David Neuzerling (@mdneuzerling) July 10, 2018I’m going to follow along with Max’s slides, making some comments along the way.
Required packages:
install.packages(c("AmesHousing", "broom", "kknn", "recipes", "rsample",
"tidyverse", "yardstick", "caret"))The data set we’ll use is the AMES IA housing data. This includes sale price (our target) along with 81 predictors, such as location, house components (eg. swimming pool), number of bedrooms, and so on. The raw data can be found at https://ww2.amstat.org/publications/jse/v19n3/decock/AmesHousing.txt but we will be using the processed version found in the AmesHousing package.
Reasons for modifying the data
Sometimes you need to do stuff to your data before you can use it. Moreover, you’re often dealing with data that’s split into train/test sets. In this case you need to work out what to do with your data based solely on the training set and then apply that—without changing your method—to the test set. If you’re dealing with $K$-fold cross-validation, then you’ve got $K$ training sets and $K$ test sets, and so you need to repeat this $K$ times.
A good example is missing value imputation, where you have some missing data in your train/test sets and you need to fill them in via some imputation method. I’m no expert on the topic (but I hope to be after the missing value imputation tutorial tomorrow!) but I’ve seen this done wrong before in StackExchange answers and in Kaggle solutions: the imputation is done before the data is split into train/test. This is called data leakage, and models assessed using the test set will appear more accurate than they are, because they’ve already had a sneak preview of the data.
So the mindset is clear: don’t touch the test set until the last possible moment. The recipes package follows this mindset. First you create a recipe, which is a blueprint for how you will process your data. At this point, no data has been modified. Then you prep the recipe using your training set, which is where the actual processing is defined and all the parameters worked out. Finally, you can bake the training set, test set, or any other data set with similar columns, and in this step the actual modification takes place.
Missing value imputation isn’t the only reason to process data, though. Processing can involve:
- Centering and scaling the predictors. Some models (K-NN, SBMs, PLS, neural networks) require that the predictor variables have the same units.
- Applying filters or PCA signal extraction to deal with correlation between predictors.
- Encoding data, such as turning factors into Boolean dummy variables, or turning dates into days of the week.
- Developing new features (ie. feature engineering).
The ames data
We load the data with the make_ames function from the AmesHousing package.
ames <- AmesHousing::make_ames()
ames %>% str
#> tibble [2,930 × 81] (S3: tbl_df/tbl/data.frame)
#> $ MS_SubClass : Factor w/ 16 levels "One_Story_1946_and_Newer_All_Styles",..: 1 1 1 1 6 6 12 12 12 6 ...
#> $ MS_Zoning : Factor w/ 7 levels "Floating_Village_Residential",..: 3 2 3 3 3 3 3 3 3 3 ...
#> $ Lot_Frontage : num [1:2930] 141 80 81 93 74 78 41 43 39 60 ...
#> $ Lot_Area : int [1:2930] 31770 11622 14267 11160 13830 9978 4920 5005 5389 7500 ...
#> $ Street : Factor w/ 2 levels "Grvl","Pave": 2 2 2 2 2 2 2 2 2 2 ...
#> $ Alley : Factor w/ 3 levels "Gravel","No_Alley_Access",..: 2 2 2 2 2 2 2 2 2 2 ...
#> $ Lot_Shape : Factor w/ 4 levels "Regular","Slightly_Irregular",..: 2 1 2 1 2 2 1 2 2 1 ...
#> $ Land_Contour : Factor w/ 4 levels "Bnk","HLS","Low",..: 4 4 4 4 4 4 4 2 4 4 ...
#> $ Utilities : Factor w/ 3 levels "AllPub","NoSeWa",..: 1 1 1 1 1 1 1 1 1 1 ...
#> $ Lot_Config : Factor w/ 5 levels "Corner","CulDSac",..: 1 5 1 1 5 5 5 5 5 5 ...
#> $ Land_Slope : Factor w/ 3 levels "Gtl","Mod","Sev": 1 1 1 1 1 1 1 1 1 1 ...
#> $ Neighborhood : Factor w/ 28 levels "North_Ames","College_Creek",..: 1 1 1 1 7 7 17 17 17 7 ...
#> $ Condition_1 : Factor w/ 9 levels "Artery","Feedr",..: 3 2 3 3 3 3 3 3 3 3 ...
#> $ Condition_2 : Factor w/ 8 levels "Artery","Feedr",..: 3 3 3 3 3 3 3 3 3 3 ...
#> $ Bldg_Type : Factor w/ 5 levels "OneFam","TwoFmCon",..: 1 1 1 1 1 1 5 5 5 1 ...
#> $ House_Style : Factor w/ 8 levels "One_and_Half_Fin",..: 3 3 3 3 8 8 3 3 3 8 ...
#> $ Overall_Qual : Factor w/ 10 levels "Very_Poor","Poor",..: 6 5 6 7 5 6 8 8 8 7 ...
#> $ Overall_Cond : Factor w/ 10 levels "Very_Poor","Poor",..: 5 6 6 5 5 6 5 5 5 5 ...
#> $ Year_Built : int [1:2930] 1960 1961 1958 1968 1997 1998 2001 1992 1995 1999 ...
#> $ Year_Remod_Add : int [1:2930] 1960 1961 1958 1968 1998 1998 2001 1992 1996 1999 ...
#> $ Roof_Style : Factor w/ 6 levels "Flat","Gable",..: 4 2 4 4 2 2 2 2 2 2 ...
#> $ Roof_Matl : Factor w/ 8 levels "ClyTile","CompShg",..: 2 2 2 2 2 2 2 2 2 2 ...
#> $ Exterior_1st : Factor w/ 16 levels "AsbShng","AsphShn",..: 4 14 15 4 14 14 6 7 6 14 ...
#> $ Exterior_2nd : Factor w/ 17 levels "AsbShng","AsphShn",..: 11 15 16 4 15 15 6 7 6 15 ...
#> $ Mas_Vnr_Type : Factor w/ 5 levels "BrkCmn","BrkFace",..: 5 4 2 4 4 2 4 4 4 4 ...
#> $ Mas_Vnr_Area : num [1:2930] 112 0 108 0 0 20 0 0 0 0 ...
#> $ Exter_Qual : Factor w/ 4 levels "Excellent","Fair",..: 4 4 4 3 4 4 3 3 3 4 ...
#> $ Exter_Cond : Factor w/ 5 levels "Excellent","Fair",..: 5 5 5 5 5 5 5 5 5 5 ...
#> $ Foundation : Factor w/ 6 levels "BrkTil","CBlock",..: 2 2 2 2 3 3 3 3 3 3 ...
#> $ Bsmt_Qual : Factor w/ 6 levels "Excellent","Fair",..: 6 6 6 6 3 6 3 3 3 6 ...
#> $ Bsmt_Cond : Factor w/ 6 levels "Excellent","Fair",..: 3 6 6 6 6 6 6 6 6 6 ...
#> $ Bsmt_Exposure : Factor w/ 5 levels "Av","Gd","Mn",..: 2 4 4 4 4 4 3 4 4 4 ...
#> $ BsmtFin_Type_1 : Factor w/ 7 levels "ALQ","BLQ","GLQ",..: 2 6 1 1 3 3 3 1 3 7 ...
#> $ BsmtFin_SF_1 : num [1:2930] 2 6 1 1 3 3 3 1 3 7 ...
#> $ BsmtFin_Type_2 : Factor w/ 7 levels "ALQ","BLQ","GLQ",..: 7 4 7 7 7 7 7 7 7 7 ...
#> $ BsmtFin_SF_2 : num [1:2930] 0 144 0 0 0 0 0 0 0 0 ...
#> $ Bsmt_Unf_SF : num [1:2930] 441 270 406 1045 137 ...
#> $ Total_Bsmt_SF : num [1:2930] 1080 882 1329 2110 928 ...
#> $ Heating : Factor w/ 6 levels "Floor","GasA",..: 2 2 2 2 2 2 2 2 2 2 ...
#> $ Heating_QC : Factor w/ 5 levels "Excellent","Fair",..: 2 5 5 1 3 1 1 1 1 3 ...
#> $ Central_Air : Factor w/ 2 levels "N","Y": 2 2 2 2 2 2 2 2 2 2 ...
#> $ Electrical : Factor w/ 6 levels "FuseA","FuseF",..: 5 5 5 5 5 5 5 5 5 5 ...
#> $ First_Flr_SF : int [1:2930] 1656 896 1329 2110 928 926 1338 1280 1616 1028 ...
#> $ Second_Flr_SF : int [1:2930] 0 0 0 0 701 678 0 0 0 776 ...
#> $ Low_Qual_Fin_SF : int [1:2930] 0 0 0 0 0 0 0 0 0 0 ...
#> $ Gr_Liv_Area : int [1:2930] 1656 896 1329 2110 1629 1604 1338 1280 1616 1804 ...
#> $ Bsmt_Full_Bath : num [1:2930] 1 0 0 1 0 0 1 0 1 0 ...
#> $ Bsmt_Half_Bath : num [1:2930] 0 0 0 0 0 0 0 0 0 0 ...
#> $ Full_Bath : int [1:2930] 1 1 1 2 2 2 2 2 2 2 ...
#> $ Half_Bath : int [1:2930] 0 0 1 1 1 1 0 0 0 1 ...
#> $ Bedroom_AbvGr : int [1:2930] 3 2 3 3 3 3 2 2 2 3 ...
#> $ Kitchen_AbvGr : int [1:2930] 1 1 1 1 1 1 1 1 1 1 ...
#> $ Kitchen_Qual : Factor w/ 5 levels "Excellent","Fair",..: 5 5 3 1 5 3 3 3 3 3 ...
#> $ TotRms_AbvGrd : int [1:2930] 7 5 6 8 6 7 6 5 5 7 ...
#> $ Functional : Factor w/ 8 levels "Maj1","Maj2",..: 8 8 8 8 8 8 8 8 8 8 ...
#> $ Fireplaces : int [1:2930] 2 0 0 2 1 1 0 0 1 1 ...
#> $ Fireplace_Qu : Factor w/ 6 levels "Excellent","Fair",..: 3 4 4 6 6 3 4 4 6 6 ...
#> $ Garage_Type : Factor w/ 7 levels "Attchd","Basment",..: 1 1 1 1 1 1 1 1 1 1 ...
#> $ Garage_Finish : Factor w/ 4 levels "Fin","No_Garage",..: 1 4 4 1 1 1 1 3 3 1 ...
#> $ Garage_Cars : num [1:2930] 2 1 1 2 2 2 2 2 2 2 ...
#> $ Garage_Area : num [1:2930] 528 730 312 522 482 470 582 506 608 442 ...
#> $ Garage_Qual : Factor w/ 6 levels "Excellent","Fair",..: 6 6 6 6 6 6 6 6 6 6 ...
#> $ Garage_Cond : Factor w/ 6 levels "Excellent","Fair",..: 6 6 6 6 6 6 6 6 6 6 ...
#> $ Paved_Drive : Factor w/ 3 levels "Dirt_Gravel",..: 2 3 3 3 3 3 3 3 3 3 ...
#> $ Wood_Deck_SF : int [1:2930] 210 140 393 0 212 360 0 0 237 140 ...
#> $ Open_Porch_SF : int [1:2930] 62 0 36 0 34 36 0 82 152 60 ...
#> $ Enclosed_Porch : int [1:2930] 0 0 0 0 0 0 170 0 0 0 ...
#> $ Three_season_porch: int [1:2930] 0 0 0 0 0 0 0 0 0 0 ...
#> $ Screen_Porch : int [1:2930] 0 120 0 0 0 0 0 144 0 0 ...
#> $ Pool_Area : int [1:2930] 0 0 0 0 0 0 0 0 0 0 ...
#> $ Pool_QC : Factor w/ 5 levels "Excellent","Fair",..: 4 4 4 4 4 4 4 4 4 4 ...
#> $ Fence : Factor w/ 5 levels "Good_Privacy",..: 5 3 5 5 3 5 5 5 5 5 ...
#> $ Misc_Feature : Factor w/ 6 levels "Elev","Gar2",..: 3 3 2 3 3 3 3 3 3 3 ...
#> $ Misc_Val : int [1:2930] 0 0 12500 0 0 0 0 0 0 0 ...
#> $ Mo_Sold : int [1:2930] 5 6 6 4 3 6 4 1 3 6 ...
#> $ Year_Sold : int [1:2930] 2010 2010 2010 2010 2010 2010 2010 2010 2010 2010 ...
#> $ Sale_Type : Factor w/ 10 levels "COD","Con","ConLD",..: 10 10 10 10 10 10 10 10 10 10 ...
#> $ Sale_Condition : Factor w/ 6 levels "Abnorml","AdjLand",..: 5 5 5 5 5 5 5 5 5 5 ...
#> $ Sale_Price : int [1:2930] 215000 105000 172000 244000 189900 195500 213500 191500 236500 189000 ...
#> $ Longitude : num [1:2930] -93.6 -93.6 -93.6 -93.6 -93.6 ...
#> $ Latitude : num [1:2930] 42.1 42.1 42.1 42.1 42.1 ...Now we will split the data into test and train. We’ll reserve 25% of of the data for testing.
library(rsample)
set.seed(4595)
data_split <- initial_split(ames, strata = "Sale_Price", p = 0.75)
ames_train <- training(data_split)
ames_test <- testing(data_split)A simple log-transform
The first of Max’s examples is a really simple log transform of Sale_Price. Suppose we use the formula log10(Sale_Price) ~ Longitude + Latitude. The steps are:
- Assign
Sale_Priceto the outcome. - Assign
LongitudeandLatittudeas predictors. - Log transform the outcome.
The way to define this in recipes is as follows:
mod_rec <- recipe(Sale_Price ~ Longitude + Latitude, data = ames_train) %>%
step_log(Sale_Price, base = 10)Infrequently occurring levels
We usually encode factors as Boolean dummy variables, with R often taking care of this in the background. If there are C levels of the factor, only C - 1 dummy variables are required. But what if you have very few values for a particular level? For example, the Neighborhood predictor in our ames data:
ames %>%
ggplot(aes(x = Neighborhood)) +
geom_bar(fill = "#6d1e3b", colour = "white") + # I don't like the default grey
coord_flip()
In fact, there’s only one data point with a Neighborhood of Landmark. This is called a “zero-variance predictor”. There are two main approaches here:
- remove any data points with infrequently occurring values, or
- group all of the infrequently occurring values into an “Other” level.
This is a job for the recipes package, and Max takes us through the example.
We can take care of the infrequently occurring levels here using the step_other function. In this case, we “other” any level that occurs fewer than 5% of the time. We can then create dummy variables for all factor variables with step_dummy:
mod_rec <- recipe(Sale_Price ~ Longitude + Latitude + Neighborhood,
data = ames_train) %>%
step_log(Sale_Price, base = 10) %>% # The log-transform from earlier
step_other(Neighborhood, threshold = 0.05) %>%
step_dummy(all_nominal())The recipes process
Recipes work in a three-step process: recipe –> prepare –> bake/juice. We can think of this as: define –> estimate –> apply. juice only applies to the original data set defined in the recipe, the idea at the core of bake is that it can be applied to an arbitrary data set.
First we prep the data using the recipe in Max’s example:
mod_rec_trained <- mod_rec %>%
prep(training = ames_train, retain = TRUE)
mod_rec_trained
#> Data Recipe
#>
#> Inputs:
#>
#> role #variables
#> outcome 1
#> predictor 3
#>
#> Training data contained 2199 data points and no missing data.
#>
#> Operations:
#>
#> Log transformation on Sale_Price [trained]
#> Collapsing factor levels for Neighborhood [trained]
#> Dummy variables from Neighborhood [trained]We can now bake the recipe, applying it to the test set we defined earlier:
ames_test_dummies <- mod_rec_trained %>% bake(new_data = ames_test)
names(ames_test_dummies)
#> [1] "Longitude" "Latitude"
#> [3] "Sale_Price" "Neighborhood_College_Creek"
#> [5] "Neighborhood_Old_Town" "Neighborhood_Edwards"
#> [7] "Neighborhood_Somerset" "Neighborhood_Northridge_Heights"
#> [9] "Neighborhood_Gilbert" "Neighborhood_Sawyer"
#> [11] "Neighborhood_other"Other uses
I have to admit that the rest got away from me a little bit, because I’m not overly familiar with all of the transformations/methods that were used (what is a Yeo-Johnson Power Transformation?!).
However, there’s a tonne of cool stuff in the slides that I’ll be coming back to later, I’m sure. Max used recipes and rsample to:
- deal with interactions between predictors,
- apply processing to all of the folds of a 10-fold cross-validation,
- train 10 linear models on that same 10-fold cross-validation,
- assess and plot the performance of those linear models, and
- train and asses 10 nearest-neighbour models on the 10-fold cross-validation.
I know I’ll be using this recipes package a lot.
devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.0.0 (2020-04-24)
#> os Ubuntu 20.04 LTS
#> system x86_64, linux-gnu
#> ui X11
#> language en_AU:en
#> collate en_AU.UTF-8
#> ctype en_AU.UTF-8
#> tz Australia/Melbourne
#> date 2020-06-13
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date lib source
#> AmesHousing 0.0.3 2017-12-17 [1] CRAN (R 4.0.0)
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0)
#> backports 1.1.7 2020-05-13 [1] CRAN (R 4.0.0)
#> blogdown 0.19 2020-05-22 [1] CRAN (R 4.0.0)
#> broom 0.5.6 2020-04-20 [1] CRAN (R 4.0.0)
#> callr 3.4.3 2020-03-28 [1] CRAN (R 4.0.0)
#> caret * 6.0-86 2020-03-20 [1] CRAN (R 4.0.0)
#> cellranger 1.1.0 2016-07-27 [1] CRAN (R 4.0.0)
#> class 7.3-17 2020-04-26 [4] CRAN (R 4.0.0)
#> cli 2.0.2 2020-02-28 [1] CRAN (R 4.0.0)
#> codetools 0.2-16 2018-12-24 [4] CRAN (R 4.0.0)
#> colorspace 1.4-1 2019-03-18 [1] CRAN (R 4.0.0)
#> crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.0)
#> data.table 1.12.8 2019-12-09 [1] CRAN (R 4.0.0)
#> DBI 1.1.0 2019-12-15 [1] CRAN (R 4.0.0)
#> dbplyr 1.4.3 2020-04-19 [1] CRAN (R 4.0.0)
#> desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.0)
#> devtools 2.3.0 2020-04-10 [1] CRAN (R 4.0.0)
#> dials 0.0.6 2020-04-03 [1] CRAN (R 4.0.0)
#> DiceDesign 1.8-1 2019-07-31 [1] CRAN (R 4.0.0)
#> digest 0.6.25 2020-02-23 [1] CRAN (R 4.0.0)
#> downlit 0.0.0.9000 2020-06-12 [1] Github (r-lib/downlit@87fb1af)
#> dplyr * 0.8.5 2020-03-07 [1] CRAN (R 4.0.0)
#> ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.0)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0)
#> fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.0)
#> farver 2.0.3 2020-01-16 [1] CRAN (R 4.0.0)
#> forcats * 0.5.0 2020-03-01 [1] CRAN (R 4.0.0)
#> foreach 1.5.0 2020-03-30 [1] CRAN (R 4.0.0)
#> fs 1.4.1 2020-04-04 [1] CRAN (R 4.0.0)
#> furrr 0.1.0 2018-05-16 [1] CRAN (R 4.0.0)
#> future 1.17.0 2020-04-18 [1] CRAN (R 4.0.0)
#> generics 0.0.2 2018-11-29 [1] CRAN (R 4.0.0)
#> ggplot2 * 3.3.0 2020-03-05 [1] CRAN (R 4.0.0)
#> globals 0.12.5 2019-12-07 [1] CRAN (R 4.0.0)
#> glue 1.4.1 2020-05-13 [1] CRAN (R 4.0.0)
#> gower 0.2.1 2019-05-14 [1] CRAN (R 4.0.0)
#> GPfit 1.0-8 2019-02-08 [1] CRAN (R 4.0.0)
#> gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.0)
#> haven 2.2.0 2019-11-08 [1] CRAN (R 4.0.0)
#> hms 0.5.3 2020-01-08 [1] CRAN (R 4.0.0)
#> htmltools 0.4.0 2019-10-04 [1] CRAN (R 4.0.0)
#> httr 1.4.1 2019-08-05 [1] CRAN (R 4.0.0)
#> hugodown 0.0.0.9000 2020-06-12 [1] Github (r-lib/hugodown@6812ada)
#> ipred 0.9-9 2019-04-28 [1] CRAN (R 4.0.0)
#> iterators 1.0.12 2019-07-26 [1] CRAN (R 4.0.0)
#> jsonlite 1.6.1 2020-02-02 [1] CRAN (R 4.0.0)
#> knitr 1.28 2020-02-06 [1] CRAN (R 4.0.0)
#> labeling 0.3 2014-08-23 [1] CRAN (R 4.0.0)
#> lattice * 0.20-41 2020-04-02 [4] CRAN (R 4.0.0)
#> lava 1.6.7 2020-03-05 [1] CRAN (R 4.0.0)
#> lhs 1.0.2 2020-04-13 [1] CRAN (R 4.0.0)
#> lifecycle 0.2.0 2020-03-06 [1] CRAN (R 4.0.0)
#> listenv 0.8.0 2019-12-05 [1] CRAN (R 4.0.0)
#> lubridate 1.7.8 2020-04-06 [1] CRAN (R 4.0.0)
#> magrittr 1.5 2014-11-22 [1] CRAN (R 4.0.0)
#> MASS 7.3-51.6 2020-04-26 [4] CRAN (R 4.0.0)
#> Matrix 1.2-18 2019-11-27 [4] CRAN (R 4.0.0)
#> memoise 1.1.0.9000 2020-05-09 [1] Github (hadley/memoise@4aefd9f)
#> ModelMetrics 1.2.2.2 2020-03-17 [1] CRAN (R 4.0.0)
#> modelr 0.1.6 2020-02-22 [1] CRAN (R 4.0.0)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.0)
#> nlme 3.1-145 2020-03-04 [4] CRAN (R 4.0.0)
#> nnet 7.3-14 2020-04-26 [4] CRAN (R 4.0.0)
#> parsnip 0.1.0 2020-04-09 [1] CRAN (R 4.0.0)
#> pillar 1.4.4 2020-05-05 [1] CRAN (R 4.0.0)
#> pkgbuild 1.0.7 2020-04-25 [1] CRAN (R 4.0.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0)
#> pkgload 1.0.2 2018-10-29 [1] CRAN (R 4.0.0)
#> plyr 1.8.6 2020-03-03 [1] CRAN (R 4.0.0)
#> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.0)
#> pROC 1.16.2 2020-03-19 [1] CRAN (R 4.0.0)
#> processx 3.4.2 2020-02-09 [1] CRAN (R 4.0.0)
#> prodlim 2019.11.13 2019-11-17 [1] CRAN (R 4.0.0)
#> ps 1.3.3 2020-05-08 [1] CRAN (R 4.0.0)
#> purrr * 0.3.4 2020-04-17 [1] CRAN (R 4.0.0)
#> R6 2.4.1 2019-11-12 [1] CRAN (R 4.0.0)
#> Rcpp 1.0.4.6 2020-04-09 [1] CRAN (R 4.0.0)
#> readr * 1.3.1 2018-12-21 [1] CRAN (R 4.0.0)
#> readxl 1.3.1 2019-03-13 [1] CRAN (R 4.0.0)
#> recipes * 0.1.10 2020-03-18 [1] CRAN (R 4.0.0)
#> remotes 2.1.1 2020-02-15 [1] CRAN (R 4.0.0)
#> reprex 0.3.0 2019-05-16 [1] CRAN (R 4.0.0)
#> reshape2 1.4.4 2020-04-09 [1] CRAN (R 4.0.0)
#> rlang 0.4.6 2020-05-02 [1] CRAN (R 4.0.0)
#> rmarkdown 2.2.3 2020-06-12 [1] Github (rstudio/rmarkdown@4ee96c8)
#> rpart 4.1-15 2019-04-12 [4] CRAN (R 4.0.0)
#> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 4.0.0)
#> rsample * 0.0.6 2020-03-31 [1] CRAN (R 4.0.0)
#> rstudioapi 0.11 2020-02-07 [1] CRAN (R 4.0.0)
#> rvest 0.3.5 2019-11-08 [1] CRAN (R 4.0.0)
#> scales 1.1.0 2019-11-18 [1] CRAN (R 4.0.0)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.0)
#> stringi 1.4.6 2020-02-17 [1] CRAN (R 4.0.0)
#> stringr * 1.4.0 2019-02-10 [1] CRAN (R 4.0.0)
#> survival 3.1-12 2020-04-10 [4] CRAN (R 4.0.0)
#> testthat 2.3.2 2020-03-02 [1] CRAN (R 4.0.0)
#> tibble * 3.0.1 2020-04-20 [1] CRAN (R 4.0.0)
#> tidyr * 1.0.2 2020-01-24 [1] CRAN (R 4.0.0)
#> tidyselect 1.0.0 2020-01-27 [1] CRAN (R 4.0.0)
#> tidyverse * 1.3.0 2019-11-21 [1] CRAN (R 4.0.0)
#> timeDate 3043.102 2018-02-21 [1] CRAN (R 4.0.0)
#> usethis 1.6.1 2020-04-29 [1] CRAN (R 4.0.0)
#> vctrs 0.3.1 2020-06-05 [1] CRAN (R 4.0.0)
#> withr 2.2.0 2020-04-20 [1] CRAN (R 4.0.0)
#> workflows 0.1.1 2020-03-17 [1] CRAN (R 4.0.0)
#> xfun 0.14 2020-05-20 [1] CRAN (R 4.0.0)
#> xml2 1.3.2 2020-04-23 [1] CRAN (R 4.0.0)
#> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0)
#>
#> [1] /home/mdneuzerling/R/x86_64-pc-linux-gnu-library/4.0
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library