recipes: A package for computing and preprocessing design matrices.

Description

The recipes package can be used to create design matrices for modeling and to conduct preprocessing of variables. It is meant to be a more extensive framework that R's formula method. Some differences between simple formula methods and recipes are that

1. Variables can have arbitrary roles in the analysis beyond predictors and outcomes.

2. A recipe consists of one or more steps that define actions on the variables.

3. Recipes can be defined sequentially using pipes as well as being modifiable and extensible.

Basic Functions

The three main functions are recipe(), prep(), and bake().

recipe() defines the operations on the data and the associated roles. Once the preprocessing steps are defined, any parameters are estimated using prep(). Once the data are ready for transformation, the bake() function applies the operations.

Step Functions

These functions are used to add new actions to the recipe and have the naming convention "step_action". For example, step_center() centers the data to have a zero mean and step_dummy() is used to create dummy variables.

Author(s)

Maintainer: Max Kuhn max@posit.co

Authors:

• Hadley Wickham hadley@posit.co

• Emil Hvitfeldt emil.hvitfeldt@posit.co

Other contributors:

• Posit Software, PBC [copyright holder, funder]

See Also

Useful links:

• https://github.com/tidymodels/recipes

• https://recipes.tidymodels.org/

• Report bugs at https://github.com/tidymodels/recipes/issues

Get types for use in recipes

Description

The .get_data_types() generic is used internally to supply types to columns used in recipes. These functions underlie the work that the user sees in selections.

Usage

.get_data_types(x)

## Default S3 method:
.get_data_types(x)

## S3 method for class 'character'
.get_data_types(x)

## S3 method for class 'ordered'
.get_data_types(x)

## S3 method for class 'factor'
.get_data_types(x)

## S3 method for class 'integer'
.get_data_types(x)

## S3 method for class 'numeric'
.get_data_types(x)

## S3 method for class 'double'
.get_data_types(x)

## S3 method for class 'Surv'
.get_data_types(x)

## S3 method for class 'logical'
.get_data_types(x)

## S3 method for class 'Date'
.get_data_types(x)

## S3 method for class 'POSIXct'
.get_data_types(x)

## S3 method for class 'list'
.get_data_types(x)

## S3 method for class 'textrecipes_tokenlist'
.get_data_types(x)

## S3 method for class 'hardhat_case_weights'
.get_data_types(x)

Arguments

Details

This function acts as an extended recipes-specific version of class(). By ignoring differences in similar types ("double" and "numeric") and allowing each element to have multiple types ("factor" returns "factor", "unordered", and "nominal", and "character" returns "string", "unordered", and "nominal") we are able to create more natural selectors such as all_nominal(), all_string() and all_integer().

The following list shows the data types for different classes, as defined by recipes. If an object has a class not supported by .get_data_types(), it will get data type "other".

• character: string, unordered, and nominal

• ordered: ordered, and nominal

• factor: factor, unordered, and nominal

• integer: integer, and numeric

• numeric: double, and numeric

• double: double, and numeric

• Surv: surv

• logical: logical

• Date: date

• POSIXct: datetime

• list: list

• textrecipes_tokenlist: tokenlist

• hardhat_case_weights: case_weights

See Also

developer_functions

Examples

data(Sacramento, package = "modeldata")
lapply(Sacramento, .get_data_types)

Estimate sparity of a recipe

Description

Estimate sparity of a recipe

Usage

.recipes_estimate_sparsity(x, ...)

Arguments

Details

Takes a untrained recipe an provides a rough estimate of the sparsity of the prepped version of the recipe.

Sampling of the input is done to avoid slowdown for larger data sets.

An estimated sparsity of the input data is calculated. Then each step where sparse = "auto" or sparse = "yes" is set, an estimate of how many predictores will be created and used to modify the estimate.

An initial sparsity of 0 will be used if a zero-row data frame is used in specification of recipe. This is likely a under-estimate of the true sparsity of the data.

Value

A recipe

Does step destroy sparsity of columns

Description

Does step destroy sparsity of columns

Usage

.recipes_preserve_sparsity(x, ...)

Arguments

Details

This function return TRUE if modifications by this step is expected to destroy sparsity of its columns. It does not know whether it will happen or not since it doesn't know if the step will select sparse columns or not.

It returns FALSE for steps that have been modified to handle sparse columns correctly.

Value

Single logical.

Toggle all auto sparse arguments

Description

Toggle all auto sparse arguments

Usage

.recipes_toggle_sparse_args(x, choice)

Arguments

Details

If a step has an argument sparse = "auto", then workflows can use this function to fill these values with the preferred action. This preferred action is calculated in workflows dependent on the model and data heuristics. Hence why it has to be passed in.

Only arguments where sparse = "auto" are affected, thus a user can set sparse = "no" and it will be respected.

Value

A recipe

Add a New Operation to the Current Recipe

Description

add_step adds a step to the last location in the recipe. add_check does the same for checks.

Usage

add_step(rec, object)

add_check(rec, object)

Arguments

Value

A updated recipe() with the new operation in the last slot.

See Also

developer_functions

Apply a trained preprocessing recipe

Description

For a recipe with at least one preprocessing operation that has been trained by prep(), apply the computations to new data.

Usage

bake(object, ...)

## S3 method for class 'recipe'
bake(object, new_data, ..., composition = "tibble")

Arguments

Details

bake() takes a trained recipe and applies its operations to a data set to create a design matrix. If you are using a recipe as a preprocessor for modeling, we highly recommend that you use a workflow() instead of manually applying a recipe (see the example in recipe()).

If the data set is not too large, time can be saved by using the retain = TRUE option of prep(). This stores the processed version of the training set. With this option set, bake(object, new_data = NULL) will return it for free.

Also, any steps with skip = TRUE will not be applied to the data when bake() is invoked with a data set in new_data. bake(object, new_data = NULL) will always have all of the steps applied.

Value

A tibble, matrix, or sparse matrix that may have different columns than the original columns in new_data.

See Also

recipe() and prep()

Examples

data(ames, package = "modeldata")

ames <- mutate(ames, Sale_Price = log10(Sale_Price))

ames_rec <-
  recipe(Sale_Price ~ ., data = ames[-(1:6), ]) %>%
  step_other(Neighborhood, threshold = 0.05) %>%
  step_dummy(all_nominal()) %>%
  step_interact(~ starts_with("Central_Air"):Year_Built) %>%
  step_ns(Longitude, Latitude, deg_free = 2) %>%
  step_zv(all_predictors()) %>%
  prep()

# return the training set (already embedded in ames_rec)
bake(ames_rec, new_data = NULL)

# apply processing to other data:
bake(ames_rec, new_data = head(ames))

# only return selected variables:
bake(ames_rec, new_data = head(ames), all_numeric_predictors())
bake(ames_rec, new_data = head(ames), starts_with(c("Longitude", "Latitude")))

Using case weights with recipes

Description

Case weights are positive numeric values that may influence how much each data point has during the preprocessing. There are a variety of situations where case weights can be used.

Details

tidymodels packages differentiate how different types of case weights should be used during the entire data analysis process, including preprocessing data, model fitting, performance calculations, etc.

The tidymodels packages require users to convert their numeric vectors to a vector class that reflects how these should be used. For example, there are some situations where the weights should not affect operations such as centering and scaling or other preprocessing operations.

The types of weights allowed in tidymodels are:

• Frequency weights via hardhat::frequency_weights()

• Importance weights via hardhat::importance_weights()

More types can be added by request.

For recipes, we distinguish between supervised and unsupervised steps. Supervised steps use the outcome in the calculations, this type of steps will use frequency and importance weights. Unsupervised steps don't use the outcome and will only use frequency weights.

There are 3 main principles about how case weights are used within recipes. First, the data set that is passed to the recipe() function should already have a case weights column in it. This column can be created beforehand using hardhat::frequency_weights() or hardhat::importance_weights(). Second, There can only be 1 case weights column in a recipe at any given time. Third, You can not modify the case weights column with most of the steps or using the update_role() and add_role() functions.

These principles ensure that you experience minimal surprises when using case weights, as the steps automatically apply case weighted operations when supported. The printing method will additionally show which steps where weighted and which steps ignored the weights because they were of an incompatible type.

See Also

frequency_weights(), importance_weights()

Helpers for steps with case weights

Description

These functions can be used to do basic calculations with or without case weights.

Usage

get_case_weights(info, .data, call = rlang::caller_env())

averages(x, wts = NULL, na_rm = TRUE)

medians(x, wts = NULL)

variances(x, wts = NULL, na_rm = TRUE)

correlations(x, wts = NULL, use = "everything", method = "pearson")

covariances(x, wts = NULL, use = "everything", method = "pearson")

pca_wts(x, wts = NULL)

are_weights_used(wts, unsupervised = FALSE)

Arguments

Details

get_case_weights() is designed for developers of recipe steps, to return a column with the role of "case weight" as a vector.

For the other functions, rows with missing case weights are removed from calculations.

For averages() and variances(), missing values in the data (not the case weights) only affect the calculations for those rows. For correlations(), the correlation matrix computation first removes rows with any missing values (equal to the "complete.obs" strategy in stats::cor()).

are_weights_used() is designed for developers of recipe steps and is used inside print method to determine how printing should be done.

See Also

developer_functions

Check variable class

Description

check_class creates a specification of a recipe check that will check if a variable is of a designated class.

Usage

check_class(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  class_nm = NULL,
  allow_additional = FALSE,
  skip = FALSE,
  class_list = NULL,
  id = rand_id("class")
)

Arguments

Details

This function can check the classes of the variables in two ways. When the class argument is provided it will check if all the variables specified are of the given class. If this argument is NULL, the check will learn the classes of each of the specified variables in prep. Both ways will break bake if the variables are not of the requested class. If a variable has multiple classes in prep, all the classes are checked. Please note that in prep the argument strings_as_factors defaults to TRUE. If the train set contains character variables the check will be break bake when strings_as_factors is TRUE.

Value

An updated version of recipe with the new check added to the sequence of any existing operations.

Tidying

When you tidy() this check, a tibble with columns terms (the selectors or variables selected) and value (the type) is returned.

Case weights

The underlying operation does not allow for case weights.

See Also

Other checks: check_cols(), check_missing(), check_new_values(), check_range()

Examples

library(dplyr)
data(Sacramento, package = "modeldata")

# Learn the classes on the train set
train <- Sacramento[1:500, ]
test <- Sacramento[501:nrow(Sacramento), ]
recipe(train, sqft ~ .) %>%
  check_class(everything()) %>%
  prep(train, strings_as_factors = FALSE) %>%
  bake(test)

# Manual specification
recipe(train, sqft ~ .) %>%
  check_class(sqft, class_nm = "integer") %>%
  check_class(city, zip, type, class_nm = "factor") %>%
  check_class(latitude, longitude, class_nm = "numeric") %>%
  prep(train, strings_as_factors = FALSE) %>%
  bake(test)

# By default only the classes that are specified
#   are allowed.
x_df <- tibble(time = c(Sys.time() - 60, Sys.time()))
x_df$time %>% class()
## Not run: 
recipe(x_df) %>%
  check_class(time, class_nm = "POSIXt") %>%
  prep(x_df) %>%
  bake_(x_df)

## End(Not run)

# Use allow_additional = TRUE if you are fine with it
recipe(x_df) %>%
  check_class(time, class_nm = "POSIXt", allow_additional = TRUE) %>%
  prep(x_df) %>%
  bake(x_df)

Check if all columns are present

Description

check_cols creates a specification of a recipe step that will check if all the columns of the training frame are present in the new data.

Usage

check_cols(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  skip = FALSE,
  id = rand_id("cols")
)

Arguments

Details

This check will break the bake function if any of the specified columns is not present in the data. If the check passes, nothing is changed to the data.

Value

An updated version of recipe with the new check added to the sequence of any existing operations.

Tidying

When you tidy() this check, a tibble with columns terms (the selectors or variables selected) and value (the type) is returned.

See Also

Other checks: check_class(), check_missing(), check_new_values(), check_range()

Examples

data(biomass, package = "modeldata")

biomass_rec <- recipe(HHV ~ ., data = biomass) %>%
  step_rm(sample, dataset) %>%
  check_cols(contains("gen")) %>%
  step_center(all_numeric_predictors())
## Not run: 
bake(biomass_rec, biomass[, c("carbon", "HHV")])

## End(Not run)

Check for missing values

Description

check_missing creates a specification of a recipe operation that will check if variables contain missing values.

Usage

check_missing(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  columns = NULL,
  skip = FALSE,
  id = rand_id("missing")
)

Arguments

Details

This check will break the bake function if any of the checked columns does contain NA values. If the check passes, nothing is changed to the data.

Value

An updated version of recipe with the new check added to the sequence of any existing operations.

tidy() results

When you tidy() this check, a tibble with column terms (the selectors or variables selected) is returned.

See Also

Other checks: check_class(), check_cols(), check_new_values(), check_range()

Examples

data(credit_data, package = "modeldata")
is.na(credit_data) %>% colSums()

# If the test passes, `new_data` is returned unaltered
recipe(credit_data) %>%
  check_missing(Age, Expenses) %>%
  prep() %>%
  bake(credit_data)

# If your training set doesn't pass, prep() will stop with an error
## Not run: 
recipe(credit_data) %>%
  check_missing(Income) %>%
  prep()

## End(Not run)

# If `new_data` contain missing values, the check will stop `bake()`

train_data <- credit_data %>% dplyr::filter(Income > 150)
test_data <- credit_data %>% dplyr::filter(Income <= 150 | is.na(Income))

rp <- recipe(train_data) %>%
  check_missing(Income) %>%
  prep()

bake(rp, train_data)
## Not run: 
bake(rp, test_data)

## End(Not run)

check that newly created variable names don't overlap

Description

check_name is to be used in the bake function to ensure that newly created variable names don't overlap with existing names. Throws an error if check fails.

Usage

check_name(
  res,
  new_data,
  object,
  newname = NULL,
  names = FALSE,
  call = caller_env()
)

Arguments

See Also

developer_functions

Check for required column at bake-time

Description

When baking a step, create an information error message when a column that is used by the step is not present in new_data.

Usage

check_new_data(req, object, new_data)

Arguments

Value

Invisible NULL. Side effects are the focus of the function.

See Also

developer_functions

Check for new values

Description

check_new_values creates a specification of a recipe operation that will check if variables contain new values.

Usage

check_new_values(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  columns = NULL,
  ignore_NA = TRUE,
  values = NULL,
  skip = FALSE,
  id = rand_id("new_values")
)

Arguments

Details

This check will break the bake function if any of the checked columns does contain values it did not contain when prep was called on the recipe. If the check passes, nothing is changed to the data.

Value

An updated version of recipe with the new check added to the sequence of any existing operations.

Tidying

When you tidy() this check, a tibble with columns terms (the selectors or variables selected) is returned.

Case weights

The underlying operation does not allow for case weights.

See Also

Other checks: check_class(), check_cols(), check_missing(), check_range()

Examples

data(credit_data, package = "modeldata")

# If the test passes, `new_data` is returned unaltered
recipe(credit_data) %>%
  check_new_values(Home) %>%
  prep() %>%
  bake(new_data = credit_data)

# If `new_data` contains values not in `x` at the [prep()] function,
# the [bake()] function will break.
## Not run: 
recipe(credit_data %>% dplyr::filter(Home != "rent")) %>%
  check_new_values(Home) %>%
  prep() %>%
  bake(new_data = credit_data)

## End(Not run)

# By default missing values are ignored, so this passes.
recipe(credit_data %>% dplyr::filter(!is.na(Home))) %>%
  check_new_values(Home) %>%
  prep() %>%
  bake(credit_data)

# Use `ignore_NA = FALSE` if you consider missing values  as a value,
# that should not occur when not observed in the train set.
## Not run: 
recipe(credit_data %>% dplyr::filter(!is.na(Home))) %>%
  check_new_values(Home, ignore_NA = FALSE) %>%
  prep() %>%
  bake(credit_data)

## End(Not run)

Check range consistency

Description

check_range creates a specification of a recipe check that will check if the range of a numeric variable changed in the new data.

Usage

check_range(
  recipe,
  ...,
  role = NA,
  skip = FALSE,
  trained = FALSE,
  slack_prop = 0.05,
  warn = FALSE,
  lower = NULL,
  upper = NULL,
  id = rand_id("range_check_")
)

Arguments

Details

The amount of slack that is allowed is determined by the slack_prop. This is a numeric of length one or two. If of length one, the same proportion will be used at both ends of the train set range. If of length two, its first value is used to compute the allowed slack at the lower end, the second to compute the allowed slack at the upper end.

Value

An updated version of recipe with the new check added to the sequence of any existing operations.

Tidying

When you tidy() this check, a tibble with columns terms (the selectors or variables selected) and value (the means) is returned.

See Also

Other checks: check_class(), check_cols(), check_missing(), check_new_values()

Examples

slack_df <- data_frame(x = 0:100)
slack_new_data <- data_frame(x = -10:110)

# this will fail the check both ends
## Not run: 
recipe(slack_df) %>%
  check_range(x) %>%
  prep() %>%
  bake(slack_new_data)

## End(Not run)

# this will fail the check only at the upper end
## Not run: 
recipe(slack_df) %>%
  check_range(x, slack_prop = c(0.1, 0.05)) %>%
  prep() %>%
  bake(slack_new_data)

## End(Not run)

# give a warning instead of an error
## Not run: 
recipe(slack_df) %>%
  check_range(x, warn = TRUE) %>%
  prep() %>%
  bake(slack_new_data)

## End(Not run)

Quantitatively check on variables

Description

This internal function is to be used in the prep function to ensure that the type of the variables matches the expectation. Throws an error if check fails.

Usage

check_type(dat, quant = TRUE, types = NULL, call = caller_env())

Arguments

Details

Using types is a more fine-tuned way to use this. function compared to using quant. types should specify all allowed types as designated by .get_data_types. Suppose you want to allow doubles, integers, characters, factors and ordered factors, then you should specify types = c("double", "integer", "string", "factor", "ordered") to get a clear error message.

See Also

developer_functions

Detect if a particular step or check is used in a recipe

Description

Detect if a particular step or check is used in a recipe

Usage

detect_step(recipe, name)

Arguments

Value

Logical indicating if recipes contains given step.

See Also

developer_functions

Examples

rec <- recipe(Species ~ ., data = iris) %>%
  step_intercept()

detect_step(rec, "intercept")

Developer functions for creating recipes steps

Description

This page provides a comprehensive list of the exported functions for creating recipes steps and guidance on how to use them.

Creating steps

add_step() and add_check() are required when creating a new step. The output of add_step() should be the return value of all steps and should have the following format:

step_example <- function(recipe,
                         ...,
                         role = NA,
                         trained = FALSE,
                         skip = FALSE,
                         id = rand_id("example")) {
  add_step(
    recipe,
    step_example_new(
      terms = enquos(...),
      role = role,
      trained = trained,
      skip = skip,
      id = id
    )
  )
}

rand_id() should be used in the arguments of step_example() to specify the argument, as we see in the above example.

recipes_pkg_check() should be used in step_example() functions together with required_pkgs() to alert users that certain other packages are required. The standard way of using this function is the following format:

recipes_pkg_check(required_pkgs.step_example())

step() and check() are used within the ⁠step_*_new()⁠ function that you use in your new step. It will be used in the following way:

step_example_new <- function(terms, role, trained, skip, id) {
  step(
    subclass = "example",
    terms = terms,
    role = role,
    trained = trained,
    skip = skip,
    id = id
  )
}

recipes_eval_select() is used within ⁠prep.step_*()⁠ functions, and are used to turn the terms object into a character vector of the selected variables.

It will most likely be used like so:

col_names <- recipes_eval_select(x$terms, training, info)

check_type() can be used within ⁠prep.step_*()⁠ functions to check that the variables passed in are the right types. We recommend that you use the types argument as it offers higher flexibility and it matches the types defined by .get_data_types(). When using types we find it better to be explicit, e.g. writing types = c("double", "integer") instead of types = "numeric", as it produces cleaner error messages.

It should be used like so:

check_type(training[, col_names], types = c("double", "integer"))

check_new_data() should be used within ⁠bake.step_*()⁠. This function is used to make check that the required columns are present in the data. It should be one of the first lines inside the function.

It should be used like so:

check_new_data(names(object$columns), object, new_data)

check_name() should be used in ⁠bake.step_*()⁠ functions for steps that add new columns to the data set. The function throws an error if the column names already exist in the data set. It should be called before adding the new columns to the data set.

get_keep_original_cols() and remove_original_cols() are used within steps with the keep_original_cols argument. get_keep_original_cols() is used in ⁠prep.step_*()⁠ functions for steps that were created before the keep_original_cols argument was added, and acts as a way to throw a warning that the user should regenerate the recipe. remove_original_cols() should be used in ⁠bake.step_*()⁠ functions to remove the original columns. It is worth noting that remove_original_cols() can remove multiple columns at once and when possible should be put outside for loops.

new_data <- remove_original_cols(new_data, object, names_of_original_cols)

recipes_remove_cols() should be used in ⁠prep.step_*()⁠ functions, and is used to remove columns from the data set, either by using the object$removals field or by using the col_names argument.

recipes_names_predictors() and recipes_names_outcomes() should be used in ⁠prep.step_*()⁠ functions, and are used to get names of predictors and outcomes.

get_case_weights() and are_weights_used() are functions that help you extract case weights and help determine if they are used or not within the step. They will typically be used within the ⁠prep.step_*()⁠ functions if the step in question supports case weights.

print_step() is used inside ⁠print.step_*()⁠ functions. This function is replacing the internally deprecated printer() function.

sel2char() is mostly used within ⁠tidy.step_*()⁠ functions to turn selections into character vectors.

names0() creates a series of num names with a common prefix. The names are numbered with leading zeros (e.g. prefix01-prefix10 instead of prefix1-prefix10). This is useful for many types of steps that produce new columns.

Interacting with recipe objects

recipes_ptype() returns the ptype, expected variables and types, that a recipe object expects at prep() and bake() time. Controlled using the stage argument. Can be used by functions that interact with recipes to verify data is correct before passing it to prep() and bake().

recipes_ptype_validate() checks that a recipe and its data are compatible using information extracted using recipes_ptype(). Can be used by functions that interact with recipes to verify data is correct before passing it to prep() and bake().

detect_step() returns a logical indicator to determine if a given step or check is included in a recipe.

fully_trained() returns a logical indicator if the recipe is fully trained. The function is_trained() can be used to check in any individual steps are trained or not.

.get_data_types() is an S3 method that is used for selections. This method can be extended to work with column types not supported by recipes.

recipes_extension_check() is recommended to be used by package authors to make sure that all steps have ⁠prep.step_*()⁠, ⁠bake.step_*()⁠, ⁠print.step_*()⁠, ⁠tidy.step_*()⁠, and ⁠required_pkgs.step_*()⁠ methods. It should be used as a test, preferably like this:

test_that("recipes_extension_check", {
  expect_snapshot(
    recipes::recipes_extension_check(
      pkg = "pkgname"
    )
  )
})

Discretize Numeric Variables

Description

discretize() converts a numeric vector into a factor with bins having approximately the same number of data points (based on a training set).

Usage

discretize(x, ...)

## Default S3 method:
discretize(x, ...)

## S3 method for class 'numeric'
discretize(
  x,
  cuts = 4,
  labels = NULL,
  prefix = "bin",
  keep_na = TRUE,
  infs = TRUE,
  min_unique = 10,
  ...
)

## S3 method for class 'discretize'
predict(object, new_data, ...)

Arguments

Details

discretize estimates the cut points from x using percentiles. For example, if cuts = 3, the function estimates the quartiles of x and uses these as the cut points. If cuts = 2, the bins are defined as being above or below the median of x.

The predict method can then be used to turn numeric vectors into factor vectors.

If keep_na = TRUE, a suffix of "_missing" is used as a factor level (see the examples below).

If infs = FALSE and a new value is greater than the largest value of x, a missing value will result.

Value

discretize returns an object of class discretize and predict.discretize returns a factor vector.

Examples

data(biomass, package = "modeldata")

biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]

median(biomass_tr$carbon)
discretize(biomass_tr$carbon, cuts = 2)
discretize(biomass_tr$carbon, cuts = 2, infs = FALSE)
discretize(biomass_tr$carbon, cuts = 2, infs = FALSE, keep_na = FALSE)
discretize(biomass_tr$carbon, cuts = 2, prefix = "maybe a bad idea to bin")

carbon_binned <- discretize(biomass_tr$carbon)
table(predict(carbon_binned, biomass_tr$carbon))

carbon_no_infs <- discretize(biomass_tr$carbon, infs = FALSE)
predict(carbon_no_infs, c(50, 100))

Helper Functions for Profile Data Sets

Description

Helper Functions for Profile Data Sets

Usage

fixed(x, pct, index, ...)

## Default S3 method:
fixed(x, pct, index, ...)

## S3 method for class 'numeric'
fixed(x, pct, index, ...)

## S3 method for class 'factor'
fixed(x, pct, index, ...)

## S3 method for class 'character'
fixed(x, pct, index, ...)

## S3 method for class 'Date'
fixed(x, pct, index, ...)

## S3 method for class 'POSIXct'
fixed(x, pct, index, ...)

prof(x, grid, ...)

## S3 method for class 'numeric'
prof(x, grid, ...)

## S3 method for class 'factor'
prof(x, grid, ...)

## S3 method for class 'character'
prof(x, grid, ...)

## S3 method for class 'Date'
prof(x, grid, ...)

## S3 method for class 'POSIXct'
prof(x, grid, ...)

Arguments

Helpers for printing step functions

Description

Helpers for printing step functions

Usage

format_ch_vec(x, sep = ", ", width = options()$width - 9)

format_selectors(x, width = options()$width - 9)

Arguments

Value

A character string

Create a formula from a prepared recipe

Description

In case a model formula is required, the formula method can be used on a recipe to show what predictors and outcome(s) could be used.

Usage

## S3 method for class 'recipe'
formula(x, ...)

Arguments

Value

A formula.

Examples

formula(recipe(Species + Sepal.Length ~ ., data = iris) %>% prep())

iris_rec <- recipe(Species ~ ., data = iris) %>%
  step_center(all_numeric()) %>%
  prep()
formula(iris_rec)

Check to see if a recipe is trained/prepared

Description

Check to see if a recipe is trained/prepared

Usage

fully_trained(x)

Arguments

Value

A logical which is true if all of the recipe steps have been run through prep. If no steps have been added to the recipe, TRUE is returned only if the recipe has been prepped.

See Also

developer_functions

Examples

rec <- recipe(Species ~ ., data = iris) %>%
  step_center(all_numeric())

rec %>% fully_trained()


rec %>%
  prep(training = iris) %>%
  fully_trained()

Get the keep_original_cols value of a recipe step

Description

Get the keep_original_cols value of a recipe step

Usage

get_keep_original_cols(object)

Arguments

Value

A logical to keep the original variables in the output

See Also

developer_functions

Role Selection

Description

has_role(), all_predictors(), and all_outcomes() can be used to select variables in a formula that have certain roles.

In most cases, the right approach for users will be use to use the predictor-specific selectors such as all_numeric_predictors() and all_nominal_predictors(). In general you should be careful about using -all_outcomes() if a ⁠*_predictors()⁠ selector would do what you want.

Similarly, has_type(), all_numeric(), all_integer(), all_double(), all_nominal(), all_ordered(), all_unordered(), all_factor(), all_string(), all_date() and all_datetime() are used to select columns based on their data type.

all_factor() captures ordered and unordered factors, all_string() captures characters, all_unordered() captures unordered factors and characters, all_ordered() captures ordered factors, all_nominal() captures characters, unordered and ordered factors.

all_integer() captures integers, all_double() captures doubles, all_numeric() captures all kinds of numeric.

all_date() captures Date() variables, all_datetime() captures POSIXct() variables.

See selections for more details.

current_info() is an internal function.

All of these functions have have limited utility outside of column selection in step functions.

Usage

has_role(match = "predictor")

has_type(match = "numeric")

all_outcomes()

all_predictors()

all_date()

all_date_predictors()

all_datetime()

all_datetime_predictors()

all_double()

all_double_predictors()

all_factor()

all_factor_predictors()

all_integer()

all_integer_predictors()

all_logical()

all_logical_predictors()

all_nominal()

all_nominal_predictors()

all_numeric()

all_numeric_predictors()

all_ordered()

all_ordered_predictors()

all_string()

all_string_predictors()

all_unordered()

all_unordered_predictors()

current_info()

Arguments

Value

Selector functions return an integer vector.

current_info() returns an environment with objects vars and data.

Examples

data(biomass, package = "modeldata")

rec <- recipe(biomass) %>%
  update_role(
    carbon, hydrogen, oxygen, nitrogen, sulfur,
    new_role = "predictor"
  ) %>%
  update_role(HHV, new_role = "outcome") %>%
  update_role(sample, new_role = "id variable") %>%
  update_role(dataset, new_role = "splitting indicator")

recipe_info <- summary(rec)
recipe_info

# Centering on all predictors except carbon
rec %>%
  step_center(all_predictors(), -carbon) %>%
  prep(training = biomass) %>%
  bake(new_data = NULL)

Extract transformed training set

Description

As of recipes version 0.1.14, juice() is superseded in favor of bake(object, new_data = NULL).

As steps are estimated by prep, these operations are applied to the training set. Rather than running bake() to duplicate this processing, this function will return variables from the processed training set.

Usage

juice(object, ..., composition = "tibble")

Arguments

Details

juice() will return the results of a recipe where all steps have been applied to the data, irrespective of the value of the step's skip argument.

juice() can only be used if a recipe was prepped with retain = TRUE. This is equivalent to bake(object, new_data = NULL) which is the preferred way to extract the transformation of the training data set.

See Also

recipe() prep() bake()

Naming Tools

Description

names0 creates a series of num names with a common prefix. The names are numbered with leading zeros (e.g. prefix01-prefix10 instead of prefix1-prefix10). dummy_names can be used for renaming unordered and ordered dummy variables (in step_dummy()).

Usage

names0(num, prefix = "x", call = rlang::caller_env())

dummy_names(var, lvl, ordinal = FALSE, sep = "_")

dummy_extract_names(var, lvl, ordinal = FALSE, sep = "_")

Arguments

Details

When using dummy_names(), factor levels that are not valid variable names (e.g. "some text with spaces") will be changed to valid names by base::make.names(); see example below. This function will also change the names of ordinal dummy variables. Instead of values such as ".L", ".Q", or "⁠^4⁠", ordinal dummy variables are given simple integer suffixes such as "⁠_1⁠", "⁠_2⁠", etc.

Value

names0 returns a character string of length num and dummy_names generates a character vector the same length as lvl.

See Also

developer_functions

Examples

names0(9, "a")
names0(10, "a")

example <- data.frame(
  x = ordered(letters[1:5]),
  y = factor(LETTERS[1:5]),
  z = factor(paste(LETTERS[1:5], 1:5))
)

dummy_names("y", levels(example$y)[-1])
dummy_names("z", levels(example$z)[-1])

after_mm <- colnames(model.matrix(~x, data = example))[-1]
after_mm
levels(example$x)

dummy_names("x", substring(after_mm, 2), ordinal = TRUE)

Estimate a preprocessing recipe

Description

For a recipe with at least one preprocessing operation, estimate the required parameters from a training set that can be later applied to other data sets.

Usage

prep(x, ...)

## S3 method for class 'recipe'
prep(
  x,
  training = NULL,
  fresh = FALSE,
  verbose = FALSE,
  retain = TRUE,
  log_changes = FALSE,
  strings_as_factors = TRUE,
  ...
)

Arguments

Details

Given a data set, this function estimates the required quantities and statistics needed by any operations. prep() returns an updated recipe with the estimates. If you are using a recipe as a preprocessor for modeling, we highly recommend that you use a workflow() instead of manually estimating a recipe (see the example in recipe()).

Note that missing data is handled in the steps; there is no global na.rm option at the recipe level or in prep().

Also, if a recipe has been trained using prep() and then steps are added, prep() will only update the new operations. If fresh = TRUE, all of the operations will be (re)estimated.

As the steps are executed, the training set is updated. For example, if the first step is to center the data and the second is to scale the data, the step for scaling is given the centered data.

Value

A recipe whose step objects have been updated with the required quantities (e.g. parameter estimates, model objects, etc). Also, the term_info object is likely to be modified as the operations are executed.

See Also

recipe() and bake()

Examples

data(ames, package = "modeldata")

library(dplyr)

ames <- mutate(ames, Sale_Price = log10(Sale_Price))

ames_rec <-
  recipe(
    Sale_Price ~ Longitude + Latitude + Neighborhood + Year_Built + Central_Air,
    data = ames
  ) %>%
  step_other(Neighborhood, threshold = 0.05) %>%
  step_dummy(all_nominal()) %>%
  step_interact(~ starts_with("Central_Air"):Year_Built) %>%
  step_ns(Longitude, Latitude, deg_free = 5)

prep(ames_rec, verbose = TRUE)

prep(ames_rec, log_changes = TRUE)

Wrapper function for preparing recipes within resampling

Description

When working with the rsample package, a simple recipe must be prepared using the prep function first. When using recipes with rsample it is helpful to have a function that can prepare a recipe across a series of split objects that are produced in this package. prepper is a wrapper function around prep that can be used to do this. See the vignette on "Recipes and rsample" for an example.

Usage

prepper(split_obj, recipe, ...)

Arguments

Details

prepper() sets the underlying prep() argument fresh to TRUE.

Print a Recipe

Description

Print a Recipe

Usage

## S3 method for class 'recipe'
print(x, form_width = 30, ...)

Arguments

Value

The original object (invisibly)

Make a random identification field for steps

Description

Make a random identification field for steps

Usage

rand_id(prefix = "step", len = 5)

Arguments

Value

A character string with the prefix and random letters separated by and underscore.

See Also

developer_functions

Create a recipe for preprocessing data

Description

A recipe is a description of the steps to be applied to a data set in order to prepare it for data analysis.

Usage

recipe(x, ...)

## Default S3 method:
recipe(x, ...)

## S3 method for class 'data.frame'
recipe(x, formula = NULL, ..., vars = NULL, roles = NULL)

## S3 method for class 'formula'
recipe(formula, data, ...)

## S3 method for class 'matrix'
recipe(x, ...)

Arguments

Details

Defining recipes

Variables in recipes can have any type of role, including outcome, predictor, observation ID, case weights, stratification variables, etc.

recipe objects can be created in several ways. If an analysis only contains outcomes and predictors, the simplest way to create one is to use a formula (e.g. y ~ x1 + x2) that does not contain inline functions such as log(x3) (see the first example below).

Alternatively, a recipe object can be created by first specifying which variables in a data set should be used and then sequentially defining their roles (see the last example). This alternative is an excellent choice when the number of variables is very high, as the formula method is memory-inefficient with many variables.

There are two different types of operations that can be sequentially added to a recipe.

• Steps can include operations like scaling a variable, creating dummy variables or interactions, and so on. More computationally complex actions such as dimension reduction or imputation can also be specified.

• Checks are operations that conduct specific tests of the data. When the test is satisfied, the data are returned without issue or modification. Otherwise, an error is thrown.

If you have defined a recipe and want to see which steps are included, use the tidy() method on the recipe object.

Note that the data passed to recipe() need not be the complete data that will be used to train the steps (by prep()). The recipe only needs to know the names and types of data that will be used. For large data sets, head() could be used to pass a smaller data set to save time and memory.

Using recipes

Once a recipe is defined, it needs to be estimated before being applied to data. Most recipe steps have specific quantities that must be calculated or estimated. For example, step_normalize() needs to compute the training set’s mean for the selected columns, while step_dummy() needs to determine the factor levels of selected columns in order to make the appropriate indicator columns.

The two most common application of recipes are modeling and stand-alone preprocessing. How the recipe is estimated depends on how it is being used.

Modeling

The best way to use use a recipe for modeling is via the workflows package. This bundles a model and preprocessor (e.g. a recipe) together and gives the user a fluent way to train the model/recipe and make predictions.

library(dplyr)
library(workflows)
library(recipes)
library(parsnip)

data(biomass, package = "modeldata")

# split data
biomass_tr <- biomass %>% filter(dataset == "Training")
biomass_te <- biomass %>% filter(dataset == "Testing")

# With only predictors and outcomes, use a formula:
rec <- recipe(HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
              data = biomass_tr)

# Now add preprocessing steps to the recipe:
sp_signed <- 
  rec %>%
  step_normalize(all_numeric_predictors()) %>%
  step_spatialsign(all_numeric_predictors())
sp_signed

## 

## -- Recipe ------------------------------------------------------------

## 

## -- Inputs

## Number of variables by role

## outcome:   1
## predictor: 5

## 

## -- Operations

## * Centering and scaling for: all_numeric_predictors()

## * Spatial sign on: all_numeric_predictors()

We can create a parsnip model, and then build a workflow with the model and recipe:

linear_mod <- linear_reg()

linear_sp_sign_wflow <- 
  workflow() %>% 
  add_model(linear_mod) %>% 
  add_recipe(sp_signed)

linear_sp_sign_wflow

## == Workflow ==========================================================
## Preprocessor: Recipe
## Model: linear_reg()
## 
## -- Preprocessor ------------------------------------------------------
## 2 Recipe Steps
## 
## * step_normalize()
## * step_spatialsign()
## 
## -- Model -------------------------------------------------------------
## Linear Regression Model Specification (regression)
## 
## Computational engine: lm

To estimate the preprocessing steps and then fit the linear model, a single call to fit() is used:

linear_sp_sign_fit <- fit(linear_sp_sign_wflow, data = biomass_tr)

When predicting, there is no need to do anything other than call predict(). This preprocesses the new data in the same manner as the training set, then gives the data to the linear model prediction code:

predict(linear_sp_sign_fit, new_data = head(biomass_te))

## # A tibble: 6 x 1
##   .pred
##   <dbl>
## 1  18.1
## 2  17.9
## 3  17.2
## 4  18.8
## 5  19.6
## 6  14.6

Stand-alone use of recipes

When using a recipe to generate data for a visualization or to troubleshoot any problems with the recipe, there are functions that can be used to estimate the recipe and apply it to new data manually.

Once a recipe has been defined, the prep() function can be used to estimate quantities required for the operations using a data set (a.k.a. the training data). prep() returns a recipe.

As an example of using PCA (perhaps to produce a plot):

# Define the recipe
pca_rec <- 
  rec %>%
  step_normalize(all_numeric_predictors()) %>%
  step_pca(all_numeric_predictors())

Now to estimate the normalization statistics and the PCA loadings:

pca_rec <- prep(pca_rec, training = biomass_tr)
pca_rec

## 

## -- Recipe ------------------------------------------------------------

## 

## -- Inputs

## Number of variables by role

## outcome:   1
## predictor: 5

## 

## -- Training information

## Training data contained 456 data points and no incomplete rows.

## 

## -- Operations

## * Centering and scaling for: carbon and hydrogen, ... | Trained

## * PCA extraction with: carbon, hydrogen, oxygen, ... | Trained

Note that the estimated recipe shows the actual column names captured by the selectors.

You can tidy.recipe() a recipe, either when it is prepped or unprepped, to learn more about its components.

tidy(pca_rec)

## # A tibble: 2 x 6
##   number operation type      trained skip  id             
##    <int> <chr>     <chr>     <lgl>   <lgl> <chr>          
## 1      1 step      normalize TRUE    FALSE normalize_AeYA4
## 2      2 step      pca       TRUE    FALSE pca_Zn1yz

You can also tidy() recipe steps with a number or id argument.

To apply the prepped recipe to a data set, the bake() function is used in the same manner that predict() would be for models. This applies the estimated steps to any data set.

bake(pca_rec, head(biomass_te))

## # A tibble: 6 x 6
##     HHV    PC1    PC2     PC3     PC4     PC5
##   <dbl>  <dbl>  <dbl>   <dbl>   <dbl>   <dbl>
## 1  18.3 0.730  -0.412 -0.495   0.333   0.253 
## 2  17.6 0.617   1.41   0.118  -0.466   0.815 
## 3  17.2 0.761   1.10  -0.0550 -0.397   0.747 
## 4  18.9 0.0400  0.950  0.158   0.405  -0.143 
## 5  20.5 0.792  -0.732  0.204   0.465  -0.148 
## 6  18.5 0.433  -0.127 -0.354  -0.0168 -0.0888

In general, the workflow interface to recipes is recommended for most applications.

Value

An object of class recipe with sub-objects:

See Also

prep() and bake()

Examples

# formula example with single outcome:
data(biomass, package = "modeldata")

# split data
biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]

# With only predictors and outcomes, use a formula
rec <- recipe(
  HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
  data = biomass_tr
)

# Now add preprocessing steps to the recipe
sp_signed <- rec %>%
  step_normalize(all_numeric_predictors()) %>%
  step_spatialsign(all_numeric_predictors())
sp_signed

# ---------------------------------------------------------------------------
# formula multivariate example:
# no need for `cbind(carbon, hydrogen)` for left-hand side

multi_y <- recipe(carbon + hydrogen ~ oxygen + nitrogen + sulfur,
  data = biomass_tr
)
multi_y <- multi_y %>%
  step_center(all_numeric_predictors()) %>%
  step_scale(all_numeric_predictors())

# ---------------------------------------------------------------------------
# example using `update_role` instead of formula:
# best choice for high-dimensional data

rec <- recipe(biomass_tr) %>%
  update_role(carbon, hydrogen, oxygen, nitrogen, sulfur,
    new_role = "predictor"
  ) %>%
  update_role(HHV, new_role = "outcome") %>%
  update_role(sample, new_role = "id variable") %>%
  update_role(dataset, new_role = "splitting indicator")
rec

Evaluate a selection with tidyselect semantics specific to recipes

Description

recipes_eval_select() is a recipes specific variant of tidyselect::eval_select() enhanced with the ability to recognize recipes selectors, such as all_numeric_predictors(). See selections for more information about the unique recipes selectors.

This is a developer tool that is only useful for creating new recipes steps.

Usage

recipes_eval_select(
  quos,
  data,
  info,
  ...,
  allow_rename = FALSE,
  check_case_weights = TRUE,
  call = caller_env()
)

Arguments

Value

A named character vector containing the evaluated selection. The names are always the same as the values, except when allow_rename = TRUE, in which case the names reflect the new names chosen by the user.

See Also

developer_functions

Examples

library(rlang)
data(scat, package = "modeldata")

rec <- recipe(Species ~ ., data = scat)

info <- summary(rec)
info

quos <- quos(all_numeric_predictors(), where(is.factor))

recipes_eval_select(quos, scat, info)

Checks that steps have all S3 methods

Description

This is a developer tool intended to help making sure all methods for each step have been created.

Usage

recipes_extension_check(
  pkg,
  exclude_steps = character(),
  exclude_methods = character()
)

Arguments

Details

It is recommended that the following test in placed in packages that add recipes steps to help keep everything up to date.

test_that("recipes_extension_check", {
  expect_snapshot(
    recipes::recipes_extension_check(
      pkg = "pkgname"
    )
  )
})

Value

cli output

See Also

developer_functions

Examples

recipes_extension_check(
  pkg = "recipes"
)

recipes_extension_check(
  pkg = "recipes",
  exclude_steps = "step_testthat_helper",
  exclude_methods = c("required_pkgs")
)

Update packages

Description

This will check to see if all required packages are installed.

Usage

recipes_pkg_check(pkg = NULL, ...)

Arguments

Value

Nothing is returned but a message is printed to the console about which packages (if any) should be installed along with code to do so.

See Also

developer_functions

Examples

## Not run: 
recipes_pkg_check("recipes")

## End(Not run)

Prototype of recipe object

Description

This helper function returns the prototype of the input data set expected by the recipe object.

Usage

recipes_ptype(x, ..., stage = "prep")

Arguments

Details

The returned ptype is a tibble of the data set that the recipe object is expecting. The specifics of which columns depend on the stage.

At prep() time, when stage = "prep", the ptype is the data passed to recipe(). The following code chunk represents a possible recipe scenario. recipes_ptype(rec_spec, stage = "prep") and recipes_ptype(rec_prep, stage = "prep") both return a ptype tibble corresponding to data_ptype. This information is used internally in prep() to verify that data_training has the right columns with the right types.

rec_spec <- recipe(outcome ~ ., data = data_ptype) %>%
  step_normalize(all_numeric_predictors()) %>%
  step_dummy(all_nominal_predictors())

rec_prep <- prep(rec_spec, training = data_training)

At bake() time, when stage = "bake", the ptype represents the data that are required for bake() to run.

data_bake <- bake(rec_prep, new_data = data_testing)

What this means in practice is that unless otherwise specified, everything but outcomes and case weights are required. These requirements can be changed with update_role_requirements(), and recipes_ptype() respects those changes.

recipes_ptype() returns NULL on recipes created prior to version 1.1.0.

Note that the order of the columns aren't guaranteed to align with data_ptype as the data internally is ordered according to roles.

Value

A zero row tibble.

See Also

developer_functions recipes_ptype_validate

Examples

training <- tibble(
  y = 1:10,
  id = 1:10,
  x1 = letters[1:10],
  x2 = factor(letters[1:10]),
  cw = hardhat::importance_weights(1:10)
)
training

rec_spec <- recipe(y ~ ., data = training)

# outcomes and case_weights are not required at bake time
recipes_ptype(rec_spec, stage = "prep")
recipes_ptype(rec_spec, stage = "bake")

rec_spec <- recipe(y ~ ., data = training) %>%
  update_role(x1, new_role = "id")

# outcomes and case_weights are not required at bake time
# "id" column is assumed to be needed
recipes_ptype(rec_spec, stage = "prep")
recipes_ptype(rec_spec, stage = "bake")

rec_spec <- recipe(y ~ ., data = training) %>%
  update_role(x1, new_role = "id") %>%
  update_role_requirements("id", bake = FALSE)

# update_role_requirements() is used to specify that "id" isn't needed
recipes_ptype(rec_spec, stage = "prep")
recipes_ptype(rec_spec, stage = "bake")

Validate prototype of recipe object

Description

This helper function validates a dataframe against the ptype of a recipe.

Usage

recipes_ptype_validate(
  x,
  new_data,
  ...,
  stage = "prep",
  call = rlang::caller_env()
)

Arguments

Value

Nothing or an error.

See Also

developer_functions recipes_ptype

Examples

rec <- recipe(mpg ~ disp, data = mtcars)

recipes_ptype_validate(rec, mtcars)

Removes columns if options apply

Description

This helper function removes columns based on character vectors.

Usage

recipes_remove_cols(new_data, object, col_names = character())

Arguments

Value

new_data with column names removed if specified by col_names or object$removals.

See Also

developer_functions

Role indicators

Description

This helper function is meant to be used in prep() methods to identify predictors and outcomes by names.

Usage

recipes_names_predictors(info)

recipes_names_outcomes(info)

Arguments

Value

Character vector of column names.

See Also

developer_functions

Objects exported from other packages

Description

These objects are imported from other packages. Follow the links below to see their documentation.

generics

required_pkgs, tidy, tunable, tune_args

hardhat

extract_fit_time, extract_parameter_dials, extract_parameter_set_dials, frequency_weights, importance_weights

magrittr

%>%

stats

update

Removes original columns if options apply

Description

This helper function should be used whenever the argument keep_original_cols is used in a function.

Usage

remove_original_cols(new_data, object, col_names)

Arguments

Value

new_data with col_names removed if get_keep_original_cols(object) == TRUE or object$preserve == TRUE.

See Also

developer_functions

S3 methods for tracking which additional packages are needed for steps.

Description

S3 methods for tracking which additional packages are needed for steps.

Usage

## S3 method for class 'step_classdist_shrunken'
required_pkgs(x, ...)

## S3 method for class 'step_depth'
required_pkgs(x, ...)

## S3 method for class 'step_ica'
required_pkgs(x, ...)

## S3 method for class 'step_isomap'
required_pkgs(x, ...)

## S3 method for class 'step_kpca'
required_pkgs(x, ...)

## S3 method for class 'step_kpca_poly'
required_pkgs(x, ...)

## S3 method for class 'step_kpca_rbf'
required_pkgs(x, ...)

## S3 method for class 'step_mutate'
required_pkgs(x, ...)

## S3 method for class 'step_nnmf'
required_pkgs(x, ...)

## S3 method for class 'step_nnmf_sparse'
required_pkgs(x, ...)

## S3 method for class 'step_pls'
required_pkgs(x, ...)

## S3 method for class 'step_poly_bernstein'
required_pkgs(x, ...)

## S3 method for class 'recipe'
required_pkgs(x, infra = TRUE, ...)

## S3 method for class 'step'
required_pkgs(x, ...)

## S3 method for class 'check'
required_pkgs(x, ...)

## S3 method for class 'step_spline_b'
required_pkgs(x, ...)

## S3 method for class 'step_spline_convex'
required_pkgs(x, ...)

## S3 method for class 'step_spline_monotone'
required_pkgs(x, ...)

## S3 method for class 'step_spline_natural'
required_pkgs(x, ...)

## S3 method for class 'step_spline_nonnegative'
required_pkgs(x, ...)

## S3 method for class 'step_window'
required_pkgs(x, ...)

Arguments

Value

A character vector

Manually alter roles

Description

update_role() alters an existing role in the recipe or assigns an initial role to variables that do not yet have a declared role.

add_role() adds an additional role to variables that already have a role in the recipe. It does not overwrite old roles, as a single variable can have multiple roles.

remove_role() eliminates a single existing role in the recipe.

Usage

add_role(recipe, ..., new_role = "predictor", new_type = NULL)

update_role(recipe, ..., new_role = "predictor", old_role = NULL)

remove_role(recipe, ..., old_role)

Arguments

Details

update_role(), add_role() and remove_role() will be applied on a recipe before any of the steps or checks, regardless of where they are located in position. This means that roles can only be changed with these three functions for columns that are already present in the original data supplied to recipe(). See the role argument in some step functions to update roles for columns created by steps.

Variables can have any arbitrary role (see the examples) but there are three special standard roles, "predictor", "outcome", and "case_weights". The first two roles are typically required when fitting a model.

update_role() should be used when a variable doesn't currently have a role in the recipe, or to replace an old_role with a new_role. add_role() only adds additional roles to variables that already have roles and will throw an error when the current role is missing (i.e. NA).

When using add_role(), if a variable is selected that already has the new_role, a warning is emitted and that variable is skipped so no duplicate roles are added.

Adding or updating roles is a useful way to group certain variables that don't fall in the standard "predictor" bucket. You can perform a step on all of the variables that have a custom role with the selector has_role().

Effects of non-standard roles

Recipes can label and retain column(s) of your data set that should not be treated as outcomes or predictors. A unique identifier column or some other ancillary data could be used to troubleshoot issues during model development but may not be either an outcome or predictor.

For example, the modeldata::biomass dataset has a column named sample with information about the specific sample type. We can change that role:

library(recipes)

data(biomass, package = "modeldata")
biomass_train <- biomass[1:100,]
biomass_test <- biomass[101:200,]

rec <- recipe(HHV ~ ., data = biomass_train) %>%
  update_role(sample, new_role = "id variable") %>%
  step_center(carbon)

rec <- prep(rec, biomass_train)

This means that sample is no longer treated as a "predictor" (the default role for columns on the right-hand side of the formula supplied to recipe()) and won't be used in model fitting or analysis, but will still be retained in the data set.

If you really aren't using sample in your recipe, we recommend that you instead remove sample from your dataset before passing it to recipe(). The reason for this is because recipes assumes that all non-standard roles are required at bake() time (or predict() time, if you are using a workflow). Since you didn't use sample in any steps of the recipe, you might think that you don't need to pass it to bake(), but this isn't true because recipes doesn't know that you didn't use it:

biomass_test$sample <- NULL

bake(rec, biomass_test)
#> Error in `bake()`:
#> x The following required columns are missing from `new_data`: `sample`.
#> i These columns have one of the following roles, which are required at `bake()`
#>   time: `id variable`.
#> i If these roles are not required at `bake()` time, use
#>   `update_role_requirements(role = "your_role", bake = FALSE)`.

As we mentioned before, the best way to avoid this issue is to not even use a role, just remove the sample column from biomass before calling recipe(). In general, predictors and non-standard roles that are supplied to recipe() should be present at both prep() and bake() time.

If you can't remove sample for some reason, then the second best way to get around this issue is to tell recipes that the "id variable" role isn't required at bake() time. You can do that by using update_role_requirements():

rec <- recipe(HHV ~ ., data = biomass_train) %>%
  update_role(sample, new_role = "id variable") %>%
  update_role_requirements("id variable", bake = FALSE) %>%
  step_center(carbon)

rec <- prep(rec, biomass_train)

# No errors!
biomass_test_baked <- bake(rec, biomass_test)

It should be very rare that you need this feature.

Value

An updated recipe object.

Examples

library(recipes)
data(biomass, package = "modeldata")

# Using the formula method, roles are created for any outcomes and predictors:
recipe(HHV ~ ., data = biomass) %>%
  summary()

# However `sample` and `dataset` aren't predictors. Since they already have
# roles, `update_role()` can be used to make changes, to any arbitrary role:
recipe(HHV ~ ., data = biomass) %>%
  update_role(sample, new_role = "id variable") %>%
  update_role(dataset, new_role = "splitting variable") %>%
  summary()

# `update_role()` cannot set a role to NA, use `remove_role()` for that
## Not run: 
recipe(HHV ~ ., data = biomass) %>%
  update_role(sample, new_role = NA_character_)

## End(Not run)

# ------------------------------------------------------------------------------

# Variables can have more than one role. `add_role()` can be used
# if the column already has at least one role:
recipe(HHV ~ ., data = biomass) %>%
  add_role(carbon, sulfur, new_role = "something") %>%
  summary()

# `update_role()` has an argument called `old_role` that is required to
# unambiguously update a role when the column currently has multiple roles.
recipe(HHV ~ ., data = biomass) %>%
  add_role(carbon, new_role = "something") %>%
  update_role(carbon, new_role = "something else", old_role = "something") %>%
  summary()

# `carbon` has two roles at the end, so the last `update_roles()` fails since
# `old_role` was not given.
## Not run: 
recipe(HHV ~ ., data = biomass) %>%
  add_role(carbon, sulfur, new_role = "something") %>%
  update_role(carbon, new_role = "something else")

## End(Not run)

# ------------------------------------------------------------------------------

# To remove a role, `remove_role()` can be used to remove a single role.
recipe(HHV ~ ., data = biomass) %>%
  add_role(carbon, new_role = "something") %>%
  remove_role(carbon, old_role = "something") %>%
  summary()

# To remove all roles, call `remove_role()` multiple times to reset to `NA`
recipe(HHV ~ ., data = biomass) %>%
  add_role(carbon, new_role = "something") %>%
  remove_role(carbon, old_role = "something") %>%
  remove_role(carbon, old_role = "predictor") %>%
  summary()

# ------------------------------------------------------------------------------

# If the formula method is not used, all columns have a missing role:
recipe(biomass) %>%
  summary()

Methods for selecting variables in step functions

Description

Tips for selecting columns in step functions.

Details

When selecting variables or model terms in step functions, dplyr-like tools are used. The selector functions can choose variables based on their name, current role, data type, or any combination of these. The selectors are passed as any other argument to the step. If the variables are explicitly named in the step function, this might look like:

  recipe( ~ ., data = USArrests) %>%
    step_pca(Murder, Assault, UrbanPop, Rape, num_comp = 3)

The first four arguments indicate which variables should be used in the PCA while the last argument is a specific argument to step_pca() about the number of components.

Note that:

1. These arguments are not evaluated until the prep function for the step is executed.

2. The dplyr-like syntax allows for negative signs to exclude variables (e.g. -Murder) and the set of selectors will processed in order.

3. A leading exclusion in these arguments (e.g. -Murder) has the effect of adding all variables to the list except the excluded variable(s), ignoring role information.

Select helpers from the tidyselect package can also be used: tidyselect::starts_with(), tidyselect::ends_with(), tidyselect::contains(), tidyselect::matches(), tidyselect::num_range(), tidyselect::everything(), tidyselect::one_of(), tidyselect::all_of(), and tidyselect::any_of()

Note that using tidyselect::everything() or any of the other tidyselect functions aren't restricted to predictors. They will thus select outcomes, ID, and predictor columns alike. This is why these functions should be used with care, and why tidyselect::everything() likely isn't what you need.

For example:

  recipe(Species ~ ., data = iris) %>%
    step_center(starts_with("Sepal"), -contains("Width"))

would only select Sepal.Length

Columns of the design matrix that may not exist when the step is coded can also be selected. For example, when using step_pca(), the number of columns created by feature extraction may not be known when subsequent steps are defined. In this case, using matches("^PC") will select all of the columns whose names start with "PC" once those columns are created.

There are sets of recipes-specific functions that can be used to select variables based on their role or type: has_role() and has_type(). For convenience, there are also functions that are more specific. The functions all_numeric() and all_nominal() select based on type, with nominal variables including both character and factor; the functions all_predictors() and all_outcomes() select based on role. The functions all_numeric_predictors() and all_nominal_predictors() select intersections of role and type. Any can be used in conjunction with the previous functions described for selecting variables using their names.

A selection like this:

  data(biomass)
  recipe(HHV ~ ., data = biomass) %>%
    step_center(all_numeric(), -all_outcomes())

is equivalent to:

  data(biomass)
  recipe(HHV ~ ., data = biomass) %>%
    step_center(all_numeric_predictors())

Both result in all the numeric predictors: carbon, hydrogen, oxygen, nitrogen, and sulfur.

If a role for a variable has not been defined, it will never be selected using role-specific selectors.

Interactions

Selectors can be used in step_interact() in similar ways but must be embedded in a model formula (as opposed to a sequence of selectors). For example, the interaction specification could be ~ starts_with("Species"):Sepal.Width. This can be useful if Species was converted to dummy variables previously using step_dummy(). The implementation of step_interact() is special, and is more restricted than the other step functions. Only the selector functions from recipes and tidyselect are allowed. User defined selector functions will not be recognized. Additionally, the tidyselect domain specific language is not recognized here, meaning that &, |, !, and - will not work.

Tips for saving recipes and filtering columns

When creating variable selections:

• If you are using column filtering steps, such as step_corr(), try to avoid hardcoding specific variable names in downstream steps in case those columns are removed by the filter. Instead, use dplyr::any_of() and dplyr::all_of().

  • dplyr::any_of() will be tolerant if a column has been removed.

  • dplyr::all_of() will fail unless all of the columns are present in the data.

• For both of these functions, if you are going to save the recipe as a binary object to use in another R session, try to avoid referring to a vector in your workspace.

  • Preferred: any_of(!!var_names)

  • Avoid: any_of(var_names)

Some examples:

some_vars <- names(mtcars)[4:6]

# No filter steps, OK for not saving the recipe
rec_1 <-
  recipe(mpg ~ ., data = mtcars) %>% 
  step_log(all_of(some_vars)) %>% 
  prep()

# No filter steps, saving the recipe
rec_2 <-
  recipe(mpg ~ ., data = mtcars) %>% 
  step_log(!!!some_vars) %>% 
  prep()

# This fails since `wt` is not in the data
try(
recipe(mpg ~ ., data = mtcars) %>% 
  step_rm(wt) %>% 
  step_log(!!!some_vars) %>% 
  prep(),
  silent = TRUE
)

# Best for filters (using any_of()) and when
# saving the recipe
rec_4 <- 
  recipe(mpg ~ ., data = mtcars) %>% 
  step_rm(wt) %>% 
  step_log(any_of(!!some_vars)) %>% 
  # equal to step_log(any_of(c("hp", "drat", "wt")))
  prep()

Using sparse data with recipes

Description

recipe(), prep(), and bake() all accept sparse tibbles from the sparsevctrs package and sparse matrices from the Matrix package. Sparse matrices are converted to sparse tibbles internally as each step expects a tibble as its input, and is expected to return a tibble as well.

Details

Several steps work with sparse data. A step can either work with sparse data, ruin sparsity, or create sparsity. The documentation for each step will indicate whether it will work with sparse data or create sparse columns. If nothing is listed it is assumed to ruin sparsity.

Sparse tibbles or data.frames will be returned from bake() if sparse columns are present in data, either from being generated in steps or because sparse data was passed into recipe(), prep(), or bake().

step sets the class of the step and check is for checks.

Description

step sets the class of the step and check is for checks.

Usage

step(subclass, ..., .prefix = "step_", call = rlang::caller_env(2))

check(subclass, ..., .prefix = "check_", call = rlang::caller_env(2))

Arguments

Value

An updated step or check with the new class.

See Also

developer_functions

Sort rows using dplyr

Description

step_arrange() creates a specification of a recipe step that will sort rows using dplyr::arrange().

Usage

step_arrange(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  inputs = NULL,
  skip = FALSE,
  id = rand_id("arrange")
)

Arguments

Details

When an object in the user's global environment is referenced in the expression defining the new variable(s), it is a good idea to use quasiquotation (e.g. ⁠!!!⁠) to embed the value of the object in the expression (to be portable between sessions). See the examples.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected

id

character, id of this step

Sparse data

This step can be applied to sparse_data such that it is preserved. Nothing needs to be done for this to happen as it is done automatically.

Case weights

The underlying operation does not allow for case weights.

See Also

Other row operation steps: step_filter(), step_impute_roll(), step_lag(), step_naomit(), step_sample(), step_shuffle(), step_slice()

Other dplyr steps: step_filter(), step_mutate(), step_mutate_at(), step_rename(), step_rename_at(), step_sample(), step_select(), step_slice()

Examples

rec <- recipe(~., data = iris) %>%
  step_arrange(desc(Sepal.Length), 1 / Petal.Length)

prepped <- prep(rec, training = iris %>% slice(1:75))
tidy(prepped, number = 1)

library(dplyr)

dplyr_train <-
  iris %>%
  as_tibble() %>%
  slice(1:75) %>%
  dplyr::arrange(desc(Sepal.Length), 1 / Petal.Length)

rec_train <- bake(prepped, new_data = NULL)
all.equal(dplyr_train, rec_train)

dplyr_test <-
  iris %>%
  as_tibble() %>%
  slice(76:150) %>%
  dplyr::arrange(desc(Sepal.Length), 1 / Petal.Length)
rec_test <- bake(prepped, iris %>% slice(76:150))
all.equal(dplyr_test, rec_test)

# When you have variables/expressions, you can create a
# list of symbols with `rlang::syms()`` and splice them in
# the call with `!!!`. See https://tidyeval.tidyverse.org

sort_vars <- c("Sepal.Length", "Petal.Length")

qq_rec <-
  recipe(~., data = iris) %>%
  # Embed the `values` object in the call using !!!
  step_arrange(!!!syms(sort_vars)) %>%
  prep(training = iris)

tidy(qq_rec, number = 1)

Impute via bagged trees

Description

Please use step_impute_bag() instead.

Usage

step_bagimpute(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  impute_with = imp_vars(all_predictors()),
  trees = 25,
  models = NULL,
  options = list(keepX = FALSE),
  seed_val = sample.int(10^4, 1),
  skip = FALSE,
  id = rand_id("impute_bag")
)

Create a factors from A dummy variable

Description

step_bin2factor() creates a specification of a recipe step that will create a two-level factor from a single dummy variable.

Usage

step_bin2factor(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  levels = c("yes", "no"),
  ref_first = TRUE,
  columns = NULL,
  skip = FALSE,
  id = rand_id("bin2factor")
)

Arguments

Details

This operation may be useful for situations where a binary piece of information may need to be represented as categorical instead of numeric. For example, naive Bayes models would do better to have factor predictors so that the binomial distribution is modeled instead of a Gaussian probability density of numeric binary data. Note that the numeric data is only verified to be numeric (and does not count levels).

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

See Also

Other dummy variable and encoding steps: step_count(), step_date(), step_dummy(), step_dummy_extract(), step_dummy_multi_choice(), step_factor2string(), step_holiday(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

data(covers, package = "modeldata")

rec <- recipe(~description, covers) %>%
  step_regex(description, pattern = "(rock|stony)", result = "rocks") %>%
  step_regex(description, pattern = "(rock|stony)", result = "more_rocks") %>%
  step_bin2factor(rocks)

tidy(rec, number = 3)

rec <- prep(rec, training = covers)
results <- bake(rec, new_data = covers)

table(results$rocks, results$more_rocks)

tidy(rec, number = 3)

Box-Cox transformation for non-negative data

Description

step_BoxCox() creates a specification of a recipe step that will transform data using a Box-Cox transformation.

Usage

step_BoxCox(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  lambdas = NULL,
  limits = c(-5, 5),
  num_unique = 5,
  skip = FALSE,
  id = rand_id("BoxCox")
)

Arguments

Details

The Box-Cox transformation, which requires a strictly positive variable, can be used to rescale a variable to be more similar to a normal distribution. In this package, the partial log-likelihood function is directly optimized within a reasonable set of transformation values (which can be changed by the user).

This transformation is typically done on the outcome variable using the residuals for a statistical model (such as ordinary least squares). Here, a simple null model (intercept only) is used to apply the transformation to the predictor variables individually. This can have the effect of making the variable distributions more symmetric.

If the transformation parameters are estimated to be very closed to the bounds, or if the optimization fails, a value of NA is used and no transformation is applied.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value , and id:

terms

character, the selectors or variables selected

value

numeric, the lambda estimate

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

References

Sakia, R. M. (1992). The Box-Cox transformation technique: A review. The Statistician, 169-178..

See Also

Other individual transformation steps: step_YeoJohnson(), step_bs(), step_harmonic(), step_hyperbolic(), step_inverse(), step_invlogit(), step_log(), step_logit(), step_mutate(), step_ns(), step_percentile(), step_poly(), step_relu(), step_sqrt()

Examples

rec <- recipe(~., data = as.data.frame(state.x77))

bc_trans <- step_BoxCox(rec, all_numeric())

bc_estimates <- prep(bc_trans, training = as.data.frame(state.x77))

bc_data <- bake(bc_estimates, as.data.frame(state.x77))

plot(density(state.x77[, "Illiteracy"]), main = "before")
plot(density(bc_data$Illiteracy), main = "after")

tidy(bc_trans, number = 1)
tidy(bc_estimates, number = 1)

B-spline basis functions

Description

step_bs() creates a specification of a recipe step that will create new columns that are basis expansions of variables using B-splines.

Usage

step_bs(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  deg_free = NULL,
  degree = 3,
  objects = NULL,
  options = list(),
  keep_original_cols = FALSE,
  skip = FALSE,
  id = rand_id("bs")
)

Arguments

Details

step_bs can create new features from a single variable that enable fitting routines to model this variable in a nonlinear manner. The extent of the possible nonlinearity is determined by the df, degree, or knots arguments of splines::bs(). The original variables are removed from the data and new columns are added. The naming convention for the new variables is varname_bs_1 and so on.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected

id

character, id of this step

Tuning Parameters

This step has 2 tuning parameters:

• deg_free: Spline Degrees of Freedom (type: integer, default: NULL)

• degree: Polynomial Degree (type: integer, default: 3)

Case weights

The underlying operation does not allow for case weights.

See Also

Other individual transformation steps: step_BoxCox(), step_YeoJohnson(), step_harmonic(), step_hyperbolic(), step_inverse(), step_invlogit(), step_log(), step_logit(), step_mutate(), step_ns(), step_percentile(), step_poly(), step_relu(), step_sqrt()

Examples

data(biomass, package = "modeldata")

biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]

rec <- recipe(
  HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
  data = biomass_tr
)

with_splines <- rec %>%
  step_bs(carbon, hydrogen)
with_splines <- prep(with_splines, training = biomass_tr)

expanded <- bake(with_splines, biomass_te)
expanded

Centering numeric data

Description

step_center() creates a specification of a recipe step that will normalize numeric data to have a mean of zero.

Usage

step_center(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  means = NULL,
  na_rm = TRUE,
  skip = FALSE,
  id = rand_id("center")
)

Arguments

Details

Centering data means that the average of a variable is subtracted from the data. step_center estimates the variable means from the data used in the training argument of prep.recipe. bake.recipe then applies the centering to new data sets using these means.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value, and id:

terms

character, the selectors or variables selected

value

numeric, the means

id

character, id of this step

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

See Also

Other normalization steps: step_normalize(), step_range(), step_scale()

Examples

data(biomass, package = "modeldata")

biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]

rec <- recipe(
  HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
  data = biomass_tr
)

center_trans <- rec %>%
  step_center(carbon, contains("gen"), -hydrogen)

center_obj <- prep(center_trans, training = biomass_tr)

transformed_te <- bake(center_obj, biomass_te)

biomass_te[1:10, names(transformed_te)]
transformed_te

tidy(center_trans, number = 1)
tidy(center_obj, number = 1)

Distances to class centroids

Description

step_classdist() creates a specification of a recipe step that will convert numeric data into Mahalanobis distance measurements to the data centroid. This is done for each value of a categorical class variable.

Usage

step_classdist(
  recipe,
  ...,
  class,
  role = "predictor",
  trained = FALSE,
  mean_func = mean,
  cov_func = cov,
  pool = FALSE,
  log = TRUE,
  objects = NULL,
  prefix = "classdist_",
  keep_original_cols = TRUE,
  skip = FALSE,
  id = rand_id("classdist")
)

Arguments

Details

step_classdist will create a new column for every unique value of the class variable. The resulting variables will not replace the original values and, by default, have the prefix classdist_. The naming format can be changed using the prefix argument.

Class-specific centroids are the multivariate averages of each predictor using the data from each class in the training set. When pre-processing a new data point, this step computes the distance from the new point to each of the class centroids. These distance features can be very effective at capturing linear class boundaries. For this reason, they can be useful to add to an existing predictor set used within a nonlinear model. If the true boundary is actually linear, the model will have an easier time learning the training data patterns.

Note that, by default, the default covariance function requires that each class should have at least as many rows as variables listed in the terms argument. If pool = TRUE, there must be at least as many data points are variables overall.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value, class , and id:

terms

character, the selectors or variables selected

value

numeric, location of centroid

class

character, name of the class

id

character, id of this step

Case weights

This step performs an supervised operation that can utilize case weights. As a result, case weights are used with frequency weights as well as importance weights. For more information,, see the documentation in case_weights and the examples on tidymodels.org.

See Also

Other multivariate transformation steps: step_classdist_shrunken(), step_depth(), step_geodist(), step_ica(), step_isomap(), step_kpca(), step_kpca_poly(), step_kpca_rbf(), step_mutate_at(), step_nnmf(), step_nnmf_sparse(), step_pca(), step_pls(), step_ratio(), step_spatialsign()

Examples

data(penguins, package = "modeldata")
penguins <- penguins[vctrs::vec_detect_complete(penguins), ]
penguins$island <- NULL
penguins$sex <- NULL

# in case of missing data...
mean2 <- function(x) mean(x, na.rm = TRUE)

# define naming convention
rec <- recipe(species ~ ., data = penguins) %>%
  step_classdist(all_numeric_predictors(),
    class = "species",
    pool = FALSE, mean_func = mean2, prefix = "centroid_"
  )

# default naming
rec <- recipe(species ~ ., data = penguins) %>%
  step_classdist(all_numeric_predictors(),
    class = "species",
    pool = FALSE, mean_func = mean2
  )

rec_dists <- prep(rec, training = penguins)

dists_to_species <- bake(rec_dists, new_data = penguins)
## on log scale:
dist_cols <- grep("classdist", names(dists_to_species), value = TRUE)
dists_to_species[, c("species", dist_cols)]

tidy(rec, number = 1)
tidy(rec_dists, number = 1)

Compute shrunken centroid distances for classification models

Description

step_classdist_shrunken creates a specification of a recipe step that will convert numeric data into Euclidean distance to the regularized class centroid. This is done for each value of a categorical class variable.

Usage

step_classdist_shrunken(
  recipe,
  ...,
  class = NULL,
  role = NA,
  trained = FALSE,
  threshold = 1/2,
  sd_offset = 1/2,
  log = TRUE,
  prefix = "classdist_",
  keep_original_cols = TRUE,
  objects = NULL,
  skip = FALSE,
  id = rand_id("classdist_shrunken")
)

Arguments

Details

Class-specific centroids are the multivariate averages of each predictor using the data from each class in the training set. When pre-processing a new data point, this step computes the distance from the new point to each of the class centroids. These distance features can be very effective at capturing linear class boundaries. For this reason, they can be useful to add to an existing predictor set used within a nonlinear model. If the true boundary is actually linear, the model will have an easier time learning the training data patterns.

Shrunken centroids use a form of regularization where the class-specific centroids are contracted to the overall class-independent centroid. If a predictor is uninformative, shrinking it may move it entirely to the overall centroid. This has the effect of removing that predictor's effect on the new distance features. However, it may not move all of the class-specific features to the center in many cases. This means that some features will only affect the classification of specific classes.

The threshold parameter can be used to optimized how much regularization should be used.

step_classdist_shrunken will create a new column for every unique value of the class variable. The resulting variables will not replace the original values and, by default, have the prefix classdist_. The naming format can be changed using the prefix argument.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value, class, type, threshold , and id:

terms

character, the selectors or variables selected

value

numeric, the centroid

class

character, name of class variable

type

character, has values "global", "by_class", and "shrunken"

threshold

numeric, value of threshold

id

character, id of this step

The first two types of centroids are in the original units while the last has been standardized.

Case weights

This step performs an supervised operation that can utilize case weights. As a result, case weights are used with frequency weights as well as importance weights. For more information,, see the documentation in case_weights and the examples on tidymodels.org.

References

Tibshirani, R., Hastie, T., Narasimhan, B., & Chu, G. (2002). Diagnosis of multiple cancer types by shrunken centroids of gene expression. Proceedings of the National Academy of Sciences, 99(10), 6567-6572.

See Also

Other multivariate transformation steps: step_classdist(), step_depth(), step_geodist(), step_ica(), step_isomap(), step_kpca(), step_kpca_poly(), step_kpca_rbf(), step_mutate_at(), step_nnmf(), step_nnmf_sparse(), step_pca(), step_pls(), step_ratio(), step_spatialsign()

Examples

data(penguins, package = "modeldata")
penguins <- penguins[vctrs::vec_detect_complete(penguins), ]
penguins$island <- NULL
penguins$sex <- NULL

# define naming convention
rec <- recipe(species ~ ., data = penguins) %>%
  step_classdist_shrunken(all_numeric_predictors(),
    class = "species",
    threshold = 1 / 4, prefix = "centroid_"
  )

# default naming
rec <- recipe(species ~ ., data = penguins) %>%
  step_classdist_shrunken(all_numeric_predictors(),
    class = "species",
    threshold = 3 / 4
  )

rec_dists <- prep(rec, training = penguins)

dists_to_species <- bake(rec_dists, new_data = penguins)
## on log scale:
dist_cols <- grep("classdist", names(dists_to_species), value = TRUE)
dists_to_species[, c("species", dist_cols)]

tidy(rec, number = 1)
tidy(rec_dists, number = 1)

High correlation filter

Description

step_corr() creates a specification of a recipe step that will potentially remove variables that have large absolute correlations with other variables.

Usage

step_corr(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  threshold = 0.9,
  use = "pairwise.complete.obs",
  method = "pearson",
  removals = NULL,
  skip = FALSE,
  id = rand_id("corr")
)

Arguments

Details

This step can potentially remove columns from the data set. This may cause issues for subsequent steps in your recipe if the missing columns are specifically referenced by name. To avoid this, see the advice in the Tips for saving recipes and filtering columns section of selections.

This step attempts to remove variables to keep the largest absolute correlation between the variables less than threshold.

When a column has a single unique value, that column will be excluded from the correlation analysis. Also, if the data set has sporadic missing values (and an inappropriate value of use is chosen), some columns will also be excluded from the filter.

The arguments use and method don't take effect if case weights are used in the recipe.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected to be removed

id

character, id of this step

Tuning Parameters

This step has 1 tuning parameters:

• threshold: Threshold (type: double, default: 0.9)

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

Author(s)

Original R code for filtering algorithm by Dong Li, modified by Max Kuhn. Contributions by Reynald Lescarbeau (for original in caret package). Max Kuhn for the step function.

See Also

Other variable filter steps: step_filter_missing(), step_lincomb(), step_nzv(), step_rm(), step_select(), step_zv()

Examples

data(biomass, package = "modeldata")

set.seed(3535)
biomass$duplicate <- biomass$carbon + rnorm(nrow(biomass))

biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]

rec <- recipe(
  HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur + duplicate,
  data = biomass_tr
)

corr_filter <- rec %>%
  step_corr(all_numeric_predictors(), threshold = .5)

filter_obj <- prep(corr_filter, training = biomass_tr)

filtered_te <- bake(filter_obj, biomass_te)
round(abs(cor(biomass_tr[, c(3:7, 9)])), 2)
round(abs(cor(filtered_te)), 2)

tidy(corr_filter, number = 1)
tidy(filter_obj, number = 1)

Create counts of patterns using regular expressions

Description

step_count() creates a specification of a recipe step that will create a variable that counts instances of a regular expression pattern in text.

Usage

step_count(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  pattern = ".",
  normalize = FALSE,
  options = list(),
  result = make.names(pattern),
  input = NULL,
  sparse = "auto",
  keep_original_cols = TRUE,
  skip = FALSE,
  id = rand_id("count")
)

Arguments

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, result , and id:

terms

character, the selectors or variables selected

result

character, the new column names

id

character, id of this step

Sparse data

This step produces sparse columns if sparse = "yes" is being set. The default value "auto" won't trigger production fo sparse columns if a recipe is prep()ed, but allows for a workflow to toggle to "yes" or "no" depending on whether the model supports sparse_data and if the model is is expected to run faster with the data.

The mechanism for determining how much sparsity is produced isn't perfect, and there will be times when you want to manually overwrite by setting sparse = "yes" or sparse = "no".

Case weights

The underlying operation does not allow for case weights.

See Also

Other dummy variable and encoding steps: step_bin2factor(), step_date(), step_dummy(), step_dummy_extract(), step_dummy_multi_choice(), step_factor2string(), step_holiday(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

data(covers, package = "modeldata")

rec <- recipe(~description, covers) %>%
  step_count(description, pattern = "(rock|stony)", result = "rocks") %>%
  step_count(description, pattern = "famil", normalize = TRUE)

rec2 <- prep(rec, training = covers)
rec2

count_values <- bake(rec2, new_data = covers)
count_values

tidy(rec, number = 1)
tidy(rec2, number = 1)

Cut a numeric variable into a factor

Description

step_cut() creates a specification of a recipe step that cuts a numeric variable into a factor based on provided boundary values.

Usage

step_cut(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  breaks,
  include_outside_range = FALSE,
  skip = FALSE,
  id = rand_id("cut")
)

Arguments

Details

Unlike the base::cut() function there is no need to specify the min and the max values in the breaks. All values before the lowest break point will end up in the first bucket, all values after the last break points will end up in the last.

step_cut() will call base::cut() in the baking step with include.lowest set to TRUE.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value , and id:

terms

character, the selectors or variables selected

value

numeric, the location of the cuts

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

See Also

Other discretization steps: step_discretize()

Examples

df <- data.frame(x = 1:10, y = 5:14)
rec <- recipe(df)

# The min and max of the variable are used as boundaries
# if they exceed the breaks
rec %>%
  step_cut(x, breaks = 5) %>%
  prep() %>%
  bake(df)

# You can use the same breaks on multiple variables
# then for each variable the boundaries are set separately
rec %>%
  step_cut(x, y, breaks = c(6, 9)) %>%
  prep() %>%
  bake(df)

# You can keep the original variables using `step_mutate` or
# `step_mutate_at`, for transforming multiple variables at once
rec %>%
  step_mutate(x_orig = x) %>%
  step_cut(x, breaks = 5) %>%
  prep() %>%
  bake(df)

# It is up to you if you want values outside the
# range learned at prep to be included
new_df <- data.frame(x = 1:11, y = 5:15)
rec %>%
  step_cut(x, breaks = 5, include_outside_range = TRUE) %>%
  prep() %>%
  bake(new_df)

rec %>%
  step_cut(x, breaks = 5, include_outside_range = FALSE) %>%
  prep() %>%
  bake(new_df)

Date feature generator

Description

step_date() creates a specification of a recipe step that will convert date data into one or more factor or numeric variables.

Usage

step_date(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  features = c("dow", "month", "year"),
  abbr = TRUE,
  label = TRUE,
  ordinal = FALSE,
  locale = clock::clock_locale()$labels,
  columns = NULL,
  keep_original_cols = TRUE,
  skip = FALSE,
  id = rand_id("date")
)

Arguments

Details

Unlike some other steps, step_date does not remove the original date variables by default. Set keep_original_cols to FALSE to remove them.

See step_time() if you want to calculate features that are smaller than days.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble with columns terms (the selectors or variables selected), value (the feature names), and ordinal (a logical) is returned.

When you tidy() this step, a tibble is returned with columns terms, value, ordinal , and id:

terms

character, the selectors or variables selected

value

character, the feature names

ordinal

logical, are factors ordered

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

See Also

Other dummy variable and encoding steps: step_bin2factor(), step_count(), step_dummy(), step_dummy_extract(), step_dummy_multi_choice(), step_factor2string(), step_holiday(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

library(lubridate)

examples <- data.frame(
  Dan = ymd("2002-03-04") + days(1:10),
  Stefan = ymd("2006-01-13") + days(1:10)
)
date_rec <- recipe(~ Dan + Stefan, examples) %>%
  step_date(all_predictors())

tidy(date_rec, number = 1)

date_rec <- prep(date_rec, training = examples)

date_values <- bake(date_rec, new_data = examples)
date_values

tidy(date_rec, number = 1)

Data depths

Description

step_depth() creates a specification of a recipe step that will convert numeric data into a measurement of data depth. This is done for each value of a categorical class variable.

Usage

step_depth(
  recipe,
  ...,
  class,
  role = "predictor",
  trained = FALSE,
  metric = "halfspace",
  options = list(),
  data = NULL,
  prefix = "depth_",
  keep_original_cols = TRUE,
  skip = FALSE,
  id = rand_id("depth")
)

Arguments

Details

Data depth metrics attempt to measure how close data a data point is to the center of its distribution. There are a number of methods for calculating depth but a simple example is the inverse of the distance of a data point to the centroid of the distribution. Generally, small values indicate that a data point not close to the centroid. step_depth can compute a class-specific depth for a new data point based on the proximity of the new value to the training set distribution.

This step requires the ddalpha package. If not installed, the step will stop with a note about installing the package.

Note that the entire training set is saved to compute future depth values. The saved data have been trained (i.e. prepared) and baked (i.e. processed) up to the point before the location that step_depth occupies in the recipe. Also, the data requirements for the different step methods may vary. For example, using metric = "Mahalanobis" requires that each class should have at least as many rows as variables listed in the terms argument.

The function will create a new column for every unique value of the class variable. The resulting variables will not replace the original values and by default have the prefix depth_. The naming format can be changed using the prefix argument.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, class , and id:

terms

character, the selectors or variables selected

class

character, name of class variable

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

See Also

Other multivariate transformation steps: step_classdist(), step_classdist_shrunken(), step_geodist(), step_ica(), step_isomap(), step_kpca(), step_kpca_poly(), step_kpca_rbf(), step_mutate_at(), step_nnmf(), step_nnmf_sparse(), step_pca(), step_pls(), step_ratio(), step_spatialsign()

Examples

# halfspace depth is the default
rec <- recipe(Species ~ ., data = iris) %>%
  step_depth(all_numeric_predictors(), class = "Species")

# use zonoid metric instead
# also, define naming convention for new columns
rec <- recipe(Species ~ ., data = iris) %>%
  step_depth(all_numeric_predictors(),
    class = "Species",
    metric = "zonoid", prefix = "zonoid_"
  )

rec_dists <- prep(rec, training = iris)

dists_to_species <- bake(rec_dists, new_data = iris)
dists_to_species

tidy(rec, number = 1)
tidy(rec_dists, number = 1)

Discretize Numeric Variables

Description

step_discretize() creates a specification of a recipe step that will convert numeric data into a factor with bins having approximately the same number of data points (based on a training set).

Usage

step_discretize(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  num_breaks = 4,
  min_unique = 10,
  objects = NULL,
  options = list(prefix = "bin"),
  skip = FALSE,
  id = rand_id("discretize")
)

Arguments

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value , and id:

terms

character, the selectors or variables selected

value

numeric, the breaks

id

character, id of this step

Tuning Parameters

This step has 2 tuning parameters:

• min_unique: Unique Value Threshold (type: integer, default: 10)

• num_breaks: Number of Cut Points (type: integer, default: 4)

Case weights

The underlying operation does not allow for case weights.

See Also

Other discretization steps: step_cut()

Examples

data(biomass, package = "modeldata")

biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]

rec <- recipe(
  HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
  data = biomass_tr
) %>%
  step_discretize(carbon, hydrogen)

rec <- prep(rec, biomass_tr)
binned_te <- bake(rec, biomass_te)
table(binned_te$carbon)

tidy(rec, 1)

Create traditional dummy variables

Description

step_dummy() creates a specification of a recipe step that will convert nominal data (e.g. factors) into one or more numeric binary model terms corresponding to the levels of the original data.

Usage

step_dummy(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  one_hot = FALSE,
  preserve = deprecated(),
  naming = dummy_names,
  levels = NULL,
  sparse = "auto",
  keep_original_cols = FALSE,
  skip = FALSE,
  id = rand_id("dummy")
)

Arguments

Details

step_dummy() will create a set of binary dummy variables from a factor variable. For example, if an unordered factor column in the data set has levels of "red", "green", "blue", the dummy variable bake will create two additional columns of 0/1 data for two of those three values (and remove the original column). For ordered factors, polynomial contrasts are used to encode the numeric values.

By default, the excluded dummy variable (i.e. the reference cell) will correspond to the first level of the unordered factor being converted. step_relevel() can be used to create a new reference level by setting the ref_level argument.

This recipe step allows for flexible naming of the resulting variables. For an unordered factor named x, with levels "a" and "b", the default naming convention would be to create a new variable called x_b. The naming format can be changed using the naming argument; the function dummy_names() is the default.

To change the type of contrast being used, change the global contrast option via options.

When the factor being converted has a missing value, all of the corresponding dummy variables are also missing. See step_unknown() for a solution.

When data to be processed contains novel levels (i.e., not contained in the training set), a missing value is assigned to the results. See step_other() for an alternative.

If no columns are selected (perhaps due to an earlier step_zv()), bake() will return the data as-is (e.g. with no dummy variables).

Note that, by default, the new dummy variable column names obey the naming rules for columns. If there are levels such as "0", dummy_names() will put a leading "X" in front of the level (since it uses make.names()). This can be changed by passing in a different function to the naming argument for this step.

Also, there are a number of contrast methods that return fractional values. The columns returned by this step are doubles (not integers) when sparse = FALSE. The columns returned when sparse = TRUE are integers.

The package vignette for dummy variables and interactions has more information.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, columns , and id:

terms

character, the selectors or variables selected

columns

character, names of resulting columns

id

character, id of this step

Sparse data

This step produces sparse columns if sparse = "yes" is being set. The default value "auto" won't trigger production fo sparse columns if a recipe is prep()ed, but allows for a workflow to toggle to "yes" or "no" depending on whether the model supports sparse_data and if the model is is expected to run faster with the data.

The mechanism for determining how much sparsity is produced isn't perfect, and there will be times when you want to manually overwrite by setting sparse = "yes" or sparse = "no".

Case weights

The underlying operation does not allow for case weights.

See Also

dummy_names()

Other dummy variable and encoding steps: step_bin2factor(), step_count(), step_date(), step_dummy_extract(), step_dummy_multi_choice(), step_factor2string(), step_holiday(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

data(Sacramento, package = "modeldata")

# Original data: city has 37 levels
length(unique(Sacramento$city))
unique(Sacramento$city) %>% sort()

rec <- recipe(~ city + sqft + price, data = Sacramento)

# Default dummy coding: 36 dummy variables
dummies <- rec %>%
  step_dummy(city) %>%
  prep(training = Sacramento)

dummy_data <- bake(dummies, new_data = NULL)

dummy_data %>%
  select(starts_with("city")) %>%
  names() # level "anything" is the reference level

# Obtain the full set of 37 dummy variables using `one_hot` option
dummies_one_hot <- rec %>%
  step_dummy(city, one_hot = TRUE) %>%
  prep(training = Sacramento)

dummy_data_one_hot <- bake(dummies_one_hot, new_data = NULL)

dummy_data_one_hot %>%
  select(starts_with("city")) %>%
  names() # no reference level


tidy(dummies, number = 1)
tidy(dummies_one_hot, number = 1)

Extract patterns from nominal data

Description

step_dummy_extract() creates a specification of a recipe step that will convert nominal data (e.g. characters or factors) into one or more integer model terms for the extracted levels.

Usage

step_dummy_extract(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  sep = NULL,
  pattern = NULL,
  threshold = 0,
  other = "other",
  naming = dummy_extract_names,
  levels = NULL,
  sparse = "auto",
  keep_original_cols = FALSE,
  skip = FALSE,
  id = rand_id("dummy_extract")
)

Arguments

Details

step_dummy_extract() will create a set of integer dummy variables from a character variable by extracting individual strings by either splitting or extracting then counting those to create count variables.

Note that threshold works in a very specific way for this step. While it is possible for one label to be present multiple times in the same row, it will only be counted once when calculating the occurrences and frequencies.

This recipe step allows for flexible naming of the resulting variables. For an unordered factor named x, with levels "a" and "b", the default naming convention would be to create a new variable called x_b. The naming format can be changed using the naming argument; the function dummy_names() is the default.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, columns , and id:

terms

character, the selectors or variables selected

columns

character, names of resulting columns

id

character, id of this step

The return value is ordered according to the frequency of columns entries in the training data set.

Sparse data

This step produces sparse columns if sparse = "yes" is being set. The default value "auto" won't trigger production fo sparse columns if a recipe is prep()ed, but allows for a workflow to toggle to "yes" or "no" depending on whether the model supports sparse_data and if the model is is expected to run faster with the data.

The mechanism for determining how much sparsity is produced isn't perfect, and there will be times when you want to manually overwrite by setting sparse = "yes" or sparse = "no".

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

See Also

dummy_extract_names()

Other dummy variable and encoding steps: step_bin2factor(), step_count(), step_date(), step_dummy(), step_dummy_multi_choice(), step_factor2string(), step_holiday(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

data(tate_text, package = "modeldata")

dummies <- recipe(~ artist + medium, data = tate_text) %>%
  step_dummy_extract(artist, medium, sep = ", ") %>%
  prep()

dummy_data <- bake(dummies, new_data = NULL)

dummy_data %>%
  select(starts_with("medium")) %>%
  names() %>%
  head()

# More detailed splitting
dummies_specific <- recipe(~medium, data = tate_text) %>%
  step_dummy_extract(medium, sep = "(, )|( and )|( on )") %>%
  prep()

dummy_data_specific <- bake(dummies_specific, new_data = NULL)

dummy_data_specific %>%
  select(starts_with("medium")) %>%
  names() %>%
  head()

tidy(dummies, number = 1)
tidy(dummies_specific, number = 1)

# pattern argument can be useful to extract harder patterns
color_examples <- tibble(
  colors = c(
    "['red', 'blue']",
    "['red', 'blue', 'white']",
    "['blue', 'blue', 'blue']"
  )
)

dummies_color <- recipe(~colors, data = color_examples) %>%
  step_dummy_extract(colors, pattern = "(?<=')[^',]+(?=')") %>%
  prep()

dummies_data_color <- dummies_color %>%
  bake(new_data = NULL)

dummies_data_color

Handle levels in multiple predictors together

Description

step_dummy_multi_choice() creates a specification of a recipe step that will convert multiple nominal data (e.g. characters or factors) into one or more numeric binary model terms for the levels of the original data.

Usage

step_dummy_multi_choice(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  threshold = 0,
  levels = NULL,
  input = NULL,
  other = "other",
  naming = dummy_names,
  prefix = NULL,
  sparse = "auto",
  keep_original_cols = FALSE,
  skip = FALSE,
  id = rand_id("dummy_multi_choice")
)

Arguments

Details

The overall proportion (or total counts) of the categories are computed. The "other" category is used in place of any categorical levels whose individual proportion (or frequency) in the training set is less than threshold.

This step produces a number of columns, based on the number of categories it finds. The naming of the columns is determined by the function based on the naming argument. The default is to return ⁠<prefix>_<category name>⁠. By default prefix is NULL, which means the name of the first column selected will be used in place.

This recipe step allows for flexible naming of the resulting variables. For an unordered factor named x, with levels "a" and "b", the default naming convention would be to create a new variable called x_b. The naming format can be changed using the naming argument; the function dummy_names() is the default.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tuning Parameters

This step has 1 tuning parameters:

• threshold: Threshold (type: double, default: 0)

Tidying

When you tidy() this step, a tibble is returned with columns terms, columns , and id:

terms

character, the selectors or variables selected

columns

character, names of resulting columns

id

character, id of this step

Sparse data

This step produces sparse columns if sparse = "yes" is being set. The default value "auto" won't trigger production fo sparse columns if a recipe is prep()ed, but allows for a workflow to toggle to "yes" or "no" depending on whether the model supports sparse_data and if the model is is expected to run faster with the data.

The mechanism for determining how much sparsity is produced isn't perfect, and there will be times when you want to manually overwrite by setting sparse = "yes" or sparse = "no".

Case weights

The underlying operation does not allow for case weights.

See Also

Other dummy variable and encoding steps: step_bin2factor(), step_count(), step_date(), step_dummy(), step_dummy_extract(), step_factor2string(), step_holiday(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

library(tibble)
languages <- tribble(
  ~lang_1,    ~lang_2,   ~lang_3,
  "English",  "Italian", NA,
  "Spanish",  NA,        "French",
  "Armenian", "English", "French",
  NA,         NA,        NA
)

dummy_multi_choice_rec <- recipe(~., data = languages) %>%
  step_dummy_multi_choice(starts_with("lang")) %>%
  prep()

bake(dummy_multi_choice_rec, new_data = NULL)
tidy(dummy_multi_choice_rec, number = 1)

dummy_multi_choice_rec2 <- recipe(~., data = languages) %>%
  step_dummy_multi_choice(starts_with("lang"),
    prefix = "lang",
    threshold = 0.2
  ) %>%
  prep()

bake(dummy_multi_choice_rec2, new_data = NULL)
tidy(dummy_multi_choice_rec2, number = 1)

Convert factors to strings

Description

step_factor2string() creates a specification of a recipe step that will convert one or more factor vectors to strings.

Usage

step_factor2string(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  columns = FALSE,
  skip = FALSE,
  id = rand_id("factor2string")
)

Arguments

Details

prep() has an option strings_as_factors that defaults to TRUE. If this step is used with the default option, the strings produced by this step will not be converted to factors.

Remember that categorical data that will be directly passed to a model should be encoded as factors. This step is helpful for ancillary columns (such as identifiers) that will not be computed on in the model.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

See Also

Other dummy variable and encoding steps: step_bin2factor(), step_count(), step_date(), step_dummy(), step_dummy_extract(), step_dummy_multi_choice(), step_holiday(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

data(Sacramento, package = "modeldata")

rec <- recipe(~ city + zip, data = Sacramento)

make_string <- rec %>%
  step_factor2string(city)

make_string <- prep(make_string,
  training = Sacramento,
  strings_as_factors = FALSE
)

make_string

# note that `city` is a string in recipe output
bake(make_string, new_data = NULL) %>% head()

# ...but remains a factor in the original data
Sacramento %>% head()

Filter rows using dplyr

Description

step_filter() creates a specification of a recipe step that will remove rows using dplyr::filter().

Usage

step_filter(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  inputs = NULL,
  skip = TRUE,
  id = rand_id("filter")
)

Arguments

Details

When an object in the user's global environment is referenced in the expression defining the new variable(s), it is a good idea to use quasiquotation (e.g. ⁠!!⁠) to embed the value of the object in the expression (to be portable between sessions). See the examples.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Row Filtering

This step can entirely remove observations (rows of data), which can have unintended and/or problematic consequences when applying the step to new data later via bake(). Consider whether skip = TRUE or skip = FALSE is more appropriate in any given use case. In most instances that affect the rows of the data being predicted, this step probably should not be applied at all; instead, execute operations like this outside and before starting a preprocessing recipe().

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected

id

character, id of this step

The expressions in terms are text representations and are not parsable.

Sparse data

This step can be applied to sparse_data such that it is preserved. Nothing needs to be done for this to happen as it is done automatically.

Case weights

The underlying operation does not allow for case weights.

See Also

Other row operation steps: step_arrange(), step_impute_roll(), step_lag(), step_naomit(), step_sample(), step_shuffle(), step_slice()

Other dplyr steps: step_arrange(), step_mutate(), step_mutate_at(), step_rename(), step_rename_at(), step_sample(), step_select(), step_slice()

Examples

rec <- recipe(~., data = iris) %>%
  step_filter(Sepal.Length > 4.5, Species == "setosa")

prepped <- prep(rec, training = iris %>% slice(1:75))

library(dplyr)

dplyr_train <-
  iris %>%
  as_tibble() %>%
  slice(1:75) %>%
  dplyr::filter(Sepal.Length > 4.5, Species == "setosa")

rec_train <- bake(prepped, new_data = NULL)
all.equal(dplyr_train, rec_train)

dplyr_test <-
  iris %>%
  as_tibble() %>%
  slice(76:150) %>%
  dplyr::filter(Sepal.Length > 4.5, Species != "setosa")
rec_test <- bake(prepped, iris %>% slice(76:150))
all.equal(dplyr_test, rec_test)

values <- c("versicolor", "virginica")

qq_rec <-
  recipe(~., data = iris) %>%
  # Embed the `values` object in the call using !!
  step_filter(Sepal.Length > 4.5, Species %in% !!values)

tidy(qq_rec, number = 1)

Missing value column filter

Description

step_filter_missing() creates a specification of a recipe step that will potentially remove variables that have too many missing values.

Usage

step_filter_missing(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  threshold = 0.1,
  removals = NULL,
  skip = FALSE,
  id = rand_id("filter_missing")
)

Arguments

Details

This step can potentially remove columns from the data set. This may cause issues for subsequent steps in your recipe if the missing columns are specifically referenced by name. To avoid this, see the advice in the Tips for saving recipes and filtering columns section of selections.

This step will remove variables if the proportion of missing values exceeds the threshold.

All variables with missing values will be removed for threshold = 0.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected

id

character, id of this step

Tuning Parameters

This step has 1 tuning parameters:

• threshold: Threshold (type: double, default: 0.1)

Sparse data

This step can be applied to sparse_data such that it is preserved. Nothing needs to be done for this to happen as it is done automatically.

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

See Also

Other variable filter steps: step_corr(), step_lincomb(), step_nzv(), step_rm(), step_select(), step_zv()

Examples

data(credit_data, package = "modeldata")

rec <- recipe(Status ~ ., data = credit_data) %>%
  step_filter_missing(all_predictors(), threshold = 0)

filter_obj <- prep(rec)

filtered_te <- bake(filter_obj, new_data = NULL)

tidy(rec, number = 1)
tidy(filter_obj, number = 1)

Distance between two locations

Description

step_geodist() creates a specification of a recipe step that will calculate the distance between points on a map to a reference location.

Usage

step_geodist(
  recipe,
  lat = NULL,
  lon = NULL,
  role = "predictor",
  trained = FALSE,
  ref_lat = NULL,
  ref_lon = NULL,
  is_lat_lon = TRUE,
  log = FALSE,
  name = "geo_dist",
  columns = NULL,
  keep_original_cols = TRUE,
  skip = FALSE,
  id = rand_id("geodist")
)

Arguments

Details

step_geodist uses the Pythagorean theorem to calculate Euclidean distances if is_lat_lon is FALSE. If is_lat_lon is TRUE, the Haversine formula is used to calculate the great-circle distance in meters.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns latitude, longitude, ref_latitude, ref_longitude, is_lat_lon, name , and id:

latitude

character, name of latitude variable

longitude

character, name of longitude variable

ref_latitude

numeric, location of latitude reference point

ref_longitude

numeric, location of longitude reference point

is_lat_lon

character, the summary function name

name

character, name of resulting variable

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

References

https://en.wikipedia.org/wiki/Haversine_formula

See Also

Other multivariate transformation steps: step_classdist(), step_classdist_shrunken(), step_depth(), step_ica(), step_isomap(), step_kpca(), step_kpca_poly(), step_kpca_rbf(), step_mutate_at(), step_nnmf(), step_nnmf_sparse(), step_pca(), step_pls(), step_ratio(), step_spatialsign()

Examples

data(Smithsonian, package = "modeldata")

# How close are the museums to Union Station?
near_station <- recipe(~., data = Smithsonian) %>%
  update_role(name, new_role = "location") %>%
  step_geodist(
    lat = latitude, lon = longitude, log = FALSE,
    ref_lat = 38.8986312, ref_lon = -77.0062457,
    is_lat_lon = TRUE
  ) %>%
  prep(training = Smithsonian)

bake(near_station, new_data = NULL) %>%
  arrange(geo_dist)

tidy(near_station, number = 1)

Add sin and cos terms for harmonic analysis

Description

step_harmonic() creates a specification of a recipe step that will add sin() and cos() terms for harmonic analysis.

Usage

step_harmonic(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  frequency = NA_real_,
  cycle_size = NA_real_,
  starting_val = NA_real_,
  keep_original_cols = FALSE,
  columns = NULL,
  skip = FALSE,
  id = rand_id("harmonic")
)

Arguments

Details

This step seeks to describe periodic components of observational data using a combination of sin and cos waves. To do this, each wave of a specified frequency is modeled using one sin and one cos term. The two terms for each frequency can then be used to estimate the amplitude and phase shift of a periodic signal in observational data. The equation relating cos waves of known frequency but unknown phase and amplitude to a sum of sin and cos terms is below:

A_j cos(\sigma_j t_i - \Phi_j) = C_j cos(\sigma_j t_i) + S_j sin(\sigma_j t_i)

Solving the equation yields C_j and S_j. the amplitude can then be obtained with:

A_j = \sqrt{C^2_j + S^2_j}

And the phase can be obtained with:

\Phi_j = \arctan{(S_j / C_j)}

where:

• \sigma_j = 2 \pi (frequency / cycle\_size))

• A_j is the amplitude of the j^{th} frequency

• \Phi_j is the phase of the j^{th} frequency

• C_j is the coefficient of the cos term for the j^{th} frequency

• S_j is the coefficient of the sin term for the j^{th} frequency

The periodic component is specified by frequency and cycle_size parameters. The cycle size relates the specified frequency to the input column(s) units. There are multiple ways to specify a wave of given frequency, for example, a POSIXct input column given a frequency of 24 and a cycle_size equal to 86400 is equivalent to a frequency of 1.0 with cycle_size equal to 3600.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tuning Parameters

This step has 1 tuning parameters:

• frequency: Harmonic Frequency (type: double, default: NA)

Tidying

When you tidy() this step, a tibble is returned with columns terms, starting_val, cycle_size, frequency, key , and id:

terms

character, the selectors or variables selected

starting_val

numeric, the starting value

cycle_size

numeric, the cycle size

frequency

numeric, the frequency

key

character, key describing the calculation

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

References

Doran, H. E., & Quilkey, J. J. (1972). Harmonic analysis of seasonal data: some important properties. American Journal of Agricultural Economics, 54, volume 4, part 1, 646-651.

Foreman, M. G. G., & Henry, R. F. (1989). The harmonic analysis of tidal model time series. Advances in water resources, 12(3), 109-120.

See Also

Other individual transformation steps: step_BoxCox(), step_YeoJohnson(), step_bs(), step_hyperbolic(), step_inverse(), step_invlogit(), step_log(), step_logit(), step_mutate(), step_ns(), step_percentile(), step_poly(), step_relu(), step_sqrt()

Examples

library(ggplot2, quietly = TRUE)
library(dplyr)

data(sunspot.year)
sunspots <-
  tibble(
    year = 1700:1988,
    n_sunspot = sunspot.year,
    type = "measured"
  ) %>%
  slice(1:75)

# sunspots period is around 11 years, sample spacing is one year
dat <- recipe(n_sunspot ~ year, data = sunspots) %>%
  step_harmonic(year, frequency = 1 / 11, cycle_size = 1) %>%
  prep() %>%
  bake(new_data = NULL)

fit <- lm(n_sunspot ~ year_sin_1 + year_cos_1, data = dat)

preds <- tibble(
  year = sunspots$year,
  n_sunspot = fit$fitted.values,
  type = "predicted"
)

bind_rows(sunspots, preds) %>%
  ggplot(aes(x = year, y = n_sunspot, color = type)) +
  geom_line()


# ------------------------------------------------------------------------------
# POSIXct example

date_time <-
  as.POSIXct(
    paste0(rep(1959:1997, each = 12), "-", rep(1:12, length(1959:1997)), "-01"),
    tz = "UTC"
  )

carbon_dioxide <- tibble(
  date_time = date_time,
  co2 = as.numeric(co2),
  type = "measured"
)

# yearly co2 fluctuations
dat <-
  recipe(co2 ~ date_time,
    data = carbon_dioxide
  ) %>%
  step_mutate(date_time_num = as.numeric(date_time)) %>%
  step_ns(date_time_num, deg_free = 3) %>%
  step_harmonic(date_time, frequency = 1, cycle_size = 86400 * 365.24) %>%
  prep() %>%
  bake(new_data = NULL)

fit <- lm(co2 ~ date_time_num_ns_1 + date_time_num_ns_2 +
  date_time_num_ns_3 + date_time_sin_1 +
  date_time_cos_1, data = dat)

preds <- tibble(
  date_time = date_time,
  co2 = fit$fitted.values,
  type = "predicted"
)

bind_rows(carbon_dioxide, preds) %>%
  ggplot(aes(x = date_time, y = co2, color = type)) +
  geom_line()

Holiday feature generator

Description

step_holiday() creates a specification of a recipe step that will convert date data into one or more binary indicator variables for common holidays.

Usage

step_holiday(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  holidays = c("LaborDay", "NewYearsDay", "ChristmasDay"),
  columns = NULL,
  sparse = "auto",
  keep_original_cols = TRUE,
  skip = FALSE,
  id = rand_id("holiday")
)

Arguments

Details

Unlike some other steps, step_holiday does not remove the original date variables by default. Set keep_original_cols to FALSE to remove them.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, holiday , and id:

terms

character, the selectors or variables selected

holiday

character, name of holidays

id

character, id of this step

Sparse data

This step produces sparse columns if sparse = "yes" is being set. The default value "auto" won't trigger production fo sparse columns if a recipe is prep()ed, but allows for a workflow to toggle to "yes" or "no" depending on whether the model supports sparse_data and if the model is is expected to run faster with the data.

The mechanism for determining how much sparsity is produced isn't perfect, and there will be times when you want to manually overwrite by setting sparse = "yes" or sparse = "no".

Case weights

The underlying operation does not allow for case weights.

See Also

timeDate::listHolidays()

Other dummy variable and encoding steps: step_bin2factor(), step_count(), step_date(), step_dummy(), step_dummy_extract(), step_dummy_multi_choice(), step_factor2string(), step_indicate_na(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

library(lubridate)

examples <- data.frame(someday = ymd("2000-12-20") + days(0:40))
holiday_rec <- recipe(~someday, examples) %>%
  step_holiday(all_predictors())

holiday_rec <- prep(holiday_rec, training = examples)
holiday_values <- bake(holiday_rec, new_data = examples)
holiday_values

Hyperbolic transformations

Description

step_hyperbolic() creates a specification of a recipe step that will transform data using a hyperbolic function.

Usage

step_hyperbolic(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  func = c("sinh", "cosh", "tanh"),
  inverse = TRUE,
  columns = NULL,
  skip = FALSE,
  id = rand_id("hyperbolic")
)

Arguments

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, inverse, func , and id:

terms

character, the selectors or variables selected

inverse

logical, is the inverse function be used

func

character, name of function. "sinh", "cosh", or "tanh"

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

See Also

Other individual transformation steps: step_BoxCox(), step_YeoJohnson(), step_bs(), step_harmonic(), step_inverse(), step_invlogit(), step_log(), step_logit(), step_mutate(), step_ns(), step_percentile(), step_poly(), step_relu(), step_sqrt()

Examples

set.seed(313)
examples <- matrix(rnorm(40), ncol = 2)
examples <- as.data.frame(examples)

rec <- recipe(~ V1 + V2, data = examples)

cos_trans <- rec %>%
  step_hyperbolic(
    all_numeric_predictors(),
    func = "cosh", inverse = FALSE
  )

cos_obj <- prep(cos_trans, training = examples)

transformed_te <- bake(cos_obj, examples)
plot(examples$V1, transformed_te$V1)

tidy(cos_trans, number = 1)
tidy(cos_obj, number = 1)

ICA signal extraction

Description

step_ica() creates a specification of a recipe step that will convert numeric data into one or more independent components.

Usage

step_ica(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  num_comp = 5,
  options = list(method = "C"),
  seed = sample.int(10000, 5),
  res = NULL,
  columns = NULL,
  prefix = "IC",
  keep_original_cols = FALSE,
  skip = FALSE,
  id = rand_id("ica")
)

Arguments

Details

Independent component analysis (ICA) is a transformation of a group of variables that produces a new set of artificial features or components. ICA assumes that the variables are mixtures of a set of distinct, non-Gaussian signals and attempts to transform the data to isolate these signals. Like PCA, the components are statistically independent from one another. This means that they can be used to combat large inter-variables correlations in a data set. Also like PCA, it is advisable to center and scale the variables prior to running ICA.

This package produces components using the "FastICA" methodology (see reference below). This step requires the dimRed and fastICA packages. If not installed, the step will stop with a note about installing these packages.

The argument num_comp controls the number of components that will be retained (the original variables that are used to derive the components are removed from the data). The new components will have names that begin with prefix and a sequence of numbers. The variable names are padded with zeros. For example, if num_comp < 10, their names will be IC1 - IC9. If num_comp = 101, the names would be IC1 - IC101.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, component, value , and id:

terms

character, the selectors or variables selected

component

character, name of component

value

numeric, the loading

id

character, id of this step

Tuning Parameters

This step has 1 tuning parameters:

• num_comp: # Components (type: integer, default: 5)

Case weights

The underlying operation does not allow for case weights.

References

Hyvarinen, A., and Oja, E. (2000). Independent component analysis: algorithms and applications. Neural Networks, 13(4-5), 411-430.

See Also

Other multivariate transformation steps: step_classdist(), step_classdist_shrunken(), step_depth(), step_geodist(), step_isomap(), step_kpca(), step_kpca_poly(), step_kpca_rbf(), step_mutate_at(), step_nnmf(), step_nnmf_sparse(), step_pca(), step_pls(), step_ratio(), step_spatialsign()

Examples

# from fastICA::fastICA
set.seed(131)
S <- matrix(runif(400), 200, 2)
A <- matrix(c(1, 1, -1, 3), 2, 2, byrow = TRUE)
X <- as.data.frame(S %*% A)

tr <- X[1:100, ]
te <- X[101:200, ]

rec <- recipe(~., data = tr)

ica_trans <- step_center(rec, V1, V2)
ica_trans <- step_scale(ica_trans, V1, V2)
ica_trans <- step_ica(ica_trans, V1, V2, num_comp = 2)

ica_estimates <- prep(ica_trans, training = tr)
ica_data <- bake(ica_estimates, te)

plot(te$V1, te$V2)
plot(ica_data$IC1, ica_data$IC2)

tidy(ica_trans, number = 3)
tidy(ica_estimates, number = 3)

Impute via bagged trees

Description

step_impute_bag() creates a specification of a recipe step that will create bagged tree models to impute missing data.

Usage

step_impute_bag(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  impute_with = imp_vars(all_predictors()),
  trees = 25,
  models = NULL,
  options = list(keepX = FALSE),
  seed_val = sample.int(10^4, 1),
  skip = FALSE,
  id = rand_id("impute_bag")
)

imp_vars(...)

Arguments

Details

For each variable requiring imputation, a bagged tree is created where the outcome is the variable of interest and the predictors are any other variables listed in the impute_with formula. One advantage to the bagged tree is that is can accept predictors that have missing values themselves. This imputation method can be used when the variable of interest (and predictors) are numeric or categorical. Imputed categorical variables will remain categorical. Also, integers will be imputed to integer too.

Note that if a variable that is to be imputed is also in impute_with, this variable will be ignored.

It is possible that missing values will still occur after imputation if a large majority (or all) of the imputing variables are also missing.

As of recipes 0.1.16, this function name changed from step_bagimpute() to step_impute_bag().

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble with columns terms (the selectors or variables selected) and model (the bagged tree object) is returned.

When you tidy() this step, a tibble is returned with columns terms, model , and id:

terms

character, the selectors or variables selected

model

list, the bagged tree object

id

character, id of this step

Tuning Parameters

This step has 1 tuning parameters:

• trees: # Trees (type: integer, default: 25)

Case weights

The underlying operation does not allow for case weights.

References

Kuhn, M. and Johnson, K. (2013). Applied Predictive Modeling. Springer Verlag.

See Also

Other imputation steps: step_impute_knn(), step_impute_linear(), step_impute_lower(), step_impute_mean(), step_impute_median(), step_impute_mode(), step_impute_roll()

Examples

data("credit_data", package = "modeldata")

## missing data per column
vapply(credit_data, function(x) mean(is.na(x)), c(num = 0))

set.seed(342)
in_training <- sample(1:nrow(credit_data), 2000)

credit_tr <- credit_data[in_training, ]
credit_te <- credit_data[-in_training, ]
missing_examples <- c(14, 394, 565)

rec <- recipe(Price ~ ., data = credit_tr)
## Not run: 
impute_rec <- rec %>%
  step_impute_bag(Status, Home, Marital, Job, Income, Assets, Debt)

imp_models <- prep(impute_rec, training = credit_tr)

imputed_te <- bake(imp_models, new_data = credit_te)

credit_te[missing_examples, ]
imputed_te[missing_examples, names(credit_te)]

tidy(impute_rec, number = 1)
tidy(imp_models, number = 1)

## Specifying which variables to imputate with

impute_rec <- rec %>%
  step_impute_bag(Status, Home, Marital, Job, Income, Assets, Debt,
    impute_with = imp_vars(Time, Age, Expenses),
    # for quick execution, nbagg lowered
    options = list(nbagg = 5, keepX = FALSE)
  )

imp_models <- prep(impute_rec, training = credit_tr)

imputed_te <- bake(imp_models, new_data = credit_te)

credit_te[missing_examples, ]
imputed_te[missing_examples, names(credit_te)]

tidy(impute_rec, number = 1)
tidy(imp_models, number = 1)

## End(Not run)

Impute via k-nearest neighbors

Description

step_impute_knn() creates a specification of a recipe step that will impute missing data using nearest neighbors.

Usage

step_impute_knn(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  neighbors = 5,
  impute_with = imp_vars(all_predictors()),
  options = list(nthread = 1, eps = 1e-08),
  ref_data = NULL,
  columns = NULL,
  skip = FALSE,
  id = rand_id("impute_knn")
)

Arguments

Details

The step uses the training set to impute any other data sets. The only distance function available is Gower's distance which can be used for mixtures of nominal and numeric data.

Once the nearest neighbors are determined, the mode is used to predictor nominal variables and the mean is used for numeric data. Note that, if the underlying data are integer, the mean will be converted to an integer too.

Note that if a variable that is to be imputed is also in impute_with, this variable will be ignored.

It is possible that missing values will still occur after imputation if a large majority (or all) of the imputing variables are also missing.

As of recipes 0.1.16, this function name changed from step_knnimpute() to step_impute_knn().

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, predictors, neighbors , and id:

terms

character, the selectors or variables selected

predictors

character, selected predictors used to impute

neighbors

integer, number of neighbors

id

character, id of this step

Tuning Parameters

This step has 1 tuning parameters:

• neighbors: # Nearest Neighbors (type: integer, default: 5)

Case weights

The underlying operation does not allow for case weights.

References

Gower, C. (1971) "A general coefficient of similarity and some of its properties," Biometrics, 857-871.

See Also

Other imputation steps: step_impute_bag(), step_impute_linear(), step_impute_lower(), step_impute_mean(), step_impute_median(), step_impute_mode(), step_impute_roll()

Examples

library(recipes)
data(biomass, package = "modeldata")

biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]
biomass_te_whole <- biomass_te

# induce some missing data at random
set.seed(9039)
carb_missing <- sample(1:nrow(biomass_te), 3)
nitro_missing <- sample(1:nrow(biomass_te), 3)

biomass_te$carbon[carb_missing] <- NA
biomass_te$nitrogen[nitro_missing] <- NA

rec <- recipe(
  HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
  data = biomass_tr
)

ratio_recipe <- rec %>%
  step_impute_knn(all_predictors(), neighbors = 3)
ratio_recipe2 <- prep(ratio_recipe, training = biomass_tr)
imputed <- bake(ratio_recipe2, biomass_te)

# how well did it work?
summary(biomass_te_whole$carbon)
cbind(
  before = biomass_te_whole$carbon[carb_missing],
  after = imputed$carbon[carb_missing]
)

summary(biomass_te_whole$nitrogen)
cbind(
  before = biomass_te_whole$nitrogen[nitro_missing],
  after = imputed$nitrogen[nitro_missing]
)

tidy(ratio_recipe, number = 1)
tidy(ratio_recipe2, number = 1)

Impute numeric variables via a linear model

Description

step_impute_linear() creates a specification of a recipe step that will create linear regression models to impute missing data.

Usage

step_impute_linear(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  impute_with = imp_vars(all_predictors()),
  models = NULL,
  skip = FALSE,
  id = rand_id("impute_linear")
)

Arguments

Details

For each variable requiring imputation, a linear model is fit where the outcome is the variable of interest and the predictors are any other variables listed in the impute_with formula. Note that if a variable that is to be imputed is also in impute_with, this variable will be ignored.

The variable(s) to be imputed must be of type numeric. The imputed values will keep the same type as their original data (i.e, model predictions are coerced to integer as needed).

Since this is a linear regression, the imputation model only uses complete cases for the training set predictors.

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, model , and id:

terms

character, the selectors or variables selected

model

list, list of fitted lm() models

id

character, id of this step

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

References

Kuhn, M. and Johnson, K. (2013). Feature Engineering and Selection https://bookdown.org/max/FES/handling-missing-data.html

See Also

Other imputation steps: step_impute_bag(), step_impute_knn(), step_impute_lower(), step_impute_mean(), step_impute_median(), step_impute_mode(), step_impute_roll()

Examples

data(ames, package = "modeldata")
set.seed(393)
ames_missing <- ames
ames_missing$Longitude[sample(1:nrow(ames), 200)] <- NA

imputed_ames <-
  recipe(Sale_Price ~ ., data = ames_missing) %>%
  step_impute_linear(
    Longitude,
    impute_with = imp_vars(Latitude, Neighborhood, MS_Zoning, Alley)
  ) %>%
  prep(ames_missing)

imputed <-
  bake(imputed_ames, new_data = ames_missing) %>%
  dplyr::rename(imputed = Longitude) %>%
  bind_cols(ames %>% dplyr::select(original = Longitude)) %>%
  bind_cols(ames_missing %>% dplyr::select(Longitude)) %>%
  dplyr::filter(is.na(Longitude))

library(ggplot2)
ggplot(imputed, aes(x = original, y = imputed)) +
  geom_abline(col = "green") +
  geom_point(alpha = .3) +
  coord_equal() +
  labs(title = "Imputed Values")

Impute numeric data below the threshold of measurement

Description

step_impute_lower() creates a specification of a recipe step designed for cases where the non-negative numeric data cannot be measured below a known value. In these cases, one method for imputing the data is to substitute the truncated value by a random uniform number between zero and the truncation point.

Usage

step_impute_lower(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  threshold = NULL,
  skip = FALSE,
  id = rand_id("impute_lower")
)

Arguments

Details

step_impute_lower estimates the variable minimums from the data used in the training argument of prep.recipe. bake.recipe then simulates a value for any data at the minimum with a random uniform value between zero and the minimum.

As of recipes 0.1.16, this function name changed from step_lowerimpute() to step_impute_lower().

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value , and id:

terms

character, the selectors or variables selected

value

numeric, the estimated value

id

character, id of this step

Case weights

The underlying operation does not allow for case weights.

See Also

Other imputation steps: step_impute_bag(), step_impute_knn(), step_impute_linear(), step_impute_mean(), step_impute_median(), step_impute_mode(), step_impute_roll()

Examples

library(recipes)
data(biomass, package = "modeldata")

## Truncate some values to emulate what a lower limit of
## the measurement system might look like

biomass$carbon <- ifelse(biomass$carbon > 40, biomass$carbon, 40)
biomass$hydrogen <- ifelse(biomass$hydrogen > 5, biomass$carbon, 5)

biomass_tr <- biomass[biomass$dataset == "Training", ]
biomass_te <- biomass[biomass$dataset == "Testing", ]

rec <- recipe(
  HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
  data = biomass_tr
)

impute_rec <- rec %>%
  step_impute_lower(carbon, hydrogen)

tidy(impute_rec, number = 1)

impute_rec <- prep(impute_rec, training = biomass_tr)

tidy(impute_rec, number = 1)

transformed_te <- bake(impute_rec, biomass_te)

plot(transformed_te$carbon, biomass_te$carbon,
  ylab = "pre-imputation", xlab = "imputed"
)

Impute numeric data using the mean

Description

step_impute_mean() creates a specification of a recipe step that will substitute missing values of numeric variables by the training set mean of those variables.

Usage

step_impute_mean(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  means = NULL,
  trim = 0,
  skip = FALSE,
  id = rand_id("impute_mean")
)

Arguments

Details

step_impute_mean estimates the variable means from the data used in the training argument of prep.recipe. bake.recipe then applies the new values to new data sets using these averages.

As of recipes 0.1.16, this function name changed from step_meanimpute() to step_impute_mean().

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value , and id:

terms

character, the selectors or variables selected

value

numeric, the mean value

id

character, id of this step

Tuning Parameters

This step has 1 tuning parameters:

• trim: Amount of Trimming (type: double, default: 0)

Sparse data

This step can be applied to sparse_data such that it is preserved. Nothing needs to be done for this to happen as it is done automatically.

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

See Also

Other imputation steps: step_impute_bag(), step_impute_knn(), step_impute_linear(), step_impute_lower(), step_impute_median(), step_impute_mode(), step_impute_roll()

Examples

data("credit_data", package = "modeldata")

## missing data per column
vapply(credit_data, function(x) mean(is.na(x)), c(num = 0))

set.seed(342)
in_training <- sample(1:nrow(credit_data), 2000)

credit_tr <- credit_data[in_training, ]
credit_te <- credit_data[-in_training, ]
missing_examples <- c(14, 394, 565)

rec <- recipe(Price ~ ., data = credit_tr)

impute_rec <- rec %>%
  step_impute_mean(Income, Assets, Debt)

imp_models <- prep(impute_rec, training = credit_tr)

imputed_te <- bake(imp_models, new_data = credit_te)

credit_te[missing_examples, ]
imputed_te[missing_examples, names(credit_te)]

tidy(impute_rec, number = 1)
tidy(imp_models, number = 1)

Impute numeric data using the median

Description

step_impute_median() creates a specification of a recipe step that will substitute missing values of numeric variables by the training set median of those variables.

Usage

step_impute_median(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  medians = NULL,
  skip = FALSE,
  id = rand_id("impute_median")
)

Arguments

Details

step_impute_median estimates the variable medians from the data used in the training argument of prep.recipe. bake.recipe then applies the new values to new data sets using these medians.

As of recipes 0.1.16, this function name changed from step_medianimpute() to step_impute_median().

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value , and id:

terms

character, the selectors or variables selected

value

numeric, the median value

id

character, id of this step

Sparse data

This step can be applied to sparse_data such that it is preserved. Nothing needs to be done for this to happen as it is done automatically.

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

See Also

Other imputation steps: step_impute_bag(), step_impute_knn(), step_impute_linear(), step_impute_lower(), step_impute_mean(), step_impute_mode(), step_impute_roll()

Examples

data("credit_data", package = "modeldata")

## missing data per column
vapply(credit_data, function(x) mean(is.na(x)), c(num = 0))

set.seed(342)
in_training <- sample(1:nrow(credit_data), 2000)

credit_tr <- credit_data[in_training, ]
credit_te <- credit_data[-in_training, ]
missing_examples <- c(14, 394, 565)

rec <- recipe(Price ~ ., data = credit_tr)

impute_rec <- rec %>%
  step_impute_median(Income, Assets, Debt)

imp_models <- prep(impute_rec, training = credit_tr)

imputed_te <- bake(imp_models, new_data = credit_te)

credit_te[missing_examples, ]
imputed_te[missing_examples, names(credit_te)]

tidy(impute_rec, number = 1)
tidy(imp_models, number = 1)

Impute nominal data using the most common value

Description

step_impute_mode() creates a specification of a recipe step that will substitute missing values of nominal variables by the training set mode of those variables.

Usage

step_impute_mode(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  modes = NULL,
  ptype = NULL,
  skip = FALSE,
  id = rand_id("impute_mode")
)

Arguments

Details

step_impute_mode estimates the variable modes from the data used in the training argument of prep.recipe. bake.recipe then applies the new values to new data sets using these values. If the training set data has more than one mode, one is selected at random.

As of recipes 0.1.16, this function name changed from step_modeimpute() to step_impute_mode().

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, value , and id:

terms

character, the selectors or variables selected

value

character, the mode value

id

character, id of this step

Case weights

This step performs an unsupervised operation that can utilize case weights. As a result, case weights are only used with frequency weights. For more information, see the documentation in case_weights and the examples on tidymodels.org.

See Also

Other imputation steps: step_impute_bag(), step_impute_knn(), step_impute_linear(), step_impute_lower(), step_impute_mean(), step_impute_median(), step_impute_roll()

Examples

data("credit_data", package = "modeldata")

## missing data per column
vapply(credit_data, function(x) mean(is.na(x)), c(num = 0))

set.seed(342)
in_training <- sample(1:nrow(credit_data), 2000)

credit_tr <- credit_data[in_training, ]
credit_te <- credit_data[-in_training, ]
missing_examples <- c(14, 394, 565)

rec <- recipe(Price ~ ., data = credit_tr)

impute_rec <- rec %>%
  step_impute_mode(Status, Home, Marital)

imp_models <- prep(impute_rec, training = credit_tr)

imputed_te <- bake(imp_models, new_data = credit_te)

table(credit_te$Home, imputed_te$Home, useNA = "always")

tidy(impute_rec, number = 1)
tidy(imp_models, number = 1)

Impute numeric data using a rolling window statistic

Description

step_impute_roll() creates a specification of a recipe step that will substitute missing values of numeric variables by the measure of location (e.g. median) within a moving window.

Usage

step_impute_roll(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  columns = NULL,
  statistic = median,
  window = 5L,
  skip = FALSE,
  id = rand_id("impute_roll")
)

Arguments

Details

On the tails, the window is shifted towards the ends. For example, for a 5-point window, the windows for the first four points are 1:5, 1:5, 1:5, and then 2:6.

When missing data are in the window, they are not passed to the function. If all of the data in the window are missing, a missing value is returned.

The statistics are calculated on the training set values before imputation. This means that if previous data within the window are missing, their imputed values are not included in the window data used for imputation. In other words, each imputation does not know anything about previous imputations in the series prior to the current point.

As of recipes 0.1.16, this function name changed from step_rollimpute() to step_impute_roll().

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms, window , and id:

terms

character, the selectors or variables selected

window

integer, window size

id

character, id of this step

Tuning Parameters

This step has 2 tuning parameters:

• statistic: Rolling Summary Statistic (type: character, default: median)

• window: Window Size (type: integer, default: 5)

Case weights

The underlying operation does not allow for case weights.

See Also

Other imputation steps: step_impute_bag(), step_impute_knn(), step_impute_linear(), step_impute_lower(), step_impute_mean(), step_impute_median(), step_impute_mode()

Other row operation steps: step_arrange(), step_filter(), step_lag(), step_naomit(), step_sample(), step_shuffle(), step_slice()

Examples

library(lubridate)

set.seed(145)
example_data <-
  data.frame(
    day = ymd("2012-06-07") + days(1:12),
    x1 = round(runif(12), 2),
    x2 = round(runif(12), 2),
    x3 = round(runif(12), 2)
  )
example_data$x1[c(1, 5, 6)] <- NA
example_data$x2[c(1:4, 10)] <- NA

library(recipes)
seven_pt <- recipe(~., data = example_data) %>%
  update_role(day, new_role = "time_index") %>%
  step_impute_roll(all_numeric_predictors(), window = 7) %>%
  prep(training = example_data)

# The training set:
bake(seven_pt, new_data = NULL)

Create missing data column indicators

Description

step_indicate_na() creates a specification of a recipe step that will create and append additional binary columns to the data set to indicate which observations are missing.

Usage

step_indicate_na(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  columns = NULL,
  prefix = "na_ind",
  sparse = "auto",
  keep_original_cols = TRUE,
  skip = FALSE,
  id = rand_id("indicate_na")
)

Arguments

Value

An updated version of recipe with the new step added to the sequence of any existing operations.

Tidying

When you tidy() this step, a tibble is returned with columns terms and id:

terms

character, the selectors or variables selected

id

character, id of this step

Sparse data

This step produces sparse columns if sparse = "yes" is being set. The default value "auto" won't trigger production fo sparse columns if a recipe is prep()ed, but allows for a workflow to toggle to "yes" or "no" depending on whether the model supports sparse_data and if the model is is expected to run faster with the data.

The mechanism for determining how much sparsity is produced isn't perfect, and there will be times when you want to manually overwrite by setting sparse = "yes" or sparse = "no".

Case weights

The underlying operation does not allow for case weights.

See Also

Other dummy variable and encoding steps: step_bin2factor(), step_count(), step_date(), step_dummy(), step_dummy_extract(), step_dummy_multi_choice(), step_factor2string(), step_holiday(), step_integer(), step_novel(), step_num2factor(), step_ordinalscore(), step_other(), step_regex(), step_relevel(), step_string2factor(), step_time(), step_unknown(), step_unorder()

Examples

data("credit_data", package = "modeldata")

## missing data per column
purrr::map_dbl(credit_data, function(x) mean(is.na(x)))

set.seed(342)
in_training <- sample(1:nrow(credit_data), 2000)

credit_tr <- credit_data[in_training, ]
credit_te <- credit_data[-in_training, ]

rec <- recipe(Price ~ ., data = credit_tr)

impute_rec <- rec %>%
  step_indicate_na(Income, Assets, Debt)

imp_models <- prep(impute_rec, training = credit_tr)

imputed_te <- bake(imp_models, new_data = credit_te)

Convert values to predefined integers

Description

step_integer() creates a specification of a recipe step that will convert new data into a set of integers based on the original data values.

Usage

step_integer(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  strict = TRUE,
  zero_based = FALSE,
  key = NULL,
  skip = FALSE,
  id = rand_id("integer")
)

Arguments