mice: Multivariate Imputation by Chained Equations

Description

The mice package implements a method to deal with missing data. The package creates multiple imputations (replacement values) for multivariate missing data. The method is based on Fully Conditional Specification, where each incomplete variable is imputed by a separate model. The MICE algorithm can impute mixes of continuous, binary, unordered categorical and ordered categorical data. In addition, MICE can impute continuous two-level data, and maintain consistency between imputations by means of passive imputation. Many diagnostic plots are implemented to inspect the quality of the imputations.

Generates Multivariate Imputations by Chained Equations (MICE)

Usage

mice(
  data,
  m = 5,
  method = NULL,
  predictorMatrix,
  ignore = NULL,
  where = NULL,
  blocks,
  visitSequence = NULL,
  formulas,
  blots = NULL,
  post = NULL,
  defaultMethod = c("pmm", "logreg", "polyreg", "polr"),
  maxit = 5,
  printFlag = TRUE,
  seed = NA,
  data.init = NULL,
  ...
)

Arguments

Details

The mice package contains functions to

• Inspect the missing data pattern

• Impute the missing data m times, resulting in m completed data sets

• Diagnose the quality of the imputed values

• Analyze each completed data set

• Pool the results of the repeated analyses

• Store and export the imputed data in various formats

• Generate simulated incomplete data

• Incorporate custom imputation methods

Generates multiple imputations for incomplete multivariate data by Gibbs sampling. Missing data can occur anywhere in the data. The algorithm imputes an incomplete column (the target column) by generating 'plausible' synthetic values given other columns in the data. Each incomplete column must act as a target column, and has its own specific set of predictors. The default set of predictors for a given target consists of all other columns in the data. For predictors that are incomplete themselves, the most recently generated imputations are used to complete the predictors prior to imputation of the target column.

A separate univariate imputation model can be specified for each column. The default imputation method depends on the measurement level of the target column. In addition to these, several other methods are provided. You can also write their own imputation functions, and call these from within the algorithm.

The data may contain categorical variables that are used in a regressions on other variables. The algorithm creates dummy variables for the categories of these variables, and imputes these from the corresponding categorical variable.

Built-in univariate imputation methods are:

These corresponding functions are coded in the mice library under names mice.impute.method, where method is a string with the name of the univariate imputation method name, for example norm. The method argument specifies the methods to be used. For the j'th column, mice() calls the first occurrence of paste('mice.impute.', method[j], sep = '') in the search path. The mechanism allows uses to write customized imputation function, mice.impute.myfunc. To call it for all columns specify method='myfunc'. To call it only for, say, column 2 specify method=c('norm','myfunc','logreg',...{}).

Skipping imputation: The user may skip imputation of a column by setting its entry to the empty method: "". For complete columns without missing data mice will automatically set the empty method. Setting t he empty method does not produce imputations for the column, so any missing cells remain NA. If column A contains NA's and is used as predictor in the imputation model for column B, then mice produces no imputations for the rows in B where A is missing. The imputed data for B may thus contain NA's. The remedy is to remove column A from the imputation model for the other columns in the data. This can be done by setting the entire column for variable A in the predictorMatrix equal to zero.

Passive imputation: mice() supports a special built-in method, called passive imputation. This method can be used to ensure that a data transform always depends on the most recently generated imputations. In some cases, an imputation model may need transformed data in addition to the original data (e.g. log, quadratic, recodes, interaction, sum scores, and so on).

Passive imputation maintains consistency among different transformations of the same data. Passive imputation is invoked if ~ is specified as the first character of the string that specifies the univariate method. mice() interprets the entire string, including the ~ character, as the formula argument in a call to model.frame(formula, data[!r[,j],]). This provides a simple mechanism for specifying deterministic dependencies among the columns. For example, suppose that the missing entries in variables data$height and data$weight are imputed. The body mass index (BMI) can be calculated within mice by specifying the string '~I(weight/height^2)' as the univariate imputation method for the target column data$bmi. Note that the ~ mechanism works only on those entries which have missing values in the target column. You should make sure that the combined observed and imputed parts of the target column make sense. An easy way to create consistency is by coding all entries in the target as NA, but for large data sets, this could be inefficient. Note that you may also need to adapt the default predictorMatrix to evade linear dependencies among the predictors that could cause errors like Error in solve.default() or Error: system is exactly singular. Though not strictly needed, it is often useful to specify visitSequence such that the column that is imputed by the ~ mechanism is visited each time after one of its predictors was visited. In that way, deterministic relation between columns will always be synchronized.

A new argument ls.meth can be parsed to the lower level .norm.draw to specify the method for generating the least squares estimates and any subsequently derived estimates. Argument ls.meth takes one of three inputs: "qr" for QR-decomposition, "svd" for singular value decomposition and "ridge" for ridge regression. ls.meth defaults to ls.meth = "qr".

Auxiliary predictors in formulas specification: For a given block, the formulas specification takes precedence over the corresponding row in the predictMatrix argument. This precedence is, however, restricted to the subset of variables specified in the terms of the block formula. Any variables not specified by formulas are imputed according to the predictMatrix specification. Variables with non-zero type values in the predictMatrix will be added as main effects to the formulas, which will act as supplementary covariates in the imputation model. It is possible to turn off this behavior by specifying the argument auxiliary = FALSE.

Value

Returns an S3 object of class mids (multiply imputed data set)

Functions

The main functions are:

Vignettes

There is a detailed series of six online vignettes that walk you through solving realistic inference problems with mice.

We suggest going through these vignettes in the following order

1. Ad hoc methods and the MICE algorithm

2. Convergence and pooling

3. Inspecting how the observed data and missingness are related

4. Passive imputation and post-processing

5. Imputing multilevel data

6. Sensitivity analysis with mice

Van Buuren, S. (2018). Boca Raton, FL.: Chapman & Hall/CRC Press. The book Flexible Imputation of Missing Data. Second Edition. contains a lot of example code.

Methodology

The mice software was published in the Journal of Statistical Software (Van Buuren and Groothuis-Oudshoorn, 2011). doi:10.18637/jss.v045.i03. The first application of the method concerned missing blood pressure data (Van Buuren et. al., 1999). The term Fully Conditional Specification was introduced in 2006 to describe a general class of methods that specify imputations model for multivariate data as a set of conditional distributions (Van Buuren et. al., 2006). Further details on mixes of variables and applications can be found in the book Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

Enhanced linear algebra

Updating the BLAS can improve speed of R, sometime considerably. The details depend on the operating system. See the discussion in the "R Installation and Administration" guide for further information.

Author(s)

Maintainer: Stef van Buuren stef.vanbuuren@tno.nl

Authors:

• Karin Groothuis-Oudshoorn c.g.m.oudshoorn@utwente.nl

Other contributors:

• Gerko Vink g.vink@uu.nl [contributor]

• Rianne Schouten R.M.Schouten@uu.nl [contributor]

• Alexander Robitzsch robitzsch@ipn.uni-kiel.de [contributor]

• Patrick Rockenschaub rockenschaub.patrick@gmail.com [contributor]

• Lisa Doove lisa.doove@ppw.kuleuven.be [contributor]

• Shahab Jolani s.jolani@maastrichtuniversity.nl [contributor]

• Margarita Moreno-Betancur margarita.moreno@mcri.edu.au [contributor]

• Ian White ian.white@ucl.ac.uk [contributor]

• Philipp Gaffert philipp.gaffert@gfk.com [contributor]

• Florian Meinfelder florian.meinfelder@uni-bamberg.de [contributor]

• Bernie Gray bfgray3@gmail.com [contributor]

• Vincent Arel-Bundock vincent.arel-bundock@umontreal.ca [contributor]

• Mingyang Cai m.cai@uu.nl [contributor]

• Thom Volker t.b.volker@uu.nl [contributor]

• Edoardo Costantini e.costantini@tilburguniversity.edu [contributor]

• Caspar van Lissa c.j.vanlissa@uu.nl [contributor]

• Hanne Oberman h.i.oberman@uu.nl [contributor]

• Stephen Wade stephematician@gmail.com [contributor]

Stef van Buuren stef.vanbuuren@tno.nl, Karin Groothuis-Oudshoorn c.g.m.oudshoorn@utwente.nl, 2000-2010, with contributions of Alexander Robitzsch, Gerko Vink, Shahab Jolani, Roel de Jong, Jason Turner, Lisa Doove, John Fox, Frank E. Harrell, and Peter Malewski.

References

van Buuren, S., Boshuizen, H.C., Knook, D.L. (1999) Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine, 18, 681–694.

van Buuren, S., Brand, J.P.L., Groothuis-Oudshoorn C.G.M., Rubin, D.B. (2006) Fully conditional specification in multivariate imputation. Journal of Statistical Computation and Simulation, 76, 12, 1049–1064.

van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1–67. doi:10.18637/jss.v045.i03

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

Van Buuren, S., Brand, J.P.L., Groothuis-Oudshoorn C.G.M., Rubin, D.B. (2006) Fully conditional specification in multivariate imputation. Journal of Statistical Computation and Simulation, 76, 12, 1049–1064.

Van Buuren, S. (2007) Multiple imputation of discrete and continuous data by fully conditional specification. Statistical Methods in Medical Research, 16, 3, 219–242.

Van Buuren, S., Boshuizen, H.C., Knook, D.L. (1999) Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine, 18, 681–694.

Brand, J.P.L. (1999) Development, implementation and evaluation of multiple imputation strategies for the statistical analysis of incomplete data sets. Dissertation. Rotterdam: Erasmus University.

See Also

mice, with.mids, pool, complete, ampute

mids, with.mids, set.seed, complete

Examples

# do default multiple imputation on a numeric matrix
imp <- mice(nhanes)
imp

# list the actual imputations for BMI
imp$imp$bmi

# first completed data matrix
complete(imp)

# imputation on mixed data with a different method per column
mice(nhanes2, meth = c("sample", "pmm", "logreg", "norm"))

## Not run: 
# example where we fit the imputation model on the train data
# and apply the model to impute the test data
set.seed(123)
ignore <- sample(c(TRUE, FALSE), size = 25, replace = TRUE, prob = c(0.3, 0.7))

# scenario 1: train and test in the same dataset
imp <- mice(nhanes2, m = 2, ignore = ignore, print = FALSE, seed = 22112)
imp.test1 <- filter(imp, ignore)
imp.test1$data
complete(imp.test1, 1)
complete(imp.test1, 2)

# scenario 2: train and test in separate datasets
traindata <- nhanes2[!ignore, ]
testdata <- nhanes2[ignore, ]
imp.train <- mice(traindata, m = 2, print = FALSE, seed = 22112)
imp.test2 <- mice.mids(imp.train, newdata = testdata)
complete(imp.test2, 1)
complete(imp.test2, 2)

## End(Not run)

Finds an imputed value from matches in the predictive metric (deprecated)

Description

This function finds matches among the observed data in the predictive mean metric. It selects the donors closest matches, randomly samples one of the donors, and returns the observed value of the match.

Usage

.pmm.match(z, yhat = yhat, y = y, donors = 5, ...)

Arguments

Details

This function is included for backward compatibility. It was used up to mice 2.21. The current mice.impute.pmm() function calls the faster C function matcher instead of .pmm.match().

Value

A scalar containing the observed value of the selected donor.

Author(s)

Stef van Buuren

References

Schenker N & Taylor JMG (1996) Partially parametric techniques for multiple imputation. Computational Statistics and Data Analysis, 22, 425-446.

Little RJA (1988) Missing-data adjustments in large surveys (with discussion). Journal of Business Economics and Statistics, 6, 287-301.

Generate missing data for simulation purposes

Description

This function generates multivariate missing data under a MCAR, MAR or MNAR missing data mechanism. Imputation of data sets containing missing values can be performed with mice.

Usage

ampute(
  data,
  prop = 0.5,
  patterns = NULL,
  freq = NULL,
  mech = "MAR",
  weights = NULL,
  std = TRUE,
  cont = TRUE,
  type = NULL,
  odds = NULL,
  bycases = TRUE,
  run = TRUE
)

Arguments

Details

This function generates missing values in complete data sets. Amputation of complete data sets is useful for the evaluation of imputation techniques, such as multiple imputation (performed with function mice in this package).

The basic strategy underlying multivariate imputation was suggested by Don Rubin during discussions in the 90's. Brand (1997) created one particular implementation, and his method found its way into the FCS paper (Van Buuren et al, 2006).

Until recently, univariate amputation procedures were used to generate missing data in complete, simulated data sets. With this approach, variables are made incomplete one variable at a time. When more than one variable needs to be amputed, the procedure is repeated multiple times.

With the univariate approach, it is difficult to relate the missingness on one variable to the missingness on another variable. A multivariate amputation procedure solves this issue and moreover, it does justice to the multivariate nature of data sets. Hence, ampute is developed to perform multivariate amputation.

The idea behind the function is the specification of several missingness patterns. Each pattern is a combination of variables with and without missing values (denoted by 0 and 1 respectively). For example, one might want to create two missingness patterns on a data set with four variables. The patterns could be something like: 0,0,1,1 and 1,0,1,0. Each combination of zeros and ones may occur.

Furthermore, the researcher specifies the proportion of missingness, either the proportion of missing cases or the proportion of missing cells, and the relative frequency each pattern occurs. Consequently, the data is split into multiple subsets, one subset per pattern. Now, each case is candidate for a certain missingness pattern, but whether the case will have missing values eventually depends on other specifications.

The first of these specifications is the missing mechanism. There are three possible mechanisms: the missingness depends completely on chance (MCAR), the missingness depends on the values of the observed variables (i.e. the variables that remain complete) (MAR) or on the values of the variables that will be made incomplete (MNAR).

When the user specifies the missingness mechanism to be "MCAR", the candidates have an equal probability of becoming incomplete. For a "MAR" or "MNAR" mechanism, weighted sum scores are calculated. These scores are a linear combination of the variables.

In order to calculate the weighted sum scores, the data is standardized. For this reason, the data has to be numeric. Second, for each case, the values in the data set are multiplied with the weights, specified by argument weights. These weighted scores will be summed, resulting in a weighted sum score for each case.

The weights may differ between patterns and they may be negative or zero as well. Naturally, in case of a MAR mechanism, the weights corresponding to the variables that will be made incomplete, have a 0. Note that this may be different for each pattern. In case of MNAR missingness, especially the weights of the variables that will be made incomplete are of importance. However, the other variables may be weighted as well.

It is the relative difference between the weights that will result in an effect in the sum scores. For example, for the first missing data pattern mentioned above, the weights for the third and fourth variables could be set to 2 and 4. However, weight values of 0.2 and 0.4 will have the exact same effect on the weighted sum score: the fourth variable is weighted twice as much as variable 3.

Based on the weighted sum scores, either a discrete or continuous distribution of probabilities is used to calculate whether a candidate will have missing values.

For a discrete distribution of probabilities, the weighted sum scores are divided into subgroups of equal size (quantiles). Thereafter, the user specifies for each subgroup the odds of being missing. Both the number of subgroups and the odds values are important for the generation of missing data. For example, for a RIGHT-like mechanism, scoring in one of the higher quantiles should have high missingness odds, whereas for a MID-like mechanism, the central groups should have higher odds. Again, not the size of the odds values are of importance, but the relative distance between the values.

The continuous distributions of probabilities are based on the logistic distribution function. The user can specify the type of missingness, which, again, may differ between patterns.

For an example and more explanation about how the arguments interact with each other, we refer to the vignette: Generate missing values with ampute.

Value

Returns an S3 object of class mads (multivariate amputed data set)

Author(s)

Rianne Schouten [aut, cre], Gerko Vink [aut], Peter Lugtig [ctb], 2016

References

Brand, J.P.L. (1999) Development, implementation and evaluation of multiple imputation strategies for the statistical analysis of incomplete data sets. pp. 110-113. Dissertation. Rotterdam: Erasmus University.

Schouten, R.M., Lugtig, P and Vink, G. (2018) Generating missing values for simulation purposes: A multivariate amputation procedure. Journal of Statistical Computation and Simulation, 88(15): 1909-1930. doi:10.1080/00949655.2018.1491577

Schouten, R.M. and Vink, G. (2018) The Dance of the Mechanisms: How Observed Information Influences the Validity of Missingness Assumptions. Sociological Methods and Research, 50(3): 1243-1258. doi:10.1177/0049124118799376

Van Buuren, S., Brand, J.P.L., Groothuis-Oudshoorn, C.G.M., Rubin, D.B. (2006) Fully conditional specification in multivariate imputation. Journal of Statistical Computation and Simulation, 76(12): 1049-1064. doi:10.1080/10629360600810434

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

Vink, G. (2016) Towards a standardized evaluation of multiple imputation routines.

See Also

mads, bwplot.mads, xyplot.mads

Examples

# start with a complete data set
compl_boys <- cc(boys)[1:3]

# Perform amputation with default settings
mads_boys <- ampute(data = compl_boys)
mads_boys$amp

# Change default matrices as desired
my_patterns <- mads_boys$patterns
my_patterns[1:3, 2] <- 0

my_weights <- mads_boys$weights
my_weights[2, 1] <- 2
my_weights[3, 1] <- 0.5

# Rerun amputation
my_mads_boys <- ampute(
  data = compl_boys, patterns = my_patterns, freq =
    c(0.3, 0.3, 0.4), weights = my_weights, type = c("RIGHT", "TAIL", "LEFT")
)
my_mads_boys$amp

Multivariate amputation based on continuous probability functions

Description

This function creates a missing data indicator for each pattern. The continuous probability distributions (Van Buuren, 2012, pp. 63, 64) will be induced on the weighted sum scores, calculated earlier in the multivariate amputation function ampute.

Usage

ampute.continuous(P, scores, prop, type)

Arguments

Value

A list containing vectors with 0 if a case should be made missing and 1 if a case should remain complete. The first vector refers to the first pattern, the second vector to the second pattern, etcetera.

Author(s)

Rianne Schouten [aut, cre], Gerko Vink [aut], Peter Lugtig [ctb], 2016

References

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

See Also

ampute, ampute.default.type

Default freq in ampute

Description

Defines the default relative frequency vector for the multivariate amputation function ampute.

Usage

ampute.default.freq(patterns)

Arguments

Value

A vector of length #patterns containing the relative frequencies with which the patterns should occur. An equal probability is given to each pattern.

Author(s)

Rianne Schouten, 2016

See Also

ampute, ampute.default.patterns

Default odds in ampute()

Description

Defines the default odds matrix for the multivariate amputation function ampute.

Usage

ampute.default.odds(patterns)

Arguments

Value

A matrix where #rows equals #patterns. Default is 4 quantiles with odds values 1, 2, 3 and 4, for each pattern, imitating a RIGHT type of missingness.

Author(s)

Rianne Schouten, 2016

See Also

ampute, ampute.default.patterns

Default patterns in ampute

Description

This function creates a default pattern matrix for the multivariate amputation function ampute().

Usage

ampute.default.patterns(n)

Arguments

Value

A square matrix of size n where 0 indicates a variable

Author(s)

Rianne Schouten, 2016

See Also

ampute, md.pattern

Default type in ampute()

Description

Defines the default type vector for the multivariate amputation function ampute.

Usage

ampute.default.type(patterns)

Arguments

Value

A string vector of length #patterns containing the missingness types. Each pattern will be amputed with a "RIGHT" missingness.

Author(s)

Rianne Schouten, 2016

See Also

ampute, ampute.default.patterns

Default weights in ampute

Description

Defines the default weights matrix for the multivariate amputation function ampute.

Usage

ampute.default.weights(patterns, mech)

Arguments

Value

A matrix of size #patterns by #variables containing the weights that will be used to calculate the weighted sum scores. Equal weights are given to all variables. When mechanism is MAR, variables that will be amputed will be weighted with 0. If it is MNAR, variables that will be observed will be weighted with 0. If mechanism is MCAR, the weights matrix will not be used. A default MAR matrix will be returned.

Author(s)

Rianne Schouten, 2016

See Also

ampute, ampute.default.patterns

Multivariate amputation based on discrete probability functions

Description

This function creates a missing data indicator for each pattern. Odds probabilities (Brand, 1999, pp. 110-113) will be induced on the weighted sum scores, calculated earlier in the multivariate amputation function ampute.

Usage

ampute.discrete(P, scores, prop, odds)

Arguments

Value

A list containing vectors with 0 if a case should be made missing and 1 if a case should remain complete. The first vector refers to the first pattern, the second vector to the second pattern, etcetera.

Author(s)

Rianne Schouten, 2016

References

Brand, J.P.L. (1999). Development, implementation and evaluation of multiple imputation strategies for the statistical analysis of incomplete data sets. Dissertation. Rotterdam: Erasmus University.

See Also

ampute, ampute.default.odds

Multivariate amputation under a MCAR mechanism

Description

This function creates a missing data indicator for each pattern, based on a MCAR missingness mechanism. The function is used in the multivariate amputation function ampute.

Usage

ampute.mcar(P, patterns, prop)

Arguments

Value

A list containing vectors with 0 if a case should be made missing and 1 if a case should remain complete. The first vector refers to the first pattern, the second vector to the second pattern, etcetera.

Author(s)

Rianne Schouten, 2016

See Also

ampute

Compare several nested models

Description

Compare several nested models

Usage

## S3 method for class 'mira'
anova(object, ..., method = "D1", use = "wald")

Arguments

Value

Object of class mice.anova

Appends specified break to the data

Description

A custom function to insert rows in long data with new pseudo-observations that are being done on the specified break ages. There should be a column called first in data with logical data that codes whether the current row is the first for subject id. Furthermore, the function assumes that columns age, occ, hgt.z, wgt.z and bmi.z are available. This function is used on the tbc data in FIMD chapter 9. Check that out to see it in action.

Usage

appendbreak(data, brk, warp.model = warp.model, id = NULL, typ = "pred")

Arguments

Value

A long data frame with additional rows for the break ages

Converts an imputed dataset (long format) into a mids object

Description

This function converts imputed data stored in long format into an object of class mids. The original incomplete dataset needs to be available so that we know where the missing data are. The function is useful to convert back operations applied to the imputed data back in a mids object. It may also be used to store multiply imputed data sets from other software into the format used by mice.

Usage

as.mids(long, where = NULL, .imp = ".imp", .id = ".id")

Arguments

Value

An object of class mids

Note

The function expects the input data long to be sorted by imputation number (variable ".imp" by default), and in the same sequence within each imputation block.

Author(s)

Gerko Vink

Examples

# impute the nhanes dataset
imp <- mice(nhanes, print = FALSE)
# extract the data in long format
X <- complete(imp, action = "long", include = TRUE)
# create dataset with .imp variable as numeric
X2 <- X

# nhanes example without .id
test1 <- as.mids(X)
is.mids(test1)
identical(complete(test1, action = "long", include = TRUE), X)

# nhanes example without .id where .imp is numeric
test2 <- as.mids(X2)
is.mids(test2)
identical(complete(test2, action = "long", include = TRUE), X)

# nhanes example, where we explicitly specify .id as column 2
test3 <- as.mids(X, .id = ".id")
is.mids(test3)
identical(complete(test3, action = "long", include = TRUE), X)

# nhanes example with .id where .imp is numeric
test4 <- as.mids(X2, .id = 6)
is.mids(test4)
identical(complete(test4, action = "long", include = TRUE), X)

# example without an .id variable
# variable .id not preserved
X3 <- X[, -6]
test5 <- as.mids(X3)
is.mids(test5)
identical(complete(test5, action = "long", include = TRUE)[, -6], X[, -6])

# where argument copies also observed data into $imp element
where <- matrix(TRUE, nrow = nrow(nhanes), ncol = ncol(nhanes))
colnames(where) <- colnames(nhanes)
test11 <- as.mids(X, where = where)
identical(complete(test11, action = "long", include = TRUE), X)

Create a mira object from repeated analyses

Description

The as.mira() function takes the results of repeated complete-data analysis stored as a list, and turns it into a mira object that can be pooled.

Usage

as.mira(fitlist)

Arguments

Value

An S3 object of class mira.

Author(s)

Stef van Buuren

See Also

mira

Converts into a mitml.result object

Description

The as.mitml.result() function takes the results of repeated complete-data analysis stored as a list, and turns it into an object of class mitml.result.

Usage

as.mitml.result(x)

Arguments

Value

An S3 object of class mitml.result, a list containing $m$ fitted analysis objects.

Author(s)

Stef van Buuren

See Also

with.mitml.list

Growth of Dutch boys

Description

Height, weight, head circumference and puberty of 748 Dutch boys.

Format

A data frame with 748 rows on the following 9 variables:

age

Decimal age (0-21 years)

hgt

Height (cm)

wgt

Weight (kg)

bmi

Body mass index

hc

Head circumference (cm)

gen

Genital Tanner stage (G1-G5)

phb

Pubic hair (Tanner P1-P6)

tv

Testicular volume (ml)

reg

Region (north, east, west, south, city)

Details

Random sample of 10% from the cross-sectional data used to construct the Dutch growth references 1997. Variables gen and phb are ordered factors. reg is a factor.

Source

Fredriks, A.M,, van Buuren, S., Burgmeijer, R.J., Meulmeester JF, Beuker, R.J., Brugman, E., Roede, M.J., Verloove-Vanhorick, S.P., Wit, J.M. (2000) Continuing positive secular growth change in The Netherlands 1955-1997. Pediatric Research, 47, 316-323.

Fredriks, A.M., van Buuren, S., Wit, J.M., Verloove-Vanhorick, S.P. (2000). Body index measurements in 1996-7 compared with 1980. Archives of Disease in Childhood, 82, 107-112.

Examples

# create two imputed data sets
imp <- mice(boys, m = 1, maxit = 2)
z <- complete(imp, 1)

# create imputations for age <8yrs
plot(z$age, z$gen,
  col = mdc(1:2)[1 + is.na(boys$gen)],
  xlab = "Age (years)", ylab = "Tanner Stage Genital"
)

# figure to show that the default imputation method does not impute BMI
# consistently
plot(z$bmi, z$wgt / (z$hgt / 100)^2,
  col = mdc(1:2)[1 + is.na(boys$bmi)],
  xlab = "Imputed BMI", ylab = "Calculated BMI"
)

# also, BMI distributions are somewhat different
oldpar <- par(mfrow = c(1, 2))
MASS::truehist(z$bmi[!is.na(boys$bmi)],
  h = 1, xlim = c(10, 30), ymax = 0.25,
  col = mdc(1), xlab = "BMI observed"
)
MASS::truehist(z$bmi[is.na(boys$bmi)],
  h = 1, xlim = c(10, 30), ymax = 0.25,
  col = mdc(2), xlab = "BMI imputed"
)
par(oldpar)

# repair the inconsistency problem by passive imputation
meth <- imp$meth
meth["bmi"] <- "~I(wgt/(hgt/100)^2)"
pred <- imp$predictorMatrix
pred["hgt", "bmi"] <- 0
pred["wgt", "bmi"] <- 0
imp2 <- mice(boys, m = 1, maxit = 2, meth = meth, pred = pred)
z2 <- complete(imp2, 1)

# show that new imputations are consistent
plot(z2$bmi, z2$wgt / (z2$hgt / 100)^2,
  col = mdc(1:2)[1 + is.na(boys$bmi)],
  ylab = "Calculated BMI"
)

# and compare distributions
oldpar <- par(mfrow = c(1, 2))
MASS::truehist(z2$bmi[!is.na(boys$bmi)],
  h = 1, xlim = c(10, 30), ymax = 0.25, col = mdc(1),
  xlab = "BMI observed"
)
MASS::truehist(z2$bmi[is.na(boys$bmi)],
  h = 1, xlim = c(10, 30), ymax = 0.25, col = mdc(2),
  xlab = "BMI imputed"
)
par(oldpar)

Brandsma school data used Snijders and Bosker (2012)

Description

Dataset with raw data from Snijders and Bosker (2012) containing data from 4106 pupils attending 216 schools. This dataset includes all pupils and schools with missing data.

Format

brandsma is a data frame with 4106 rows and 14 columns:

sch

School number

pup

Pupil ID

iqv

IQ verbal

iqp

IQ performal

sex

Sex of pupil

ses

SES score of pupil

min

Minority member 0/1

rpg

Number of repeated groups, 0, 1, 2

lpr

language score PRE

lpo

language score POST

apr

Arithmetic score PRE

apo

Arithmetic score POST

den

Denomination classification 1-4 - at school level

ssi

School SES indicator - at school level

Note

This dataset is constructed from the raw data. There are a few differences with the data set used in Chapter 4 and 5 of Snijders and Bosker:

1. All schools are included, including the five school with missing values on langpost.

2. Missing denomina codes are left as missing.

3. Aggregates are undefined in the presence of missing data in the underlying values. Variables ses, iqv and iqp are in their original scale, and not globally centered. No aggregate variables at the school level are included.

4. There is a wider selection of original variables. Note however that the source data contain an even wider set of variables.

Source

Constructed from MLbook_2nded_total_4106-99.sav from https://www.stats.ox.ac.uk/~snijders/mlbook.htm by function data-raw/R/brandsma.R

References

Brandsma, HP and Knuver, JWM (1989), Effects of school and classroom characteristics on pupil progress in language and arithmetic. International Journal of Educational Research, 13(7), 777 - 788.

Snijders, TAB and Bosker RJ (2012). Multilevel Analysis, 2nd Ed. Sage, Los Angeles, 2012.

Box-and-whisker plot of amputed and non-amputed data

Description

Plotting method to investigate the relation between the data variables and the amputed data. The function shows how the amputed values are related to the variable values.

Usage

## S3 method for class 'mads'
bwplot(
  x,
  data,
  which.pat = NULL,
  standardized = TRUE,
  descriptives = TRUE,
  layout = NULL,
  ...
)

Arguments

Value

A list containing the box-and-whisker plots. Note that a new pattern will always be shown in a new plot.

Note

The mads object contains all the information you need to make any desired plots. Check mads or the vignette Multivariate Amputation using Ampute to understand the contents of class object mads.

Author(s)

Rianne Schouten, 2016

See Also

ampute, bwplot, mads

Box-and-whisker plot of observed and imputed data

Description

Plotting methods for imputed data using lattice. bwplot() produces box-and-whisker plots. The function automatically separates the observed and imputed data. The functions extend the usual features of lattice.

Usage

## S3 method for class 'mids'
bwplot(
  x,
  data,
  na.groups = NULL,
  groups = NULL,
  as.table = TRUE,
  theme = mice.theme(),
  mayreplicate = TRUE,
  allow.multiple = TRUE,
  outer = TRUE,
  drop.unused.levels = lattice::lattice.getOption("drop.unused.levels"),
  ...,
  subscripts = TRUE,
  subset = TRUE
)

Arguments

Details

The argument na.groups may be used to specify (combinations of) missingness in any of the variables. The argument groups can be used to specify groups based on the variable values themselves. Only one of both may be active at the same time. When both are specified, na.groups takes precedence over groups.

Use the subset and na.groups together to plots parts of the data. For example, select the first imputed data set by by subset=.imp==1.

Graphical parameters like col, pch and cex can be specified in the arguments list to alter the plotting symbols. If length(col)==2, the color specification to define the observed and missing groups. col[1] is the color of the 'observed' data, col[2] is the color of the missing or imputed data. A convenient color choice is col=mdc(1:2), a transparent blue color for the observed data, and a transparent red color for the imputed data. A good choice is col=mdc(1:2), pch=20, cex=1.5. These choices can be set for the duration of the session by running mice.theme().

Value

The high-level functions documented here, as well as other high-level Lattice functions, return an object of class "trellis". The update.trellis method can be used to subsequently update components of the object, and the print.trellis method (usually called by default) will plot it on an appropriate plotting device.

Note

The first two arguments (x and data) are reversed compared to the standard Trellis syntax implemented in lattice. This reversal was necessary in order to benefit from automatic method dispatch.

In mice the argument x is always a mids object, whereas in lattice the argument x is always a formula.

In mice the argument data is always a formula object, whereas in lattice the argument data is usually a data frame.

All other arguments have identical interpretation.

Generates a box-and-whisker plot for each numerical variable in a mids object. This extends the bwplot generic.

Author(s)

Stef van Buuren

References

Sarkar, Deepayan (2008) Lattice: Multivariate Data Visualization with R, Springer.

van Buuren S and Groothuis-Oudshoorn K (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

See Also

mice, bwplot

Examples

imp <- mice(boys, maxit = 1)

### box-and-whisker plot per imputation of all numerical variables
bwplot(imp)

### tv (testicular volume), conditional on region
bwplot(imp, tv ~ .imp | reg)

### same data, organized in a different way
bwplot(imp, tv ~ reg | .imp, theme = list())

Combine R objects by rows and columns

Description

Functions cbind() and rbind() are defined in the mice package in order to enable dispatch to cbind.mids() and rbind.mids() when one of the arguments is a data.frame.

Usage

cbind(...)

rbind(...)

Arguments

Details

The standard base::cbind() and base::rbind() always dispatch to base::cbind.data.frame() or base::rbind.data.frame() if one of the arguments is a data.frame. The versions defined in the mice package intercept the user command and test whether the first argument has class "mids". If so, function calls cbind.mids(), respectively rbind.mids(). In all other cases, the call is forwarded to standard functions in the base package.

The cbind.mids() function combines two mids objects columnwise into a single object of class mids, or combines a single mids object with a vector, matrix, factor or data.frame columnwise into a mids object.

If both arguments of cbind.mids() are mids-objects, the data list components should have the same number of rows. Also, the number of imputations (m) should be identical. If the second argument is a matrix, factor or vector, it is transformed into a data.frame. The number of rows should match with the data component of the first argument.

The cbind.mids() function renames any duplicated variable or block names by appending ".1", ".2" to duplicated names.

The rbind.mids() function combines two mids objects rowwise into a single mids object, or combines a mids object with a vector, matrix, factor or data frame rowwise into a mids object.

If both arguments of rbind.mids() are mids objects, then rbind.mids() requires that both have the same number of multiple imputations. In addition, their data components should match.

If the second argument of rbind.mids() is not a mids object, the columns of the arguments should match. The where matrix for the second argument is set to FALSE, signalling that any missing values in that argument were not imputed. The ignore vector for the second argument is set to FALSE. Rows inherited from the second argument will therefore influence the parameter estimation of the imputation model in any future iterations.

Value

An S3 object of class mids

Note

The cbind.mids() function constructs the elements of the new mids object as follows:

The rbind.mids() function constructs the elements of the new mids object as follows:

Author(s)

Karin Groothuis-Oudshoorn, Stef van Buuren

References

van Buuren S and Groothuis-Oudshoorn K (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

See Also

cbind, ibind, mids

Examples

# --- cbind ---
# impute four variables at once (default)
imp <- mice(nhanes, m = 1, maxit = 1, print = FALSE)
imp$predictorMatrix

# impute two by two
data1 <- nhanes[, c("age", "bmi")]
data2 <- nhanes[, c("hyp", "chl")]
imp1 <- mice(data1, m = 2, maxit = 1, print = FALSE)
imp2 <- mice(data2, m = 2, maxit = 1, print = FALSE)

# Append two solutions
imp12 <- cbind(imp1, imp2)

# This is a different imputation model
imp12$predictorMatrix

# Append the other way around
imp21 <- cbind(imp2, imp1)
imp21$predictorMatrix

# Append 'forgotten' variable chl
data3 <- nhanes[, 1:3]
imp3 <- mice(data3, maxit = 1, m = 2, print = FALSE)
imp4 <- cbind(imp3, chl = nhanes$chl)

# Of course, chl was not imputed
head(complete(imp4))

# Combine mids object with data frame
imp5 <- cbind(imp3, nhanes2)
head(complete(imp5))

# --- rbind ---
imp1 <- mice(nhanes[1:13, ], m = 2, maxit = 1, print = FALSE)
imp5 <- mice(nhanes[1:13, ], m = 2, maxit = 2, print = FALSE)
mylist <- list(age = NA, bmi = NA, hyp = NA, chl = NA)

nrow(complete(rbind(imp1, imp5)))
nrow(complete(rbind(imp1, mylist)))

nrow(complete(rbind(imp1, data.frame(mylist))))
nrow(complete(rbind(imp1, complete(imp5))))

Select complete cases

Description

Extracts the complete cases, also known as listwise deletion. cc(x) is similar to na.omit(x), but returns an object of the same class as the input data. Dimensions are not dropped. For extracting incomplete cases, use ici.

Usage

cc(x)

Arguments

Value

A vector, matrix or data.frame containing the data of the complete cases.

Author(s)

Stef van Buuren, 2017.

See Also

na.omit, cci, ici

Examples

# cc(nhanes)   # get the 13 complete cases
# cc(nhanes$bmi) # extract complete bmi

Complete case indicator

Description

The complete case indicator is useful for extracting the subset of complete cases. The function cci(x) calls complete.cases(x). The companion function ici() selects the incomplete cases.

Usage

cci(x)

Arguments

Value

Logical vector indicating the complete cases.

Author(s)

Stef van Buuren, 2017.

See Also

complete.cases, ici, cc

Examples

cci(nhanes) # indicator for 13 complete cases
cci(mice(nhanes, maxit = 0))
f <- cci(nhanes[, c("bmi", "hyp")]) # complete data for bmi and hyp
nhanes[f, ] # obtain all data from those with complete bmi and hyp

Extracts the completed data from a mids object

Description

Takes an object of class mids, fills in the missing data, and returns the completed data in a specified format.

Usage

## S3 method for class 'mids'
complete(
  data,
  action = 1L,
  include = FALSE,
  mild = FALSE,
  order = c("last", "first"),
  ...
)

Arguments

Details

The argument action can be length-1 character, which is matched to one of the following keywords:

"all"

produces a mild object of imputed data sets. When include = TRUE, then the original data are appended as the first list element;

"long"

produces a data set where imputed data sets are stacked vertically. The columns are added: 1) .imp, integer, referring the imputation number, and 2) .id, character, the row names of data$data;

"stacked"

same as "long" but without the two additional columns;

"broad"

produces a data set with where imputed data sets are stacked horizontally. Columns are ordered as in the original data. The imputation number is appended to each column name;

"repeated"

same as "broad", but with columns in a different order.

Value

Complete data set with missing values replaced by imputations. A data.frame, or a list of data frames of class mild.

Note

Technical note: mice 3.7.5 renamed the complete() function to complete.mids() and exported it as an S3 method of the generic tidyr::complete(). Name clashes between mice::complete() and tidyr::complete() should no longer occur.

See Also

mice, mids

Examples

# obtain first imputed data set
sum(is.na(nhanes2))
imp <- mice(nhanes2, print = FALSE, maxit = 1)
dat <- complete(imp)
sum(is.na(dat))

# obtain stacked third and fifth imputation
dat <- complete(imp, c(3, 5))

# obtain all datasets, with additional identifiers
head(complete(imp, "long"))

# same, but now as list, mild object
dslist <- complete(imp, "all")
length(dslist)

# same, but also include the original data
dslist <- complete(imp, "all", include = TRUE)
length(dslist)

# select original + 3 + 5, store as mild
dslist <- complete(imp, c(0, 3, 5), mild = TRUE)
names(dslist)

Construct blocks from formulas and predictorMatrix

Description

This helper function attempts to find blocks of variables in the specification of the formulas and/or predictorMatrix objects. Blocks specified by formulas may consist of multiple variables. Blocks specified by predictorMatrix are assumed to consist of single variables. Any duplicates in names are removed, and the formula specification is preferred. predictorMatrix and formulas. When both arguments specify models for the same block, the model for the predictMatrix is removed, and priority is given to the specification given in formulas.

Usage

construct.blocks(formulas = NULL, predictorMatrix = NULL)

Arguments

Value

A blocks object.

See Also

make.blocks, name.blocks

Examples

form <- list(bmi + hyp ~ chl + age, chl ~ bmi)
pred <- make.predictorMatrix(nhanes[, c("age", "chl")])
construct.blocks(formulas = form, pred = pred)

Computes convergence diagnostics for a mids object

Description

Takes an object of class mids, computes the autocorrelation and/or potential scale reduction factor, and returns a data.frame with the specified diagnostic(s) per iteration.

Usage

convergence(data, diagnostic = "all", parameter = "mean", ...)

Arguments

Details

The argument diagnostic can be length-1 character, which is matched to one of the following keywords:

"all"

computes both the lag-1 autocorrelation as well as the potential scale reduction factor (cf. Vehtari et al., 2021) per iteration of the MICE algorithm;

"ac"

computes only the autocorrelation per iteration;

"psrf"

computes only the potential scale reduction factor per iteration;

"gr"

same as psrf, the potential scale reduction factor is colloquially called the Gelman-Rubin diagnostic.

In the unlikely event of perfect convergence, the autocorrelation equals zero and the potential scale reduction factor equals one. To interpret the convergence diagnostic(s) in the output of the function, it is recommended to plot the diagnostics (ac and/or psrf) against the iteration number (.it) per imputed variable (vrb). A persistently decreasing trend across iterations indicates potential non-convergence.

Value

A data.frame with the autocorrelation and/or potential scale reduction factor per iteration of the MICE algorithm.

References

Vehtari, A., Gelman, A., Simpson, D., Carpenter, B., & Burkner, P.-C. (2021). Rank-Normalization, Folding, and Localization: An Improved R for Assessing Convergence of MCMC. Bayesian Analysis, 1(1), 1-38. https://doi.org/10.1214/20-BA1221

See Also

mice, mids

Examples

## Not run: 
# obtain imputed data set
imp <- mice(nhanes2, print = FALSE)
# compute convergence diagnostics
convergence(imp)

## End(Not run)

Compare two nested models using D1-statistic

Description

The D1-statistics is the multivariate Wald test.

Usage

D1(fit1, fit0 = NULL, dfcom = NULL, df.com = NULL)

Arguments

Note

Warning: 'D1()' assumes that the order of the variables is the same in different models. See https://github.com/amices/mice/issues/420 for details.

References

Li, K. H., T. E. Raghunathan, and D. B. Rubin. 1991. Large-Sample Significance Levels from Multiply Imputed Data Using Moment-Based Statistics and an F Reference Distribution. Journal of the American Statistical Association, 86(416): 1065–73.

https://stefvanbuuren.name/fimd/sec-multiparameter.html#sec:wald

See Also

testModels

Examples

# Compare two linear models:
imp <- mice(nhanes2, seed = 51009, print = FALSE)
mi1 <- with(data = imp, expr = lm(bmi ~ age + hyp + chl))
mi0 <- with(data = imp, expr = lm(bmi ~ age + hyp))
D1(mi1, mi0)
## Not run: 
# Compare two logistic regression models
imp <- mice(boys, maxit = 2, print = FALSE)
fit1 <- with(imp, glm(gen > levels(gen)[1] ~ hgt + hc + reg, family = binomial))
fit0 <- with(imp, glm(gen > levels(gen)[1] ~ hgt + hc, family = binomial))
D1(fit1, fit0)

## End(Not run)

Compare two nested models using D2-statistic

Description

The D2-statistic pools test statistics from the repeated analyses. The method is less powerful than the D1- and D3-statistics.

Usage

D2(fit1, fit0 = NULL, use = "wald")

Arguments

Note

Warning: 'D2()' assumes that the order of the variables is the same in different models. See https://github.com/amices/mice/issues/420 for details.

References

Li, K. H., X. L. Meng, T. E. Raghunathan, and D. B. Rubin. 1991. Significance Levels from Repeated p-Values with Multiply-Imputed Data. Statistica Sinica 1 (1): 65–92.

https://stefvanbuuren.name/fimd/sec-multiparameter.html#sec:chi

See Also

testModels

Examples

# Compare two linear models:
imp <- mice(nhanes2, seed = 51009, print = FALSE)
mi1 <- with(data = imp, expr = lm(bmi ~ age + hyp + chl))
mi0 <- with(data = imp, expr = lm(bmi ~ age + hyp))
D2(mi1, mi0)
## Not run: 
# Compare two logistic regression models
imp <- mice(boys, maxit = 2, print = FALSE)
fit1 <- with(imp, glm(gen > levels(gen)[1] ~ hgt + hc + reg, family = binomial))
fit0 <- with(imp, glm(gen > levels(gen)[1] ~ hgt + hc, family = binomial))
D2(fit1, fit0)

## End(Not run)

Compare two nested models using D3-statistic

Description

The D3-statistic is a likelihood-ratio test statistic.

Usage

D3(fit1, fit0 = NULL, dfcom = NULL, df.com = NULL)

Arguments

Details

The D3() function implement the LR-method by Meng and Rubin (1992). The implementation of the method relies on the broom package, the standard update mechanism for statistical models in R and the offset function.

The function calculates m repetitions of the full (or null) models, calculates the mean of the estimates of the (fixed) parameter coefficients \beta. For each imputed imputed dataset, it calculates the likelihood for the model with the parameters constrained to \beta.

The mitml::testModels() function offers similar functionality for a subset of statistical models. Results of mice::D3() and mitml::testModels() differ in multilevel models because the testModels() also constrains the variance components parameters. For more details on

Value

An object of class mice.anova

References

Meng, X. L., and D. B. Rubin. 1992. Performing Likelihood Ratio Tests with Multiply-Imputed Data Sets. Biometrika, 79 (1): 103–11.

https://stefvanbuuren.name/fimd/sec-multiparameter.html#sec:likelihoodratio

http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#setting-residual-variances-to-a-fixed-value-zero-or-other

See Also

fix.coef

Examples

# Compare two linear models:
imp <- mice(nhanes2, seed = 51009, print = FALSE)
mi1 <- with(data = imp, expr = lm(bmi ~ age + hyp + chl))
mi0 <- with(data = imp, expr = lm(bmi ~ age + hyp))
D3(mi1, mi0)
## Not run: 
# Compare two logistic regression models
imp <- mice(boys, maxit = 2, print = FALSE)
fit1 <- with(imp, glm(gen > levels(gen)[1] ~ hgt + hc + reg, family = binomial))
fit0 <- with(imp, glm(gen > levels(gen)[1] ~ hgt + hc, family = binomial))
D3(fit1, fit0)

## End(Not run)

Density plot of observed and imputed data

Description

Plotting methods for imputed data using lattice. densityplot produces plots of the densities. The function automatically separates the observed and imputed data. The functions extend the usual features of lattice.

Usage

## S3 method for class 'mids'
densityplot(
  x,
  data,
  na.groups = NULL,
  groups = NULL,
  as.table = TRUE,
  plot.points = FALSE,
  theme = mice.theme(),
  mayreplicate = TRUE,
  thicker = 2.5,
  allow.multiple = TRUE,
  outer = TRUE,
  drop.unused.levels = lattice::lattice.getOption("drop.unused.levels"),
  panel = lattice::lattice.getOption("panel.densityplot"),
  default.prepanel = lattice::lattice.getOption("prepanel.default.densityplot"),
  ...,
  subscripts = TRUE,
  subset = TRUE
)

Arguments

Details

The argument na.groups may be used to specify (combinations of) missingness in any of the variables. The argument groups can be used to specify groups based on the variable values themselves. Only one of both may be active at the same time. When both are specified, na.groups takes precedence over groups.

Use the subset and na.groups together to plots parts of the data. For example, select the first imputed data set by by subset=.imp==1.

Graphical parameters like col, pch and cex can be specified in the arguments list to alter the plotting symbols. If length(col)==2, the color specification to define the observed and missing groups. col[1] is the color of the 'observed' data, col[2] is the color of the missing or imputed data. A convenient color choice is col=mdc(1:2), a transparent blue color for the observed data, and a transparent red color for the imputed data. A good choice is col=mdc(1:2), pch=20, cex=1.5. These choices can be set for the duration of the session by running mice.theme().

Value

The high-level functions documented here, as well as other high-level Lattice functions, return an object of class "trellis". The update.trellis method can be used to subsequently update components of the object, and the print.trellis method (usually called by default) will plot it on an appropriate plotting device.

Note

The first two arguments (x and data) are reversed compared to the standard Trellis syntax implemented in lattice. This reversal was necessary in order to benefit from automatic method dispatch.

In mice the argument x is always a mids object, whereas in lattice the argument x is always a formula.

In mice the argument data is always a formula object, whereas in lattice the argument data is usually a data frame.

All other arguments have identical interpretation.

densityplot errs on empty groups, which occurs if all observations in the subgroup contain NA. The relevant error message is: Error in density.default: ... need at least 2 points to select a bandwidth automatically. There is yet no workaround for this problem. Use the more robust bwplot or stripplot as a replacement.

Author(s)

Stef van Buuren

References

Sarkar, Deepayan (2008) Lattice: Multivariate Data Visualization with R, Springer.

van Buuren S and Groothuis-Oudshoorn K (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Examples

imp <- mice(boys, maxit = 1)

### density plot of head circumference per imputation
### blue is observed, red is imputed
densityplot(imp, ~ hc | .imp)

### All combined in one panel.
densityplot(imp, ~hc)

Employee selection data

Description

A toy example from Craig Enders.

Usage

employee

Format

A data frame with 20 rows and 3 variables:

IQ

candidate IQ score

wbeing

candidate well-being score

jobperf

candidate job performance score

Details

Enders describes these data as follows: I designed these data to mimic an employee selection scenario in which prospective employees complete an IQ test and a psychological well-being questionnaire during their interview. The company subsequently hires the applications that score in the upper half of the IQ distribution, and a supervisor rates their job performance following a 6-month probationary period. Note that the job performance scores are missing at random (MAR) (i.e. individuals in the lower half of the IQ distribution were never hired, and thus have no performance rating). In addition, I randomly deleted three of the well-being scores in order to mimic a situation where the applicant's well-being questionnaire is inadvertently lost.

A larger version of this data set in present as data.enders.employee.

Source

Enders (2010), Applied Missing Data Analysis, p. 218

Computes least squares parameters

Description

This function computes least squares estimates, variance/covariance matrices, residuals and degrees of freedom according to ridge regression, QR decomposition or Singular Value Decomposition. This function is internally called by .norm.draw(), but can be called by any user-specified imputation function.

Usage

estimice(x, y, ls.meth = "qr", ridge = 1e-05, ...)

Arguments

Details

When calculating the inverse of the crossproduct of the predictor matrix, problems may arise. For example, taking the inverse is not possible when the predictor matrix is rank deficient, or when the estimation problem is computationally singular. This function detects such error cases and automatically falls back to adding a ridge penalty to the diagonal of the crossproduct to allow for proper calculation of the inverse.

Value

A list containing components c (least squares estimate), r (residuals), v (variance/covariance matrix) and df (degrees of freedom).

Note

This functions adds a star to variable names in the mice iteration history to signal that a ridge penalty was added. In that case, it also adds an entry to loggedEvents.

Author(s)

Gerko Vink, 2018

Extends a formula with predictors

Description

Extends a formula with predictors

Usage

extend.formula(
  formula = ~0,
  predictors = NULL,
  auxiliary = TRUE,
  include.intercept = FALSE,
  ...
)

Arguments

Value

A formula

Extends formula's with predictor matrix settings

Description

Extends formula's with predictor matrix settings

Usage

extend.formulas(
  formulas,
  data,
  blocks,
  predictorMatrix = NULL,
  auxiliary = TRUE,
  include.intercept = FALSE,
  ...
)

Arguments

Value

A list of formula's

Extract broken stick estimates from a lmer object

Description

Extract broken stick estimates from a lmer object

Usage

extractBS(fit)

Arguments

Value

A matrix containing broken stick estimates

Author(s)

Stef van Buuren, 2012

SE Fireworks disaster data

Description

Multiple outcomes of a randomized study to reduce post-traumatic stress.

Format

fdd is a data frame with 52 rows and 65 columns:

id

Client number

trt

Treatment (E=EMDR, C=CBT)

pp

Per protocol (Y/N)

trtp

Number of parental treatments

sex

Sex: M/F

etn

Ethnicity: NL/OTHER

age

Age (years)

trauma

Trauma count (1-5)

prop1

PROPS total score T1

prop2

PROPS total score T2

prop3

PROPS total score T3

crop1

CROPS total score T1

crop2

CROPS total score T2

crop3

CROPS total score T3

masc1

MASC score T1

masc2

MASC score T2

masc3

MASC score T3

cbcl1

CBCL T1

cbcl3

CBCL T3

prs1

PRS total score T1

prs2

PRS total score T2

prs3

PRS total score T3

ypa1

PTSD-RI B intrusive recollection parent T1

ypb1

PTSD-RI C avoidant/numbing parent T1

ypc1

PTSD-RI D hyper-arousal parent T1

yp1

PTSD-RI B+C+D parent T1

ypa2

PTSD-RI B intrusive recollection parent T2

ypb2

PTSD-RI C avoidant/numbing parent T2

ypc2

PTSD-RI D hyper-arousal parent T2

yp2

PTSD-RI B+C+D parent T1

ypa3

PTSD-RI B intrusive recollection parent T3

ypb3

PTSD-RI C avoidant/numbing parent T3

ypc3

PTSD-RI D hyper-arousal parent T3

yp3

PTSD-RI B+C+D parent T3

yca1

PTSD-RI B intrusive recollection child T1

ycb1

PTSD-RI C avoidant/numbing child T1

ycc1

PTSD-RI D hyper-arousal child T1

yc1

PTSD-RI B+C+D child T1

yca2

PTSD-RI B intrusive recollection child T2

ycb2

PTSD-RI C avoidant/numbing child T2

ycc2

PTSD-RI D hyper-arousal child T2

yc2

PTSD-RI B+C+D child T2

yca3

PTSD-RI B intrusive recollection child T3

ycb3

PTSD-RI C avoidant/numbing child T3

ycc3

PTSD-RI D hyper-arousal child T3

yc3

PTSD-RI B+C+D child T3

ypf1

PTSD-RI parent full T1

ypf2

PTSD-RI parent full T2

ypf3

PTSD-RI parent full T3

ypp1

PTSD parent partial T1

ypp2

PTSD parent partial T2

ypp3

PTSD parent partial T3

ycf1

PTSD child full T1

ycf2

PTSD child full T2

ycf3

PTSD child full T3

ycp1

PTSD child partial T1

ycp2

PTSD child partial T2

ycp3

PTSD child partial T3

cbin1

CBCL Internalizing T1

cbin3

CBCL Internalizing T3

cbex1

CBCL Externalizing T1

cbex3

CBCL Externalizing T3

bir1

Birlison T1

bir2

Birlison T2

bir3

Birlison T3

fdd.pred is the 65 by 65 binary predictor matrix used to impute fdd.

Details

Data from a randomized experiment to reduce post-traumatic stress by two treatments: Eye Movement Desensitization and Reprocessing (EMDR) (experimental treatment), and cognitive behavioral therapy (CBT) (control treatment). 52 children were randomized to one of these two treatments. Outcomes were measured at three time points: at baseline (pre-treatment, T1), post-treatment (T2, 4-8 weeks), and at follow-up (T3, 3 months). For more details, see de Roos et al (2011). Some person covariates were reshuffled. The imputation methodology is explained in Chapter 9 of van Buuren (2012).

Source

de Roos, C., Greenwald, R., den Hollander-Gijsman, M., Noorthoorn, E., van Buuren, S., de Jong, A. (2011). A Randomised Comparison of Cognitive Behavioral Therapy (CBT) and Eye Movement Desensitisation and Reprocessing (EMDR) in disaster-exposed children. European Journal of Psychotraumatology, 2, 5694.

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL. Boca Raton, FL.: Chapman & Hall/CRC Press.

Examples

data <- fdd
md.pattern(fdd)

Fifth Dutch growth study 2009

Description

Age, height, weight and region of 10030 children measured within the Fifth Dutch Growth Study 2009

Format

fdgs is a data frame with 10030 rows and 8 columns:

id

Person number

reg

Region (factor, 5 levels)

age

Age (years)

sex

Sex (boy, girl)

hgt

Height (cm)

wgt

Weight (kg)

hgt.z

Height Z-score

wgt.z

Weight Z-score

Details

The data set contains data from children of Dutch descent (biological parents are born in the Netherlands). Children with growth-related diseases were excluded. The data were used to construct new growth charts of children of Dutch descent (Schonbeck 2013), and to calculate overweight and obesity prevalence (Schonbeck 2011).

Some groups were underrepresented. Multiple imputation was used to create synthetic cases that were used to correct for the nonresponse. See Van Buuren (2012), chapter 8 for details.

Source

Schonbeck, Y., Talma, H., van Dommelen, P., Bakker, B., Buitendijk, S. E., Hirasing, R. A., van Buuren, S. (2011). Increase in prevalence of overweight in Dutch children and adolescents: A comparison of nationwide growth studies in 1980, 1997 and 2009. PLoS ONE, 6(11), e27608.

Schonbeck, Y., Talma, H., van Dommelen, P., Bakker, B., Buitendijk, S. E., Hirasing, R. A., van Buuren, S. (2013). The world's tallest nation has stopped growing taller: the height of Dutch children from 1955 to 2009. Pediatric Research, 73(3), 371-377.

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Boca Raton, FL.: Chapman & Hall/CRC Press.

Examples

data <- data(fdgs)
summary(data)

Fraction of incomplete cases among cases with observed

Description

FICO is an outbound statistic defined by the fraction of incomplete cases among cases with Yj observed (White and Carlin, 2010).

Usage

fico(data)

Arguments

Value

A vector of length ncol(data) of FICO statistics.

Author(s)

Stef van Buuren, 2012

References

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

White, I.R., Carlin, J.B. (2010). Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Statistics in Medicine, 29, 2920-2931.

See Also

fluxplot, flux, md.pattern

Subset rows of a mids object

Description

This function takes a mids object and returns a new mids object that pertains to the subset of the data identified by the expression in .... The expression may use column values from the incomplete data in .data$data.

Usage

## S3 method for class 'mids'
filter(.data, ..., .preserve = FALSE)

Arguments

Value

An S3 object of class mids

Note

The function calculates a logical vector include of length nrow(.data$data). The function constructs the elements of the filtered mids object as follows:

Author(s)

Patrick Rockenschaub

See Also

filter

Examples

imp <- mice(nhanes, m = 2, maxit = 1, print = FALSE)

# example with external logical vector
imp_f <- filter(imp, c(rep(TRUE, 13), rep(FALSE, 12)))

nrow(complete(imp))
nrow(complete(imp_f))

# example with calculated include vector
imp_f2 <- filter(imp, age >= 2 & hyp == 1)
nrow(complete(imp_f2)) # should be 5

Fix coefficients and update model

Description

Refits a model with a specified set of coefficients.

Usage

fix.coef(model, beta = NULL)

Arguments

Details

The function calculates the linear predictor using the new coefficients, and reformulates the model using the offset argument. The linear predictor is called offset, and its coefficient will be 1 by definition. The new model only fits the intercept, which should be 0 if we set beta = coef(model).

Value

An updated R model object

Author(s)

Stef van Buuren, 2018

Examples

model0 <- lm(Volume ~ Girth + Height, data = trees)
formula(model0)
coef(model0)
deviance(model0)

# refit same model
model1 <- fix.coef(model0)
formula(model1)
coef(model1)
deviance(model1)

# change the beta's
model2 <- fix.coef(model0, beta = c(-50, 5, 1))
coef(model2)
deviance(model2)

# compare predictions
plot(predict(model0), predict(model1))
abline(0, 1)
plot(predict(model0), predict(model2))
abline(0, 1)

# compare proportion explained variance
cor(predict(model0), predict(model0) + residuals(model0))^2
cor(predict(model1), predict(model1) + residuals(model1))^2
cor(predict(model2), predict(model2) + residuals(model2))^2

# extract offset from constrained model
summary(model2$offset)

# it also works with factors and missing data
model0 <- lm(bmi ~ age + hyp + chl, data = nhanes2)
model1 <- fix.coef(model0)
model2 <- fix.coef(model0, beta = c(15, -8, -8, 2, 0.2))

Influx and outflux of multivariate missing data patterns

Description

Influx and outflux are statistics of the missing data pattern. These statistics are useful in selecting predictors that should go into the imputation model.

Usage

flux(data, local = names(data))

Arguments

Details

Infux and outflux have been proposed by Van Buuren (2018), chapter 4.

Influx is equal to the number of variable pairs (Yj , Yk) with Yj missing and Yk observed, divided by the total number of observed data cells. Influx depends on the proportion of missing data of the variable. Influx of a completely observed variable is equal to 0, whereas for completely missing variables we have influx = 1. For two variables with the same proportion of missing data, the variable with higher influx is better connected to the observed data, and might thus be easier to impute.

Outflux is equal to the number of variable pairs with Yj observed and Yk missing, divided by the total number of incomplete data cells. Outflux is an indicator of the potential usefulness of Yj for imputing other variables. Outflux depends on the proportion of missing data of the variable. Outflux of a completely observed variable is equal to 1, whereas outflux of a completely missing variable is equal to 0. For two variables having the same proportion of missing data, the variable with higher outflux is better connected to the missing data, and thus potentially more useful for imputing other variables.

FICO is an outbound statistic defined by the fraction of incomplete cases among cases with Yj observed (White and Carlin, 2010).

Value

A data frame with ncol(data) rows and six columns: pobs = Proportion observed, influx = Influx outflux = Outflux ainb = Average inbound statistic aout = Average outbound statistic fico = Fraction of incomplete cases among cases with Yj observed

Author(s)

Stef van Buuren, 2012

References

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

White, I.R., Carlin, J.B. (2010). Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Statistics in Medicine, 29, 2920-2931.

See Also

fluxplot, md.pattern, fico

Fluxplot of the missing data pattern

Description

Influx and outflux are statistics of the missing data pattern. These statistics are useful in selecting predictors that should go into the imputation model.

Usage

fluxplot(
  data,
  local = names(data),
  plot = TRUE,
  labels = TRUE,
  xlim = c(0, 1),
  ylim = c(0, 1),
  las = 1,
  xlab = "Influx",
  ylab = "Outflux",
  main = paste("Influx-outflux pattern for", deparse(substitute(data))),
  eqscplot = TRUE,
  pty = "s",
  lwd = 1,
  ...
)

Arguments

Details

Infux and outflux have been proposed by Van Buuren (2012), chapter 4.

Influx is equal to the number of variable pairs (Yj , Yk) with Yj missing and Yk observed, divided by the total number of observed data cells. Influx depends on the proportion of missing data of the variable. Influx of a completely observed variable is equal to 0, whereas for completely missing variables we have influx = 1. For two variables with the same proportion of missing data, the variable with higher influx is better connected to the observed data, and might thus be easier to impute.

Outflux is equal to the number of variable pairs with Yj observed and Yk missing, divided by the total number of incomplete data cells. Outflux is an indicator of the potential usefulness of Yj for imputing other variables. Outflux depends on the proportion of missing data of the variable. Outflux of a completely observed variable is equal to 1, whereas outflux of a completely missing variable is equal to 0. For two variables having the same proportion of missing data, the variable with higher outflux is better connected to the missing data, and thus potentially more useful for imputing other variables.

Value

An invisible data frame with ncol(data) rows and six columns: pobs = Proportion observed, influx = Influx outflux = Outflux ainb = Average inbound statistic aout = Average outbound statistic fico = Fraction of incomplete cases among cases with Yj observed

Author(s)

Stef van Buuren, 2012

References

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

White, I.R., Carlin, J.B. (2010). Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Statistics in Medicine, 29, 2920-2931.

See Also

flux, md.pattern, fico

Wrapper function that runs MICE in parallel

Description

This is a wrapper function for mice, using multiple cores to execute mice in parallel. As a result, the imputation procedure can be sped up, which may be useful in general. By default, futuremice distributes the number of imputations m about equally over the cores.

Usage

futuremice(
  data,
  m = 5,
  parallelseed = NA,
  n.core = NULL,
  seed = NA,
  use.logical = TRUE,
  future.plan = "multisession",
  packages = NULL,
  globals = NULL,
  ...
)

Arguments

Details

This function relies on package furrr, which is a package for R versions 3.2.0 and later. We have chosen to use furrr function future_map to allow the use of futuremice on Mac, Linux and Windows systems.

This wrapper function combines the output of future_map with function ibind from the mice package. A mids object is returned and can be used for further analyses.

A seed value can be specified in the global environment, which will yield reproducible results. A seed value can also be specified within the futuremice call, through specifying the argument parallelseed. If parallelseed is not specified, a seed value is drawn randomly by default, and accessible through $parallelseed in the output object. Hence, results will always be reproducible, regardless of whether the seed is specified in the global environment, or by setting the same seed within the function (potentially by extracting the seed from the futuremice output object.

Value

A mids object as defined by mids-class

Author(s)

Thom Benjamin Volker, Gerko Vink

References

Volker, T.B. and Vink, G. (2022). futuremice: The future starts today. https://www.gerkovink.com/miceVignettes/futuremice/Vignette_futuremice.html

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

See Also

future, furrr, future_map, plan, mice, mids-class

Examples

# 150 imputations in dataset nhanes, performed by 3 cores
## Not run: 
imp1 <- futuremice(data = nhanes, m = 150, n.core = 3)
# Making use of arguments in mice.
imp2 <- futuremice(data = nhanes, m = 100, method = "norm.nob")
imp2$method
fit <- with(imp2, lm(bmi ~ hyp))
pool(fit)

## End(Not run)

Extract list of fitted models

Description

Function getfit() returns the list of objects containing the repeated analysis results, or optionally, one of these fitted objects. The function looks for a list element called analyses, and return this component as a list with mira class. If element analyses is not found in x, then it returns x as a mira object.

Usage

getfit(x, i = -1L, simplify = FALSE)

Arguments

Details

No checking is done for validity of objects. The function also processes objects of class mitml.result from the mitml package.

Value

If i = -1 an object of class mira containing all analyses. If i selects one of the analyses, then it return an object whose with class inherited from that element.

Author(s)

Stef van Buuren, 2012, 2020

See Also

mira, with.mids

Examples

imp <- mice(nhanes, print = FALSE, seed = 21443)
fit <- with(imp, lm(bmi ~ chl + hyp))
f1 <- getfit(fit)
class(f1)
f2 <- getfit(fit, 2)
class(f2)

Extract estimate from mipo object

Description

getqbar returns a named vector of pooled estimates.

Usage

getqbar(x)

Arguments

Glance method to extract information from a 'mipo' object

Description

Glance method to extract information from a 'mipo' object

Usage

## S3 method for class 'mipo'
glance(x, ...)

Arguments

Value

a dataframe with one row and the following columns:

• nimp

• nobs

Note

If x contains 'lm' models, R2 and Adj.R2 are included in the output

Generalized linear model for mids object

Description

Applies glm() to a multiply imputed data set

Usage

glm.mids(formula, family = gaussian, data, ...)

Arguments

Details

This function is included for backward compatibility with V1.0. The function is superseded by with.mids.

Value

An objects of class mira, which stands for 'multiply imputed repeated analysis'. This object contains data$m distinct glm.objects, plus some descriptive information.

Author(s)

Stef van Buuren, Karin Groothuis-Oudshoorn, 2000

References

Van Buuren, S., Groothuis-Oudshoorn, C.G.M. (2000) Multivariate Imputation by Chained Equations: MICE V1.0 User's manual. Leiden: TNO Quality of Life.

See Also

with.mids, glm, mids, mira

Examples

imp <- mice(nhanes)

# logistic regression on the imputed data
fit <- glm.mids((hyp == 2) ~ bmi + chl, data = imp, family = binomial)
fit

Enlarge number of imputations by combining mids objects

Description

This function combines two mids objects x and y into a single mids object, with the objective of increasing the number of imputed data sets. If the number of imputations in x and y are m(x) and m(y), then the combined object will have m(x)+m(y) imputations.

Usage

ibind(x, y)

Arguments

Details

The two mids objects are required to have the same underlying multiple imputation model and should be fitted on the same data.

Value

An S3 object of class mids

Author(s)

Karin Groothuis-Oudshoorn, Stef van Buuren

See Also

mids

Examples

data(nhanes)
imp1 <- mice(nhanes, m = 1, maxit = 2, print = FALSE)
imp1$m

imp2 <- mice(nhanes, m = 3, maxit = 3, print = FALSE)
imp2$m

imp12 <- ibind(imp1, imp2)
imp12$m
plot(imp12)

Select incomplete cases

Description

Extracts incomplete cases from a data set. The companion function for selecting the complete cases is cc.

Usage

ic(x)

Arguments

Value

A vector, matrix or data.frame containing the data of the complete cases.

Author(s)

Stef van Buuren, 2017.

See Also

cc, ici

Examples

ic(nhanes) # get the 12 rows with incomplete cases
ic(nhanes[1:10, ]) # incomplete cases within the first ten rows
ic(nhanes[, c("bmi", "hyp")]) # restrict extraction to variables bmi and hyp

Incomplete case indicator

Description

This array is useful for extracting the subset of incomplete cases. The companion function cci() selects the complete cases.

Usage

ici(x)

Arguments

Value

Logical vector indicating the incomplete cases,

Author(s)

Stef van Buuren, 2017.

See Also

cci, ic

Examples

ici(nhanes) # indicator for 12 rows with incomplete cases

Conditional imputation helper

Description

Sorry, the ifdo() function is not yet implemented.

Usage

ifdo(cond, action)

Arguments

Value

Currently returns an error message.

Author(s)

Stef van Buuren, 2012

Check for mads object

Description

Check for mads object

Usage

is.mads(x)

Arguments

Value

A logical indicating whether x is an object of class mads

Check for mids object

Description

Check for mids object

Usage

is.mids(x)

Arguments

Value

A logical indicating whether x is an object of class mids

Check for mipo object

Description

Check for mipo object

Usage

is.mipo(x)

Arguments

Value

A logical indicating whether x is an object of class mipo

Check for mira object

Description

Check for mira object

Usage

is.mira(x)

Arguments

Value

A logical indicating whether x is an object of class mira

Check for mitml.result object

Description

Check for mitml.result object

Usage

is.mitml.result(x)

Arguments

Value

A logical indicating whether x is an object of class mitml.result

Leiden 85+ study

Description

Subset of data from the Leiden 85+ study

Format

leiden85 is a data frame with 956 rows and 336 columns.

Details

The data set concerns of subset of 956 members of a very old (85+) cohort in Leiden.

Multiple imputation of this data set has been described in Boshuizen et al (1998), Van Buuren et al (1999) and Van Buuren (2012), chapter 7.

The data set is not available as part of mice.

Source

Lagaay, A. M., van der Meij, J. C., Hijmans, W. (1992). Validation of medical history taking as part of a population based survey in subjects aged 85 and over. Brit. Med. J., 304(6834), 1091-1092.

Izaks, G. J., van Houwelingen, H. C., Schreuder, G. M., Ligthart, G. J. (1997). The association between human leucocyte antigens (HLA) and mortality in community residents aged 85 and older. Journal of the American Geriatrics Society, 45(1), 56-60.

Boshuizen, H. C., Izaks, G. J., van Buuren, S., Ligthart, G. J. (1998). Blood pressure and mortality in elderly people aged 85 and older: Community based study. Brit. Med. J., 316(7147), 1780-1784.

Van Buuren, S., Boshuizen, H.C., Knook, D.L. (1999) Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine, 18, 681–694.

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

Linear regression for mids object

Description

Applies lm() to multiply imputed data set

Usage

lm.mids(formula, data, ...)

Arguments

Details

This function is included for backward compatibility with V1.0. The function is superseded by with.mids.

Value

An objects of class mira, which stands for 'multiply imputed repeated analysis'. This object contains data$m distinct lm.objects, plus some descriptive information.

Author(s)

Stef van Buuren, Karin Groothuis-Oudshoorn, 2000

References

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

See Also

lm, mids, mira

Examples

imp <- mice(nhanes)
fit <- lm.mids(bmi ~ hyp + chl, data = imp)
fit

Multivariate amputed data set (mads)

Description

The mads object is an S3 class that contains an amputed dataset, i.e., a dataset with simulated missing values. The ampute() function produces a mads object. The mads() function is the S3 constructor. The S3 class mads has the following methods: bwplot.mads(),print(), summary(), and xyplot.mads().

Usage

mads(
  call,
  prop,
  patterns,
  freq,
  mech,
  weights,
  cont,
  type,
  odds,
  amp,
  cand,
  scores,
  data
)

## S3 method for class 'mads'
print(x, ...)

## S3 method for class 'mads'
summary(object, ...)

Arguments

Value

An object of class "mads".

print() returns the input object invisibly.

summary() returns the input object invisibly.

Contents

call:

The function call.

prop:

Proportion of cases with missing values. Note: even when the proportion is entered as the proportion of missing cells (when bycases == TRUE), this object contains the proportion of missing cases.

patterns:

A data frame of size #patterns by #variables where 0 indicates a variable has missing values and 1 indicates a variable remains complete.

freq:

A vector of length #patterns containing the relative frequency with which the patterns occur. For example, if the vector is c(0.4, 0.4, 0.2), this means that of all cases with missing values, 40 percent is candidate for pattern 1, 40 percent for pattern 2 and 20 percent for pattern 3. The vector sums to 1.

mech:

A string specifying the missingness mechanism, either "MCAR", "MAR" or "MNAR".

weights:

A data frame of size #patterns by #variables. It contains the weights that were used to calculate the weighted sum scores. The weights may differ between patterns and between variables.

cont:

Logical, whether probabilities are based on continuous logit functions or on discrete odds distributions.

type:

A vector of strings containing the type of missingness for each pattern. Either "LEFT", "MID", "TAIL" or "RIGHT". The first type refers to the first pattern, the second type to the second pattern, etc.

odds:

A matrix where #patterns defines the #rows. Each row contains the odds of being missing for the corresponding pattern. The amount of odds values defines in how many quantiles the sum scores were divided. The values are relative probabilities: a quantile with odds value 4 will have a probability of being missing that is four times higher than a quantile with odds 1. The #quantiles may differ between patterns, NA is used for cells remaining empty.

amp:

A data frame containing the input data with NAs for the amputed values.

cand:

A vector that contains the pattern number for each case. A value between 1 and #patterns is given. For example, a case with value 2 is candidate for missing data pattern 2.

scores:

A list containing vectors with weighted sum scores of the candidates. The first vector refers to the candidates of the first pattern, the second vector refers to the candidates of the second pattern, etc. The length of the vectors differ because the number of candidates is different for each pattern.

data:

The complete data set that was entered in ampute.

Author(s)

Rianne Schouten, 2016

See Also

ampute, Vignette titled "Multivariate Amputation using Ampute".

Creates a blocks argument

Description

This helper function generates a list of the type needed for blocks argument in the [=mice]{mice} function.

Usage

make.blocks(
  data,
  partition = c("scatter", "collect", "void"),
  calltype = "pred"
)

Arguments

Details

Choices "scatter" and "collect" represent to two extreme scenarios for assigning variables to imputation blocks. Use "scatter" to create an imputation model based on fully conditionally specification (FCS). Use "collect" to gather all variables to be imputed by a joint model (JM). Scenario's in-between these two extremes represent hybrid imputation models that combine FCS and JM.

Any variable not listed in will not be imputed. Specification "void" represents the extreme scenario that skips imputation of all variables.

A variable may be a member of multiple blocks. The variable will be re-imputed in each block, so the final imputations for variable will come from the last block that was executed. This scenario may be useful where the same complete background factors appear in multiple imputation blocks.

A variable may appear multiple times within a given block. If a univariate imputation model is applied to such a block, then the variable is re-imputed each time as it appears in the block.

Value

A named list of character vectors with variables names.

Examples

make.blocks(nhanes)
make.blocks(c("age", "sex", "edu"))

Creates a blots argument

Description

This helper function creates a valid blots object. The blots object is an argument to the mice function. The name blots is a contraction of blocks-dots. Through blots, the user can specify any additional arguments that are specifically passed down to the lowest level imputation function.

Usage

make.blots(data, blocks = make.blocks(data))

Arguments

Value

A matrix

See Also

make.blocks

Examples

make.predictorMatrix(nhanes)
make.blots(nhanes, blocks = name.blocks(c("age", "hyp"), "xxx"))

Creates a formulas argument

Description

This helper function creates a valid formulas object. The formulas object is an argument to the mice function. It is a list of formula's that specifies the target variables and the predictors by means of the standard ~ operator.

Usage

make.formulas(data, blocks = make.blocks(data), predictorMatrix = NULL)

Arguments

Value

A list of formula's.

See Also

make.blocks, make.predictorMatrix

Examples

f1 <- make.formulas(nhanes)
f1
f2 <- make.formulas(nhanes, blocks = make.blocks(nhanes, "collect"))
f2

# for editing, it may be easier to work with the character vector
c1 <- as.character(f1)
c1

# fold it back into a formula list
f3 <- name.formulas(lapply(c1, as.formula))
f3

Creates a method argument

Description

This helper function creates a valid method vector. The method vector is an argument to the mice function that specifies the method for each block.

Usage

make.method(
  data,
  where = make.where(data),
  blocks = make.blocks(data),
  defaultMethod = c("pmm", "logreg", "polyreg", "polr")
)

Arguments

Value

Vector of length(blocks) element with method names

See Also

mice

Examples

make.method(nhanes2)

Creates a post argument

Description

This helper function creates a valid post vector. The post vector is an argument to the mice function that specifies post-processing for a variable after each iteration of imputation.

Usage

make.post(data)

Arguments

Value

Character vector of ncol(data) element

See Also

mice

Examples

make.post(nhanes2)

Creates a predictorMatrix argument

Description

This helper function creates a valid predictMatrix. The predictorMatrix is an argument to the mice function. It specifies the target variable or block in the rows, and the predictor variables on the columns. An entry of 0 means that the column variable is NOT used to impute the row variable or block. A nonzero value indicates that it is used.

Usage

make.predictorMatrix(data, blocks = make.blocks(data), predictorMatrix = NULL)

Arguments

Value

A matrix

See Also

make.blocks

Examples

make.predictorMatrix(nhanes)
make.predictorMatrix(nhanes, blocks = make.blocks(nhanes, "collect"))

Creates a visitSequence argument

Description

This helper function creates a valid visitSequence. The visitSequence is an argument to the mice function that specifies the sequence in which blocks are imputed.

Usage

make.visitSequence(data = NULL, blocks = NULL)

Arguments

Value

Vector containing block names

See Also

mice

Examples

make.visitSequence(nhanes)

Creates a where argument

Description

This helper function creates a valid where matrix. The where matrix is an argument to the mice function. It has the same size as data and specifies which values are to be imputed (TRUE) or nor (FALSE).

Usage

make.where(data, keyword = c("missing", "all", "none", "observed"))

Arguments

Value

A matrix with logical

See Also

make.blocks, make.predictorMatrix

Examples

head(make.where(nhanes), 3)

# create & analyse synthetic data
where <- make.where(nhanes2, "all")
imp <- mice(nhanes2,
  m = 10, where = where,
  print = FALSE, seed = 123
)
fit <- with(imp, lm(chl ~ bmi + age + hyp))
summary(pool.syn(fit))

Mammal sleep data

Description

Dataset from Allison and Cicchetti (1976) of 62 mammal species on the interrelationship between sleep, ecological, and constitutional variables. The dataset contains missing values on five variables.

Format

mammalsleep is a data frame with 62 rows and 11 columns:

species

Species of animal

bw

Body weight (kg)

brw

Brain weight (g)

sws

Slow wave ("nondreaming") sleep (hrs/day)

ps

Paradoxical ("dreaming") sleep (hrs/day)

ts

Total sleep (hrs/day) (sum of slow wave and paradoxical sleep)

mls

Maximum life span (years)

gt

Gestation time (days)

pi

Predation index (1-5), 1 = least likely to be preyed upon

sei

Sleep exposure index (1-5), 1 = least exposed (e.g. animal sleeps in a well-protected den), 5 = most exposed

odi

Overall danger index (1-5) based on the above two indices and other information, 1 = least danger (from other animals), 5 = most danger (from other animals)

Details

Allison and Cicchetti (1976) investigated the interrelationship between sleep, ecological, and constitutional variables. They assessed these variables for 39 mammalian species. The authors concluded that slow-wave sleep is negatively associated with a factor related to body size. This suggests that large amounts of this sleep phase are disadvantageous in large species. Also, paradoxical sleep (REM sleep) was associated with a factor related to predatory danger, suggesting that large amounts of this sleep phase are disadvantageous in prey species.

Source

Allison, T., Cicchetti, D.V. (1976). Sleep in Mammals: Ecological and Constitutional Correlates. Science, 194(4266), 732-734.

Examples

sleep <- data(mammalsleep)

Find index of matched donor units

Description

Find index of matched donor units

Usage

matchindex(d, t, k = 5L)

Arguments

Details

For each element in t, the method finds the k nearest neighbours in d, randomly draws one of these neighbours, and returns its position in vector d.

Fast predictive mean matching algorithm in seven steps:

1. Shuffle records to remove effects of ties

2. Obtain sorting order on shuffled data

3. Calculate index on input data and sort it

4. Pre-sample vector h with values between 1 and k

For each of the n0 elements in t:

5. find the two adjacent neighbours

6. find the h_i'th nearest neighbour

7. store the index of that neighbour

Return vector of n0 positions in d.

We may use the function to perform predictive mean matching under a given predictive model. To do so, specify both d and t as predictions from the same model. Suppose that y contains the observed outcomes of the donor cases (in the same sequence as d), then y[matchindex(d, t)] returns one matched outcome for every target case.

See https://github.com/amices/mice/issues/236. This function is a replacement for the matcher() function that has been in default in mice since version 2.22 (June 2014).

Value

An integer vector with length(t) elements. Each element is an index in the array d.

Author(s)

Stef van Buuren, Nasinski Maciej, Alexander Robitzsch

Examples

set.seed(1)

# Inputs need not be sorted
d <- c(-5, 5, 0, 10, 12)
t <- c(-6, -4, 0, 2, 4, -2, 6)

# Index (in vector a) of closest match
idx <- matchindex(d, t, 1)
idx

# To check: show values of closest match

# Random draw among indices of the 5 closest predictors
matchindex(d, t)

# An example
train <- mtcars[1:20, ]
test <- mtcars[21:32, ]
fit <- lm(mpg ~ disp + cyl, data = train)
d <- fitted.values(fit)
t <- predict(fit, newdata = test)  # note: not using mpg
idx <- matchindex(d, t)

# Borrow values from train to produce 12 synthetic values for mpg in test.
# Synthetic values are plausible values that could have been observed if
# they had been measured.
train$mpg[idx]

# Exercise: Create a distribution of 1000 plausible values for each of the
# twelve mpg entries in test, and count how many times the true value
# (which we know here) is located within the inter-quartile range of each
# distribution. Is your count anywhere close to 500? Why? Why not?

Jamshidian and Jalal's Non-Parametric MCAR Test

Description

Test whether missingness is contingent upon the observed variables, according to the methodology developed by Jamshidian and Jalal (2010) (see Details).

Usage

mcar(
  x,
  imputed = mice(x, method = "norm"),
  min_n = 6,
  method = "auto",
  replications = 10000,
  use_chisq = 30,
  alpha = 0.05
)

Arguments

Details

Three types of missingness have been distinguished in the literature (Rubin, 1976): Missing completely at random (MCAR), which means that missingness is random; missing at random (MAR), which means that missingness is contingent on the observed; and missing not at random (MNAR), which means that missingness is related to unobserved data.

Jamshidian and Jalal's non-parametric MCAR test assumes that the missing data are either MCAR or MAR, and tests whether the missingness is independent of the observed values. If so, the covariance matrices of the imputed data will be equal accross groups with different patterns of missingness. This test consists of the following procedure:

1. Data are imputed.

2. The imputed data are split into k groups according to the k missing data patterns in the original data (see md.pattern()).

3. Perform Hawkins' test for equality of covariances across the k groups.

4. If the test is not significant, conclude that there is no evidence against multivariate normality of the data, nor against MCAR.

5. If the test is significant, and multivariate normality of the data can be assumed, then it can be concluded that missingness is MAR.

6. If multivariate normality cannot be assumed, then perform the Anderson-Darling non-parametric test for equality of covariances across the k groups.

7. If the Anderson-Darling test is not significant, this is evidence against multivariate normality - but no evidence against MCAR.

8. If the Anderson-Darling test is significant, this is evidence it can be concluded that missingness is MAR.

Note that, despite its name in common parlance, an MCAR test can only indicate whether missingness is MCAR or MAR. The procedure cannot distinguish MCAR from MNAR, so a non-significant result does not rule out MNAR.

This is a re-implementation of the function TestMCARNormality, which was originally published in the R-packgage MissMech, which has been removed from CRAN. This new implementation is faster, as its backend is written in C++. It also enhances the functionality of the original:

• Multiply imputed data can now be used; the median p-value and test statistic across replications is then reported, as suggested by Eekhout, Wiel, and Heymans (2017).

• The printing method for an mcar_object gives a warning when at least one p-value of either test was significant. In this case, it is recommended to inspect the range of p-values, and consider potential violations of MCAR.

• A plotting method for an mcar_object is provided.

• A plotting method for the ⁠$md.pattern⁠ element of an mcar_object is provided.

Value

An object of class mcar_object.

Author(s)

Caspar J. Van Lissa

References

Rubin, D. B. (1976). Inference and Missing Data. Biometrika, Vol. 63, No. 3, pp. 581-592. doi:10.2307/2335739

Eekhout, I., M. A. Wiel, & M. W. Heymans (2017). Methods for Significance Testing of Categorical Covariates in Logistic Regression Models After Multiple Imputation: Power and Applicability Analysis. BMC Medical Research Methodology 17 (1): 129.

Jamshidian, M., & Jalal, S. (2010). Tests of homoscedasticity, normality, and missing completely at random for incomplete multivariate data. Psychometrika, 75(4), 649–674. doi:10.1007/s11336-010-9175-3

Examples

res <- mcar(nhanes)
# Examine test results
res
# Plot p-values across imputed data sets
plot(res)
# Plot md patterns used for the test
plot(res, type = "md.pattern")
# Note difference with the raw md.patterns:
md.pattern(nhanes)

Missing data pattern by variable pairs

Description

Number of observations per variable pair.

Usage

md.pairs(data)

Arguments

Details

The four components in the output value is have the following interpretation:

list('rr')

response-response, both variables are observed

list('rm')

response-missing, row observed, column missing

list('mr')

missing -response, row missing, column observed

list('mm')

missing -missing, both variables are missing

Value

A list of four components named rr, rm, mr and mm. Each component is square numerical matrix containing the number observations within four missing data pattern.

Author(s)

Stef van Buuren, Karin Groothuis-Oudshoorn, 2009

References

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Examples

pat <- md.pairs(nhanes)
pat

# show that these four matrices decompose the total sample size
# for each pair
pat$rr + pat$rm + pat$mr + pat$mm

# percentage of usable cases to impute row variable from column variable
round(100 * pat$mr / (pat$mr + pat$mm))

Missing data pattern

Description

Display missing-data patterns.

Usage

md.pattern(x, plot = TRUE, rotate.names = FALSE)

Arguments

Details

This function is useful for investigating any structure of missing observations in the data. In specific case, the missing data pattern could be (nearly) monotone. Monotonicity can be used to simplify the imputation model. See Schafer (1997) for details. Also, the missing pattern could suggest which variables could potentially be useful for imputation of missing entries.

Value

A matrix with ncol(x)+1 columns, in which each row corresponds to a missing data pattern (1=observed, 0=missing). Rows and columns are sorted in increasing amounts of missing information. The last column and row contain row and column counts, respectively.

Author(s)

Gerko Vink, 2018, based on an earlier version of the same function by Stef van Buuren, Karin Groothuis-Oudshoorn, 2000

References

Schafer, J.L. (1997), Analysis of multivariate incomplete data. London: Chapman&Hall.

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Examples

md.pattern(nhanes)
#     age hyp bmi chl
#  13   1   1   1   1  0
#   1   1   1   0   1  1
#   3   1   1   1   0  1
#   1   1   0   0   1  2
#   7   1   0   0   0  3
#   0   8   9  10 27

Graphical parameter for missing data plots

Description

mdc returns colors used to distinguish observed, missing and combined data in plotting. mice.theme return a partial list of named objects that can be used as a theme in stripplot, bwplot, densityplot and xyplot.

Usage

mdc(
  r = "observed",
  s = "symbol",
  transparent = TRUE,
  cso = grDevices::hcl(240, 100, 40, 0.7),
  csi = grDevices::hcl(0, 100, 40, 0.7),
  csc = "gray50",
  clo = grDevices::hcl(240, 100, 40, 0.8),
  cli = grDevices::hcl(0, 100, 40, 0.8),
  clc = "gray50"
)

Arguments

Details

This function eases consistent use of colors in plots. The default follows the Abayomi convention, which uses blue for observed data, red for missing or imputed data, and black for combined data.

Value

mdc() returns a vector containing color definitions. The length of the output vector is calculate from the length of r and s. Elements of the input vectors are repeated if needed.

Author(s)

Stef van Buuren, sept 2012.

References

Sarkar, Deepayan (2008) Lattice: Multivariate Data Visualization with R, Springer.

See Also

hcl, rgb, xyplot, trellis.par.set

Examples

# all six colors
mdc(1:6)

# lines color for observed and missing data
mdc(c("obs", "mis"), "lin")

Imputation by a two-level logistic model using glmer

Description

Imputes univariate systematically and sporadically missing data using a two-level logistic model using lme4::glmer()

Usage

mice.impute.2l.bin(y, ry, x, type, wy = NULL, intercept = TRUE, ...)

Arguments

Details

Data are missing systematically if they have not been measured, e.g., in the case where we combine data from different sources. Data are missing sporadically if they have been partially observed.

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Shahab Jolani, 2015; adapted to mice, SvB, 2018

References

Jolani S., Debray T.P.A., Koffijberg H., van Buuren S., Moons K.G.M. (2015). Imputation of systematically missing predictors in an individual participant data meta-analysis: a generalized approach using MICE. Statistics in Medicine, 34:1841-1863.

See Also

Other univariate-2l: mice.impute.2l.lmer(), mice.impute.2l.norm(), mice.impute.2l.pan()

Examples

library(tidyr)
library(dplyr)
data("toenail2")
data <- tidyr::complete(toenail2, patientID, visit) %>%
  tidyr::fill(treatment) %>%
  dplyr::select(-time) %>%
  dplyr::mutate(patientID = as.integer(patientID))
## Not run: 
pred <- mice(data, print = FALSE, maxit = 0, seed = 1)$pred
pred["outcome", "patientID"] <- -2
imp <- mice(data, method = "2l.bin", pred = pred, maxit = 1, m = 1, seed = 1)

## End(Not run)

Imputation by a two-level normal model using lmer

Description

Imputes univariate systematically and sporadically missing data using a two-level normal model using lme4::lmer().

Usage

mice.impute.2l.lmer(y, ry, x, type, wy = NULL, intercept = TRUE, ...)

Arguments

Details

Data are missing systematically if they have not been measured, e.g., in the case where we combine data from different sources. Data are missing sporadically if they have been partially observed.

While the method is fully Bayesian, it may fix parameters of the variance-covariance matrix or the random effects to their estimated value in cases where creating draws from the posterior is not possible. The procedure throws a warning when this happens.

If lme4::lmer() fails, the procedure prints the warning "lmer does not run. Simplify imputation model" and returns the current imputation. If that happens we see flat lines in the trace line plots. Thus, the appearance of flat trace lines should be taken as an additional alert to a problem with imputation model fitting.

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Shahab Jolani, 2017

References

Jolani S. (2017) Hierarchical imputation of systematically and sporadically missing data: An approximate Bayesian approach using chained equations. Forthcoming.

Jolani S., Debray T.P.A., Koffijberg H., van Buuren S., Moons K.G.M. (2015). Imputation of systematically missing predictors in an individual participant data meta-analysis: a generalized approach using MICE. Statistics in Medicine, 34:1841-1863.

Van Buuren, S. (2011) Multiple imputation of multilevel data. In Hox, J.J. and and Roberts, J.K. (Eds.), The Handbook of Advanced Multilevel Analysis, Chapter 10, pp. 173–196. Milton Park, UK: Routledge.

See Also

Other univariate-2l: mice.impute.2l.bin(), mice.impute.2l.norm(), mice.impute.2l.pan()

Imputation by a two-level normal model

Description

Imputes univariate missing data using a two-level normal model

Usage

mice.impute.2l.norm(y, ry, x, type, wy = NULL, intercept = TRUE, ...)

Arguments

Details

Implements the Gibbs sampler for the linear multilevel model with heterogeneous with-class variance (Kasim and Raudenbush, 1998). Imputations are drawn as an extra step to the algorithm. For simulation work see Van Buuren (2011).

The random intercept is automatically added in mice.impute.2L.norm(). A model within a random intercept can be specified by mice(..., intercept = FALSE).

Value

Vector with imputed data, same type as y, and of length sum(wy)

Note

Added June 25, 2012: The currently implemented algorithm does not handle predictors that are specified as fixed effects (type=1). When using mice.impute.2l.norm(), the current advice is to specify all predictors as random effects (type=2).

Warning: The assumption of heterogeneous variances requires that in every class at least one observation has a response in y.

Author(s)

Roel de Jong, 2008

References

Kasim RM, Raudenbush SW. (1998). Application of Gibbs sampling to nested variance components models with heterogeneous within-group variance. Journal of Educational and Behavioral Statistics, 23(2), 93–116.

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Van Buuren, S. (2011) Multiple imputation of multilevel data. In Hox, J.J. and and Roberts, J.K. (Eds.), The Handbook of Advanced Multilevel Analysis, Chapter 10, pp. 173–196. Milton Park, UK: Routledge.

See Also

Other univariate-2l: mice.impute.2l.bin(), mice.impute.2l.lmer(), mice.impute.2l.pan()

Imputation by a two-level normal model using pan

Description

Imputes univariate missing data using a two-level normal model with homogeneous within group variances. Aggregated group effects (i.e. group means) can be automatically created and included as predictors in the two-level regression (see argument type). This function needs the pan package.

Usage

mice.impute.2l.pan(
  y,
  ry,
  x,
  type,
  intercept = TRUE,
  paniter = 500,
  groupcenter.slope = FALSE,
  ...
)

Arguments

Details

Implements the Gibbs sampler for the linear two-level model with homogeneous within group variances which is a special case of a multivariate linear mixed effects model (Schafer & Yucel, 2002). For a two-level imputation with heterogeneous within-group variances see mice.impute.2l.norm. The random intercept is automatically added in mice.impute.2l.norm().

Value

A vector of length nmis with imputations.

Note

This function does not implement the where functionality. It always produces nmis imputation, irrespective of the where argument of the mice function.

Author(s)

Alexander Robitzsch (IPN - Leibniz Institute for Science and Mathematics Education, Kiel, Germany), robitzsch@ipn.uni-kiel.de

Alexander Robitzsch (IPN - Leibniz Institute for Science and Mathematics Education, Kiel, Germany), robitzsch@ipn.uni-kiel.de.

References

Schafer J L, Yucel RM (2002). Computational strategies for multivariate linear mixed-effects models with missing values. Journal of Computational and Graphical Statistics. 11, 437-457.

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

See Also

Other univariate-2l: mice.impute.2l.bin(), mice.impute.2l.lmer(), mice.impute.2l.norm()

Examples

# simulate some data
# two-level regression model with fixed slope

# number of groups
G <- 250
# number of persons
n <- 20
# regression parameter
beta <- .3
# intraclass correlation
rho <- .30
# correlation with missing response
rho.miss <- .10
# missing proportion
missrate <- .50
y1 <- rep(rnorm(G, sd = sqrt(rho)), each = n) + rnorm(G * n, sd = sqrt(1 - rho))
x <- rnorm(G * n)
y <- y1 + beta * x
dfr0 <- dfr <- data.frame("group" = rep(1:G, each = n), "x" = x, "y" = y)
dfr[rho.miss * x + rnorm(G * n, sd = sqrt(1 - rho.miss)) < qnorm(missrate), "y"] <- NA

# empty imputation in mice
imp0 <- mice(as.matrix(dfr), maxit = 0)
predM <- imp0$predictorMatrix
impM <- imp0$method

# specify predictor matrix and method
predM1 <- predM
predM1["y", "group"] <- -2
predM1["y", "x"] <- 1 # fixed x effects imputation
impM1 <- impM
impM1["y"] <- "2l.pan"

# multilevel imputation
imp1 <- mice(as.matrix(dfr),
  m = 1, predictorMatrix = predM1,
  method = impM1, maxit = 1
)

# multilevel analysis
library(lme4)
mod <- lmer(y ~ (1 + x | group) + x, data = complete(imp1))
summary(mod)

# Examples of predictorMatrix specification

# random x effects
# predM1["y","x"] <- 2

# fixed x effects and group mean of x
# predM1["y","x"] <- 3

# random x effects and group mean of x
# predM1["y","x"] <- 4

Imputation of most likely value within the class

Description

Method 2lonly.mean replicates the most likely value within a class of a second-level variable. It works for numeric and factor data. The function is primarily useful as a quick fixup for data in which the second-level variable is inconsistent.

Usage

mice.impute.2lonly.mean(y, ry, x, type, wy = NULL, ...)

Arguments

Details

Observed values in y are averaged within the class, and replicated to the missing y within that class. This function is primarily useful for repairing incomplete data that are constant within the class, but vary over classes.

For numeric variables, mice.impute.2lonly.mean() imputes the class mean of y. If y is a second-level variable, then conventionally all observed y will be identical within the class, and the function just provides a quick fix for any missing y by filling in the class mean.

For factor variables, mice.impute.2lonly.mean() imputes the most frequently occuring category within the class.

If there are no observed y in the class, all entries of the class are set to NA. Note that this may produce problems later on in mice if imputation routines are called that expects predictor data to be complete. Methods designed for imputing this type of second-level variables include mice.impute.2lonly.norm and mice.impute.2lonly.pmm.

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Gerko Vink, Stef van Buuren, 2019

References

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Boca Raton, FL.: Chapman & Hall/CRC Press.

See Also

Other univariate-2lonly: mice.impute.2lonly.norm(), mice.impute.2lonly.pmm()

Imputation at level 2 by Bayesian linear regression

Description

Imputes univariate missing data at level 2 using Bayesian linear regression analysis. Variables are level 1 are aggregated at level 2. The group identifier at level 2 must be indicated by type = -2 in the predictorMatrix.

Usage

mice.impute.2lonly.norm(y, ry, x, type, wy = NULL, ...)

Arguments

Details

This function allows in combination with mice.impute.2l.pan switching regression imputation between level 1 and level 2 as described in Yucel (2008) or Gelman and Hill (2007, p. 541).

The function checks for partial missing level-2 data. Level-2 data are assumed to be constant within the same cluster. If one or more entries are missing, then the procedure aborts with an error message that identifies the cluster with incomplete level-2 data. In such cases, one may first fill in the cluster mean (or mode) by the 2lonly.mean method to remove inconsistencies.

Value

A vector of length nmis with imputations.

Note

For a more general approach, see miceadds::mice.impute.2lonly.function().

Author(s)

Alexander Robitzsch (IPN - Leibniz Institute for Science and Mathematics Education, Kiel, Germany), robitzsch@ipn.uni-kiel.de

References

Gelman, A. and Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge, Cambridge University Press.

Yucel, RM (2008). Multiple imputation inference for multivariate multilevel continuous data with ignorable non-response. Philosophical Transactions of the Royal Society A, 366, 2389-2404.

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

See Also

mice.impute.norm, mice.impute.2lonly.pmm, mice.impute.2l.pan, mice.impute.2lonly.mean

Other univariate-2lonly: mice.impute.2lonly.mean(), mice.impute.2lonly.pmm()

Examples

# simulate some data
# x,y ... level 1 variables
# v,w ... level 2 variables

G <- 250 # number of groups
n <- 20 # number of persons
beta <- .3 # regression coefficient
rho <- .30 # residual intraclass correlation
rho.miss <- .10 # correlation with missing response
missrate <- .50 # missing proportion
y1 <- rep(rnorm(G, sd = sqrt(rho)), each = n) + rnorm(G * n, sd = sqrt(1 - rho))
w <- rep(round(rnorm(G), 2), each = n)
v <- rep(round(runif(G, 0, 3)), each = n)
x <- rnorm(G * n)
y <- y1 + beta * x + .2 * w + .1 * v
dfr0 <- dfr <- data.frame("group" = rep(1:G, each = n), "x" = x, "y" = y, "w" = w, "v" = v)
dfr[rho.miss * x + rnorm(G * n, sd = sqrt(1 - rho.miss)) < qnorm(missrate), "y"] <- NA
dfr[rep(rnorm(G), each = n) < qnorm(missrate), "w"] <- NA
dfr[rep(rnorm(G), each = n) < qnorm(missrate), "v"] <- NA

# empty mice imputation
imp0 <- mice(as.matrix(dfr), maxit = 0)
predM <- imp0$predictorMatrix
impM <- imp0$method

# multilevel imputation
predM1 <- predM
predM1[c("w", "y", "v"), "group"] <- -2
predM1["y", "x"] <- 1 # fixed x effects imputation
impM1 <- impM
impM1[c("y", "w", "v")] <- c("2l.pan", "2lonly.norm", "2lonly.pmm")

# y ... imputation using pan
# w ... imputation at level 2 using norm
# v ... imputation at level 2 using pmm

imp1 <- mice(as.matrix(dfr),
  m = 1, predictorMatrix = predM1,
  method = impM1, maxit = 1, paniter = 500
)

# Demonstration that 2lonly.norm aborts for partial missing data.
# Better use 2lonly.mean for repair.
data <- data.frame(
  patid = rep(1:4, each = 5),
  sex = rep(c(1, 2, 1, 2), each = 5),
  crp = c(
    68, 78, 93, NA, 143,
    5, 7, 9, 13, NA,
    97, NA, 56, 52, 34,
    22, 30, NA, NA, 45
  )
)
pred <- make.predictorMatrix(data)
pred[, "patid"] <- -2
# only missing value (out of five) for patid == 1
data[3, "sex"] <- NA
## Not run: 
# The following fails because 2lonly.norm found partially missing
# level-2 data
# imp <- mice(data, method = c("", "2lonly.norm", "2l.pan"),
#             predictorMatrix = pred, maxit = 1, m = 2)
# > iter imp variable
# > 1   1  sex  crpError in .imputation.level2(y = y, ... :
# >   Method 2lonly.norm found the following clusters with partially missing
# >    level-2 data: 1
# > Method 2lonly.mean can fix such inconsistencies.

## End(Not run)

# In contrast, if all sex values are missing for patid == 1, it runs fine,
# except on r-patched-solaris-x86. I used dontrun to evade CRAN errors.
## Not run: 
data[1:5, "sex"] <- NA
imp <- mice(data,
  method = c("", "2lonly.norm", "2l.pan"),
  predictorMatrix = pred, maxit = 1, m = 2
)

## End(Not run)

Imputation at level 2 by predictive mean matching

Description

Imputes univariate missing data at level 2 using predictive mean matching. Variables are level 1 are aggregated at level 2. The group identifier at level 2 must be indicated by type = -2 in the predictorMatrix.

Usage

mice.impute.2lonly.pmm(y, ry, x, type, wy = NULL, ...)

Arguments

Details

This function allows in combination with mice.impute.2l.pan switching regression imputation between level 1 and level 2 as described in Yucel (2008) or Gelman and Hill (2007, p. 541).

The function checks for partial missing level-2 data. Level-2 data are assumed to be constant within the same cluster. If one or more entries are missing, then the procedure aborts with an error message that identifies the cluster with incomplete level-2 data. In such cases, one may first fill in the cluster mean (or mode) by the 2lonly.mean method to remove inconsistencies.

Value

A vector of length nmis with imputations.

Note

The extension to categorical variables transforms a dependent factor variable by means of the as.integer() function. This may make sense for categories that are approximately ordered, but less so for pure nominal measures.

For a more general approach, see miceadds::mice.impute.2lonly.function().

Author(s)

Alexander Robitzsch (IPN - Leibniz Institute for Science and Mathematics Education, Kiel, Germany), robitzsch@ipn.uni-kiel.de

References

Gelman, A. and Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge, Cambridge University Press.

Yucel, RM (2008). Multiple imputation inference for multivariate multilevel continuous data with ignorable non-response. Philosophical Transactions of the Royal Society A, 366, 2389-2404.

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

See Also

mice.impute.pmm, mice.impute.2lonly.norm, mice.impute.2l.pan, mice.impute.2lonly.mean

Other univariate-2lonly: mice.impute.2lonly.mean(), mice.impute.2lonly.norm()

Examples

# simulate some data
# x,y ... level 1 variables
# v,w ... level 2 variables

G <- 250 # number of groups
n <- 20 # number of persons
beta <- .3 # regression coefficient
rho <- .30 # residual intraclass correlation
rho.miss <- .10 # correlation with missing response
missrate <- .50 # missing proportion
y1 <- rep(rnorm(G, sd = sqrt(rho)), each = n) + rnorm(G * n, sd = sqrt(1 - rho))
w <- rep(round(rnorm(G), 2), each = n)
v <- rep(round(runif(G, 0, 3)), each = n)
x <- rnorm(G * n)
y <- y1 + beta * x + .2 * w + .1 * v
dfr0 <- dfr <- data.frame("group" = rep(1:G, each = n), "x" = x, "y" = y, "w" = w, "v" = v)
dfr[rho.miss * x + rnorm(G * n, sd = sqrt(1 - rho.miss)) < qnorm(missrate), "y"] <- NA
dfr[rep(rnorm(G), each = n) < qnorm(missrate), "w"] <- NA
dfr[rep(rnorm(G), each = n) < qnorm(missrate), "v"] <- NA

# empty mice imputation
imp0 <- mice(as.matrix(dfr), maxit = 0)
predM <- imp0$predictorMatrix
impM <- imp0$method

# multilevel imputation
predM1 <- predM
predM1[c("w", "y", "v"), "group"] <- -2
predM1["y", "x"] <- 1 # fixed x effects imputation
impM1 <- impM
impM1[c("y", "w", "v")] <- c("2l.pan", "2lonly.norm", "2lonly.pmm")

# turn v into a categorical variable
dfr$v <- as.factor(dfr$v)
levels(dfr$v) <- LETTERS[1:4]

# y ... imputation using pan
# w ... imputation at level 2 using norm
# v ... imputation at level 2 using pmm

# skip imputation on solaris
is.solaris <- function() grepl("SunOS", Sys.info()["sysname"])
if (!is.solaris()) {
  imp <- mice(dfr,
    m = 1, predictorMatrix = predM1,
    method = impM1, maxit = 1, paniter = 500
  )
}

Imputation by classification and regression trees

Description

Imputes univariate missing data using classification and regression trees.

Usage

mice.impute.cart(y, ry, x, wy = NULL, minbucket = 5, cp = 1e-04, ...)

Arguments

Details

Imputation of y by classification and regression trees. The procedure is as follows:

1. Fit a classification or regression tree by recursive partitioning;

2. For each ymis, find the terminal node they end up according to the fitted tree;

3. Make a random draw among the member in the node, and take the observed value from that draw as the imputation.

Value

Vector with imputed data, same type as y, and of length sum(wy)

Numeric vector of length sum(!ry) with imputations

Author(s)

Lisa Doove, Stef van Buuren, Elise Dusseldorp, 2012

References

Doove, L.L., van Buuren, S., Dusseldorp, E. (2014), Recursive partitioning for missing data imputation in the presence of interaction Effects. Computational Statistics & Data Analysis, 72, 92-104.

Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. (1984), Classification and regression trees, Monterey, CA: Wadsworth & Brooks/Cole Advanced Books & Software.

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

See Also

mice, mice.impute.rf, rpart, rpart.control

Other univariate imputation functions: mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Examples

imp <- mice(nhanes2, meth = "cart", minbucket = 4)
plot(imp)

Multivariate multilevel imputation using jomo

Description

This function is a wrapper around the jomoImpute function from the mitml package so that it can be called to impute blocks of variables in mice. The mitml::jomoImpute function provides an interface to the jomo package for multiple imputation of multilevel data https://CRAN.R-project.org/package=jomo. Imputations can be generated using type or formula, which offer different options for model specification.

Usage

mice.impute.jomoImpute(
  data,
  formula,
  type,
  m = 1,
  silent = TRUE,
  format = "imputes",
  ...
)

Arguments

Value

A list of imputations for all incomplete variables in the model, that can be stored in the the imp component of the mids object.

Note

The number of imputations m is set to 1, and the function is called m times so that it fits within the mice iteration scheme.

This is a multivariate imputation function using a joint model.

Author(s)

Stef van Buuren, 2018, building on work of Simon Grund, Alexander Robitzsch and Oliver Luedtke (authors of mitml package) and Quartagno and Carpenter (authors of jomo package).

References

Grund S, Luedtke O, Robitzsch A (2016). Multiple Imputation of Multilevel Missing Data: An Introduction to the R Package pan. SAGE Open.

Quartagno M and Carpenter JR (2015). Multiple imputation for IPD meta-analysis: allowing for heterogeneity and studies with missing covariates. Statistics in Medicine, 35:2938-2954, 2015.

See Also

jomoImpute

Other multivariate-2l: mice.impute.panImpute()

Examples

## Not run: 
# Note: Requires mitml 0.3-5.7
blocks <- list(c("bmi", "chl", "hyp"), "age")
method <- c("jomoImpute", "pmm")
ini <- mice(nhanes, blocks = blocks, method = method, maxit = 0)
pred <- ini$pred
pred["B1", "hyp"] <- -2
imp <- mice(nhanes, blocks = blocks, method = method, pred = pred, maxit = 1)

## End(Not run)

Imputation by direct use of lasso logistic regression

Description

Imputes univariate missing binary data using lasso logistic regression with bootstrap.

Usage

mice.impute.lasso.logreg(y, ry, x, wy = NULL, nfolds = 10, ...)

Arguments

Details

The method consists of the following steps:

1. For a given y variable under imputation, draw a bootstrap version y* with replacement from the observed cases y[ry], and stores in x* the corresponding values from x[ry, ].

2. Fit a regularised (lasso) logistic regression with y* as the outcome, and x* as predictors. A vector of regression coefficients bhat is obtained. All of these coefficients are considered random draws from the imputation model parameters posterior distribution. Same of these coefficients will be shrunken to 0.

3. Compute predicted scores for m.d., i.e. logit-1(X bhat)

4. Compare the score to a random (0,1) deviate, and impute.

The method is based on the Direct Use of Regularized Regression (DURR) proposed by Zhao & Long (2016) and Deng et al (2016).

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Edoardo Costantini, 2021

References

Deng, Y., Chang, C., Ido, M. S., & Long, Q. (2016). Multiple imputation for general missing data patterns in the presence of high-dimensional data. Scientific reports, 6(1), 1-10.

Zhao, Y., & Long, Q. (2016). Multiple imputation in the presence of high-dimensional data. Statistical Methods in Medical Research, 25(5), 2021-2035.

See Also

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by direct use of lasso linear regression

Description

Imputes univariate missing normal data using lasso linear regression with bootstrap.

Usage

mice.impute.lasso.norm(y, ry, x, wy = NULL, nfolds = 10, ...)

Arguments

Details

The method consists of the following steps:

1. For a given y variable under imputation, draw a bootstrap version y* with replacement from the observed cases y[ry], and stores in x* the corresponding values from x[ry, ].

2. Fit a regularised (lasso) linear regression with y* as the outcome, and x* as predictors. A vector of regression coefficients bhat is obtained. All of these coefficients are considered random draws from the imputation model parameters posterior distribution. Same of these coefficients will be shrunken to 0.

3. Draw the imputed values from the predictive distribution defined by the original (non-bootstrap) data, bhat, and estimated error variance.

The method is based on the Direct Use of Regularized Regression (DURR) proposed by Zhao & Long (2016) and Deng et al (2016).

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Edoardo Costantini, 2021

References

Deng, Y., Chang, C., Ido, M. S., & Long, Q. (2016). Multiple imputation for general missing data patterns in the presence of high-dimensional data. Scientific reports, 6(1), 1-10.

Zhao, Y., & Long, Q. (2016). Multiple imputation in the presence of high-dimensional data. Statistical Methods in Medical Research, 25(5), 2021-2035.

See Also

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by indirect use of lasso logistic regression

Description

Imputes univariate missing data using logistic regression following a preprocessing lasso variable selection step.

Usage

mice.impute.lasso.select.logreg(y, ry, x, wy = NULL, nfolds = 10, ...)

Arguments

Details

The method consists of the following steps:

1. For a given y variable under imputation, fit a linear regression with lasso penalty using y[ry] as dependent variable and x[ry, ] as predictors. The coefficients that are not shrunk to 0 define the active set of predictors that will be used for imputation.

2. Fit a logit with the active set of predictors, and find (bhat, V(bhat))

3. Draw BETA from N(bhat, V(bhat))

4. Compute predicted scores for m.d., i.e. logit-1(X BETA)

5. Compare the score to a random (0,1) deviate, and impute.

The user can specify a predictorMatrix in the mice call to define which predictors are provided to this univariate imputation method. The lasso regularization will select, among the variables indicated by the user, the ones that are important for imputation at any given iteration. Therefore, users may force the exclusion of a predictor from a given imputation model by speficing a 0 entry. However, a non-zero entry does not guarantee the variable will be used, as this decision is ultimately made by the lasso variable selection procedure.

The method is based on the Indirect Use of Regularized Regression (IURR) proposed by Zhao & Long (2016) and Deng et al (2016).

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Edoardo Costantini, 2021

References

Deng, Y., Chang, C., Ido, M. S., & Long, Q. (2016). Multiple imputation for general missing data patterns in the presence of high-dimensional data. Scientific reports, 6(1), 1-10.

Zhao, Y., & Long, Q. (2016). Multiple imputation in the presence of high-dimensional data. Statistical Methods in Medical Research, 25(5), 2021-2035.

See Also

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by indirect use of lasso linear regression

Description

Imputes univariate missing data using Bayesian linear regression following a preprocessing lasso variable selection step.

Usage

mice.impute.lasso.select.norm(y, ry, x, wy = NULL, nfolds = 10, ...)

Arguments

Details

The method consists of the following steps:

1. For a given y variable under imputation, fit a linear regression with lasso penalty using y[ry] as dependent variable and x[ry, ] as predictors. Coefficients that are not shrunk to 0 define an active set of predictors that will be used for imputation

2. Define a Bayesian linear model using y[ry] as the dependent variable, the active set of x[ry, ] as predictors, and standard non-informative priors

3. Draw parameter values for the intercept, regression weights, and error variance from their posterior distribution

4. Draw imputations from the posterior predictive distribution

The user can specify a predictorMatrix in the mice call to define which predictors are provided to this univariate imputation method. The lasso regularization will select, among the variables indicated by the user, the ones that are important for imputation at any given iteration. Therefore, users may force the exclusion of a predictor from a given imputation model by specifying a 0 entry. However, a non-zero entry does not guarantee the variable will be used, as this decision is ultimately made by the lasso variable selection procedure.

The method is based on the Indirect Use of Regularized Regression (IURR) proposed by Zhao & Long (2016) and Deng et al (2016).

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Edoardo Costantini, 2021

References

Deng, Y., Chang, C., Ido, M. S., & Long, Q. (2016). Multiple imputation for general missing data patterns in the presence of high-dimensional data. Scientific reports, 6(1), 1-10.

Zhao, Y., & Long, Q. (2016). Multiple imputation in the presence of high-dimensional data. Statistical Methods in Medical Research, 25(5), 2021-2035.

See Also

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by linear discriminant analysis

Description

Imputes univariate missing data using linear discriminant analysis

Usage

mice.impute.lda(y, ry, x, wy = NULL, ...)

Arguments

Details

Imputation of categorical response variables by linear discriminant analysis. This function uses the Venables/Ripley functions lda() and predict.lda() to compute posterior probabilities for each incomplete case, and draws the imputations from this posterior.

This function can be called from within the Gibbs sampler by specifying "lda" in the method argument of mice(). This method is usually faster and uses fewer resources than calling the function, but the statistical properties may not be as good (Brand, 1999). mice.impute.polyreg.

Value

Vector with imputed data, of type factor, and of length sum(wy)

Warning

The function does not incorporate the variability of the discriminant weight, so it is not 'proper' in the sense of Rubin. For small samples and rare categories in the y, variability of the imputed data could therefore be underestimated.

Added: SvB June 2009 Tried to include bootstrap, but disabled since bootstrapping may easily lead to constant variables within groups.

Author(s)

Stef van Buuren, Karin Groothuis-Oudshoorn, 2000

References

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Brand, J.P.L. (1999). Development, Implementation and Evaluation of Multiple Imputation Strategies for the Statistical Analysis of Incomplete Data Sets. Ph.D. Thesis, TNO Prevention and Health/Erasmus University Rotterdam. ISBN 90-74479-08-1.

Venables, W.N. & Ripley, B.D. (1997). Modern applied statistics with S-PLUS (2nd ed). Springer, Berlin.

See Also

mice, mice.impute.polyreg, lda

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by logistic regression

Description

Imputes univariate missing data using logistic regression.

Usage

mice.impute.logreg(y, ry, x, wy = NULL, ...)

Arguments

Details

Imputation for binary response variables by the Bayesian logistic regression model (Rubin 1987, p. 169-170). The Bayesian method consists of the following steps:

1. Fit a logit, and find (bhat, V(bhat))

2. Draw BETA from N(bhat, V(bhat))

3. Compute predicted scores for m.d., i.e. logit-1(X BETA)

4. Compare the score to a random (0,1) deviate, and impute.

The method relies on the standard glm.fit function. Warnings from glm.fit are suppressed. Perfect prediction is handled by the data augmentation method.

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Stef van Buuren, Karin Groothuis-Oudshoorn

References

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Brand, J.P.L. (1999). Development, Implementation and Evaluation of Multiple Imputation Strategies for the Statistical Analysis of Incomplete Data Sets. Ph.D. Thesis, TNO Prevention and Health/Erasmus University Rotterdam. ISBN 90-74479-08-1.

Venables, W.N. & Ripley, B.D. (1997). Modern applied statistics with S-Plus (2nd ed). Springer, Berlin.

White, I., Daniel, R. and Royston, P (2010). Avoiding bias due to perfect prediction in multiple imputation of incomplete categorical variables. Computational Statistics and Data Analysis, 54:22672275.

See Also

mice, glm, glm.fit

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by logistic regression using the bootstrap

Description

Imputes univariate missing data using logistic regression by a bootstrapped logistic regression model. The bootstrap method draws a simple bootstrap sample with replacement from the observed data y[ry] and x[ry, ].

Usage

mice.impute.logreg.boot(y, ry, x, wy = NULL, ...)

Arguments

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Stef van Buuren, Karin Groothuis-Oudshoorn, 2000, 2011

References

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

See Also

mice, glm, glm.fit

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by the mean

Description

Imputes the arithmetic mean of the observed data

Usage

mice.impute.mean(y, ry, x = NULL, wy = NULL, ...)

Arguments

Value

Vector with imputed data, same type as y, and of length sum(wy)

Warning

Imputing the mean of a variable is almost never appropriate. See Little and Rubin (2002, p. 61-62) or Van Buuren (2012, p. 10-11)

References

Van Buuren, S., Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. doi:10.18637/jss.v045.i03

Little, R.J.A. and Rubin, D.B. (2002). Statistical Analysis with Missing Data. New York: John Wiley and Sons.

Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

See Also

mice, mean

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by predictive mean matching with distance aided donor selection

Description

Imputes univariate missing data using predictive mean matching.

Usage

mice.impute.midastouch(
  y,
  ry,
  x,
  wy = NULL,
  ridge = 1e-05,
  midas.kappa = NULL,
  outout = TRUE,
  neff = NULL,
  debug = NULL,
  ...
)

Arguments

Details

Imputation of y by predictive mean matching, based on Rubin (1987, p. 168, formulas a and b) and Siddique and Belin 2008. The procedure is as follows:

1. Draw a bootstrap sample from the donor pool.

2. Estimate a beta matrix on the bootstrap sample by the leave one out principle.

3. Compute type II predicted values for yobs (nobs x 1) and ymis (nmis x nobs).

4. Calculate the distance between all yobs and the corresponding ymis.

5. Convert the distances in drawing probabilities.

6. For each recipient draw a donor from the entire pool while considering the probabilities from the model.

7. Take its observed value in y as the imputation.

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Philipp Gaffert, Florian Meinfelder, Volker Bosch 2015

References

Gaffert, P., Meinfelder, F., Bosch V. (2018) Towards an MI-proper Predictive Mean Matching, JSM 2018. Discussion Paper.

Little, R.J.A. (1988), Missing data adjustments in large surveys (with discussion), Journal of Business Economics and Statistics, 6, 287–301.

Parzen, M., Lipsitz, S. R., Fitzmaurice, G. M. (2005), A note on reducing the bias of the approximate Bayesian bootstrap imputation variance estimator. Biometrika 92, 4, 971–974.

Rubin, D.B. (1987), Multiple imputation for nonresponse in surveys. New York: Wiley.

Siddique, J., Belin, T.R. (2008), Multiple imputation using an iterative hot-deck with distance-based donor selection. Statistics in medicine, 27, 1, 83–102

Van Buuren, S., Brand, J.P.L., Groothuis-Oudshoorn C.G.M., Rubin, D.B. (2006), Fully conditional specification in multivariate imputation. Journal of Statistical Computation and Simulation, 76, 12, 1049–1064.

Van Buuren, S., Groothuis-Oudshoorn, K. (2011), mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45, 3, 1–67. doi:10.18637/jss.v045.i03

See Also

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Examples

# do default multiple imputation on a numeric matrix
imp <- mice(nhanes, method = "midastouch")
imp

# list the actual imputations for BMI
imp$imp$bmi

# first completed data matrix
complete(imp)

# imputation on mixed data with a different method per column
mice(nhanes2, method = c("sample", "midastouch", "logreg", "norm"))

Imputation under MNAR mechanism by NARFCS

Description

Imputes univariate data under a user-specified MNAR mechanism by linear or logistic regression and NARFCS. Sensitivity analysis under different model specifications may shed light on the impact of different MNAR assumptions on the conclusions.

Usage

mice.impute.mnar.logreg(y, ry, x, wy = NULL, ums = NULL, umx = NULL, ...)

mice.impute.mnar.norm(y, ry, x, wy = NULL, ums = NULL, umx = NULL, ...)

Arguments

Details

This function imputes data that are thought to be Missing Not at Random (MNAR) by the NARFCS method. The NARFCS procedure (Tompsett et al, 2018) generalises the so-called \delta-adjustment sensitivity analysis method of Van Buuren, Boshuizen & Knook (1999) to the case with multiple incomplete variables within the FCS framework. In practical terms, the NARFCS procedure shifts the imputations drawn at each iteration of mice by a user-specified quantity that can vary across subjects, to reflect systematic departures of the missing data from the data distribution imputed under MAR.

Specification of the NARFCS model is done by the blots argument of mice(). The blots parameter is a named list. For each variable to be imputed by mice.impute.mnar.norm() or mice.impute.mnar.logreg() the corresponding element in blots is a list with at least one argument ums and, optionally, a second argument umx. For example, the high-level call might like something like mice(nhanes[, c(2, 4)], method = c("pmm", "mnar.norm"), blots = list(chl = list(ums = "-3+2*bmi"))).

The ums parameter is required, and might look like this: "-4+1*Y". The ums specifcation must have the following characteristics:

1. A single term corresponding to the intercept (constant) term, not multiplied by any variable name, must be included in the expression;

2. Each term in the expression (corresponding to the intercept or a predictor variable) must be separated by either a "+" or "-" sign, depending on the sign of the sensitivity parameter;

3. Within each non-intercept term, the sensitivity parameter value comes first and the predictor variable comes second, and these must be separated by a "*" sign;

4. For categorical predictors, for example a variable Z with K + 1 categories ("Cat0","Cat1", ...,"CatK"), K category-specific terms are needed, and those not in umx (see below) must be specified by concatenating the variable name with the name of the category (e.g. ZCat1) as this is how they are named in the design matrix (argument x) passed to the univariate imputation function. An example is "2+1*ZCat1-3*ZCat2".

If given, the umx specification must have the following characteristics:

1. It contains only complete variables, with no missing values;

2. It is a numeric matrix. In particular, categorical variables must be represented as dummy indicators with names corresponding to what is used in ums to refer to the category-specific terms (see above);

3. It has the same number of rows as the data argument passed on to the main mice function;

4. It does not contain variables that were already predictors in the identifiable part of the model for the variable under imputation.

Limitation: The present implementation can only condition on variables that appear in the identifiable part of the imputation model (x) or in complete auxiliary variables passed on via the umx argument. It is not possible to specify models where the offset depends on incomplete auxiliary variables.

For an MNAR alternative see also mice.impute.ri.

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Margarita Moreno-Betancur, Stef van Buuren, Ian R. White, 2020.

References

Tompsett, D. M., Leacy, F., Moreno-Betancur, M., Heron, J., & White, I. R. (2018). On the use of the not-at-random fully conditional specification (NARFCS) procedure in practice. Statistics in Medicine, 37(15), 2338-2353. doi:10.1002/sim.7643.

Van Buuren, S., Boshuizen, H.C., Knook, D.L. (1999) Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine, 18, 681–694.

See Also

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mpmm(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Examples

# 1: Example with no auxiliary data: only pass unidentifiable model specification (ums)

# Specify argument to pass on to mnar imputation functions via "blots" argument
mnar.blot <- list(X = list(ums = "-4"), Y = list(ums = "2+1*ZCat1-3*ZCat2"))

# Run NARFCS by using mnar imputation methods and passing argument via blots
impNARFCS <- mice(mnar_demo_data,
  method = c("mnar.logreg", "mnar.norm", ""),
  blots = mnar.blot, seed = 234235, print = FALSE
)

# Obtain MI results: Note they coincide with those from old version at
# https://github.com/moreno-betancur/NARFCS
pool(with(impNARFCS, lm(Y ~ X + Z)))$pooled$estimate

# 2: Example passing also auxiliary data to MNAR procedure (umx)
# Assumptions:
# - Auxiliary data are complete, no missing values
# - Auxiliary data are a numeric matrix
# - Auxiliary data have same number of rows as x
# - Auxiliary data have no overlapping variable names with x

# Specify argument to pass on to mnar imputation functions via "blots" argument
aux <- matrix(0:1, nrow = nrow(mnar_demo_data))
dimnames(aux) <- list(NULL, "even")
mnar.blot <- list(
  X = list(ums = "-4"),
  Y = list(ums = "2+1*ZCat1-3*ZCat2+0.5*even", umx = aux)
)

# Run NARFCS by using mnar imputation methods and passing argument via blots
impNARFCS <- mice(mnar_demo_data,
  method = c("mnar.logreg", "mnar.norm", ""),
  blots = mnar.blot, seed = 234235, print = FALSE
)

# Obtain MI results: As expected they differ (slightly) from those
# from old version at https://github.com/moreno-betancur/NARFCS
pool(with(impNARFCS, lm(Y ~ X + Z)))$pooled$estimate

Imputation by multivariate predictive mean matching

Description

Imputes multivariate incomplete data among which there are specific relations, for instance, polynomials, interactions, range restrictions and sum scores.

Usage

mice.impute.mpmm(data, format = "imputes", ...)

Arguments

Details

This function implements the predictive mean matching and applies canonical regression analysis to select donors fora set of missing variables. In general, canonical regressionanalysis looks for a linear combination of covariates that predicts a linear combination of outcomes (a set of missing variables) optimally in a least-square sense (Israels, 1987). The predicted value of the linear combination of the set of missing variables would be applied to perform predictive mean matching.

Value

A matrix with imputed data, which has ncol(y) columns and sum(wy) rows.

Note

The function requires variables in the block have the same missingness pattern. If there are more than one missingness pattern, the function will return a warning.

Author(s)

Mingyang Cai and Gerko Vink

See Also

mice.impute.pmm Van Buuren, S. (2018). Flexible Imputation of Missing Data. Second Edition. Chapman & Hall/CRC. Boca Raton, FL.

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.norm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Examples

# simulate data
beta2 <- beta1 <- .5
x <- rnorm(1000)
e <- rnorm(1000, 0, 1)
y <- beta1 * x + beta2 * x^2 + e
dat <- data.frame(y = y, x = x, x2 = x^2)
m <- as.logical(rbinom(1000, 1, 0.25))
dat[m, c("x", "x2")] <- NA

# impute
blk <- list("y", c("x", "x2"))
meth <- c("", "mpmm")
imp <- mice(dat, blocks = blk, method = meth, print = FALSE,
    m = 2, maxit = 2)

# analyse and check
summary(pool(with(imp, lm(y ~ x + x2))))
with(dat, plot(x, x2, col = mdc(1)))
with(complete(imp), points(x[m], x2[m], col = mdc(2)))

Imputation by Bayesian linear regression

Description

Calculates imputations for univariate missing data by Bayesian linear regression, also known as the normal model.

Usage

mice.impute.norm(y, ry, x, wy = NULL, ...)

Arguments

Details

Imputation of y by the normal model by the method defined by Rubin (1987, p. 167). The procedure is as follows:

1. Calculate the cross-product matrix S=X_{obs}'X_{obs}.

2. Calculate V = (S+{diag}(S)\kappa)^{-1}, with some small ridge parameter \kappa.

3. Calculate regression weights \hat\beta = VX_{obs}'y_{obs}.

4. Draw a random variable \dot g \sim \chi^2_\nu with \nu=n_1 - q.

5. Calculate \dot\sigma^2 = (y_{obs} - X_{obs}\hat\beta)'(y_{obs} - X_{obs}\hat\beta)/\dot g.

6. Draw q independent N(0,1) variates in vector \dot z_1.

7. Calculate V^{1/2} by Cholesky decomposition.

8. Calculate \dot\beta = \hat\beta + \dot\sigma\dot z_1 V^{1/2}.

9. Draw n_0 independent N(0,1) variates in vector \dot z_2.

10. Calculate the n_0 values y_{imp} = X_{mis}\dot\beta + \dot z_2\dot\sigma.

Using mice.impute.norm for all columns emulates Schafer's NORM method (Schafer, 1997).

Value

Vector with imputed data, same type as y, and of length sum(wy)

Author(s)

Stef van Buuren, Karin Groothuis-Oudshoorn

References

Rubin, D.B (1987). Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons.

Schafer, J.L. (1997). Analysis of incomplete multivariate data. London: Chapman & Hall.

See Also

Other univariate imputation functions: mice.impute.cart(), mice.impute.lasso.logreg(), mice.impute.lasso.norm(), mice.impute.lasso.select.logreg(), mice.impute.lasso.select.norm(), mice.impute.lda(), mice.impute.logreg(), mice.impute.logreg.boot(), mice.impute.mean(), mice.impute.midastouch(), mice.impute.mnar.logreg(), mice.impute.mpmm(), mice.impute.norm.boot(), mice.impute.norm.nob(), mice.impute.norm.predict(), mice.impute.pmm(), mice.impute.polr(), mice.impute.polyreg(), mice.impute.quadratic(), mice.impute.rf(), mice.impute.ri()

Imputation by linear regression, bootstrap method

Description

Imputes univariate missing data using linear regression with bootstrap

Usage

mice.impute.norm.boot(y, ry, x, wy = NULL, ...)