High correlation check

[markseeto]

I've seen some behaviour from mice that seems strange to me - high correlation logged events for predictor variables that are not used in the same model. I'm using mice version 3.11.0.

In the following example, x1 and x2 have very high correlation but they are not used in the same model, so I would not have expected a problem.

set.seed(1)

d <- data.frame(x1 = 1:10,
                x2 = c(1:9, 10.1))

cor(d)
##           x1        x2
## x1 1.0000000 0.9999608
## x2 0.9999608 1.0000000

d$y1 <- d$x1 + rnorm(nrow(d), 0, 0.1)
d$y2 <- d$x2 + rnorm(nrow(d), 0, 0.1)

d[1, "y1"] <- NA
d[2, "y2"] <- NA

d

##    x1   x2       y1        y2
## 1   1  1.0       NA  1.151178
## 2   2  2.0 2.018364        NA
## 3   3  3.0 2.916437  2.937876
## 4   4  4.0 4.159528  3.778530
## 5   5  5.0 5.032951  5.112493
## 6   6  6.0 5.917953  5.995507
## 7   7  7.0 7.048743  6.998381
## 8   8  8.0 8.073832  8.094384
## 9   9  9.0 9.057578  9.082122
## 10 10 10.1 9.969461 10.159390


meth <- c(x1 = "", x2 = "", y1 = "norm", y2 = "norm")

predMat <- matrix(0, nrow = ncol(d), ncol = ncol(d))
rownames(predMat) <- colnames(predMat) <- names(d)
predMat["y1", "x1"] <- 1
predMat["y2", "x2"] <- 1

predMat

##    x1 x2 y1 y2
## x1  0  0  0  0
## x2  0  0  0  0
## y1  1  0  0  0
## y2  0  1  0  0

mice.out <- mice(data = d, m = 5, maxit = 10,
                 method = meth, predictorMatrix = predMat)

## Warning message:
## Number of logged events: 51

mice.out$loggedEvents

##    it im dep      meth                                                       out
## 1   0  0     collinear                                                        x2
## 2   1  1  y1      norm All predictors are constant or have too high correlation.

# (other rows omitted)

complete(mice.out, action = "all")  # bad imputed values

Since x1 and x2 don't appear as predictor variables in the same model, I don't understand why their high correlation causes a logged event. We also see that the imputed values are bad.

Of course this is an extreme example and we could run mice separately for y1 and y2, but in another situation they could have predictor variables in common.

Maybe I'm misunderstanding something. Is this behaviour intended?

Thanks.

[gerkovink]

You have basically recreated x1 and x2, but with extra steps.

set.seed(1)
d <- data.frame(x1 = 1:10,
                x2 = c(1:9, 10.1))
d$y1 <- d$x1 + rnorm(nrow(d), 0, 0.1)
d$y2 <- d$x2 + rnorm(nrow(d), 0, 0.1)

cor(d)
#>           x1        x2        y1        y2
#> x1 1.0000000 0.9999608 0.9996862 0.9993470
#> x2 0.9999608 1.0000000 0.9995650 0.9994010
#> y1 0.9996862 0.9995650 1.0000000 0.9985884
#> y2 0.9993470 0.9994010 0.9985884 1.0000000

^{Created on 2020-10-09 by the reprex package (v0.3.0)}

Add more variance and up the threshold to avoid x2 being kicked out.

library(mice)
#> 
#> Attaching package: 'mice'
#> The following objects are masked from 'package:base':
#> 
#>     cbind, rbind
set.seed(1)
d <- data.frame(x1 = 1:10,
                x2 = c(1:9, 10.1))
d$y1 <- d$x1 + rnorm(nrow(d), 0, 2)
d$y2 <- d$x2 + rnorm(nrow(d), 0, 2)

cor(d)
#>           x1        x2        y1        y2
#> x1 1.0000000 0.9999608 0.9104953 0.8441215
#> x2 0.9999608 1.0000000 0.9091064 0.8443825
#> y1 0.9104953 0.9091064 1.0000000 0.6746252
#> y2 0.8441215 0.8443825 0.6746252 1.0000000

meth <- c(x1 = "", x2 = "", y1 = "norm", y2 = "norm")

predMat <- matrix(0, nrow = ncol(d), ncol = ncol(d))
rownames(predMat) <- colnames(predMat) <- names(d)
predMat["y1", "x1"] <- 1
predMat["y2", "x2"] <- 1

predMat
#>    x1 x2 y1 y2
#> x1  0  0  0  0
#> x2  0  0  0  0
#> y1  1  0  0  0
#> y2  0  1  0  0

mice.out <- mice(data = d, 
                 m = 5, 
                 maxit = 10,
                 method = meth, 
                 predictorMatrix = predMat, 
                 print = FALSE, 
                 threshold = 1) # multicollinearity threshold
mice.out$predictorMatrix
#>    x1 x2 y1 y2
#> x1  0  0  0  0
#> x2  0  0  0  0
#> y1  1  0  0  0
#> y2  0  1  0  0

mice.out$loggedEvents
#> NULL

^{Created on 2020-10-09 by the reprex package (v0.3.0)}

[markseeto]

Thanks for your reply Gerko. I will use the threshold argument.

What I was suggesting was that, as far as I can see, the high correlation between x1 and x2 is not a problem because none of the imputation models contain both x1 and x2 as predictors. So to me it would seem preferable that neither of x1 and x2 is kicked out even when threshold has not been changed from its default value. But maybe there is another problem with high correlations that I'm not aware of, even when the highly correlated variables are not predictors in the same model.

But I can accept that I will have to set threshold.

Thanks.

[stefvanbuuren]

The behaviour is as intended. There are two different things happening in your code:

1. mice() first checks the data for collinearity at initialisation. All variables that are a predictor anywhere in the imputation model and that are not passive enter this check (lines 58-61 in edit.setup.R). In your example, x2 is a predictor, so it is part of the check. mice() is not smart enough to do separate checks per specified imputation model. If you do not want to check at all, specify remove.collinear = FALSE.
2. The message All predictors are constant or have too high correlation. results from find.collinear() during the iterations. You may evade these by increasing threshold, as indicated by @gerkovink.

[markseeto]

Thanks for the explanation Stef.

[stefvanbuuren]

Related #150

[stefvanbuuren]

On rereading, I think there is an error in my advice above: It is the remove.lindep() function that runs during the iterations (not find.collinear()). Hence use the maxcor argument to change variable exclusion settings during imputation.