---
title: "Checking a measurement-error correction"
description: "The measurement-error toolkit assumes you know the error size. Estimating reliability from replicates, checking whether to correct, and sensitivity analysis."
date: "2026-05-28 09:00"
date-modified: "2026-09-27"
categories: ["R", "ecology tutorial", "regression", "measurement error", "reproducibility"]
image: thumbnail.png
image-alt: "Single panel line chart: corrected slope against assumed reliability, falling from about 2.8 at a reliability of 0.5, steeply at first and then more gently, to about 1.4 at 1.0; a dashed horizontal line marks the true slope of 2.0, and the one highlighted point sits on that line at a reliability of 0.7."
---
*Corrected 27 September 2026: the summary advised leaving the estimate uncorrected at high reliability, which the simulation here does not support (the corrected slope still has the lower error at a reliability of 0.99); it now says the gain shrinks to thousandths and the trade should be checked.*
Every correction in this cluster took one number as given. The moment correction divided by the reliability. Deming regression asked for the ratio of the two error variances. SIMEX calibrated its whole simulation in multiples of the measurement-error variance. None of them can start without a number describing how large the error is, and none of them can recover that number from the error-prone data alone. This closing post is about where the number comes from, whether the correction is worth making once you have it, and what to do when you cannot get it at all.
```{r}
#| label: setup
#| code-fold: true
#| code-summary: "Setup: packages, colours and plot theme"
#| results: hide
#| message: false
#| warning: false
suppressMessages({library(ggplot2); library(dplyr); library(tidyr)})
theme_te <- theme_minimal(base_size = 12) +
theme(
plot.background = element_rect(fill = "#f5f4ee", colour = NA),
panel.background = element_rect(fill = "#f5f4ee", colour = NA),
panel.grid.minor = element_blank(),
plot.title = element_text(face = "bold"), legend.position = "top")
col_naive <- "#b5534e"; col_corr <- "#2f8f63"; col_true <- "#275139"; col_acc <- "#cda23f"
```
## Estimating reliability from repeated measurements
The clean way to learn the size of measurement error is to measure the same units more than once. If you weigh each animal three times, or send each soil sample to the lab in triplicate, the spread *within* a unit is measurement error and the spread *between* units is real variation. A one-way analysis of variance separates the two.
```{r}
#| label: replicates
set.seed(7314)
m <- 80 # units measured repeatedly
r <- 3 # measurements per unit
sdx <- 3; sdu <- 2 # true between-unit sd 3, error sd 2
xt <- rnorm(m, 10, sdx) # latent true value of each unit
dat <- data.frame(
unit = factor(rep(1:m, each = r)),
w = rep(xt, each = r) + rnorm(m * r, 0, sdu)
)
fit <- aov(w ~ unit, data = dat)
ms <- summary(fit)[[1]][, "Mean Sq"]
ms_b <- ms[1]; ms_w <- ms[2]
var_u_hat <- ms_w # within-unit mean square estimates error variance
var_x_hat <- (ms_b - ms_w) / r # between minus within, scaled, estimates true variance
icc <- var_x_hat / (var_x_hat + var_u_hat)
```
The within-unit mean square estimates the measurement-error variance directly, at `r round(var_u_hat, 2)` against a true value of `4`. The between-unit component recovers `r round(var_x_hat, 2)` for a true `9`. The between-unit share of the total of the two, `r round(icc, 3)` here against a true `0.692`, is the reliability. This quantity is the intraclass correlation coefficient, and it is exactly the reliability the first post divided by. The estimate carries its own sampling error, wider for the between-unit part because it rests on only `r m` unit means, which is itself worth remembering: the reliability you plug into a correction is a guess with a confidence interval, not a constant.
Replicates buy a second thing. Averaging the `r r` measurements per unit gives a predictor whose error variance is smaller by a factor of `r r`, lifting its reliability from about `r round(icc, 2)` for a single reading to `r round(var_x_hat / (var_x_hat + var_u_hat / r), 3)` for the mean. Measuring twice and averaging is often a cheaper route to a usable predictor than any correction applied afterwards.
## Is the correction worth making?
A correction is not free. It removes bias, but it inflates the variance of the estimate, because it divides by a reliability that is itself estimated and less than one. When the error is small the bias it removes is tiny, so whatever margin it wins shrinks towards nothing, and whether it is still worth having is a question to check rather than assume. The way to see the trade is to compare the naive and corrected slopes by root mean squared error, which folds bias and variance into one number, across a range of reliabilities.
```{r}
#| label: worth-it
b1 <- 2; b0 <- 1; n <- 200; sde <- 2; sdx2 <- 3; varx <- sdx2^2
rel_levels <- c(0.5, 0.6, 0.7, 0.8, 0.9, 0.95, 0.99)
reps <- 600
res <- data.frame()
set.seed(202)
for (rel in rel_levels) {
varu <- varx * (1 - rel) / rel
bn <- numeric(reps); bc <- numeric(reps)
for (k in seq_len(reps)) {
x <- rnorm(n, 10, sdx2)
y <- b0 + b1 * x + rnorm(n, 0, sde)
w <- x + rnorm(n, 0, sqrt(varu))
bn[k] <- coef(lm(y ~ w))[2]
lam <- (var(w) - varu) / var(w)
bc[k] <- bn[k] / lam
}
res <- rbind(res, data.frame(rel,
rmse_naive = sqrt(mean((bn - b1)^2)),
rmse_corr = sqrt(mean((bc - b1)^2))))
}
```
```{r}
#| label: worth-numbers
#| include: false
r_lo <- res[res$rel == 0.5, ]; r_hi <- res[res$rel == 0.99, ]
```
At a reliability of `0.5` the naive slope is badly biased and its error is `r round(r_lo$rmse_naive, 3)`, while the corrected slope cuts that to `r round(r_lo$rmse_corr, 3)`: correcting is clearly worth it. At a reliability of `0.99` the margin has nearly gone. The naive error is already down to `r round(r_hi$rmse_naive, 3)` and the corrected version sits at `r round(r_hi$rmse_corr, 3)`, so correction is still ahead, but by thousandths rather than by a factor of three. The variance cost is real and present at every reliability here: the corrected slope always has the wider spread. It stays ahead because even a reliability of `0.99` leaves enough attenuation to outweigh that extra spread, which is a reminder of how stubborn the bias is, not a licence to skip the check.
```{r}
#| label: fig-worth
#| fig-cap: "Root mean squared error of the slope against the reliability of the predictor, for the naive estimate (red) and the moment-corrected estimate (green). Correction wins decisively when reliability is low and keeps a narrowing lead as reliability approaches one, where there is little bias left to remove."
#| fig-alt: "Two descending curves of root mean squared error against reliability from 0.5 to 0.99. The red naive curve starts high near one and falls steeply. The green corrected curve stays low and flat throughout. The gap between them is large on the left and narrows almost to nothing on the right, with the green curve still slightly lower throughout."
#| fig-width: 7
#| fig-height: 4.6
d1 <- pivot_longer(res, c(rmse_naive, rmse_corr),
names_to = "estimator", values_to = "rmse")
d1$estimator <- factor(d1$estimator, levels = c("rmse_naive", "rmse_corr"),
labels = c("naive (biased)", "corrected"))
ggplot(d1, aes(rel, rmse, colour = estimator)) +
geom_line(linewidth = 1.1) + geom_point(size = 2.4) +
scale_colour_manual(values = c("naive (biased)" = col_naive, "corrected" = col_corr),
name = NULL) +
labs(title = "When is correcting worth it?",
x = "reliability of the predictor",
y = "root mean squared error of the slope") +
theme_te
```
## When you cannot measure the error
Sometimes there are no replicates and no validation sample, and the error variance is genuinely unknown. You are not stuck, but you cannot produce a single corrected number and call it the answer. What you can do is show how the corrected estimate moves as the assumed reliability moves, and let the reader see the dependence rather than hiding it.
```{r}
#| label: sensitivity
set.seed(551)
rel_true <- 0.7
varu_true <- varx * (1 - rel_true) / rel_true
x <- rnorm(n, 10, sdx2)
y <- b0 + b1 * x + rnorm(n, 0, sde)
w <- x + rnorm(n, 0, sqrt(varu_true))
b_naive <- coef(lm(y ~ w))[2]
assumed_rel <- seq(0.5, 1.0, by = 0.05)
corrected <- b_naive / assumed_rel # correction under each assumed reliability
```
The naive slope is `r round(b_naive, 3)`. Sweeping the assumed reliability from `0.5` to `1.0` sends the corrected slope from `r round(max(corrected), 3)` down to `r round(min(corrected), 3)`, the latter being just the naive value when you assume no error at all. Only at the true reliability of `0.7` does the correction land near the real slope, at `r round(b_naive / rel_true, 3)`. The band is wide, and that width is the honest result: with an unknown error variance the data are consistent with a broad range of slopes, and the right output is that range together with a statement of what reliability each end assumes.
```{r}
#| label: fig-sensitivity
#| fig-cap: "The corrected slope as a function of the reliability you assume, for a single dataset. The dashed line is the true slope; the dark point on the right is the naive estimate, which assumes no error; the green point marks the true reliability. Without external information to fix the reliability, the whole curve is admissible."
#| fig-alt: "A gold curve of corrected slope falling as assumed reliability rises from 0.5 to 1.0. A horizontal dashed line marks the true slope of two, crossed by the curve near reliability 0.7 where a green point sits. A dark point at the far right, at reliability one, marks the naive estimate below the true slope."
#| fig-width: 7
#| fig-height: 4.6
sens <- data.frame(assumed_rel, corrected)
ggplot(sens, aes(assumed_rel, corrected)) +
geom_hline(yintercept = b1, colour = col_true, linewidth = 0.5, linetype = "22") +
geom_line(colour = col_acc, linewidth = 1.2) +
geom_point(colour = col_acc, size = 2.4) +
annotate("point", x = 1.0, y = b_naive, colour = "#16241d", size = 3) +
annotate("point", x = rel_true, y = b_naive / rel_true, colour = col_true, size = 3.2) +
annotate("text", x = 0.99, y = b_naive, label = "naive\n(assume no error)",
hjust = 1, vjust = 1.3, size = 3.3, lineheight = 0.9) +
annotate("text", x = rel_true, y = b_naive / rel_true, label = "true reliability",
hjust = -0.08, vjust = -1.3, size = 3.3, colour = col_true) +
scale_y_continuous(expand = expansion(mult = c(0.12, 0.05))) +
labs(title = "Sensitivity when the error variance is a guess",
x = "assumed reliability", y = "corrected slope") +
theme_te
```
## What to take away
Measurement-error correction is only as trustworthy as the error information behind it. If you can measure the same units more than once, one-way analysis of variance turns the replicates into a reliability estimate and, by averaging, into a better predictor at the same time. Before reaching for any correction, ask whether it is worth it: at high reliability the bias is small and the gain from correcting shrinks to thousandths, so the correction changes little either way; check the trade rather than assume it. And when the error variance cannot be pinned down, do not manufacture a single corrected figure. Report the corrected estimate across a plausible range of reliabilities and state the assumption at each end, so the reader can judge how much the conclusion leans on a number you had to guess.
That is the thread running through the whole cluster. Regression dilution, errors-in-variables, and SIMEX are different machines for the same task, and each needs to be told how large the error is. The measurement of the measurement error is not a preliminary to the analysis; it is the part that makes the analysis possible.
## References
- Fuller 1987 Measurement Error Models, ISBN 978-0-471-86187-4
- Carroll, Ruppert, Stefanski, Crainiceanu 2006 Measurement Error in Nonlinear Models, 2nd ed, ISBN 978-1-58488-633-4
- Nakagawa, Schielzeth 2010 Biological Reviews 85(4):935-956 (10.1111/j.1469-185X.2010.00141.x)
- Bartlett, Keogh 2018 Statistical Methods in Medical Research 27(6):1695-1708 (10.1177/0962280216667764)
## Related tutorials
- [Measurement error and regression dilution](../measurement-error-regression-dilution/)
- [Errors-in-variables and Deming regression](../errors-in-variables-deming-regression/)
- [Correcting measurement error with SIMEX](../simex-measurement-error-correction/)
- [Latent variables in an SEM, from scratch in R](../latent-variables-in-an-sem/)
