Errors-in-variables and Deming regression

ecology tutorial

regression

measurement error

When both variables carry error, ordinary regression is biased either way. Deming regression uses the error-variance ratio to recover the true slope in R.

Author

Tidy Ecology

Published

2026-07-30

The previous post put measurement error on one variable and watched the slope shrink. Field data rarely obliges by keeping one axis clean. Compare two field methods for canopy cover and both are noisy. Calibrate a cheap sensor against a reference and the reference has error too. Fit wing length against body mass and both traits were measured with callipers. When error sits on both variables, ordinary least squares no longer has a known direction of bias you can reason about from one side. It has two answers, one for each way you run it, and the truth lies between them.

This post shows the bracketing, then uses Deming regression to recover the slope when the ratio of the two error variances is known. It ends on the assumption that ratio represents, because that assumption is the whole ballgame.

Both axes carry error

Two quantities sit on an exact line: y equals 2 plus 1.5 times x. We measure neither directly. We record w, which is x plus error, and z, which is y plus error. The two errors have different sizes; that difference will matter.

set.seed(5231)
n   <- 200
a0  <- 2; b1 <- 1.5
xt  <- rnorm(n, mean = 12, sd = 4)      # true, unobserved
yt  <- a0 + b1 * xt                      # exact functional relationship
w   <- xt + rnorm(n, 0, sd = 2)          # recorded x, error variance 4
z   <- yt + rnorm(n, 0, sd = 4)          # recorded y, error variance 16

Run ordinary regression both ways. Regressing z on w treats the horizontal axis as error-free and, as before, returns an attenuated slope. Regressing w on z treats the vertical axis as error-free; inverting that fit to express it as a slope of z on w returns something too steep. The true slope is trapped between them.

f_yx  <- lm(z ~ w)                       # forward: attenuated
f_xy  <- lm(w ~ z)                       # inverse
b_fwd <- unname(coef(f_yx)[2])
b_inv <- unname(1 / coef(f_xy)[2])       # inverse fit expressed as z-on-w slope

Forward regression gives 1.239 and the inverted inverse regression gives 2.188, so the true 1.5 is bracketed: 1.239 below and 2.188 above. Neither is the answer, and picking whichever you ran first is picking a bias.

Deming regression with a known error ratio

Deming regression fits the line by accounting for error in both axes. It needs one piece of outside information: the ratio of the two error variances, $\delta = \sigma_v^2 / \sigma_u^2$, error on y over error on x. Given $\delta$, the slope has a closed form,

\[ \hat\beta \;=\; \frac{(s_{zz} - \delta\, s_{ww}) + \sqrt{(s_{zz} - \delta\, s_{ww})^2 + 4\,\delta\, s_{wz}^2}}{2\, s_{wz}}, \]

using the sample variances and covariance of the recorded data. In our design the error variances are 16 and 4, so $\delta$ = 4. A jackknife supplies the standard error.

delta_true <- 4
dem <- deming(w, z, delta_true)

jk     <- sapply(seq_len(n), function(i) deming(w[-i], z[-i], delta_true)["slope"])
se_dem <- sqrt((n - 1) / n * sum((jk - mean(jk))^2))

Deming returns a slope of 1.546 (jackknife standard error 0.101), back at the true 1.5 within noise. The line sits between the two ordinary fits, exactly where the truth is.

inv_a <- mean(z) - b_inv * mean(w)
lines <- data.frame(
  method    = factor(c("true", "OLS z on w", "OLS w on z", "Deming"),
                     levels = c("true", "OLS z on w", "OLS w on z", "Deming")),
  intercept = c(a0, coef(f_yx)[1], inv_a, unname(dem["intercept"])),
  slope     = c(b1, b_fwd, b_inv, b_dem)
)
ggplot(data.frame(w = w, z = z), aes(w, z)) +
  geom_point(alpha = 0.35, size = 1.1, colour = "#5d6b61") +
  geom_abline(data = lines, aes(intercept = intercept, slope = slope, colour = method),
              linewidth = 1.0) +
  scale_colour_manual(values = c("true" = col_true, "OLS z on w" = col_fwd,
                                 "OLS w on z" = col_inv, "Deming" = col_dem), name = NULL) +
  labs(title = "Ordinary regression brackets the truth; Deming recovers it",
       x = "recorded x (with error)", y = "recorded y (with error)") +
  theme_te

Scatter of recorded y against recorded x with four fitted lines through a diffuse point cloud. A green true line has a red shallower line below it and a blue steeper line above it, one on each side. A gold Deming line overlaps the green true line. — Figure 1: Recorded y against recorded x. Ordinary regression of z on w (red) is too shallow and of w on z (blue) is too steep; they bracket the true line (green). Deming regression with the correct error ratio (gold) lands on the truth.

The ratio is an assumption, not a result

Two special cases of Deming are common in ecology. Setting $\delta$ = 1 gives the major axis, which minimises perpendicular distances and assumes the two errors are equal in size. The standardised major axis (reduced major axis) sets the slope to the ratio of standard deviations, sd(z) / sd(w), and is the usual tool for allometric scaling. Both are Deming under a particular guess at $\delta$.

Here the errors are not equal (16 against 4), so the major axis, at 1.906, overshoots the truth by 0.406, and the standardised major axis gives 1.646. Neither is wrong as a method; each answers the question its assumption poses. The point is that the answer moves with the assumption.

Sweeping $\delta$ makes this concrete. The Deming slope slides monotonically from the inverse regression (all error on x) to the forward regression (all error on y), passing through the major axis at $\delta$ = 1 and hitting the truth only at the correct ratio.

delta_grid <- 10^seq(-1.5, 1.5, length.out = 60)
d2 <- data.frame(delta = delta_grid,
                 slope = sapply(delta_grid, function(d) deming(w, z, d)["slope"]))
marks <- data.frame(delta = c(1, 4),
                    slope = c(b_ma, b_dem),
                    lab   = c("major axis (ratio 1)", "correct ratio 4"))
ggplot(d2, aes(delta, slope)) +
  geom_hline(yintercept = b1, colour = col_true, linewidth = 0.5) +
  geom_line(colour = col_dem, linewidth = 1.1) +
  geom_point(data = marks, size = 3, colour = "#16241d") +
  geom_text(data = marks, aes(label = lab), vjust = -1, size = 3.3, colour = "#16241d") +
  scale_x_log10() +
  labs(title = "The Deming slope depends entirely on the assumed error ratio",
       x = "assumed error-variance ratio (log scale)", y = "Deming slope") +
  theme_te

Curve of Deming slope against assumed error-variance ratio on a logarithmic axis. The curve falls smoothly from left to right and crosses a horizontal true-slope line at one point. Two marked points show the major-axis value at ratio one, above the truth, and the correct-ratio value on the truth. — Figure 2: Deming slope as the assumed error ratio is varied on a log scale. The horizontal line is the true slope; the estimate matches it only at the correct ratio, and drifts elsewhere. The two limits are the two ordinary regressions.

What to take away

With error on both axes, ordinary regression brackets the truth but never delivers it, and which side you land on is decided by which variable you happen to call the response. Deming regression removes that arbitrariness once you supply the ratio of error variances. Everything then rests on that ratio: the major axis assumes it is one, the standardised major axis assumes error scales with spread, and a Deming fit assumes whatever value you feed it.

So the honest boundary is sharp. If replicate measurements or instrument specifications pin the error variances, Deming is a clean fix. If the ratio is unknown, the model is not identified from the recorded data alone; you are choosing a slope by choosing an assumption, and the choice should be stated rather than buried. The next post keeps error on a single predictor but drops the tidy variance formula, correcting the bias by simulation when no closed form is at hand.

References

Deming 1943 Statistical Adjustment of Data, ISBN 978-0-486-64685-5
Warton, Wright, Falster, Westoby 2006 Biological Reviews 81(2):259-291 (10.1017/S1464793106007007)
Carroll, Ruppert, Stefanski, Crainiceanu 2006 Measurement Error in Nonlinear Models, 2nd ed, ISBN 978-1-58488-633-4
Linnet 1993 Clinical Chemistry 39(3):424-432 (10.1093/clinchem/39.3.424)

Where to go next

When the model is more elaborate than a straight line, or when the error affects the predictor through a nonlinear term, the closed-form corrections stop applying. A simulation-based method sidesteps that by adding error on purpose and extrapolating back. The closing post then returns to the question every correction here quietly assumed: where the error variance actually comes from.

--- title: "Errors-in-variables and Deming regression" description: "When both variables carry error, ordinary regression is biased either way. Deming regression uses the error-variance ratio to recover the true slope in R." date: "2026-07-30 10:00" categories: ["R", "ecology tutorial", "regression", "measurement error"] image: thumbnail.png --- The previous post put measurement error on one variable and watched the slope shrink. Field data rarely obliges by keeping one axis clean. Compare two field methods for canopy cover and both are noisy. Calibrate a cheap sensor against a reference and the reference has error too. Fit wing length against body mass and both traits were measured with callipers. When error sits on *both* variables, ordinary least squares no longer has a known direction of bias you can reason about from one side. It has two answers, one for each way you run it, and the truth lies between them. This post shows the bracketing, then uses Deming regression to recover the slope when the ratio of the two error variances is known. It ends on the assumption that ratio represents, because that assumption is the whole ballgame. ```{r} #| label: setup #| include: false suppressMessages({library(ggplot2); library(dplyr); library(tidyr)}) theme_te <- theme_minimal(base_size = 12) + theme(panel.grid.minor = element_blank(), plot.title = element_text(face = "bold"), legend.position = "top") col_true <- "#2f8f63"; col_fwd <- "#b5534e"; col_inv <- "#4f6d8c"; col_dem <- "#cda23f" deming <- function(w, z, delta) { sxx <- var(w); syy <- var(z); sxy <- cov(w, z) b <- ((syy - delta*sxx) + sqrt((syy - delta*sxx)^2 + 4*delta*sxy^2)) / (2*sxy) c(intercept = mean(z) - b*mean(w), slope = b) } ``` ## Both axes carry error Two quantities sit on an exact line: `y` equals `2` plus `1.5` times `x`. We measure neither directly. We record `w`, which is `x` plus error, and `z`, which is `y` plus error. The two errors have different sizes; that difference will matter. ```{r} #| label: simulate set.seed(5231) n <- 200 a0 <- 2; b1 <- 1.5 xt <- rnorm(n, mean = 12, sd = 4) # true, unobserved yt <- a0 + b1 * xt # exact functional relationship w <- xt + rnorm(n, 0, sd = 2) # recorded x, error variance 4 z <- yt + rnorm(n, 0, sd = 4) # recorded y, error variance 16 ``` Run ordinary regression both ways. Regressing `z` on `w` treats the horizontal axis as error-free and, as before, returns an attenuated slope. Regressing `w` on `z` treats the vertical axis as error-free; inverting that fit to express it as a slope of `z` on `w` returns something too steep. The true slope is trapped between them. ```{r} #| label: ols-both f_yx <- lm(z ~ w) # forward: attenuated f_xy <- lm(w ~ z) # inverse b_fwd <- unname(coef(f_yx)[2]) b_inv <- unname(1 / coef(f_xy)[2]) # inverse fit expressed as z-on-w slope ``` ```{r} #| label: ols-numbers #| include: false ``` Forward regression gives `r round(b_fwd, 3)` and the inverted inverse regression gives `r round(b_inv, 3)`, so the true `1.5` is bracketed: `r round(b_fwd, 3)` below and `r round(b_inv, 3)` above. Neither is the answer, and picking whichever you ran first is picking a bias. ## Deming regression with a known error ratio Deming regression fits the line by accounting for error in both axes. It needs one piece of outside information: the ratio of the two error variances, $\delta = \sigma_v^2 / \sigma_u^2$, error on `y` over error on `x`. Given $\delta$, the slope has a closed form, $$ \hat\beta \;=\; \frac{(s_{zz} - \delta\, s_{ww}) + \sqrt{(s_{zz} - \delta\, s_{ww})^2 + 4\,\delta\, s_{wz}^2}}{2\, s_{wz}}, $$ using the sample variances and covariance of the recorded data. In our design the error variances are `16` and `4`, so $\delta$ = `4`. A jackknife supplies the standard error. ```{r} #| label: deming-fit delta_true <- 4 dem <- deming(w, z, delta_true) jk <- sapply(seq_len(n), function(i) deming(w[-i], z[-i], delta_true)["slope"]) se_dem <- sqrt((n - 1) / n * sum((jk - mean(jk))^2)) ``` ```{r} #| label: deming-numbers #| include: false b_dem <- unname(dem["slope"]) b_ma <- unname(deming(w, z, 1)["slope"]) b_sma <- sign(cov(w, z)) * sd(z) / sd(w) ``` Deming returns a slope of `r round(b_dem, 3)` (jackknife standard error `r round(se_dem, 3)`), back at the true `1.5` within noise. The line sits between the two ordinary fits, exactly where the truth is. ```{r} #| label: fig-bracket #| fig-cap: "Recorded y against recorded x. Ordinary regression of z on w (red) is too shallow and of w on z (blue) is too steep; they bracket the true line (green). Deming regression with the correct error ratio (gold) lands on the truth." #| fig-alt: "Scatter of recorded y against recorded x with four fitted lines through a diffuse point cloud. A green true line has a red shallower line below it and a blue steeper line above it, one on each side. A gold Deming line overlaps the green true line." #| fig-width: 7 #| fig-height: 4.6 inv_a <- mean(z) - b_inv * mean(w) lines <- data.frame( method = factor(c("true", "OLS z on w", "OLS w on z", "Deming"), levels = c("true", "OLS z on w", "OLS w on z", "Deming")), intercept = c(a0, coef(f_yx)[1], inv_a, unname(dem["intercept"])), slope = c(b1, b_fwd, b_inv, b_dem) ) ggplot(data.frame(w = w, z = z), aes(w, z)) + geom_point(alpha = 0.35, size = 1.1, colour = "#5d6b61") + geom_abline(data = lines, aes(intercept = intercept, slope = slope, colour = method), linewidth = 1.0) + scale_colour_manual(values = c("true" = col_true, "OLS z on w" = col_fwd, "OLS w on z" = col_inv, "Deming" = col_dem), name = NULL) + labs(title = "Ordinary regression brackets the truth; Deming recovers it", x = "recorded x (with error)", y = "recorded y (with error)") + theme_te ``` ## The ratio is an assumption, not a result Two special cases of Deming are common in ecology. Setting $\delta$ = `1` gives the **major axis**, which minimises perpendicular distances and assumes the two errors are equal in size. The **standardised major axis** (reduced major axis) sets the slope to the ratio of standard deviations, `sd(z) / sd(w)`, and is the usual tool for allometric scaling. Both are Deming under a particular guess at $\delta$. Here the errors are not equal (`16` against `4`), so the major axis, at `r round(b_ma, 3)`, overshoots the truth by `r round(b_ma - b1, 3)`, and the standardised major axis gives `r round(b_sma, 3)`. Neither is wrong as a method; each answers the question its assumption poses. The point is that the answer moves with the assumption. Sweeping $\delta$ makes this concrete. The Deming slope slides monotonically from the inverse regression (all error on `x`) to the forward regression (all error on `y`), passing through the major axis at $\delta$ = `1` and hitting the truth only at the correct ratio. ```{r} #| label: fig-sensitivity #| fig-cap: "Deming slope as the assumed error ratio is varied on a log scale. The horizontal line is the true slope; the estimate matches it only at the correct ratio, and drifts elsewhere. The two limits are the two ordinary regressions." #| fig-alt: "Curve of Deming slope against assumed error-variance ratio on a logarithmic axis. The curve falls smoothly from left to right and crosses a horizontal true-slope line at one point. Two marked points show the major-axis value at ratio one, above the truth, and the correct-ratio value on the truth." #| fig-width: 7 #| fig-height: 4.6 delta_grid <- 10^seq(-1.5, 1.5, length.out = 60) d2 <- data.frame(delta = delta_grid, slope = sapply(delta_grid, function(d) deming(w, z, d)["slope"])) marks <- data.frame(delta = c(1, 4), slope = c(b_ma, b_dem), lab = c("major axis (ratio 1)", "correct ratio 4")) ggplot(d2, aes(delta, slope)) + geom_hline(yintercept = b1, colour = col_true, linewidth = 0.5) + geom_line(colour = col_dem, linewidth = 1.1) + geom_point(data = marks, size = 3, colour = "#16241d") + geom_text(data = marks, aes(label = lab), vjust = -1, size = 3.3, colour = "#16241d") + scale_x_log10() + labs(title = "The Deming slope depends entirely on the assumed error ratio", x = "assumed error-variance ratio (log scale)", y = "Deming slope") + theme_te ``` ## What to take away With error on both axes, ordinary regression brackets the truth but never delivers it, and which side you land on is decided by which variable you happen to call the response. Deming regression removes that arbitrariness once you supply the ratio of error variances. Everything then rests on that ratio: the major axis assumes it is one, the standardised major axis assumes error scales with spread, and a Deming fit assumes whatever value you feed it. So the honest boundary is sharp. If replicate measurements or instrument specifications pin the error variances, Deming is a clean fix. If the ratio is unknown, the model is not identified from the recorded data alone; you are choosing a slope by choosing an assumption, and the choice should be stated rather than buried. The next post keeps error on a single predictor but drops the tidy variance formula, correcting the bias by simulation when no closed form is at hand. ## References - Deming 1943 Statistical Adjustment of Data, ISBN 978-0-486-64685-5 - Warton, Wright, Falster, Westoby 2006 Biological Reviews 81(2):259-291 (10.1017/S1464793106007007) - Carroll, Ruppert, Stefanski, Crainiceanu 2006 Measurement Error in Nonlinear Models, 2nd ed, ISBN 978-1-58488-633-4 - Linnet 1993 Clinical Chemistry 39(3):424-432 (10.1093/clinchem/39.3.424) ## Where to go next When the model is more elaborate than a straight line, or when the error affects the predictor through a nonlinear term, the closed-form corrections stop applying. A simulation-based method sidesteps that by adding error on purpose and extrapolating back. The closing post then returns to the question every correction here quietly assumed: where the error variance actually comes from. ## Related tutorials - [Measurement error and regression dilution](../measurement-error-regression-dilution/) - [Correcting measurement error with SIMEX](../simex-measurement-error-correction/) - [Checking a measurement-error correction](../checking-measurement-error-corrections/) - [Linear models: t-test and ANOVA as regression](../linear-models-t-test-anova/)