---
title: "Heteroscedasticity and limiting factors"
description: "When an ecological variable sets an upper bound, the mean slope is about half of it. Upper quantiles recover the limiting relationship in base R."
date: "2026-08-06 10:00"
categories: [quantile regression, limiting factors, R, ecology tutorial, ggplot2]
image: thumbnail.png
image-alt: "A wedge-shaped scatter under a dotted ceiling, with upper quantile lines tracking the bound and a shallow least-squares line below."
---
Many ecological scatter plots are wedge-shaped: points spread widely beneath an upper edge that rises with the predictor. The standard reading is the principle of limiting factors. At any site the response is held down by whichever resource is scarcest; the measured variable sets a ceiling on what is possible, and other unmeasured factors push most sites below it. The ecological information then lives in the upper edge, not the middle. A mean regression, which chases the centre, systematically understates the effect of the measured factor when that factor is not the active constraint (Thomson and colleagues 1996; Cade and Noon 2003).
This post shows the pattern with a simple generating model, measures how badly least squares understates the ceiling, and uses upper quantiles to recover it. It also states the honest limit: the same wedge is produced by ordinary changing variance, and the data alone cannot tell the two apart.
## A ceiling with scatter beneath it
We build a ceiling that rises with the gradient and let each point fall a random fraction of the way below it. With the fraction drawn uniformly, the tau-quantile of the response given `x` is exactly tau times the ceiling, so the quantile slope should equal tau times the ceiling slope.
```{r}
#| label: setup
#| echo: false
#| message: false
#| warning: false
library(ggplot2); library(dplyr); library(tidyr)
ink <- "#16241d"; body <- "#2c3a31"; forest <- "#275139"
sage <- "#93a87f"; paper <- "#f5f4ee"; line <- "#dad9ca"
faint <- "#5d6b61"; warm <- "#b5534e"; gold <- "#c9b458"
theme_te <- function(base_size = 12) {
theme_minimal(base_size = base_size) +
theme(plot.background = element_rect(fill = paper, colour = NA),
panel.background = element_rect(fill = paper, colour = NA),
panel.grid.major = element_line(colour = line, linewidth = 0.3),
panel.grid.minor = element_blank(),
axis.text = element_text(colour = faint),
axis.title = element_text(colour = body),
plot.subtitle = element_text(colour = "#46604a"),
legend.position = "top", legend.key = element_blank(),
legend.text = element_text(colour = body),
legend.title = element_text(colour = body))
}
rho <- function(u, tau) u * (tau - (u < 0))
qslope <- function(x, y, tau) {
obj <- function(b) sum(rho(y - (b[1] + b[2] * x), tau))
optim(unname(coef(lm(y ~ x))), obj, method = "Nelder-Mead",
control = list(reltol = 1e-12, maxit = 8000))$par[2]
}
```
```{r}
#| label: data
set.seed(4203)
n <- 300
x <- runif(n, 1, 20) # resource gradient (for example patch area)
u <- runif(n) # everything else that can limit a site
b0 <- 2; b1 <- 1.5 # ceiling = b0 + b1 * x
y <- (b0 + b1 * x) * u # each site sits somewhere below its ceiling
ols <- lm(y ~ x)
coef(ols)[2]
```
```{r}
#| label: ols-scalars
#| echo: false
r_ols <- round(unname(coef(ols)[2]), 3)
r_ratio <- round(unname(coef(ols)[2]) / b1, 3)
```
The least-squares slope is `r r_ols`, against a true ceiling slope of `r b1`. The ratio is `r r_ratio`, close to one half, exactly what you expect: averaging over a uniform fraction of the ceiling gives half the ceiling slope. Read as an effect size, the mean regression halves the strength of the relationship.
## Upper quantiles recover the ceiling
Fitting quantile slopes across a range of tau shows the slope climbing toward the ceiling slope as tau increases.
```{r}
#| label: quantiles
taus <- c(0.5, 0.75, 0.9, 0.95, 0.99)
qs <- sapply(taus, function(t) qslope(x, y, t))
names(qs) <- taus
round(qs, 3)
```
```{r}
#| label: q-scalars
#| echo: false
r_qs <- round(qs, 3)
r_qth <- round(taus * b1, 3)
```
The slopes are `r r_qs[1]` at the median, rising to `r r_qs[3]` at the ninetieth and `r r_qs[5]` at the ninety-ninth percentile, approaching the ceiling slope of `r b1`. The theoretical values, tau times the ceiling slope, are `r r_qth[1]`, `r r_qth[3]` and `r r_qth[5]`. The upper quantiles estimate the limiting relationship that the mean misses.
```{r}
#| label: fig-wedge
#| echo: false
#| fig-cap: "A wedge under a rising ceiling. Upper quantiles track the bound; least squares (dashed) sits far below it."
#| fig-alt: "Wedge-shaped point cloud beneath a dotted ceiling line, with median, ninetieth and ninety-fifth quantile lines rising toward the ceiling and a shallow dashed least-squares line."
#| fig-width: 7
#| fig-height: 4.6
tt <- c(0.5, 0.9, 0.95)
co <- t(sapply(tt, function(t) { obj <- function(b) sum(rho(y - (b[1] + b[2]*x), t))
optim(unname(coef(lm(y~x))), obj, control = list(reltol = 1e-12, maxit = 8000))$par }))
xg <- seq(1, 20, length.out = 60)
qdf <- do.call(rbind, lapply(seq_along(tt), function(i)
data.frame(x = xg, y = co[i, 1] + co[i, 2] * xg, tau = factor(tt[i]))))
ggplot() +
geom_point(aes(x, y), colour = sage, alpha = 0.6, size = 1.5) +
geom_line(data = data.frame(x = xg, y = b0 + b1 * xg), aes(x, y),
colour = ink, linetype = "dotted", linewidth = 0.8) +
geom_abline(intercept = coef(ols)[1], slope = coef(ols)[2],
colour = faint, linetype = "dashed", linewidth = 0.9) +
geom_line(data = qdf, aes(x, y, colour = tau), linewidth = 1) +
scale_colour_manual(values = c("0.5" = gold, "0.9" = forest, "0.95" = warm),
name = "quantile") +
labs(x = "resource gradient", y = "response",
subtitle = "dotted = true ceiling; dashed = least squares") +
theme_te()
```
## The fanning is the signature, and a test for it
Under constant variance every quantile slope is the same and the fits are parallel. A slope that climbs with tau is the fingerprint of a changing spread, here driven by the limiting-factor structure. A case-resampling bootstrap gives a confidence interval for the difference between an upper and a central slope, which tests whether the fan is real.
```{r}
#| label: bootstrap
set.seed(101)
B <- 1200
bs <- replicate(B, {
i <- sample(n, n, TRUE)
c(qslope(x[i], y[i], 0.5), qslope(x[i], y[i], 0.9))
})
diff_ci <- quantile(bs[2, ] - bs[1, ], c(.025, .975))
round(diff_ci, 3)
```
```{r}
#| label: boot-scalars
#| echo: false
r_ci50 <- round(quantile(bs[1, ], c(.025, .975)), 3)
r_ci90 <- round(quantile(bs[2, ], c(.025, .975)), 3)
r_dci <- round(diff_ci, 3)
```
The median slope has interval [`r r_ci50[1]`, `r r_ci50[2]`] and the ninetieth [`r r_ci90[1]`, `r r_ci90[2]`]. Their difference is [`r r_dci[1]`, `r r_dci[2]`], well clear of zero, so the slopes genuinely differ across the distribution. The whole quantile process makes the point visually.
```{r}
#| label: fig-coef-path
#| echo: false
#| fig-cap: "Slope as a function of tau, with a bootstrap band. It rises toward the ceiling slope; the mean slope (dotted) sits near the middle."
#| fig-alt: "Slope on the vertical axis against quantile level on the horizontal axis, rising steadily with a shaded confidence band, a dashed theoretical line and a dotted horizontal least-squares slope."
#| fig-width: 7
#| fig-height: 4.6
tg <- seq(0.1, 0.95, 0.05)
sg <- sapply(tg, function(t) qslope(x, y, t))
set.seed(303); Bp <- 400
band <- sapply(tg, function(t)
quantile(replicate(Bp, { i <- sample(n, n, TRUE); qslope(x[i], y[i], t) }),
c(.025, .975)))
pdf <- data.frame(tau = tg, slope = sg, lo = band[1, ], hi = band[2, ])
ggplot(pdf, aes(tau, slope)) +
geom_ribbon(aes(ymin = lo, ymax = hi), fill = sage, alpha = 0.35) +
geom_line(aes(y = tau * b1), colour = warm, linetype = "dashed", linewidth = 0.8) +
geom_line(colour = forest, linewidth = 1) +
geom_hline(yintercept = coef(ols)[2], colour = faint, linetype = "dotted") +
labs(x = "quantile", y = "slope",
subtitle = "dashed = tau times ceiling slope; dotted = least-squares slope") +
theme_te()
```
## The honest limit
The upper quantile is an honest description of the upper edge of the data. Reading it as a limiting factor is an ecological interpretation, not a statistical result. A wedge like this can equally be produced by a mean relationship with variance that happens to grow with `x`, with no ceiling and no limiting mechanism at all. The quantile fit is the same in both cases; the fanning slope is the same; the bootstrap interval is the same. Nothing in the data distinguishes a real constraint from ordinary heteroscedasticity. That separation has to come from the ecology: an argument about which resource is scarce, an experiment that manipulates it, or independent knowledge of the mechanism. Quantile regression tells you where the edge is and how steep it is; it does not tell you why the points sit below it (Cade, Noon and Flather 2005).
## References
Cade and Noon 2003 Frontiers in Ecology and the Environment 1(8):412-420 (10.1890/1540-9295(2003)001[0412:AGITQR]2.0.CO;2)
Cade, Terrell and Schroeder 1999 Ecology 80(1):311-323 (10.1890/0012-9658(1999)080[0311:EEOLFW]2.0.CO;2)
Thomson, Weiblen, Thomson, Alfaro and Legendre 1996 Ecology 77(6):1698-1715 (10.2307/2265776)
Scharf, Juanes and Sutherland 1998 Ecology 79(2):448-460 (10.1890/0012-9658(1998)079[0448:IERFTE]2.0.CO;2)
Cade, Noon and Flather 2005 Ecology 86(3):786-800 (10.1890/04-0785)
## Related tutorials
- [Quantile regression in ecology: beyond the mean](../quantile-regression-in-ecology/)
- [Distributional regression with location and scale](../distributional-regression-location-scale/)
- [Checking a quantile regression](../checking-a-quantile-regression/)
- [Species response curves with GAMs](../gam-species-response-curves/)