Detrending and bandwidth choice in early warning signals

early warning signals

time series

resilience

ecology tutorial

The same collapsing series, four defensible detrending choices, and a variance early warning that flips sign in R: how analyst decisions drive the trend statistic.

Author

Tidy Ecology

Published

2026-08-15

Early warning indicators are not computed on the raw series. They are computed on the residuals left after a slow trend is removed, because the trend itself inflates both variance and lag-1 autocorrelation. That detrending step, and the rolling-window width that follows it, are analyst choices with no objective setting. This post takes the collapsing grazing series from the critical slowing tutorial and shows how much those two choices move the verdict. In one case they flip the sign of a variance warning on identical data.

The same series as before

We regenerate the pre-transition record used earlier: a slow approach to a fold, sampled at unit intervals.

r <- 1; K <- 10; h <- 1
f <- function(x, c) r*x*(1 - x/K) - c*x^2/(x^2 + h^2)
sim_grazing <- function(seed, T = 600, dt = 0.01, sigma = 0.30,
                        c0 = 1.0, c1 = 2.72, x0 = 8.9){
  set.seed(seed); nstep <- T/dt; cvec <- seq(c0, c1, length.out = nstep)
  x <- numeric(nstep); x[1] <- x0
  for(i in 2:nstep){ x[i] <- x[i-1] + f(x[i-1], cvec[i-1])*dt + sigma*sqrt(dt)*rnorm(1)
    if(x[i] < 0.001) x[i] <- 0.001 }
  keep <- seq(1, nstep, by = 1/dt); data.frame(t = (keep-1)*dt, x = x[keep], c = cvec[keep])
}
d <- sim_grazing(seed = 4238)
tr_idx <- which(d$x < 3.5)[1]
seg <- d$x[1:(tr_idx - 10)]; n <- length(seg)

Four defensible ways to detrend

A Gaussian kernel smoother removes a local mean; its bandwidth sets how much wobble counts as trend. A small bandwidth follows the data closely and strips out slow structure; a large one leaves slow structure behind. First-differencing removes any trend without a bandwidth at all, but it reshapes the correlation structure. Centring only (no detrending) leaves the full trend in place. All four appear in the literature.

gauss_detrend <- function(y, bw){ tg <- seq_along(y)
  trend <- sapply(tg, function(i){ w <- dnorm(tg, i, bw); sum(w*y)/sum(w) }); y - trend }
roll_ar1 <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1)
  sapply(s, function(k){ w <- res[k:(k+win-1)]
    ar.ols(w, order.max = 1, aic = FALSE, demean = TRUE, intercept = FALSE)$ar[1] }) }
roll_var <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1)
  sapply(s, function(k) var(res[k:(k+win-1)])) }
kt <- function(v) cor(seq_along(v), v, method = "kendall")

res_raw  <- seg - mean(seg)             # no detrending
res_bw05 <- gauss_detrend(seg, 0.05*n)  # small bandwidth
res_bw25 <- gauss_detrend(seg, 0.25*n)  # large bandwidth
res_fd   <- diff(seg)                   # first difference

tau_ar_raw  <- kt(roll_ar1(res_raw));  tau_ar_bw05 <- kt(roll_ar1(res_bw05))
tau_ar_bw25 <- kt(roll_ar1(res_bw25)); tau_ar_fd   <- kt(roll_ar1(res_fd))
tau_var_raw  <- kt(roll_var(res_raw));  tau_var_bw05 <- kt(roll_var(res_bw05))
tau_var_bw25 <- kt(roll_var(res_bw25)); tau_var_fd   <- kt(roll_var(res_fd))

For the autocorrelation indicator every choice gives a positive trend: +0.78 (raw), +0.72 (bandwidth 5%), +0.64 (bandwidth 25%), +0.89 (first-difference). The autocorrelation warning survives the choice here.

The variance indicator does not. It reads +0.80 on the raw series, +0.70 and +0.64 under the two bandwidths, but -0.49 under first-differencing. The sign flips. First-differencing removes the slow build-up of variance that is the whole point of the indicator, so the same data now say the variance is falling. The two indicators also disagree with each other under first-differencing: autocorrelation gives the strongest warning of all four choices while variance gives the only negative one.

A dot plot with four detrending choices on the x axis and Kendall tau on the y axis. Autocorrelation points sit between 0.6 and 0.9. Variance points sit near 0.8, 0.7 and 0.64 for the first three choices but drop below zero to about minus 0.49 for first-difference. — Figure 1: Kendall trend statistic for each indicator under four detrending choices. Autocorrelation stays positive throughout; variance flips to negative under first-differencing. The dashed line is zero, the boundary between a rising and a falling warning.

How far the choices reach

The two bandwidths above are just two points. Sweeping the Gaussian bandwidth across a plausible range, and separately sweeping the rolling-window width, shows the full spread of the autocorrelation trend statistic that an analyst could report from this one series.

bws <- seq(0.03, 0.40, by = 0.01)
tau_bw <- sapply(bws, function(b) kt(roll_ar1(gauss_detrend(seg, b*n))))
res10 <- gauss_detrend(seg, 0.10*n)
wins <- seq(0.25, 0.75, by = 0.05)
tau_win <- sapply(wins, function(wf) kt(roll_ar1(res10, wf)))
bw_lo <- min(tau_bw); bw_hi <- max(tau_bw)
win_lo <- min(tau_win); win_hi <- max(tau_win)

The bandwidth choice alone moves the autocorrelation trend statistic across 0.63 to 0.77. The window width moves it further, from 0.46 at a quarter-length window to 0.83 at a three-quarter-length one. None of these settings is wrong. Each is a defensible default, and each yields a different number to put in a paper.

Two panels. Left: Kendall tau against detrending bandwidth, wandering between about 0.63 and 0.77. Right: Kendall tau against window width, rising from about 0.46 at a quarter length to about 0.83 at three-quarter length. — Figure 2: Sensitivity of the autocorrelation trend statistic to the two analyst choices. Left: Gaussian detrending bandwidth as a fraction of record length. Right: rolling-window width as a fraction of record length. Both span a wide range without any wrong answer.

Reporting honestly

None of this means the indicators are useless. It means a single trend statistic, computed with one detrending method and one window width, is not a result on its own. Two habits help. Fix the analysis choices before looking at the outcome, so the bandwidth is not tuned towards the answer you hoped for. And report the sensitivity: show the trend statistic across a range of bandwidths and windows, not one number. On short ecological records, where there is little data to pin down the trend, this sensitivity is larger still. A warning that holds only for one bandwidth is not a warning; it is a choice. The checking tutorial adds the other half of the reckoning: whether a trend statistic, once chosen, is distinguishable from chance at all.

References

Carpenter and Brock 2006 Ecology Letters 9(3):311-318 (10.1111/j.1461-0248.2005.00877.x)
Dakos et al. 2012 PLoS ONE 7(7):e41010 (10.1371/journal.pone.0041010)
Boettiger and Hastings 2012 Journal of the Royal Society Interface 9(75):2527-2539 (10.1098/rsif.2012.0125)
Dakos et al. 2015 Philosophical Transactions of the Royal Society B 370(1659):20130263 (10.1098/rstb.2013.0263)

--- title: "Detrending and bandwidth choice in early warning signals" description: "The same collapsing series, four defensible detrending choices, and a variance early warning that flips sign in R: how analyst decisions drive the trend statistic." date: "2026-08-15 10:00" categories: [early warning signals, time series, R, resilience, ecology tutorial] image: thumbnail.png image-alt: "Kendall tau of a variance indicator under four detrending choices, one of which flips the sign" --- Early warning indicators are not computed on the raw series. They are computed on the *residuals* left after a slow trend is removed, because the trend itself inflates both variance and lag-1 autocorrelation. That detrending step, and the rolling-window width that follows it, are analyst choices with no objective setting. This post takes the collapsing grazing series from the [critical slowing tutorial](../early-warning-signals-critical-slowing/) and shows how much those two choices move the verdict. In one case they flip the sign of a variance warning on identical data. ## The same series as before We regenerate the pre-transition record used earlier: a slow approach to a fold, sampled at unit intervals. ```{r} #| label: series r <- 1; K <- 10; h <- 1 f <- function(x, c) r*x*(1 - x/K) - c*x^2/(x^2 + h^2) sim_grazing <- function(seed, T = 600, dt = 0.01, sigma = 0.30, c0 = 1.0, c1 = 2.72, x0 = 8.9){ set.seed(seed); nstep <- T/dt; cvec <- seq(c0, c1, length.out = nstep) x <- numeric(nstep); x[1] <- x0 for(i in 2:nstep){ x[i] <- x[i-1] + f(x[i-1], cvec[i-1])*dt + sigma*sqrt(dt)*rnorm(1) if(x[i] < 0.001) x[i] <- 0.001 } keep <- seq(1, nstep, by = 1/dt); data.frame(t = (keep-1)*dt, x = x[keep], c = cvec[keep]) } d <- sim_grazing(seed = 4238) tr_idx <- which(d$x < 3.5)[1] seg <- d$x[1:(tr_idx - 10)]; n <- length(seg) ``` ## Four defensible ways to detrend A Gaussian kernel smoother removes a local mean; its bandwidth sets how much wobble counts as trend. A small bandwidth follows the data closely and strips out slow structure; a large one leaves slow structure behind. First-differencing removes any trend without a bandwidth at all, but it reshapes the correlation structure. Centring only (no detrending) leaves the full trend in place. All four appear in the literature. ```{r} #| label: detrend gauss_detrend <- function(y, bw){ tg <- seq_along(y) trend <- sapply(tg, function(i){ w <- dnorm(tg, i, bw); sum(w*y)/sum(w) }); y - trend } roll_ar1 <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1) sapply(s, function(k){ w <- res[k:(k+win-1)] ar.ols(w, order.max = 1, aic = FALSE, demean = TRUE, intercept = FALSE)$ar[1] }) } roll_var <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1) sapply(s, function(k) var(res[k:(k+win-1)])) } kt <- function(v) cor(seq_along(v), v, method = "kendall") res_raw <- seg - mean(seg) # no detrending res_bw05 <- gauss_detrend(seg, 0.05*n) # small bandwidth res_bw25 <- gauss_detrend(seg, 0.25*n) # large bandwidth res_fd <- diff(seg) # first difference tau_ar_raw <- kt(roll_ar1(res_raw)); tau_ar_bw05 <- kt(roll_ar1(res_bw05)) tau_ar_bw25 <- kt(roll_ar1(res_bw25)); tau_ar_fd <- kt(roll_ar1(res_fd)) tau_var_raw <- kt(roll_var(res_raw)); tau_var_bw05 <- kt(roll_var(res_bw05)) tau_var_bw25 <- kt(roll_var(res_bw25)); tau_var_fd <- kt(roll_var(res_fd)) ``` For the autocorrelation indicator every choice gives a positive trend: `r sprintf("%+.2f", tau_ar_raw)` (raw), `r sprintf("%+.2f", tau_ar_bw05)` (bandwidth 5%), `r sprintf("%+.2f", tau_ar_bw25)` (bandwidth 25%), `r sprintf("%+.2f", tau_ar_fd)` (first-difference). The autocorrelation warning survives the choice here. The variance indicator does not. It reads `r sprintf("%+.2f", tau_var_raw)` on the raw series, `r sprintf("%+.2f", tau_var_bw05)` and `r sprintf("%+.2f", tau_var_bw25)` under the two bandwidths, but `r sprintf("%+.2f", tau_var_fd)` under first-differencing. The sign flips. First-differencing removes the slow build-up of variance that is the whole point of the indicator, so the same data now say the variance is falling. The two indicators also disagree with each other under first-differencing: autocorrelation gives the strongest warning of all four choices while variance gives the only negative one. ```{r} #| label: setup-theme #| echo: false #| message: false #| warning: false library(ggplot2) ink <- "#16241d"; body <- "#2c3a31"; forest <- "#275139"; label <- "#46604a" sage <- "#93a87f"; paper <- "#f5f4ee"; line <- "#dad9ca"; faint <- "#5d6b61" amber <- "#cda23f"; brick <- "#b5534e" theme_te <- function(){ theme_minimal(base_size = 11) + theme(plot.background = element_rect(fill = "white", colour = NA), panel.background = element_rect(fill = "white", colour = NA), panel.grid.minor = element_blank(), panel.grid.major = element_line(colour = line, linewidth = 0.3), axis.title = element_text(colour = body), axis.text = element_text(colour = faint), plot.title = element_text(colour = ink, face = "bold", size = 12), plot.subtitle = element_text(colour = label, size = 10), legend.position = "top", legend.title = element_blank(), legend.text = element_text(colour = body)) } ``` ```{r} #| label: fig-flip #| echo: false #| fig-width: 7.6 #| fig-height: 4 #| fig-cap: "Kendall trend statistic for each indicator under four detrending choices. Autocorrelation stays positive throughout; variance flips to negative under first-differencing. The dashed line is zero, the boundary between a rising and a falling warning." #| fig-alt: "A dot plot with four detrending choices on the x axis and Kendall tau on the y axis. Autocorrelation points sit between 0.6 and 0.9. Variance points sit near 0.8, 0.7 and 0.64 for the first three choices but drop below zero to about minus 0.49 for first-difference." choice <- c("raw", "bandwidth 5%", "bandwidth 25%", "first-diff") df_flip <- data.frame( choice = factor(rep(choice, 2), levels = choice), indicator = rep(c("autocorrelation", "variance"), each = 4), tau = c(tau_ar_raw, tau_ar_bw05, tau_ar_bw25, tau_ar_fd, tau_var_raw, tau_var_bw05, tau_var_bw25, tau_var_fd)) ggplot(df_flip, aes(choice, tau, colour = indicator, group = indicator)) + geom_hline(yintercept = 0, colour = brick, linetype = "dashed", linewidth = 0.4) + geom_line(linewidth = 0.5) + geom_point(size = 3.2) + scale_colour_manual(values = c("autocorrelation" = forest, "variance" = amber)) + labs(title = "The detrending choice can flip the verdict", x = NULL, y = "Kendall tau with time") + theme_te() ``` ## How far the choices reach The two bandwidths above are just two points. Sweeping the Gaussian bandwidth across a plausible range, and separately sweeping the rolling-window width, shows the full spread of the autocorrelation trend statistic that an analyst could report from this one series. ```{r} #| label: sweeps bws <- seq(0.03, 0.40, by = 0.01) tau_bw <- sapply(bws, function(b) kt(roll_ar1(gauss_detrend(seg, b*n)))) res10 <- gauss_detrend(seg, 0.10*n) wins <- seq(0.25, 0.75, by = 0.05) tau_win <- sapply(wins, function(wf) kt(roll_ar1(res10, wf))) bw_lo <- min(tau_bw); bw_hi <- max(tau_bw) win_lo <- min(tau_win); win_hi <- max(tau_win) ``` The bandwidth choice alone moves the autocorrelation trend statistic across `r sprintf("%.2f", bw_lo)` to `r sprintf("%.2f", bw_hi)`. The window width moves it further, from `r sprintf("%.2f", win_lo)` at a quarter-length window to `r sprintf("%.2f", win_hi)` at a three-quarter-length one. None of these settings is wrong. Each is a defensible default, and each yields a different number to put in a paper. ```{r} #| label: fig-sweeps #| echo: false #| fig-width: 8 #| fig-height: 3.6 #| fig-cap: "Sensitivity of the autocorrelation trend statistic to the two analyst choices. Left: Gaussian detrending bandwidth as a fraction of record length. Right: rolling-window width as a fraction of record length. Both span a wide range without any wrong answer." #| fig-alt: "Two panels. Left: Kendall tau against detrending bandwidth, wandering between about 0.63 and 0.77. Right: Kendall tau against window width, rising from about 0.46 at a quarter length to about 0.83 at three-quarter length." d_bw <- data.frame(bw = bws, tau = tau_bw) d_wn <- data.frame(win = wins, tau = tau_win) p_bw <- ggplot(d_bw, aes(bw, tau)) + geom_line(colour = forest, linewidth = 0.6) + geom_point(colour = forest, size = 1.3) + labs(title = "Bandwidth sweep", x = "bandwidth (fraction of length)", y = "AR(1) Kendall tau") + theme_te() + theme(legend.position = "none") p_wn <- ggplot(d_wn, aes(win, tau)) + geom_line(colour = amber, linewidth = 0.6) + geom_point(colour = amber, size = 1.3) + labs(title = "Window sweep", x = "window (fraction of length)", y = "AR(1) Kendall tau") + theme_te() + theme(legend.position = "none") grid::grid.newpage() grid::pushViewport(grid::viewport(layout = grid::grid.layout(1, 2))) print(p_bw, vp = grid::viewport(layout.pos.row = 1, layout.pos.col = 1)) print(p_wn, vp = grid::viewport(layout.pos.row = 1, layout.pos.col = 2)) ``` ## Reporting honestly None of this means the indicators are useless. It means a single trend statistic, computed with one detrending method and one window width, is not a result on its own. Two habits help. Fix the analysis choices before looking at the outcome, so the bandwidth is not tuned towards the answer you hoped for. And report the sensitivity: show the trend statistic across a range of bandwidths and windows, not one number. On short ecological records, where there is little data to pin down the trend, this sensitivity is larger still. A warning that holds only for one bandwidth is not a warning; it is a choice. The [checking tutorial](../checking-early-warning-signals/) adds the other half of the reckoning: whether a trend statistic, once chosen, is distinguishable from chance at all. ## References - Carpenter and Brock 2006 Ecology Letters 9(3):311-318 (10.1111/j.1461-0248.2005.00877.x) - Dakos et al. 2012 PLoS ONE 7(7):e41010 (10.1371/journal.pone.0041010) - Boettiger and Hastings 2012 Journal of the Royal Society Interface 9(75):2527-2539 (10.1098/rsif.2012.0125) - Dakos et al. 2015 Philosophical Transactions of the Royal Society B 370(1659):20130263 (10.1098/rstb.2013.0263) ## Related tutorials - [Early warning signals and critical slowing](../early-warning-signals-critical-slowing/) - [Spatial early warning signals](../spatial-early-warning-signals/) - [Checking early warning signals](../checking-early-warning-signals/) - [Autocorrelation in time series](../autocorrelation-in-time-series/)