Detrending and bandwidth in early warning signals

early warning signals

time series

resilience

ecology tutorial

The same collapsing series, four defensible detrending choices, and a variance warning that flips sign in R: how analyst decisions drive the trend statistic.

Author

Tidy Ecology

Published

2026-06-06

Early warning indicators are not computed on the raw series. They are computed on the residuals left after a slow trend is removed, because the trend itself inflates both variance and lag-1 autocorrelation. That detrending step, and the rolling-window width that follows it, are analyst choices with no objective setting. This post takes the collapsing grazing series from the critical slowing tutorial and shows how much those two choices move the verdict. In one case they flip the sign of a variance warning on identical data.

The same series as before

We regenerate the pre-transition record used earlier: a slow approach to a fold, sampled at unit intervals.

r <- 1; K <- 10; h <- 1
f <- function(x, c) r*x*(1 - x/K) - c*x^2/(x^2 + h^2)
sim_grazing <- function(seed, T = 600, dt = 0.01, sigma = 0.30,
                        c0 = 1.0, c1 = 2.72, x0 = 8.9){
  set.seed(seed); nstep <- T/dt; cvec <- seq(c0, c1, length.out = nstep)
  x <- numeric(nstep); x[1] <- x0
  for(i in 2:nstep){ x[i] <- x[i-1] + f(x[i-1], cvec[i-1])*dt + sigma*sqrt(dt)*rnorm(1)
    if(x[i] < 0.001) x[i] <- 0.001 }
  keep <- seq(1, nstep, by = 1/dt); data.frame(t = (keep-1)*dt, x = x[keep], c = cvec[keep])
}
d <- sim_grazing(seed = 4238)
tr_idx <- which(d$x < 3.5)[1]
seg <- d$x[1:(tr_idx - 10)]; n <- length(seg)

Four defensible ways to detrend

A Gaussian kernel smoother removes a local mean; its bandwidth sets how much wobble counts as trend. A small bandwidth follows the data closely and strips out slow structure; a large one leaves slow structure behind. First-differencing removes any trend without a bandwidth at all, but it reshapes the correlation structure. Centring only (no detrending) leaves the full trend in place. All four appear in the literature.

gauss_detrend <- function(y, bw){ tg <- seq_along(y)
  trend <- sapply(tg, function(i){ w <- dnorm(tg, i, bw); sum(w*y)/sum(w) }); y - trend }
roll_ar1 <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1)
  sapply(s, function(k){ w <- res[k:(k+win-1)]
    ar.ols(w, order.max = 1, aic = FALSE, demean = TRUE, intercept = FALSE)$ar[1] }) }
roll_var <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1)
  sapply(s, function(k) var(res[k:(k+win-1)])) }
kt <- function(v) cor(seq_along(v), v, method = "kendall")

res_raw  <- seg - mean(seg)             # no detrending
res_bw05 <- gauss_detrend(seg, 0.05*n)  # small bandwidth
res_bw25 <- gauss_detrend(seg, 0.25*n)  # large bandwidth
res_fd   <- diff(seg)                   # first difference

tau_ar_raw  <- kt(roll_ar1(res_raw));  tau_ar_bw05 <- kt(roll_ar1(res_bw05))
tau_ar_bw25 <- kt(roll_ar1(res_bw25)); tau_ar_fd   <- kt(roll_ar1(res_fd))
tau_var_raw  <- kt(roll_var(res_raw));  tau_var_bw05 <- kt(roll_var(res_bw05))
tau_var_bw25 <- kt(roll_var(res_bw25)); tau_var_fd   <- kt(roll_var(res_fd))

For the autocorrelation indicator every choice gives a positive trend: +0.78 (raw), +0.72 (bandwidth 5%), +0.64 (bandwidth 25%), +0.89 (first-difference). The autocorrelation warning survives the choice here.

The variance indicator does not. It reads +0.80 on the raw series, +0.70 and +0.64 under the two bandwidths, but -0.49 under first-differencing. Here the sign flips. First-differencing removes the slow build-up of variance that is the whole point of the indicator.

A dot plot with four detrending choices on the x axis and Kendall tau on the y axis. Autocorrelation points sit between 0.6 and 0.9. Variance points sit near 0.8, 0.7 and 0.64 for the first three choices but drop below zero to about minus 0.49 for first-difference. — Figure 1: Kendall trend statistic for each indicator under four detrending choices. Autocorrelation stays positive throughout; variance flips to negative under first-differencing. The dashed line is zero, the boundary between a rising and a falling warning.

How far the choices reach

The two bandwidths above are just two points. Sweeping the Gaussian bandwidth across a plausible range, and separately sweeping the rolling-window width, shows the full spread of the autocorrelation trend statistic that an analyst could report from this one series.

bws <- seq(0.03, 0.40, by = 0.01)
tau_bw <- sapply(bws, function(b) kt(roll_ar1(gauss_detrend(seg, b*n))))
res10 <- gauss_detrend(seg, 0.10*n)
wins <- seq(0.25, 0.75, by = 0.05)
tau_win <- sapply(wins, function(wf) kt(roll_ar1(res10, wf)))
bw_lo <- min(tau_bw); bw_hi <- max(tau_bw)
win_lo <- min(tau_win); win_hi <- max(tau_win)
win_at_hi <- wins[which.max(tau_win)]

The bandwidth choice alone moves the autocorrelation trend statistic across 0.63 to 0.77. The window width moves it from 0.46 to 0.83, the top of that range falling at a window of 55 per cent of the record. None of these settings is wrong. Each is a defensible default, and each yields a different number to put in a paper.

Two panels, each with the Kendall tau of the autocorrelation indicator on the y axis. Left: tau against the Gaussian detrending bandwidth as a fraction of record length, running from 0.03 to 0.40, drawn as a line with a point at each bandwidth. Right: tau against the rolling-window width as a fraction of record length, running from 0.25 to 0.75 in steps of 0.05, drawn as a line with a point at each width. In both panels the line moves up and down rather than following a straight trend, over the ranges of tau given in the text above. — Figure 2: Sensitivity of the autocorrelation trend statistic to the two analyst choices. Left: Gaussian detrending bandwidth as a fraction of record length. Right: rolling-window width as a fraction of record length.

Reporting honestly

None of this means the indicators are useless. It means a single trend statistic, computed with one detrending method and one window width, is not a result on its own. Two habits help. Fix the analysis choices before looking at the outcome, so the bandwidth is not tuned towards the answer you hoped for. And report the sensitivity: show the trend statistic across a range of bandwidths and windows, not one number. On short ecological records, where there is little data to pin down the trend, this sensitivity is larger still. A warning that holds only for one bandwidth is not a warning; it is a choice. The checking tutorial adds the other half of the reckoning: whether a trend statistic, once chosen, is distinguishable from chance at all.

References

Carpenter and Brock 2006 Ecology Letters 9(3):311-318 (10.1111/j.1461-0248.2005.00877.x)
Dakos et al. 2012 PLoS ONE 7(7):e41010 (10.1371/journal.pone.0041010)
Boettiger and Hastings 2012 Journal of the Royal Society Interface 9(75):2527-2539 (10.1098/rsif.2012.0125)
Dakos et al. 2015 Philosophical Transactions of the Royal Society B 370(1659):20130263 (10.1098/rstb.2013.0263)

--- title: "Detrending and bandwidth in early warning signals" description: "The same collapsing series, four defensible detrending choices, and a variance warning that flips sign in R: how analyst decisions drive the trend statistic." date: "2026-06-06 12:00" categories: [early warning signals, time series, R, resilience, ecology tutorial] image: thumbnail.png image-alt: "Kendall tau of a variance indicator under four detrending choices, one of which flips the sign" --- Early warning indicators are not computed on the raw series. They are computed on the *residuals* left after a slow trend is removed, because the trend itself inflates both variance and lag-1 autocorrelation. That detrending step, and the rolling-window width that follows it, are analyst choices with no objective setting. This post takes the collapsing grazing series from the [critical slowing tutorial](../early-warning-signals-critical-slowing/) and shows how much those two choices move the verdict. In one case they flip the sign of a variance warning on identical data. ## The same series as before We regenerate the pre-transition record used earlier: a slow approach to a fold, sampled at unit intervals. ```{r} #| label: series r <- 1; K <- 10; h <- 1 f <- function(x, c) r*x*(1 - x/K) - c*x^2/(x^2 + h^2) sim_grazing <- function(seed, T = 600, dt = 0.01, sigma = 0.30, c0 = 1.0, c1 = 2.72, x0 = 8.9){ set.seed(seed); nstep <- T/dt; cvec <- seq(c0, c1, length.out = nstep) x <- numeric(nstep); x[1] <- x0 for(i in 2:nstep){ x[i] <- x[i-1] + f(x[i-1], cvec[i-1])*dt + sigma*sqrt(dt)*rnorm(1) if(x[i] < 0.001) x[i] <- 0.001 } keep <- seq(1, nstep, by = 1/dt); data.frame(t = (keep-1)*dt, x = x[keep], c = cvec[keep]) } d <- sim_grazing(seed = 4238) tr_idx <- which(d$x < 3.5)[1] seg <- d$x[1:(tr_idx - 10)]; n <- length(seg) ``` ## Four defensible ways to detrend A Gaussian kernel smoother removes a local mean; its bandwidth sets how much wobble counts as trend. A small bandwidth follows the data closely and strips out slow structure; a large one leaves slow structure behind. First-differencing removes any trend without a bandwidth at all, but it reshapes the correlation structure. Centring only (no detrending) leaves the full trend in place. All four appear in the literature. ```{r} #| label: detrend gauss_detrend <- function(y, bw){ tg <- seq_along(y) trend <- sapply(tg, function(i){ w <- dnorm(tg, i, bw); sum(w*y)/sum(w) }); y - trend } roll_ar1 <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1) sapply(s, function(k){ w <- res[k:(k+win-1)] ar.ols(w, order.max = 1, aic = FALSE, demean = TRUE, intercept = FALSE)$ar[1] }) } roll_var <- function(res, wf = 0.5){ win <- round(wf*length(res)); s <- 1:(length(res)-win+1) sapply(s, function(k) var(res[k:(k+win-1)])) } kt <- function(v) cor(seq_along(v), v, method = "kendall") res_raw <- seg - mean(seg) # no detrending res_bw05 <- gauss_detrend(seg, 0.05*n) # small bandwidth res_bw25 <- gauss_detrend(seg, 0.25*n) # large bandwidth res_fd <- diff(seg) # first difference tau_ar_raw <- kt(roll_ar1(res_raw)); tau_ar_bw05 <- kt(roll_ar1(res_bw05)) tau_ar_bw25 <- kt(roll_ar1(res_bw25)); tau_ar_fd <- kt(roll_ar1(res_fd)) tau_var_raw <- kt(roll_var(res_raw)); tau_var_bw05 <- kt(roll_var(res_bw05)) tau_var_bw25 <- kt(roll_var(res_bw25)); tau_var_fd <- kt(roll_var(res_fd)) ``` For the autocorrelation indicator every choice gives a positive trend: `r sprintf("%+.2f", tau_ar_raw)` (raw), `r sprintf("%+.2f", tau_ar_bw05)` (bandwidth 5%), `r sprintf("%+.2f", tau_ar_bw25)` (bandwidth 25%), `r sprintf("%+.2f", tau_ar_fd)` (first-difference). The autocorrelation warning survives the choice here. The variance indicator does not. It reads `r sprintf("%+.2f", tau_var_raw)` on the raw series, `r sprintf("%+.2f", tau_var_bw05)` and `r sprintf("%+.2f", tau_var_bw25)` under the two bandwidths, but `r sprintf("%+.2f", tau_var_fd)` under first-differencing. Here the sign flips. First-differencing removes the slow build-up of variance that is the whole point of the indicator. ```{r} #| label: setup-theme #| echo: false #| message: false #| warning: false library(ggplot2) ink <- "#16241d"; body <- "#2c3a31"; forest <- "#275139"; label <- "#46604a" sage <- "#93a87f"; paper <- "#f5f4ee"; line <- "#dad9ca"; faint <- "#5d6b61" amber <- "#cda23f"; brick <- "#b5534e" theme_te <- function(){ theme_minimal(base_size = 11) + theme(plot.background = element_rect(fill = "#f5f4ee", colour = NA), panel.background = element_rect(fill = "#f5f4ee", colour = NA), panel.grid.minor = element_blank(), panel.grid.major = element_line(colour = line, linewidth = 0.3), axis.title = element_text(colour = body), axis.text = element_text(colour = faint), plot.title = element_text(colour = ink, face = "bold", size = 12), plot.subtitle = element_text(colour = label, size = 10), legend.position = "top", legend.title = element_blank(), legend.text = element_text(colour = body)) } ``` ```{r} #| label: fig-flip #| echo: false #| fig-width: 7.6 #| fig-height: 4 #| fig-cap: "Kendall trend statistic for each indicator under four detrending choices. Autocorrelation stays positive throughout; variance flips to negative under first-differencing. The dashed line is zero, the boundary between a rising and a falling warning." #| fig-alt: "A dot plot with four detrending choices on the x axis and Kendall tau on the y axis. Autocorrelation points sit between 0.6 and 0.9. Variance points sit near 0.8, 0.7 and 0.64 for the first three choices but drop below zero to about minus 0.49 for first-difference." choice <- c("raw", "bandwidth 5%", "bandwidth 25%", "first-diff") df_flip <- data.frame( choice = factor(rep(choice, 2), levels = choice), indicator = rep(c("autocorrelation", "variance"), each = 4), tau = c(tau_ar_raw, tau_ar_bw05, tau_ar_bw25, tau_ar_fd, tau_var_raw, tau_var_bw05, tau_var_bw25, tau_var_fd)) ggplot(df_flip, aes(choice, tau, colour = indicator, group = indicator)) + geom_hline(yintercept = 0, colour = brick, linetype = "dashed", linewidth = 0.4) + geom_line(linewidth = 0.5) + geom_point(size = 3.2) + scale_colour_manual(values = c("autocorrelation" = forest, "variance" = amber)) + labs(title = "The detrending choice can flip the verdict", x = NULL, y = "Kendall tau with time") + theme_te() ``` ## How far the choices reach The two bandwidths above are just two points. Sweeping the Gaussian bandwidth across a plausible range, and separately sweeping the rolling-window width, shows the full spread of the autocorrelation trend statistic that an analyst could report from this one series. ```{r} #| label: sweeps bws <- seq(0.03, 0.40, by = 0.01) tau_bw <- sapply(bws, function(b) kt(roll_ar1(gauss_detrend(seg, b*n)))) res10 <- gauss_detrend(seg, 0.10*n) wins <- seq(0.25, 0.75, by = 0.05) tau_win <- sapply(wins, function(wf) kt(roll_ar1(res10, wf))) bw_lo <- min(tau_bw); bw_hi <- max(tau_bw) win_lo <- min(tau_win); win_hi <- max(tau_win) win_at_hi <- wins[which.max(tau_win)] ``` The bandwidth choice alone moves the autocorrelation trend statistic across `r sprintf("%.2f", bw_lo)` to `r sprintf("%.2f", bw_hi)`. The window width moves it from `r sprintf("%.2f", win_lo)` to `r sprintf("%.2f", win_hi)`, the top of that range falling at a window of `r sprintf("%.0f", 100 * win_at_hi)` per cent of the record. None of these settings is wrong. Each is a defensible default, and each yields a different number to put in a paper. ```{r} #| label: fig-sweeps #| echo: false #| fig-width: 8 #| fig-height: 3.6 #| fig-cap: "Sensitivity of the autocorrelation trend statistic to the two analyst choices. Left: Gaussian detrending bandwidth as a fraction of record length. Right: rolling-window width as a fraction of record length." #| fig-alt: "Two panels, each with the Kendall tau of the autocorrelation indicator on the y axis. Left: tau against the Gaussian detrending bandwidth as a fraction of record length, running from 0.03 to 0.40, drawn as a line with a point at each bandwidth. Right: tau against the rolling-window width as a fraction of record length, running from 0.25 to 0.75 in steps of 0.05, drawn as a line with a point at each width. In both panels the line moves up and down rather than following a straight trend, over the ranges of tau given in the text above." d_bw <- data.frame(bw = bws, tau = tau_bw) d_wn <- data.frame(win = wins, tau = tau_win) p_bw <- ggplot(d_bw, aes(bw, tau)) + geom_line(colour = forest, linewidth = 0.6) + geom_point(colour = forest, size = 1.3) + labs(title = "Bandwidth sweep", x = "bandwidth (fraction of length)", y = "AR(1) Kendall tau") + theme_te() + theme(legend.position = "none") p_wn <- ggplot(d_wn, aes(win, tau)) + geom_line(colour = amber, linewidth = 0.6) + geom_point(colour = amber, size = 1.3) + labs(title = "Window sweep", x = "window (fraction of length)", y = "AR(1) Kendall tau") + theme_te() + theme(legend.position = "none") grid::grid.newpage() grid::pushViewport(grid::viewport(layout = grid::grid.layout(1, 2))) print(p_bw, vp = grid::viewport(layout.pos.row = 1, layout.pos.col = 1)) print(p_wn, vp = grid::viewport(layout.pos.row = 1, layout.pos.col = 2)) ``` ## Reporting honestly None of this means the indicators are useless. It means a single trend statistic, computed with one detrending method and one window width, is not a result on its own. Two habits help. Fix the analysis choices before looking at the outcome, so the bandwidth is not tuned towards the answer you hoped for. And report the sensitivity: show the trend statistic across a range of bandwidths and windows, not one number. On short ecological records, where there is little data to pin down the trend, this sensitivity is larger still. A warning that holds only for one bandwidth is not a warning; it is a choice. The [checking tutorial](../checking-early-warning-signals/) adds the other half of the reckoning: whether a trend statistic, once chosen, is distinguishable from chance at all. ## References - Carpenter and Brock 2006 Ecology Letters 9(3):311-318 (10.1111/j.1461-0248.2005.00877.x) - Dakos et al. 2012 PLoS ONE 7(7):e41010 (10.1371/journal.pone.0041010) - Boettiger and Hastings 2012 Journal of the Royal Society Interface 9(75):2527-2539 (10.1098/rsif.2012.0125) - Dakos et al. 2015 Philosophical Transactions of the Royal Society B 370(1659):20130263 (10.1098/rstb.2013.0263) ## Related tutorials - [Early warning signals and critical slowing](../early-warning-signals-critical-slowing/) - [Spatial early warning signals](../spatial-early-warning-signals/) - [Checking early warning signals](../checking-early-warning-signals/) - [Autocorrelation in time series](../autocorrelation-in-time-series/)