Checking a circular analysis

R
circular statistics
model checking
ecology tutorial
ggplot2
The mean direction, the concentration, the uniformity verdict and the overlap coefficient are all decisions in disguise. A base R look at what circular summaries hide, and how to report them honestly.
Author

Tidy Ecology

Published

2026-08-18

The three preceding posts each ended on a warning, and this one collects them. A circular analysis produces clean-looking numbers, a mean direction, a concentration, a p-value for uniformity, a coefficient of overlap, and every one of them depends on an assumption or a tuning choice that the number itself does not show. Report the number alone and you report a decision as though it were a fact. This post takes each summary in turn, in base R, and asks what it hides and how to check it.

The mean direction can be a mirage

The mean direction and the mean resultant length come as a pair, and the resultant length is the warning label. When it is close to zero, the mean direction is not a weak preference, it is meaningless: a two-way or multi-way pattern produces a near-zero resultant no matter how tightly the sub-groups cluster. Take a strongly bimodal sample, animals moving along an axis, and resample it to see how stable its mean direction is.

set.seed(4253)
unimodal <- rvm(60, mu = pi / 2, kappa = 2)   # one preferred direction
bimodal  <- raxial(60, mu = pi / 2, kappa = 4) # a two-way axis

boot_dir <- function(x, B = 2000) { set.seed(7); replicate(B, circ_mean(sample(x, replace = TRUE)) %% (2 * pi)) }
sd_deg <- function(ang) sqrt(-2 * log(circ_R(ang))) * 180 / pi
du <- boot_dir(unimodal); db <- boot_dir(bimodal)
c(unimodal_Rbar = circ_R(unimodal), bimodal_Rbar = circ_R(bimodal),
  unimodal_dir_SD = sd_deg(du), bimodal_dir_SD = sd_deg(db))
  unimodal_Rbar    bimodal_Rbar unimodal_dir_SD  bimodal_dir_SD 
     0.68967108      0.05374539      6.73811613     81.75848303 

The unimodal sample has a resultant length of 0.69, and its mean direction is stable: across bootstrap resamples the direction varies by a circular standard deviation of only 7 degrees. The bimodal sample has a resultant length of 0.05, near zero, and its mean direction is worthless: it wanders across 82 degrees between resamples, essentially picking a random point on the circle each time. The lesson from the first post, stated as a check: never report a mean direction without its resultant length, and treat a small resultant not as a weak signal but as a sign that a single direction is the wrong summary.

The uniformity verdict depends on the test

The same bimodal sample exposes the second warning. Whether it counts as evidence of a preferred direction depends entirely on which uniformity test you run.

set.seed(808)
null60 <- t(replicate(5000, allstats(runif(60, 0, 2 * pi))))
pvals <- sapply(1:4, function(j) mean(null60[, j] >= allstats(bimodal)[j]))
names(pvals) <- c("Rayleigh", "Rao", "Kuiper", "Watson")
round(pvals, 4)
Rayleigh      Rao   Kuiper   Watson 
  0.8454   0.0002   0.0030   0.0066 

The Rayleigh test returns a p-value of 0.85 and finds nothing, because the near-zero resultant it is built on carries no signal. Rao, Kuiper and Watson all reject uniformity at the five per cent level. Same data, opposite conclusions, and the difference is not sampling noise but the question each statistic encodes. The check here is to choose the test before seeing the data, guided by the biology: if two-way or multi-way patterns are plausible, an omnibus test is required and the Rayleigh test alone will mislead. This is exactly the shift recent ecological work recommends (Landler, Ruxton and Malkemper 2018, 2019).

Left panel shows two density curves of bootstrap mean directions in degrees: a tall narrow green peak and a low flat amber distribution spread across all angles. Right panel is a bar chart of four p-values with a dashed line at 0.05; the Rayleigh bar is well above it and the Rao, Kuiper and Watson bars are below.
Figure 1: Left: bootstrap distributions of the mean direction. The unimodal sample (green) concentrates tightly; the bimodal sample (amber) spreads across the whole circle, so its mean direction is meaningless. Right: uniformity p-values for the bimodal sample. The Rayleigh test sits above the five per cent line (fails to reject); the three omnibus tests reject.

The concentration is biased upward in small samples

The concentration parameter kappa has a quieter problem: its maximum-likelihood estimate is biased, and the bias is large exactly when samples are small, which is when ecologists most want a number. Simulate from a known concentration and watch the estimate.

kappa_true <- 2
kbias <- do.call(rbind, lapply(c(10, 15, 25, 50, 100), function(n) {
  set.seed(500 + n)
  est <- t(replicate(2500, { x <- rvm(n, 0, kappa_true); kh <- kappa_mle(circ_R(x)); c(kh, kappa_bc(kh, n)) }))
  data.frame(n = n, mle = mean(est[is.finite(est[, 1]), 1]), corrected = mean(est[is.finite(est[, 2]), 2]))
}))
round(kbias, 2)
    n  mle corrected
1  10 2.54      1.95
2  15 2.35      1.99
3  25 2.20      2.01
4  50 2.10      2.01
5 100 2.04      2.00

At 10 observations the maximum-likelihood estimate averages 2.54 when the truth is 2, an overestimate of around 27 per cent, so the animals look more tightly oriented than they are. The bias fades as the sample grows, reaching 2.04 by 100 observations. The bias correction of Best and Fisher (1981), applied in the second column, removes almost all of it at every sample size. The check is simple: with fewer than a few dozen angles, report the bias-corrected concentration, not the raw maximum-likelihood value.

The overlap coefficient is smoothed and biased

The overlap coefficient carries both a bias and a bandwidth dependence, and only a simulation against a known truth exposes them. Build two activity densities whose true overlap can be computed exactly, then estimate it from finite samples with the recommended bandwidth that tightens as the sample grows.

hour2rad <- function(h) 2 * pi * h / 24
fa <- function(t) dvm(t, hour2rad(0), 2)
fb <- function(t) 0.5 * dvm(t, hour2rad(6), 4) + 0.5 * dvm(t, hour2rad(18), 4)
gT <- seq(0, 2 * pi, length.out = 20001)[-20001]
D_true <- sum(pmin(fa(gT), fb(gT))) * (2 * pi / 20000)

oD4 <- function(A, B, kbw, ng = 256) { g <- seq(0, 2 * pi, length.out = ng + 1)[-(ng + 1)]
  sum(pmin(ckde(g, A, kbw), ckde(g, B, kbw))) * (2 * pi / ng) }
oD1 <- function(A, B, kbw) 0.5 * (mean(pmin(1, ckde(A, B, kbw) / ckde(A, A, kbw))) +
                                  mean(pmin(1, ckde(B, A, kbw) / ckde(B, B, kbw))))
kbw_adaptive <- function(n) 3.17 * n^0.4                 # tightens as n grows

obias <- do.call(rbind, lapply(c(20, 50, 100, 300), function(n) {
  set.seed(900 + n)
  res <- t(replicate(800, { A <- rvm(n, hour2rad(0), 2); B <- rmix(n, hour2rad(c(6, 18)), c(4, 4), c(0.5, 0.5))
    kb <- kbw_adaptive(n); c(oD4(A, B, kb), oD1(A, B, kb)) }))
  data.frame(n = n, Dhat4 = mean(res[, 1]), Dhat1 = mean(res[, 2]), SD4 = sd(res[, 1]))
}))
round(cbind(obias, true = D_true), 3)
    n Dhat4 Dhat1   SD4 true
1  20 0.405 0.365 0.087 0.36
2  50 0.395 0.368 0.062 0.36
3 100 0.393 0.372 0.048 0.36
4 300 0.382 0.369 0.027 0.36

The true overlap is 0.36. The grid estimator overshoots it at every sample size, by about 0.04 at 20 detections, shrinking towards 0.02 at 300. The upward bias is the smoothing showing through: kernel densities are wider than the truth, and wider curves share more area. The data-point estimator carries far less bias, which is exactly why Ridout and Linkie recommend it for small samples. On top of the bias sits the variability, a standard deviation of about 0.09 at the smallest sample, which is what made the bootstrap interval in the previous post so wide.

Left panel plots two curves of estimated concentration against sample size: the maximum-likelihood curve starts near 2.5 and falls towards 2, the corrected curve sits on the dashed line at 2. Right panel plots two overlap curves against sample size: the grid estimator sits above the dashed true value and declines slowly, the data-point estimator sits close to the dashed line.
Figure 2: Left: the mean maximum-likelihood concentration (points) overshoots the true value of two (dashed) at small samples, while the bias-corrected estimate tracks it. Right: the grid overlap estimator sits above the true overlap (dashed) and converges slowly, whereas the data-point estimator stays close to the truth.

The through-line

None of these are exotic failures; they are the ordinary behaviour of circular summaries, and the same pattern runs through every checking post on this blog. A number that looks like a measurement, the mean direction, the concentration, the p-value, the overlap, is really the output of a choice: an assumption about how many directions the biology has, a decision about which test to run, a bandwidth, a sample size the estimator was never going to be precise at. The honest report shows the choice alongside the number. Give the resultant length with the mean direction, the test with the uniformity claim, the bandwidth and an interval with the overlap. The circle is unforgiving of summaries that pretend a distribution is simpler than it is, and the checks above are mostly ways of refusing that pretence.

References

Batschelet E 1981 Circular Statistics in Biology. Academic Press. ISBN 978-0-12-081050-9.

Fisher NI 1993 Statistical Analysis of Circular Data. Cambridge University Press. ISBN 978-0-521-56890-6.

Mardia KV, Jupp PE 2000 Directional Statistics. Wiley. ISBN 978-0-471-95333-3.

Pewsey A, Neuhauser M, Ruxton GD 2013 Circular Statistics in R. Oxford University Press. ISBN 978-0-19-967113-7.

Best DJ, Fisher NI 1981 Communications in Statistics - Simulation and Computation 10(5):493-502 (10.1080/03610918108812225).

Watson GS 1961 Biometrika 48(1-2):109-114 (10.1093/biomet/48.1-2.109).

Ridout MS, Linkie M 2009 Journal of Agricultural, Biological, and Environmental Statistics 14(3):322-337 (10.1198/jabes.2009.08038).

Landler L, Ruxton GD, Malkemper EP 2018 Behavioral Ecology and Sociobiology 72(8):128 (10.1007/s00265-018-2538-y).

Landler L, Ruxton GD, Malkemper EP 2019 BMC Ecology 19:30 (10.1186/s12898-019-0246-8).

Newsletter

Get new tutorials by email

New R and QGIS tutorials for ecologists, straight to your inbox. No spam; unsubscribe anytime.

By subscribing you agree to receive these emails and confirm your address once. See the privacy policy.