Summarising ecological data by group

dplyr

data wrangling

ecology tutorial

Grouped summaries in R with dplyr: group_by and summarise, and why the mean of per-site proportions is not the pooled proportion when effort varies.

Author

Tidy Ecology

Published

2026-07-19

Once your field data is a tidy table, most questions become grouped summaries: mean abundance per habitat, species richness per site, the proportion of a focal species per plot. R makes this a one-liner with dplyr, and that is exactly why it is dangerous. The one-liner hides a choice, and the wrong choice gives you a number that answers a different question than the one you asked.

This post covers group_by() and summarise(), then spends most of its time on one measurable trap that catches ecologists constantly: the mean of per-site ratios is not the same as the ratio computed on the pooled totals, and when sampling effort differs between sites the two can differ by nearly a factor of two.

group_by() then summarise()

The pattern is always the same. You split the rows into groups, then collapse each group to one number. Here is a focal-species survey across six quadrats, where each quadrat has a total arthropod count (the sampling effort varies) and a count of the focal beetle:

library(dplyr)
surv <- data.frame(
  quadrat = c("Q1", "Q2", "Q3", "Q4", "Q5", "Q6"),
  total   = c(200,  180,   40,   30,   25,  220),   # all arthropods counted
  focal   = c( 20,   27,   20,   18,   10,   22)    # the focal beetle
)
surv

  quadrat total focal
1      Q1   200    20
2      Q2   180    27
3      Q3    40    20
4      Q4    30    18
5      Q5    25    10
6      Q6   220    22

To get the focal proportion in each quadrat, add a column and summarise. group_by() marks the grouping; summarise() reduces each group to a row; n() counts rows; sum() and mean() do the obvious:

surv <- surv |> mutate(prop = focal / total)
round(surv$prop, 3)

[1] 0.10 0.15 0.50 0.60 0.40 0.10

The six proportions run from 0.10 to 0.60. Now suppose you want a single headline number: what fraction of the community is the focal beetle? There are two ways to compute it, and they are not the same number.

The mean of ratios is not the ratio of means

summary_tbl <- surv |>
  summarise(
    pooled         = sum(focal) / sum(total),  # ratio of the pooled totals
    mean_of_ratios = mean(focal / total),      # average of the per-quadrat proportions
    n              = n()
  )
summary_tbl

     pooled mean_of_ratios n
1 0.1683453      0.3083333 6

The pooled proportion is 0.168: of the 695 individuals counted across all quadrats, 117 were the focal beetle. The mean of ratios is 0.308, the plain average of the six per-quadrat proportions. They differ by a factor of 1.83, and neither is a rounding artefact.

The reason is that the quadrats are not the same size and the focal proportion depends on size. The correlation between quadrat total and focal proportion is -0.95: the beetle makes up a large share of the small, sparse quadrats and a small share of the big ones. The pooled proportion is dominated by the big quadrats (they contribute most of the individuals), so it lands low. The mean of ratios gives every quadrat one equal vote regardless of size, so the small high-proportion quadrats pull it up.

Neither number is wrong. They answer different questions:

The pooled proportion answers “across all the individuals I counted, what fraction is the focal species?” It is a population-level quantity, weighted by how many individuals each quadrat contributed.
The mean of ratios answers “in a typical quadrat, what fraction is the focal species?” It weights quadrats equally, which is what you want if the quadrat is your unit of replication and their sizes are incidental.

The mistake is reporting one while describing the other: writing “the focal beetle is 31 percent of the community” (the mean of ratios) when “the community” means the pooled individuals (17 percent), or claiming “a typical quadrat is 17 percent focal” when the per-quadrat proportions average 31 percent.

The pooled proportion is a weighted mean

The two summaries are not unrelated. The pooled proportion is exactly the mean of the per-quadrat proportions weighted by quadrat total:

weighted.mean(surv$prop, w = surv$total)   # equals the pooled proportion

[1] 0.1683453

This lands on 0.168, the pooled value again. So the real question behind the trap is: what should the weights be? If each individual should count equally, weight by effort (the pooled proportion). If each site should count equally, do not weight (the mean of ratios). Deciding that is ecology, not arithmetic, and summarise() will happily give you either without asking.

Bar chart of focal proportion for six quadrats ranging from 0.10 to 0.60, with a solid horizontal line near 0.17 for the pooled proportion and a dashed horizontal line near 0.31 for the mean of proportions. — Figure 1: Focal-species proportion in each quadrat (bars), with the pooled proportion (solid line) and the mean of proportions (dashed line). The two summaries straddle the bars because the small high-proportion quadrats pull the unweighted mean up, while the pooled value follows the large quadrats.

Where to go next

Grouped summaries assume the table is already assembled, but survey data usually arrives in two tables, one for sites and one for observations, that you have to join. The next trap lives there: an inner join silently drops the sites where the species was absent, which biases exactly the kind of mean you just computed. The joining tutorial takes that apart.

References

Wickham H, Averick M, Bryan J, et al 2019. Journal of Open Source Software 4(43):1686 (10.21105/joss.01686).

Wickham H, Cetinkaya-Rundel M, Grolemund G 2023. R for Data Science, 2nd ed. O’Reilly. ISBN 978-1-492-09740-2.

--- title: "Summarising ecological data by group" description: "Grouped summaries in R with dplyr: group_by and summarise, and why the mean of per-site proportions is not the pooled proportion when effort varies." date: 2026-07-19 10:00 categories: [R, dplyr, data wrangling, ecology tutorial] image: thumbnail.png image-alt: "Bar chart of a focal species' proportion in six quadrats, with two horizontal reference lines for the pooled proportion and the mean of proportions." --- Once your field data is a tidy table, most questions become grouped summaries: mean abundance per habitat, species richness per site, the proportion of a focal species per plot. R makes this a one-liner with `dplyr`, and that is exactly why it is dangerous. The one-liner hides a choice, and the wrong choice gives you a number that answers a different question than the one you asked. This post covers `group_by()` and `summarise()`, then spends most of its time on one measurable trap that catches ecologists constantly: the mean of per-site ratios is not the same as the ratio computed on the pooled totals, and when sampling effort differs between sites the two can differ by nearly a factor of two. ## group_by() then summarise() The pattern is always the same. You split the rows into groups, then collapse each group to one number. Here is a focal-species survey across six quadrats, where each quadrat has a total arthropod count (the sampling effort varies) and a count of the focal beetle: ```{r} #| label: data #| message: false library(dplyr) surv <- data.frame( quadrat = c("Q1", "Q2", "Q3", "Q4", "Q5", "Q6"), total = c(200, 180, 40, 30, 25, 220), # all arthropods counted focal = c( 20, 27, 20, 18, 10, 22) # the focal beetle ) surv ``` To get the focal proportion in each quadrat, add a column and summarise. `group_by()` marks the grouping; `summarise()` reduces each group to a row; `n()` counts rows; `sum()` and `mean()` do the obvious: ```{r} #| label: per-quadrat surv <- surv |> mutate(prop = focal / total) round(surv$prop, 3) ``` The six proportions run from `r sprintf("%.2f", min((c(20,27,20,18,10,22))/c(200,180,40,30,25,220)))` to `r sprintf("%.2f", max((c(20,27,20,18,10,22))/c(200,180,40,30,25,220)))`. Now suppose you want a single headline number: what fraction of the community is the focal beetle? There are two ways to compute it, and they are not the same number. ## The mean of ratios is not the ratio of means ```{r} #| label: two-summaries summary_tbl <- surv |> summarise( pooled = sum(focal) / sum(total), # ratio of the pooled totals mean_of_ratios = mean(focal / total), # average of the per-quadrat proportions n = n() ) summary_tbl ``` ```{r} #| label: gap-numbers #| include: false pooled <- sum(surv$focal) / sum(surv$total) mor <- mean(surv$focal / surv$total) ratio <- mor / pooled rho <- cor(surv$total, surv$prop) wm <- weighted.mean(surv$prop, surv$total) ``` The **pooled** proportion is `r sprintf("%.3f", pooled)`: of the `r sum(surv$total)` individuals counted across all quadrats, `r sum(surv$focal)` were the focal beetle. The **mean of ratios** is `r sprintf("%.3f", mor)`, the plain average of the six per-quadrat proportions. They differ by a factor of `r sprintf("%.2f", ratio)`, and neither is a rounding artefact. The reason is that the quadrats are not the same size and the focal proportion depends on size. The correlation between quadrat total and focal proportion is `r sprintf("%.2f", rho)`: the beetle makes up a large share of the small, sparse quadrats and a small share of the big ones. The pooled proportion is dominated by the big quadrats (they contribute most of the individuals), so it lands low. The mean of ratios gives every quadrat one equal vote regardless of size, so the small high-proportion quadrats pull it up. Neither number is wrong. They answer different questions: - The **pooled** proportion answers "across all the individuals I counted, what fraction is the focal species?" It is a population-level quantity, weighted by how many individuals each quadrat contributed. - The **mean of ratios** answers "in a typical quadrat, what fraction is the focal species?" It weights quadrats equally, which is what you want if the quadrat is your unit of replication and their sizes are incidental. The mistake is reporting one while describing the other: writing "the focal beetle is `r sprintf("%.0f", mor*100)` percent of the community" (the mean of ratios) when "the community" means the pooled individuals (`r sprintf("%.0f", pooled*100)` percent), or claiming "a typical quadrat is `r sprintf("%.0f", pooled*100)` percent focal" when the per-quadrat proportions average `r sprintf("%.0f", mor*100)` percent. ## The pooled proportion is a weighted mean The two summaries are not unrelated. The pooled proportion is exactly the mean of the per-quadrat proportions **weighted by quadrat total**: ```{r} #| label: weighted-mean weighted.mean(surv$prop, w = surv$total) # equals the pooled proportion ``` This lands on `r sprintf("%.3f", wm)`, the pooled value again. So the real question behind the trap is: what should the weights be? If each individual should count equally, weight by effort (the pooled proportion). If each site should count equally, do not weight (the mean of ratios). Deciding that is ecology, not arithmetic, and `summarise()` will happily give you either without asking. ```{r} #| label: fig-props #| echo: false #| fig-width: 6.4 #| fig-height: 3.8 #| fig-cap: "Focal-species proportion in each quadrat (bars), with the pooled proportion (solid line) and the mean of proportions (dashed line). The two summaries straddle the bars because the small high-proportion quadrats pull the unweighted mean up, while the pooled value follows the large quadrats." #| fig-alt: "Bar chart of focal proportion for six quadrats ranging from 0.10 to 0.60, with a solid horizontal line near 0.17 for the pooled proportion and a dashed horizontal line near 0.31 for the mean of proportions." library(ggplot2) te_green <- "#275139" ggplot(surv, aes(x = quadrat, y = prop)) + geom_col(fill = te_green, width = 0.66) + geom_hline(yintercept = pooled, linewidth = 0.9, colour = "grey15") + geom_hline(yintercept = mor, linewidth = 0.9, linetype = "dashed", colour = "grey15") + annotate("text", x = 6.35, y = pooled, label = "pooled", hjust = 1, vjust = -0.5, size = 3.3, colour = "grey15") + annotate("text", x = 6.35, y = mor, label = "mean of ratios", hjust = 1, vjust = -0.5, size = 3.3, colour = "grey15") + scale_y_continuous(expand = expansion(mult = c(0, 0.08))) + labs(x = NULL, y = "focal proportion") + theme_minimal(base_size = 13) + theme(plot.background = element_rect(fill = "white", colour = NA), panel.background = element_rect(fill = "white", colour = NA), panel.grid.minor = element_blank(), panel.grid.major.x = element_blank(), axis.title = element_text(colour = "grey25")) ``` ## Where to go next Grouped summaries assume the table is already assembled, but survey data usually arrives in two tables, one for sites and one for observations, that you have to join. The next trap lives there: an inner join silently drops the sites where the species was absent, which biases exactly the kind of mean you just computed. The joining tutorial takes that apart. ## Related tutorials - [Reading field data into R](../reading-field-data-into-r/) - [Wide and long: reshaping species data](../wide-and-long-species-data/) - [Joining ecological tables without losing zeros](../joining-ecological-tables/) - [From a field sheet to your first analysis](../from-field-sheet-to-first-analysis/) ## References Wickham H, Averick M, Bryan J, et al 2019. Journal of Open Source Software 4(43):1686 (10.21105/joss.01686). Wickham H, Cetinkaya-Rundel M, Grolemund G 2023. R for Data Science, 2nd ed. O'Reilly. ISBN 978-1-492-09740-2.