Strategies in the iterated game

evolutionary ecology

evolutionary game theory

social evolution

ecology tutorial

Evolutionary dynamics of iterated prisoner’s dilemma strategies in R: why cooperation and defection are both stable, and the start that decides which wins.

Author

Tidy Ecology

Published

2026-08-07

Tit-for-tat resists defection once the future counts enough, but that is not the same as saying cooperation always wins. In the repeated game both cooperation and defection can be stable, and which one a population ends up in depends on where it starts. This post runs the evolutionary dynamics of a few classic strategies in base R and finds the tipping point between a cooperative world and a selfish one.

A tournament of strategies

Take three strategies for the repeated prisoner’s dilemma: always defect, always cooperate, and tit-for-tat. Using the same memory-one payoff engine as the previous post, build the matrix of per-round payoffs, each strategy against each, at a continuation probability of 0.9.

Tt <- 5; R <- 3; P <- 1; S <- 0
payoff_vec <- c(R, S, Tt, P); swap <- c(1, 3, 2, 4)
strat <- function(p, open = 1) list(p = p, open = open)
strategies <- list(ALLD = strat(c(0, 0, 0, 0), 0),
                   ALLC = strat(c(1, 1, 1, 1), 1),
                   TFT  = strat(c(1, 0, 1, 0), 1))
per_round <- function(A, B, w = 0.9) {
  a0 <- A$open; b0 <- B$open
  pi0 <- c(a0*b0, a0*(1-b0), (1-a0)*b0, (1-a0)*(1-b0))
  M <- matrix(0, 4, 4)
  for (s in 1:4) { ca <- A$p[s]; cb <- B$p[swap[s]]
    M[s, ] <- c(ca*cb, ca*(1-cb), (1-ca)*cb, (1-ca)*(1-cb)) }
  as.numeric((1 - 0.9) * pi0 %*% solve(diag(4) - 0.9 * M) %*% payoff_vec)
}
nm <- names(strategies)
G <- outer(seq_along(nm), seq_along(nm),
           Vectorize(function(i, j) per_round(strategies[[i]], strategies[[j]])))
dimnames(G) <- list(nm, nm)
round(G, 2)

     ALLD ALLC TFT
ALLD  1.0    5 1.4
ALLC  0.0    3 3.0
TFT   0.9    3 3.0

Read the matrix by row: tit-for-tat earns 3 against itself and against a cooperator, but only 0.9 against a defector. A defector earns 5 against a naive cooperator, but just 1.4 against tit-for-tat and 1 against another defector. Exploiting cooperators is lucrative; meeting reciprocators or fellow defectors is not.

Two stable worlds

Both defection and tit-for-tat are evolutionarily stable. A defector in a defector world earns 1, more than the 0.9 a lone tit-for-tat would get, so defection resists invasion. A tit-for-tat in a tit-for-tat world earns 3, far above the 1.4 a defector would get, so cooperation resists too.

cat(sprintf("all-defect world:    ALLD earns %.2f, a TFT invader %.2f  -> defection stable\n", G["ALLD","ALLD"], G["TFT","ALLD"]))

all-defect world:    ALLD earns 1.00, a TFT invader 0.90  -> defection stable

cat(sprintf("tit-for-tat world:   TFT earns %.2f, an ALLD invader %.2f  -> cooperation stable\n", G["TFT","TFT"], G["ALLD","TFT"]))

tit-for-tat world:   TFT earns 3.00, an ALLD invader 1.40  -> cooperation stable

With two stable states the outcome hinges on the starting mix, exactly like alternative stable states in community ecology. Run the replicator dynamics for defectors against tit-for-tat from many starting frequencies and watch where each ends up.

replicator2 <- function(xD, dt = 0.05, tmax = 2000) {
  M <- G[c("ALLD","TFT"), c("ALLD","TFT")]; x <- c(xD, 1 - xD)
  for (i in seq_len(tmax/dt)) { f <- as.numeric(M %*% x); x <- x + dt*x*(f - sum(x*f))
    x <- pmax(x, 1e-12); x <- x/sum(x) }
  x[1]
}
thr <- (G["ALLD","TFT"] - G["TFT","TFT"]) /
       ((G["ALLD","TFT"] - G["TFT","TFT"]) + (G["TFT","ALLD"] - G["ALLD","ALLD"]))
cat(sprintf("unstable threshold: defectors win only above %.3f of the population\n", thr))

unstable threshold: defectors win only above 0.941 of the population

cat(sprintf("start 90%% defectors -> %.0f%% defectors;  start 97%% -> %.0f%% defectors\n",
            100*replicator2(0.90), 100*replicator2(0.97)))

start 90% defectors -> 0% defectors;  start 97% -> 100% defectors

The separatrix sits at 0.941. Defection takes over only if it already makes up more than 94% of the population; a population that is 90% defectors still slides all the way back to cooperation, while one at 97% tips into universal defection. Tit-for-tat has an enormous basin of attraction. A handful of reciprocators is usually enough to pull a population toward cooperation, which is the core reason reciprocity is a plausible route out of the dilemma.

library(ggplot2)
x0 <- seq(0.80, 1.0, 0.01)
d <- data.frame(start = x0, final = sapply(x0, replicator2))
ggplot(d, aes(start, final)) +
  geom_vline(xintercept = thr, linetype = "dashed", colour = "#93a87f") +
  geom_line(colour = "#275139", linewidth = 0.9) + geom_point(colour = "#16241d", size = 1.4) +
  labs(x = "Starting fraction of defectors", y = "Final fraction of defectors") +
  theme_minimal(base_size = 12)

A step function: final defector fraction stays near zero until the starting fraction reaches about 0.94, then jumps to one. — Figure 1: Final fraction of defectors against their starting fraction, from the replicator dynamics of always-defect versus tit-for-tat. The outcome jumps at 0.941: below it cooperation wins, above it defection does. Two stable states, a high threshold between them.

The weak point: naive cooperators

That large basin has a leak. A population of tit-for-tat earns 3 against itself, but so does always-cooperate: reciprocators and unconditional cooperators are indistinguishable while no defector is present, so nothing stops always-cooperate from drifting in by chance. Once enough naive cooperators accumulate, they become food, and a defector that could not have invaded pure tit-for-tat now finds easy meals. Cooperation in this model is not a fortress; it is a stable state that slowly erodes from the inside and can then be toppled, which is why real cooperative systems tend to need active policing, not just reciprocity.

An honest limit

The dynamics here assume error-free play, a fixed continuation probability, and a small hand-picked set of strategies. All three matter. The strategy set is not exhaustive, and a strategy outside it can overturn the ranking. The clean bistability dissolves once players make mistakes, because a slip against an unforgiving partner triggers a feud, which the checking post examines. And the whole picture is built at one value of the continuation probability; lower it below the reciprocity threshold and cooperation stops being stable at all.

References

Axelrod R 1984. The Evolution of Cooperation. Basic Books. ISBN 978-0465021215

Nowak MA, Sigmund K 1992. Nature 355(6357):250-253 (10.1038/355250a0)

Nowak MA, May RM 1992. Nature 359(6398):826-829 (10.1038/359826a0)

--- title: "Strategies in the iterated game" description: "Evolutionary dynamics of iterated prisoner's dilemma strategies in R: why cooperation and defection are both stable, and the start that decides which wins." date: "2026-08-07 11:00" categories: [evolutionary ecology, evolutionary game theory, social evolution, R, ecology tutorial] image: thumbnail.png image-alt: "Final fraction of defectors against their starting fraction, jumping from zero to one at a high threshold near 0.94" --- [Tit-for-tat resists defection](../direct-reciprocity-and-tit-for-tat/) once the future counts enough, but that is not the same as saying cooperation always wins. In the repeated game both cooperation and defection can be stable, and which one a population ends up in depends on where it starts. This post runs the evolutionary dynamics of a few classic strategies in base R and finds the tipping point between a cooperative world and a selfish one. ## A tournament of strategies Take three strategies for the repeated prisoner's dilemma: always defect, always cooperate, and tit-for-tat. Using the same memory-one payoff engine as the [previous post](../direct-reciprocity-and-tit-for-tat/), build the matrix of per-round payoffs, each strategy against each, at a continuation probability of 0.9. ```{r} #| label: engine Tt <- 5; R <- 3; P <- 1; S <- 0 payoff_vec <- c(R, S, Tt, P); swap <- c(1, 3, 2, 4) strat <- function(p, open = 1) list(p = p, open = open) strategies <- list(ALLD = strat(c(0, 0, 0, 0), 0), ALLC = strat(c(1, 1, 1, 1), 1), TFT = strat(c(1, 0, 1, 0), 1)) per_round <- function(A, B, w = 0.9) { a0 <- A$open; b0 <- B$open pi0 <- c(a0*b0, a0*(1-b0), (1-a0)*b0, (1-a0)*(1-b0)) M <- matrix(0, 4, 4) for (s in 1:4) { ca <- A$p[s]; cb <- B$p[swap[s]] M[s, ] <- c(ca*cb, ca*(1-cb), (1-ca)*cb, (1-ca)*(1-cb)) } as.numeric((1 - 0.9) * pi0 %*% solve(diag(4) - 0.9 * M) %*% payoff_vec) } nm <- names(strategies) G <- outer(seq_along(nm), seq_along(nm), Vectorize(function(i, j) per_round(strategies[[i]], strategies[[j]]))) dimnames(G) <- list(nm, nm) round(G, 2) ``` Read the matrix by row: tit-for-tat earns 3 against itself and against a cooperator, but only 0.9 against a defector. A defector earns 5 against a naive cooperator, but just 1.4 against tit-for-tat and 1 against another defector. Exploiting cooperators is lucrative; meeting reciprocators or fellow defectors is not. ## Two stable worlds Both defection and tit-for-tat are evolutionarily stable. A defector in a defector world earns 1, more than the 0.9 a lone tit-for-tat would get, so defection resists invasion. A tit-for-tat in a tit-for-tat world earns 3, far above the 1.4 a defector would get, so cooperation resists too. ```{r} #| label: ess cat(sprintf("all-defect world: ALLD earns %.2f, a TFT invader %.2f -> defection stable\n", G["ALLD","ALLD"], G["TFT","ALLD"])) cat(sprintf("tit-for-tat world: TFT earns %.2f, an ALLD invader %.2f -> cooperation stable\n", G["TFT","TFT"], G["ALLD","TFT"])) ``` With two stable states the outcome hinges on the starting mix, exactly like [alternative stable states](../priority-effects-and-alternative-states/) in community ecology. Run the replicator dynamics for defectors against tit-for-tat from many starting frequencies and watch where each ends up. ```{r} #| label: bistable replicator2 <- function(xD, dt = 0.05, tmax = 2000) { M <- G[c("ALLD","TFT"), c("ALLD","TFT")]; x <- c(xD, 1 - xD) for (i in seq_len(tmax/dt)) { f <- as.numeric(M %*% x); x <- x + dt*x*(f - sum(x*f)) x <- pmax(x, 1e-12); x <- x/sum(x) } x[1] } thr <- (G["ALLD","TFT"] - G["TFT","TFT"]) / ((G["ALLD","TFT"] - G["TFT","TFT"]) + (G["TFT","ALLD"] - G["ALLD","ALLD"])) cat(sprintf("unstable threshold: defectors win only above %.3f of the population\n", thr)) cat(sprintf("start 90%% defectors -> %.0f%% defectors; start 97%% -> %.0f%% defectors\n", 100*replicator2(0.90), 100*replicator2(0.97))) ``` The separatrix sits at 0.941. Defection takes over only if it already makes up more than 94% of the population; a population that is 90% defectors still slides all the way back to cooperation, while one at 97% tips into universal defection. Tit-for-tat has an enormous basin of attraction. A handful of reciprocators is usually enough to pull a population toward cooperation, which is the core reason reciprocity is a plausible route out of the dilemma. ```{r} #| label: fig-bistable #| fig-cap: "Final fraction of defectors against their starting fraction, from the replicator dynamics of always-defect versus tit-for-tat. The outcome jumps at 0.941: below it cooperation wins, above it defection does. Two stable states, a high threshold between them." #| fig-alt: "A step function: final defector fraction stays near zero until the starting fraction reaches about 0.94, then jumps to one." library(ggplot2) x0 <- seq(0.80, 1.0, 0.01) d <- data.frame(start = x0, final = sapply(x0, replicator2)) ggplot(d, aes(start, final)) + geom_vline(xintercept = thr, linetype = "dashed", colour = "#93a87f") + geom_line(colour = "#275139", linewidth = 0.9) + geom_point(colour = "#16241d", size = 1.4) + labs(x = "Starting fraction of defectors", y = "Final fraction of defectors") + theme_minimal(base_size = 12) ``` ## The weak point: naive cooperators That large basin has a leak. A population of tit-for-tat earns 3 against itself, but so does always-cooperate: reciprocators and unconditional cooperators are indistinguishable while no defector is present, so nothing stops always-cooperate from drifting in by chance. Once enough naive cooperators accumulate, they become food, and a defector that could not have invaded pure tit-for-tat now finds easy meals. Cooperation in this model is not a fortress; it is a stable state that slowly erodes from the inside and can then be toppled, which is why real cooperative systems tend to need active policing, not just reciprocity. ## An honest limit The dynamics here assume error-free play, a fixed continuation probability, and a small hand-picked set of strategies. All three matter. The strategy set is not exhaustive, and a strategy outside it can overturn the ranking. The clean bistability dissolves once players make mistakes, because a slip against an unforgiving partner triggers a feud, which the [checking post](../checking-a-reciprocity-model/) examines. And the whole picture is built at one value of the continuation probability; lower it below the [reciprocity threshold](../direct-reciprocity-and-tit-for-tat/) and cooperation stops being stable at all. ## Related tutorials - [The prisoner's dilemma](../the-prisoners-dilemma/) - [Direct reciprocity and tit-for-tat](../direct-reciprocity-and-tit-for-tat/) - [Assessment rules for reputation](../assessment-rules-and-reputation/) - [Priority effects and alternative stable states](../priority-effects-and-alternative-states/) ## References Axelrod R 1984. The Evolution of Cooperation. Basic Books. ISBN 978-0465021215 Nowak MA, Sigmund K 1992. Nature 355(6357):250-253 (10.1038/355250a0) Nowak MA, May RM 1992. Nature 359(6398):826-829 (10.1038/359826a0)