Strategies in the iterated game

evolutionary ecology

evolutionary game theory

social evolution

ecology tutorial

Evolutionary dynamics of iterated prisoner’s dilemma strategies in R: why cooperation and defection are both stable, and the start that decides which wins.

Author

Tidy Ecology

Published

2026-07-08

Modified

2026-10-14

Corrected 14 October 2026: the post said that the clean bistability dissolves once players make mistakes; it now says that mistakes shrink the basin of attraction of tit-for-tat, while at small error rates defection and tit-for-tat both stay stable against each other.

Corrected 7 October 2026: the post said that in its model cooperation slowly erodes from the inside and can then be toppled, which is why real cooperative systems tend to need active policing; it now says that deterministic replicator dynamics like the ones it runs contain no drift and cannot show that erosion, which needs a finite population, and it no longer draws the conclusion about policing.

Corrected 1 October 2026: the post said a handful of reciprocators is usually enough to pull a population toward cooperation; in this deterministic model it is any share of reciprocators above the threshold the post computes, and the post now gives that share.

Tit-for-tat resists defection once the future counts enough, but that is not the same as saying cooperation always wins. In the repeated game both cooperation and defection can be stable, and which one a population ends up in depends on where it starts. This post runs the evolutionary dynamics of a few classic strategies in base R and finds the tipping point between a cooperative world and a selfish one.

A tournament of strategies

Take three strategies for the repeated prisoner’s dilemma: always defect, always cooperate, and tit-for-tat. Using the same memory-one payoff engine as the previous post, build the matrix of per-round payoffs, each strategy against each, at a continuation probability of 0.9.

Tt <- 5; R <- 3; P <- 1; S <- 0
payoff_vec <- c(R, S, Tt, P); swap <- c(1, 3, 2, 4)
strat <- function(p, open = 1) list(p = p, open = open)
strategies <- list(ALLD = strat(c(0, 0, 0, 0), 0),
                   ALLC = strat(c(1, 1, 1, 1), 1),
                   TFT  = strat(c(1, 0, 1, 0), 1))
per_round <- function(A, B, w = 0.9) {
  a0 <- A$open; b0 <- B$open
  pi0 <- c(a0*b0, a0*(1-b0), (1-a0)*b0, (1-a0)*(1-b0))
  M <- matrix(0, 4, 4)
  for (s in 1:4) { ca <- A$p[s]; cb <- B$p[swap[s]]
    M[s, ] <- c(ca*cb, ca*(1-cb), (1-ca)*cb, (1-ca)*(1-cb)) }
  as.numeric((1 - w) * pi0 %*% solve(diag(4) - w * M) %*% payoff_vec)
}
nm <- names(strategies)
G <- outer(seq_along(nm), seq_along(nm),
           Vectorize(function(i, j) per_round(strategies[[i]], strategies[[j]])))
dimnames(G) <- list(nm, nm)
round(G, 2)

     ALLD ALLC TFT
ALLD  1.0    5 1.4
ALLC  0.0    3 3.0
TFT   0.9    3 3.0

Read the matrix by row: tit-for-tat earns 3 against itself and against a cooperator, but only 0.9 against a defector. A defector earns 5 against a naive cooperator, but just 1.4 against tit-for-tat and 1 against another defector. Exploiting cooperators is lucrative; meeting reciprocators or fellow defectors is not.

Two stable worlds

Defection is evolutionarily stable, and tit-for-tat is stable against defectors, which is not quite the same thing. A defector in a defector world earns 1, more than the 0.9 a lone tit-for-tat would get, so defection resists invasion. A tit-for-tat in a tit-for-tat world earns 3, far above the 1.4 a defector would get, so cooperation resists defection too. It is not a strict ESS: an unconditional cooperator also earns 3 against tit-for-tat and 3 against another cooperator, so nothing selects it out while defectors are absent, which is the leak the next section picks up.

cat(sprintf("all-defect world:    ALLD earns %.2f, a TFT invader %.2f  -> defection stable\n", G["ALLD","ALLD"], G["TFT","ALLD"]))

all-defect world:    ALLD earns 1.00, a TFT invader 0.90  -> defection stable

cat(sprintf("tit-for-tat world:   TFT earns %.2f, an ALLD invader %.2f  -> cooperation stable\n", G["TFT","TFT"], G["ALLD","TFT"]))

tit-for-tat world:   TFT earns 3.00, an ALLD invader 1.40  -> cooperation stable

With two stable states the outcome hinges on the starting mix, exactly like alternative stable states in community ecology. Run the replicator dynamics for defectors against tit-for-tat from many starting frequencies and watch where each ends up.

replicator2 <- function(xD, dt = 0.05, tmax = 2000) {
  M <- G[c("ALLD","TFT"), c("ALLD","TFT")]; x <- c(xD, 1 - xD)
  for (i in seq_len(tmax/dt)) { f <- as.numeric(M %*% x); x <- x + dt*x*(f - sum(x*f))
    x <- pmax(x, 1e-12); x <- x/sum(x) }
  x[1]
}
thr <- (G["ALLD","TFT"] - G["TFT","TFT"]) /
       ((G["ALLD","TFT"] - G["TFT","TFT"]) + (G["TFT","ALLD"] - G["ALLD","ALLD"]))
cat(sprintf("unstable threshold: defectors win only above %.3f of the population\n", thr))

unstable threshold: defectors win only above 0.941 of the population

cat(sprintf("start 90%% defectors -> %.0f%% defectors;  start 97%% -> %.0f%% defectors\n",
            100*replicator2(0.90), 100*replicator2(0.97)))

start 90% defectors -> 0% defectors;  start 97% -> 100% defectors

The separatrix sits at 0.941. Defection takes over only if it already makes up more than 94% of the population; a population that is 90% defectors still slides all the way back to cooperation, while one at 97% tips into universal defection. Tit-for-tat has an enormous basin of attraction. Any share of reciprocators above 5.9% is enough to pull the population all the way to cooperation, which is the core reason reciprocity is a plausible route out of the dilemma.

library(ggplot2)
te_canvas <- theme(plot.background  = element_rect(fill = "#f5f4ee", colour = NA),
                   panel.background = element_rect(fill = "#f5f4ee", colour = NA))
x0 <- seq(0.80, 1.0, 0.01)
d <- data.frame(start = x0, final = sapply(x0, replicator2))
ggplot(d, aes(start, final)) +
  geom_vline(xintercept = thr, linetype = "dashed", colour = "#93a87f") +
  geom_line(colour = "#275139", linewidth = 0.9) + geom_point(colour = "#16241d", size = 1.4) +
  labs(x = "Starting fraction of defectors", y = "Final fraction of defectors") +
  theme_minimal(base_size = 12) + te_canvas

A step function: final defector fraction stays near zero until the starting fraction reaches about 0.94, then jumps to one. — Figure 1: Final fraction of defectors against their starting fraction, from the replicator dynamics of always-defect versus tit-for-tat. The outcome jumps at 0.941: below it cooperation wins, above it defection does. Two stable states, a high threshold between them.

The weak point: naive cooperators

That large basin has a leak. A population of tit-for-tat earns 3 against itself, but so does always-cooperate: reciprocators and unconditional cooperators are indistinguishable while no defector is present, so nothing stops always-cooperate from drifting in by chance. Once enough naive cooperators accumulate, they become food, and a defector that could not have invaded pure tit-for-tat now finds easy meals. Replicator dynamics like those above cannot show this, because they contain no drift: with always-cooperate added and no defectors present, its share stays wherever it starts, and a rare defector can begin to spread only once unconditional cooperators are common enough for its gains from them to outweigh its losses against tit-for-tat. In a finite population, where chance can carry always-cooperate that far, cooperation is not a fortress; it is a stable state that can slowly erode from the inside and then be toppled.

An honest limit

The dynamics here assume error-free play, a fixed continuation probability, and a small hand-picked set of strategies. All three matter. The strategy set is not exhaustive, and a strategy outside it can overturn the ranking. Mistakes shrink the basin of attraction of tit-for-tat: a slip against an unforgiving partner triggers a feud, which the checking post examines, and tit-for-tat then earns less against itself, although at small error rates defection and tit-for-tat both stay stable against each other. And the whole picture is built at one value of the continuation probability; lower it below the reciprocity threshold and cooperation stops being stable at all.

References

Axelrod R 1984. The Evolution of Cooperation. Basic Books. ISBN 978-0465021215

Nowak MA, Sigmund K 1992. Nature 355(6357):250-253 (10.1038/355250a0)

Nowak MA, May RM 1992. Nature 359(6398):826-829 (10.1038/359826a0)

--- title: "Strategies in the iterated game" description: "Evolutionary dynamics of iterated prisoner's dilemma strategies in R: why cooperation and defection are both stable, and the start that decides which wins." date: "2026-07-08 09:00" date-modified: "2026-10-14" categories: [evolutionary ecology, evolutionary game theory, social evolution, R, ecology tutorial] image: thumbnail.png image-alt: "Final fraction of defectors against their starting fraction, jumping from zero to one at a high threshold near 0.94" --- *Corrected 14 October 2026: the post said that the clean bistability dissolves once players make mistakes; it now says that mistakes shrink the basin of attraction of tit-for-tat, while at small error rates defection and tit-for-tat both stay stable against each other.* *Corrected 7 October 2026: the post said that in its model cooperation slowly erodes from the inside and can then be toppled, which is why real cooperative systems tend to need active policing; it now says that deterministic replicator dynamics like the ones it runs contain no drift and cannot show that erosion, which needs a finite population, and it no longer draws the conclusion about policing.* *Corrected 1 October 2026: the post said a handful of reciprocators is usually enough to pull a population toward cooperation; in this deterministic model it is any share of reciprocators above the threshold the post computes, and the post now gives that share.* [Tit-for-tat resists defection](../direct-reciprocity-and-tit-for-tat/) once the future counts enough, but that is not the same as saying cooperation always wins. In the repeated game both cooperation and defection can be stable, and which one a population ends up in depends on where it starts. This post runs the evolutionary dynamics of a few classic strategies in base R and finds the tipping point between a cooperative world and a selfish one. ## A tournament of strategies Take three strategies for the repeated prisoner's dilemma: always defect, always cooperate, and tit-for-tat. Using the same memory-one payoff engine as the [previous post](../direct-reciprocity-and-tit-for-tat/), build the matrix of per-round payoffs, each strategy against each, at a continuation probability of 0.9. ```{r} #| label: engine Tt <- 5; R <- 3; P <- 1; S <- 0 payoff_vec <- c(R, S, Tt, P); swap <- c(1, 3, 2, 4) strat <- function(p, open = 1) list(p = p, open = open) strategies <- list(ALLD = strat(c(0, 0, 0, 0), 0), ALLC = strat(c(1, 1, 1, 1), 1), TFT = strat(c(1, 0, 1, 0), 1)) per_round <- function(A, B, w = 0.9) { a0 <- A$open; b0 <- B$open pi0 <- c(a0*b0, a0*(1-b0), (1-a0)*b0, (1-a0)*(1-b0)) M <- matrix(0, 4, 4) for (s in 1:4) { ca <- A$p[s]; cb <- B$p[swap[s]] M[s, ] <- c(ca*cb, ca*(1-cb), (1-ca)*cb, (1-ca)*(1-cb)) } as.numeric((1 - w) * pi0 %*% solve(diag(4) - w * M) %*% payoff_vec) } nm <- names(strategies) G <- outer(seq_along(nm), seq_along(nm), Vectorize(function(i, j) per_round(strategies[[i]], strategies[[j]]))) dimnames(G) <- list(nm, nm) round(G, 2) ``` Read the matrix by row: tit-for-tat earns 3 against itself and against a cooperator, but only 0.9 against a defector. A defector earns 5 against a naive cooperator, but just 1.4 against tit-for-tat and 1 against another defector. Exploiting cooperators is lucrative; meeting reciprocators or fellow defectors is not. ## Two stable worlds Defection is evolutionarily stable, and tit-for-tat is stable against defectors, which is not quite the same thing. A defector in a defector world earns 1, more than the 0.9 a lone tit-for-tat would get, so defection resists invasion. A tit-for-tat in a tit-for-tat world earns 3, far above the 1.4 a defector would get, so cooperation resists defection too. It is not a strict ESS: an unconditional cooperator also earns 3 against tit-for-tat and 3 against another cooperator, so nothing selects it out while defectors are absent, which is the leak the next section picks up. ```{r} #| label: ess cat(sprintf("all-defect world: ALLD earns %.2f, a TFT invader %.2f -> defection stable\n", G["ALLD","ALLD"], G["TFT","ALLD"])) cat(sprintf("tit-for-tat world: TFT earns %.2f, an ALLD invader %.2f -> cooperation stable\n", G["TFT","TFT"], G["ALLD","TFT"])) ``` With two stable states the outcome hinges on the starting mix, exactly like [alternative stable states](../priority-effects-and-alternative-states/) in community ecology. Run the replicator dynamics for defectors against tit-for-tat from many starting frequencies and watch where each ends up. ```{r} #| label: bistable replicator2 <- function(xD, dt = 0.05, tmax = 2000) { M <- G[c("ALLD","TFT"), c("ALLD","TFT")]; x <- c(xD, 1 - xD) for (i in seq_len(tmax/dt)) { f <- as.numeric(M %*% x); x <- x + dt*x*(f - sum(x*f)) x <- pmax(x, 1e-12); x <- x/sum(x) } x[1] } thr <- (G["ALLD","TFT"] - G["TFT","TFT"]) / ((G["ALLD","TFT"] - G["TFT","TFT"]) + (G["TFT","ALLD"] - G["ALLD","ALLD"])) cat(sprintf("unstable threshold: defectors win only above %.3f of the population\n", thr)) cat(sprintf("start 90%% defectors -> %.0f%% defectors; start 97%% -> %.0f%% defectors\n", 100*replicator2(0.90), 100*replicator2(0.97))) ``` The separatrix sits at 0.941. Defection takes over only if it already makes up more than 94% of the population; a population that is 90% defectors still slides all the way back to cooperation, while one at 97% tips into universal defection. Tit-for-tat has an enormous basin of attraction. Any share of reciprocators above `r sprintf("%.1f", 100 * (1 - thr))`% is enough to pull the population all the way to cooperation, which is the core reason reciprocity is a plausible route out of the dilemma. ```{r} #| label: fig-bistable #| fig-cap: "Final fraction of defectors against their starting fraction, from the replicator dynamics of always-defect versus tit-for-tat. The outcome jumps at 0.941: below it cooperation wins, above it defection does. Two stable states, a high threshold between them." #| fig-alt: "A step function: final defector fraction stays near zero until the starting fraction reaches about 0.94, then jumps to one." library(ggplot2) te_canvas <- theme(plot.background = element_rect(fill = "#f5f4ee", colour = NA), panel.background = element_rect(fill = "#f5f4ee", colour = NA)) x0 <- seq(0.80, 1.0, 0.01) d <- data.frame(start = x0, final = sapply(x0, replicator2)) ggplot(d, aes(start, final)) + geom_vline(xintercept = thr, linetype = "dashed", colour = "#93a87f") + geom_line(colour = "#275139", linewidth = 0.9) + geom_point(colour = "#16241d", size = 1.4) + labs(x = "Starting fraction of defectors", y = "Final fraction of defectors") + theme_minimal(base_size = 12) + te_canvas ``` ## The weak point: naive cooperators That large basin has a leak. A population of tit-for-tat earns 3 against itself, but so does always-cooperate: reciprocators and unconditional cooperators are indistinguishable while no defector is present, so nothing stops always-cooperate from drifting in by chance. Once enough naive cooperators accumulate, they become food, and a defector that could not have invaded pure tit-for-tat now finds easy meals. Replicator dynamics like those above cannot show this, because they contain no drift: with always-cooperate added and no defectors present, its share stays wherever it starts, and a rare defector can begin to spread only once unconditional cooperators are common enough for its gains from them to outweigh its losses against tit-for-tat. In a finite population, where chance can carry always-cooperate that far, cooperation is not a fortress; it is a stable state that can slowly erode from the inside and then be toppled. ## An honest limit The dynamics here assume error-free play, a fixed continuation probability, and a small hand-picked set of strategies. All three matter. The strategy set is not exhaustive, and a strategy outside it can overturn the ranking. Mistakes shrink the basin of attraction of tit-for-tat: a slip against an unforgiving partner triggers a feud, which the [checking post](../checking-a-reciprocity-model/) examines, and tit-for-tat then earns less against itself, although at small error rates defection and tit-for-tat both stay stable against each other. And the whole picture is built at one value of the continuation probability; lower it below the [reciprocity threshold](../direct-reciprocity-and-tit-for-tat/) and cooperation stops being stable at all. ## References Axelrod R 1984. The Evolution of Cooperation. Basic Books. ISBN 978-0465021215 Nowak MA, Sigmund K 1992. Nature 355(6357):250-253 (10.1038/355250a0) Nowak MA, May RM 1992. Nature 359(6398):826-829 (10.1038/359826a0) ## Related tutorials - [The prisoner's dilemma](../the-prisoners-dilemma/) - [Direct reciprocity and tit-for-tat](../direct-reciprocity-and-tit-for-tat/) - [Assessment rules for reputation](../assessment-rules-and-reputation/) - [Priority effects and alternative stable states](../priority-effects-and-alternative-states/)