Direct reciprocity and tit-for-tat

How repeated interaction rescues cooperation in R: the shadow of the future, tit-for-tat, and the continuation probability at which reciprocity beats defection.

In a single prisoner’s dilemma defection always wins. The escape, first argued by Trivers (1971) and made famous by Axelrod and Hamilton (1981), is that animals meet again. If today’s partner is tomorrow’s too, cheating now invites retaliation later, and cooperation can pay. This is direct reciprocity. This post codes the repeated game in base R, builds tit-for-tat, and finds the exact point where the shadow of the future is long enough to make reciprocity beat defection.

The repeated game

Suppose after each round the same two players meet again with probability w, the continuation probability. A strategy is a rule for what to play given the last round. The simplest and most famous is tit-for-tat: cooperate first, then copy whatever the opponent just did. We can compute the exact long-run payoff of any pair of memory-one strategies (those depending only on the previous round) from a small Markov chain over the four outcomes.

Tt <- 5; R <- 3; P <- 1; S <- 0
payoff_vec <- c(R, S, Tt, P)          # states CC, CD, DC, DD (focal move, opponent move)
swap <- c(1, 3, 2, 4)                 # same state seen from the opponent's side
strat <- function(p, open = 1) list(p = p, open = open)   # p = P(cooperate) after CC,CD,DC,DD
ALLD <- strat(c(0, 0, 0, 0), open = 0)
TFT  <- strat(c(1, 0, 1, 0), open = 1)
per_round <- function(A, B, w = 0.9) {
  a0 <- A$open; b0 <- B$open
  pi0 <- c(a0 * b0, a0 * (1 - b0), (1 - a0) * b0, (1 - a0) * (1 - b0))
  M <- matrix(0, 4, 4)
  for (s in 1:4) {
    ca <- A$p[s]; cb <- B$p[swap[s]]
    M[s, ] <- c(ca * cb, ca * (1 - cb), (1 - ca) * cb, (1 - ca) * (1 - cb))
  }
  as.numeric((1 - w) * pi0 %*% solve(diag(4) - w * M) %*% payoff_vec)   # discounted per-round payoff
}

Tit-for-tat against a defector

The test of reciprocity is whether a population of tit-for-tat players can keep out defectors. Compare what tit-for-tat earns against itself with what a defector earns by invading it.

for (w in c(0.4, 0.9)) {
  cat(sprintf("w = %.1f:  TFT vs TFT = %.2f   ALLD vs TFT = %.2f   ->  %s\n",
              w, per_round(TFT, TFT, w), per_round(ALLD, TFT, w),
              ifelse(per_round(TFT, TFT, w) > per_round(ALLD, TFT, w), "TFT holds", "ALLD invades")))
}

w = 0.4:  TFT vs TFT = 3.00   ALLD vs TFT = 3.40   ->  ALLD invades
w = 0.9:  TFT vs TFT = 3.00   ALLD vs TFT = 1.40   ->  TFT holds

When encounters rarely repeat (w of 0.4), a defector invading tit-for-tat earns 3.40 per round against tit-for-tat’s own 3.00: it grabs the temptation payoff and mostly walks away before the retaliation matters, so defection wins. When encounters usually repeat (w of 0.9), the defector earns only 1.40: tit-for-tat punishes it in every future round, and reciprocity holds.

The exact threshold

The switch happens at a precise continuation probability. Tit-for-tat resists defection when its payoff against itself beats the defector’s, which works out to

\[ w > \frac{T - R}{T - P}. \]

wstar <- (Tt - R) / (Tt - P)
cat(sprintf("reciprocity beats defection when w > (T-R)/(T-P) = %.2f\n", wstar))

reciprocity beats defection when w > (T-R)/(T-P) = 0.50

With these payoffs the threshold is 0.5: the future must count for at least half as much as the present. Below it, encounters are too fleeting and defection dominates as in the one-shot game; above it, the shadow of the future is long enough to make cheating unprofitable. The whole logic of reciprocity lives in that inequality: cooperation needs a high enough chance of meeting again.

library(ggplot2)
ws <- seq(0.01, 0.95, 0.02)
d <- rbind(data.frame(w = ws, pay = sapply(ws, function(w) per_round(TFT, TFT, w)), who = "TFT vs TFT"),
           data.frame(w = ws, pay = sapply(ws, function(w) per_round(ALLD, TFT, w)), who = "ALLD invading TFT"))
ggplot(d, aes(w, pay, colour = who)) +
  geom_vline(xintercept = wstar, linetype = "dashed", colour = "#93a87f") +
  geom_line(linewidth = 0.9) +
  scale_colour_manual(values = c("TFT vs TFT" = "#275139", "ALLD invading TFT" = "#b5534e")) +
  annotate("text", x = wstar + 0.02, y = 4.5, label = "w* = 0.5", colour = "#46604a", size = 3.4, hjust = 0) +
  labs(x = "Continuation probability w", y = "Per-round payoff", colour = NULL) +
  theme_minimal(base_size = 12) + theme(legend.position = "top")

Figure 1: Per-round payoff of tit-for-tat against itself (flat at 3) and of a defector invading it, as the continuation probability rises. They cross at w = 0.5; above it reciprocity resists defection.

An honest limit

The threshold assumes players actually meet again, recognise each other, and remember the last round. Take any of those away and reciprocity weakens: in a large, well-mixed population where partners are strangers, w is effectively zero and defection returns. Tit-for-tat is also unforgiving, which becomes a liability once mistakes happen, a problem the checking post takes up. And reciprocity is only one way out of the dilemma; where partners are relatives rather than repeat acquaintances, kin selection does the work instead. Repeated interaction is a powerful escape, but it buys cooperation only under conditions that not every species meets.

Related tutorials

• The prisoner’s dilemma
• Strategies in the iterated game
• Indirect reciprocity and reputation
• The replicator equation

References

Trivers RL 1971. Quarterly Review of Biology 46(1):35-57 (10.1086/406755)

Axelrod R, Hamilton WD 1981. Science 211(4489):1390-1396 (10.1126/science.7466396)

Axelrod R 1984. The Evolution of Cooperation. Basic Books. ISBN 978-0465021215