Indirect reciprocity and reputation

Indirect reciprocity in R: how reputation lets cooperation pay among strangers, the q > c/b threshold for helping, and the separatrix that splits the outcomes.

Direct reciprocity needs the same two individuals to meet again. Plenty of cooperation happens where that never occurs: a cleaner fish serves a client it may never see twice, a person gives blood for a stranger. Nowak and Sigmund (1998) proposed the escape: help someone who has helped others, and let the population keep score. Your partner today is not the one who repays you; someone who watched you is. This is indirect reciprocity, and it runs on reputation.

This post builds the simplest version in base R, finds the exact amount of reputation information needed to make helping pay, and shows why the outcome depends on where the population starts.

The donation game

Strip the interaction down. A donor can pay a cost c to give a recipient a benefit b, with b > c. Donors and recipients are drawn at random from a large population, and nobody meets the same partner twice. Two strategies compete. A discriminator helps a recipient it knows to be good and refuses one it knows to be bad; when it has no information it helps anyway. A defector never helps.

The one parameter that matters is q, the probability that the donor knows the recipient’s reputation. Set q to zero and nobody has any information, so the donation game collapses to a one-shot prisoner’s dilemma and defection wins. Set q to one and every donor sees every recipient’s record. The interesting question is where in between the switch happens.

For now assume reputation tracks type: discriminators are seen as good, defectors as bad. That assumption is doing real work, and the next post takes it apart.

b <- 3; cost <- 1                    # benefit to the recipient, cost to the donor

pay_disc <- function(x, q) {
  give <- x + (1 - x) * (1 - q)      # helps every discriminator, plus defectors it cannot identify
  get  <- b * x                      # every discriminator donor helps a good recipient
  -cost * give + get
}
pay_defect <- function(x, q) b * x * (1 - q)   # gets help only from uninformed discriminators

round(c(disc = pay_disc(0.5, 0.6), defect = pay_defect(0.5, 0.6)), 3)

  disc defect 
   0.8    0.6 

At half discriminators and reputation known six times out of ten, the discriminator earns 0.8 and the defector 0.6. The discriminator is ahead, but not by much, and the margin depends on both numbers.

Where the threshold sits

Take the difference of the two payoffs and the algebra tidies to something short. Writing x for the fraction of discriminators, the advantage of discriminating over defecting is

\[G(x, q) = x\,q\,(b - c) - c\,(1 - q).\]

The first term is what reputation buys: a discriminator meets other discriminators a fraction x of the time, and with probability q its own record is visible when it does. The second term is what reputation costs: whenever a discriminator cannot identify a defector, it wastes a donation. A defector never pays that.

gain <- function(x, q) x * q * (b - cost) - cost * (1 - q)
stopifnot(all.equal(gain(0.5, 0.6), pay_disc(0.5, 0.6) - pay_defect(0.5, 0.6)))

for (q in c(0.2, 1/3, 0.5, 0.9))
  cat(sprintf("q = %.2f   advantage in a discriminator population: %+.3f\n", q, gain(1, q)))

q = 0.20   advantage in a discriminator population: -0.400
q = 0.33   advantage in a discriminator population: -0.000
q = 0.50   advantage in a discriminator population: +0.500
q = 0.90   advantage in a discriminator population: +1.700

In a population made entirely of discriminators the advantage is q*b - c, which is positive exactly when

\[q > \frac{c}{b}.\]

With b = 3 and c = 1 that is q > 0.333. Below it a rare defector does better than the residents and cooperation cannot hold; above it discrimination resists invasion. At q exactly one third the advantage is zero, which the loop prints as -0.000.

That threshold has a familiar shape. Direct reciprocity beat defection when the continuation probability exceeded (T - R) / (T - P), and under donation-game payoffs (T = b, R = b - c, P = 0, S = -c) that expression is also c / b. The two routes to cooperation demand the same thing in different currencies: direct reciprocity needs a high enough chance of meeting the same partner again, indirect reciprocity a high enough chance that someone else knows what you did.

library(ggplot2)
qs <- seq(0, 1, 0.01)
d <- data.frame(q = qs, adv = gain(1, qs))
ggplot(d, aes(q, adv)) +
  geom_hline(yintercept = 0, colour = "#8d8d80") +
  geom_vline(xintercept = cost / b, linetype = "dashed", colour = "#93a87f") +
  geom_line(colour = "#275139", linewidth = 0.9) +
  annotate("text", x = cost / b + 0.03, y = -1.2,
           label = "q* = c/b = 0.333", colour = "#46604a", size = 3.4, hjust = 0) +
  labs(x = "Probability that reputation is known (q)",
       y = "Advantage of discriminating") +
  theme_minimal(base_size = 12)

Figure 1: Payoff advantage of a discriminator over a defector in a population of discriminators, as the probability that reputation is known rises. It crosses zero at q = c/b = 1/3.

Two stable outcomes, not one

The threshold above assumes the population is already full of discriminators. That is only half the story, because G(x, q) also rises with x: the more discriminators there are, the more it pays to be one. A game with that property has two stable states, exactly as the iterated prisoner’s dilemma did. Setting G(x, q) to zero gives the dividing line between them:

\[x^* = \frac{c\,(1 - q)}{q\,(b - c)}.\]

Start above x* and discriminators take over; start below it and defection sweeps. When x* exceeds one there is no interior root at all and defection wins from anywhere, which happens precisely when q falls under c / b.

separatrix <- function(q) cost * (1 - q) / (q * (b - cost))
for (q in c(0.4, 0.5, 0.6, 0.8, 0.9))
  cat(sprintf("q = %.1f   separatrix at x* = %.3f\n", q, separatrix(q)))

q = 0.4   separatrix at x* = 0.750
q = 0.5   separatrix at x* = 0.500
q = 0.6   separatrix at x* = 0.333
q = 0.8   separatrix at x* = 0.125
q = 0.9   separatrix at x* = 0.056

Better information does two things at once. It moves the threshold, and it shrinks the basin of attraction of defection: at q = 0.4 a population needs three quarters discriminators to get going, while at q = 0.9 five and a half per cent is enough.

A replicator run confirms that the interior root really is a watershed rather than an attractor. The replicator equation here reduces to one line, since the advantage G is already the payoff difference the dynamics need.

replicate_run <- function(x0, q, dt = 0.01, steps = 20000) {
  x <- x0
  for (i in seq_len(steps)) {
    x <- x + dt * x * (1 - x) * gain(x, q)
    x <- min(max(x, 0), 1)
  }
  x
}
starts <- c(0.20, 0.30, 0.40, 0.60)
data.frame(start = starts, final = round(sapply(starts, replicate_run, q = 0.6), 3))

  start final
1   0.2     0
2   0.3     0
3   0.4     1
4   0.6     1

At q = 0.6 the separatrix sits at one third, and the four runs split cleanly around it: 0.20 and 0.30 fall to zero, 0.40 and 0.60 climb to one. Nothing settles in between.

grid <- data.frame(q = seq(0.02, 1, 0.005))
grid$x <- pmin(separatrix(grid$q), 1)      # clamped: above q = 1/3 there is no interior root
ggplot(grid, aes(q, x)) +
  geom_ribbon(aes(ymin = x, ymax = 1), fill = "#dfe7d5") +
  geom_ribbon(aes(ymin = 0, ymax = x), fill = "#f0dedc") +
  geom_line(colour = "#275139", linewidth = 0.9) +
  annotate("text", x = 0.72, y = 0.72, label = "discrimination wins",
           colour = "#275139", size = 3.6) +
  annotate("text", x = 0.72, y = 0.12, label = "defection wins",
           colour = "#b5534e", size = 3.6) +
  coord_cartesian(xlim = c(0, 1), ylim = c(0, 1)) +
  labs(x = "Probability that reputation is known (q)",
       y = "Starting fraction of discriminators") +
  theme_minimal(base_size = 12)

Figure 2: The separatrix as a function of information. Populations starting above the curve evolve to full discrimination; those below collapse to defection. Left of q = 1/3 the curve sits at the top of the plot, so defection wins from any start.

An honest limit

The clean threshold rests on one assumption that the model never checks: that a discriminator is always seen as good and a defector as bad. Real bookkeeping is not that kind. A discriminator that correctly refuses a defector has, on the face of it, just withheld help, and under some scoring rules that is enough to lose its own standing. Once you write down how observers actually update their opinions, some rules keep the reputations accurate and others destroy them within a few dozen rounds, and the q > c/b result survives only under the first kind. That is the subject of the next post.

Two more assumptions are worth naming. Everyone here shares the same view of everyone else, which requires the population to observe the same events, and private information changes the answer for some rules and not others. And q is treated as a free dial, when in nature it is set by group size, by how much individuals watch each other, and by whether gossip travels. Estimating q in a real system is much harder than solving for the threshold.

Related tutorials

• Direct reciprocity and tit-for-tat
• Assessment rules for reputation
• Public and private reputation
• Checking a reputation model

References

Nowak MA, Sigmund K 1998. Nature 393(6685):573-577 (10.1038/31225)

Nowak MA, Sigmund K 2005. Nature 437(7063):1291-1298 (10.1038/nature04131)

Wedekind C, Milinski M 2000. Science 288(5467):850-852 (10.1126/science.288.5467.850)

Leimar O, Hammerstein P 2001. Proceedings of the Royal Society B 268(1468):745-753 (10.1098/rspb.2000.1573)