1. Introduction

The evolutionary origin of cooperative behavior is an enduring puzzle. For several decades, various disciplines have tried to answer why an organism performs an act that is costly to itself and beneficial to another (e.g., Axelrod, 1984, Axelrod & Dion, 1988, Fehr & Fischbacher, 2003, Gintis et al., 2003, Hamilton, 1964a, Hamilton, 1964b, Trivers, 1971). The Prisoner's Dilemma Game (PDG) is the prevailing and orthodox framework for studying the evolution of cooperation towards nonkin, but it suffers from a procedural flaw when compared with the real-life situations that it is designed to model. In a standard PDG, players face a binary choice: whether to cooperate or to defect. Cooperation is, however, seldom in an “all-or-nothing” case (Frean, 1996, Killingback et al., 1999, Wahl & Nowak, 1999a, Wahl & Nowak, 1999b). Individuals can decide about the extent of their cooperation. Bats may vary cooperativeness in terms of the quantity of a meal shared (Wilkinson, 1984), impala may vary grooming time (Hart & Hart, 1992), and fish may vary cooperation by increasing or decreasing the inspection distance of a predator (Milinski, Lüthi, Eggler, & Parker, 1997). In line with Roberts and Renwick (2003), we explore in this study how human participants gradually adjust the level of their cooperativeness throughout a series of interactions. We specifically investigate whether the strategy pursued by the opponent (subtle cheating vs. no cheating) affects cooperation and how this is moderated by communication noise.

If interactions with variable investment levels are considered, a new kind of cheating is possible because some cooperators may be less generous than others are (Sherratt & Roberts, 1998). Rather than a plain defection (in the dichotomous iterated PDG), individuals may invest a little less than their partners (in an iterated PDG that allows variable investment). In principle, this “subtle cheating” could gradually erode cooperation. Natural selection should, however, favor the ability to detect and discriminate against subtle cheaters (Trivers, 1971). Cosmides and Tooby argued that humans have evolved domain-specific cognitive capacities that allow them to detect cheaters (Cosmides, 1989, Cosmides & Tooby, 1992). There is even some evidence that cheaters may look different from cooperators (Yamagishi, Tanida, Mashima, Shimoma, & Kanazawa, 2003). The capacity to distinguish defectors and cooperators allows humans to select partners that will reciprocate.

How do evolutionary strategies (and hence human cooperativeness) survive the potentially destabilizing effects of this type of subtle exploitation? Roberts and Sherratt (1998) simulated interactions between a number of simple strategies (including cheater strategies) and found that cooperation emerged. It did so through a strategy of “Raise-The-Stakes” (RTS). RTS invests a minimal amount at first (“testing-the-water”) and then gradually increases the investment with a small amount in subsequent interactions, if and only if the partner matches or betters RTS's investment. RTS allows individuals to take advantage of cooperative opportunities while minimizing the risk of being exploited. Compare this strategy to the Tit-for-Tat (TFT) strategy that typically outperforms other strategies in the standard dichotomous PDG (Axelrod, 1984). TFT is less cautious on its initial move by starting with a cooperative choice. Such a strategy makes a giant “leap of faith” on the first move and is vulnerable to exploitation by cheaters in a PDG with variable investment.

Roberts and Renwick (2003) tested whether the RTS strategy is adopted by human participants in an iterated PDG with variable investment. The experiment supported the prediction of Roberts and Sherratt (1998) that investment in cooperation increases over the course of an interaction. Participants increased donations over successive rounds when the partner reciprocated the investment. On the first move, however, participants donated considerably more than the minimal amount required to “test the water”. This aspect of human behavior differed from the simulations but has an interesting implication. According to the simulations of Roberts and Sherratt (1998), RTS's major advantage over TFT was its low initial cooperativeness, which immunizes the RTS strategy against exploitation. Therefore, an important question is whether the “faithful” RTS strategy adopted by human participants is robust against subtle cheating. The question of what would happen when they are confronted with cheaters is legitimate, because natural populations are likely to contain individuals who do not cooperate (Doebeli et al., 2004, Kurzban & Houser, 2005, Sherratt & Roberts, 2001). Starting at an intermediate level makes the faithful RTS-like strategy vulnerable to exploitation. Killingback and Doebeli (1998) have questioned the robustness of RTS, and there has been no direct experimental test of whether the faithful RTS-like strategy adopted by the participants (Roberts & Renwick, 2003) is robust against subtle cheating.

Humans following a faithful RTS strategy have three options when subtly cheated on. They can either ignore the subtle cheating and maintain their cooperation level without raising the stakes, they can harshly punish by investing nothing on the subsequent round, or they can switch to TFT and match the partner's moves. Maintaining cooperation levels indefinitely leads to a large relative fitness disadvantage because of an infinitely enduring exploitation by the subtle cheater. Turning to noncooperation in reaction to a subtle defection has relatively high opportunity costs, but this “punishment” could turn some subtle cheaters into cooperators. Theory predicts, however, that a partner's low investment will be punished by matching the partner's choice (Killingback & Doebeli, 2002, Roberts & Sherratt, 1998, Sherratt & Roberts, 2002). Playing TFT signals that exploitation will be futile and, at the same time, reduces opportunity costs for the actor to a minimum. We therefore expect human participants to pursue a TFT-like strategy when confronted with a cheater, but an RTS strategy when confronted with a reciprocator.

The robustness of RTS is determined not only by the strategy pursued by the opponent, but also by situational and contextual variables. Errors are likely to occur in reality and a single mistake can be calamitous for reciprocal altruism (e.g., Van Lange, Ouwerkerk & Tazelaar, 2002). The escalation of retaliation among TFT players after an occasional mistake gradually breaks down cooperation. A good way to cope with noise is allowing some percentage of the other player's defections to go unpunished (Nowak & Sigmund, 1992, Nowak & Sigmund, 1993, Wu & Axelrod, 1995). Generosity might prevent a single error from eliciting a sequence of defections. Barrett (1999) and Cosmides and Tooby (in press) predict that people would be tolerant towards a nonintentional act of defection and make a distinction between an inadvertent mistake and intentional cheating. Interaction with a cheater strategy in a situation in which the perceived likelihood of mistakes is higher could therefore result in higher levels of cooperation, compared with a game in which noise is absent. Evolutionary simulations (Sherratt & Roberts, 1999), however, have demonstrated that RTS-like strategies tend to become less generous as the probability of mistakes increases. To disentangle the two different hypotheses, we explore the effect of noise on RTS strategies adopted by human participants. Based on modeling studies (Sherratt & Roberts, 1999), we predict that human participants will be less willing to raise stakes in noisy situations (both against subtle cheaters and noncheaters). Based on the cheater detection literature, we could expect human participants to display more generosity towards subtle cheaters as the perceived likelihood of inadvertent mistakes rises.

2. Material and methods

3. Results

A 2 (Strategy)×2 (Noise) within-subjects analysis of variance revealed a main effect for strategy, F(1,168)=180.79, P<.0001. The positive correlation (M=0.57, Fisher transformed Pearson's correlation coefficient) between round number and level of cooperation among participants interacting with the TFT strategy indicates that participants increased cooperation against a strategy that matched the player's investment. The interaction with a TFT-1 strategy yielded negative correlations (M=−0.71) between round number and level of cooperation, which indicates that participants decreased cooperation over the course of an interaction with a strategy that undercut the player's level of investment. The analysis also revealed a significant main effect of noise, F(1,168)=4.78, P<.05. When in a noisy situation, participants were less likely to raise stakes (M=−0.17) than when noise was absent (M=0.03). Participants became less generous as the perceived probability of mistakes increased. The strategy-by-noise interaction did not yield a significant effect, F(1,168)=0.16, ns. Fig. 1 shows the median cooperation level across the 10 trials for the four conditions. Fig. 2 shows the average correlations between round number and cooperation in the four conditions.

Fig. 1. Amounts donated per round in the four conditions: (A) TFT and no noise; (B) TFT and noise; (C) TFT-1 and no noise; (D) TFT-1 and noise. Data are plotted as medians across 57 participants, with 25th and 75th percentiles.

Fig. 2. RTS coefficients for all conditions. Bars represent the mean normalized correlation between round number and amount donated; error bars are standard errors.

Additionally, we compared the total amount of coins gathered throughout the experiment by the participant versus the preprogrammed computer strategy. Playing against TFT, participants (M=162.6) earned significantly less than the opponent did (M=165.1), t(113)=−2.76, P<.01. In contrast, playing against TFT-1, participants (M=131.9) did not earn less than the opponent did (M=131.1), t(113)=0.88, ns.

4. Discussion

In line with the findings of Roberts and Renwick (2003), we found that cooperation increased throughout the interaction when participants' investments were matched. Participants raised the stakes when they played against a preprogrammed computer strategy that gave exactly the same number of coins that the participant gave in the previous interaction round (i.e., TFT). However, if participants employ RTS against a matching strategy, they receive a small sucker's payoff for each rise in cooperation and will gain less than the preprogrammed strategy. RTS is thus “willing to be exploited” for eventual greater gains.

While previous research has shown that RTS is capable of taking advantage of cooperative opportunities, RTS's reaction to defectors is understudied. This question is relevant because humans seem to adopt a faithful RTS strategy: They start at an intermediate level, which makes them vulnerable to exploitation. Populations are likely to contain a class of individuals that temporarily or permanently lack the time, energy, resources, ability, or willingness to cooperate (Sherratt & Roberts, 2001). Theory predicts that low investments of the partner are punished by matching the partner's level of cooperation (Sherratt & Roberts, 2002). Such a transformation of RTS into a TFT-like strategy when human participants interact with a “subtle cheater” has, to our knowledge, not been studied yet. We found that cooperation decreased gradually throughout the interaction whenever the partner undercut the cooperation of the participant. Rather than investing zero or maintaining an intermediate level of cooperation when confronted with a cheater, participants adjusted their level of cooperation gradually downwards. This shift in strategy suggests that participants discriminate cheaters and reciprocators, as proposed by Cosmides and Tooby (1992) and Trivers (1971). Indeed, on average, human participants are not exploited by a subtle cheater, whereas they increase cooperation against a cooperator. RTS is a robust strategy.

In our experiment, we simulated network problems with the computers on which the experimental task was completed. Modeling studies (Sherratt & Roberts, 1999) suggest that individuals become less generous as the probability of mistakes increases. Cheater detection theory, in contrast, suggests that noise would reduce the negative effects of subtle cheating strategies. In this way, humans would suffer lower opportunity costs. Our data are in line with the first prediction. As participants perceived a greater likelihood of mistakes, they were less willing to raise stakes against a reciprocator and less willing to give a cheater a new chance. In line with Sherratt and Roberts (1999), we interpret the noncooperative behavior in noisy situations as participants being more cautious when the perceived probability of errors rises. It seems as if the potential exploitation costs outweigh the opportunity costs in noisy situations.

In contrast with the initial stimulations Roberts and Sherratt (1998), but consistent with Robert and Renwick (2003) and Van Lange et al. (2002), the initial move in the PDG was substantially higher than the minimal amount needed to “test-the-water”. RTS-like strategies forego and delay potential gains by investing only a small amount on the first move, whereas TFT-like strategies are vulnerable to exploitation by starting out with a cooperative move. The potential cost of an initial cooperative move is underestimated in a standard PDG but could be quite substantial in a nondiscrete PDG. A strategy cancelling out the opportunity cost carried by RTS strategies and minimizing the risk of exploitation carried by TFT strategies should start out with a moderately cooperative move. Initial moves of human participants could be the result of a combination of concerns about losing gains (i.e., opportunity costs) and exploitation, and this could explain the moderately high levels of cooperation on the first round. A strategy that is informed about the probabilities of meeting a cheater versus a reciprocator and adapt its initial move to these probabilities might be more successful than low starting RTS or high starting TFT strategies. Recent experimental evidence has shown that people do adjust cooperation according to their expectations (Smeesters, Warlop, Van Avermaet, Corneillie, & Yzerbyt, 2003).

In the present study, the opponent was programmed to play TFT. But as people adopt an RTS strategy, the question arises to what extent human participants double their stakes when they interact with RTS, creating an even more rapid upward spiral of increasing cooperation. Some cooperators, however, may be less generous than others (Sherratt & Roberts, 1998), and participants might differ in their willingness to raise stakes: Personality variables, such as social value orientation (e.g., Van Lange, 1999), could explain interindividual differences in the willingness to raise the stakes when the opponent matches. Additionally, future research could investigate whether other situational factors might affect the adoption of RTS. Participants knowing that their choices in a PDG will be made public at the end of the game might act more generously than participants whose choices are made anonymously. Participants may perceive that being cooperative would enhance one's reputation (e.g., Barclay, 2004, Nowak & Sigmund, 1998). The possibility of indirect reciprocity might boost RTS. Finally, future research could explore why situational noise does not buffer the negative effects, as cheater detection theory suggests. Possibly, noise buffers a decrease in cooperation only when it is even more subtle. Occasional undercutting of the opponent's contribution rather than continuous undercutting might provide a fairer test of the predictions derived from the cheater detection theory.