1. Introduction

It is widely believed that the human face reveals information about the self—that even without prior acquaintance, one can deduce personality traits, moral virtues, or social characteristics from the face of another (Hassin & Trope, 2000, Liggett, 1974). In Europe, from the time of ancient Greece, cultivated men practiced physiognomy, the art of reading traits from faces. In the second half of the 18th century, Lavater's physiognomy came into fashion and played a significant role among intellectual circles of the time (Zebrowitz, 1997). Despite continual criticism that physiognomy is analogous with charlatanism and despite the political dangers that its stereotypes tend to foster, the belief and practice of physiognomy continue to the present day.

Experimental studies in social cognition suggest that people can, in fact, infer personality traits from another's face. Studies indicate that face-based judgments are often more accurate than skeptics would like to think. Bond, Berry, and Omar (1994) found a reliable relation between face-based impressions of honesty and the willingness to deceive others, and Borkenau, Mauer, Riemann, Spinath, and Angleitner (2004) found that inferences of intelligence and personality from thin slices of behavior strongly predicted intelligence and personality test scores. These studies in social psychology document the so-called kernel-of-truth hypothesis, which maintains that there is some validity to social judgments based upon facial appearances (Bond et al., 1994). More precisely, this hypothesis suggests that people are able to distinguish personality traits from faces with a greater-than-chance accuracy rate. This rate is likely to increase if the target information contains more dynamic features (a 90-s videotape), if the target information is event related (telling a lie), or if people actually meet one another (a 30-min interaction). In settings of a less contrived nature, where more relevant information is available, social impressions concerning an individual's personality are likely to converge.

Face-based impressions do prove more complicated, however. Again and again, lie detection research has revealed our poor capacity to identify liars (DePaulo & Rosenthal, 1979, Ekman et al., 1999). Even in real-life settings, unskilled perceivers fail to pick up relevant signals that are likely to display untruthfulness. Considerable evidence demonstrates that our social cognition system is not fine-tuned enough to receive and process deception cues that make it easier to identify a lie. Moreover, while evolutionary psychologists attach great significance to the human capacity of cheating detection (Cosmides, 1989, Cosmides & Tooby, 1992), individuals fail to discriminate defectors from cooperators when confronted with neutral-expression pictures taken from subjects who played a prisoner's dilemma game (PDG). Perceivers fail to identify the type of photographed targets with an accuracy any better than that of chance (Yamagishi et al., unpublished data). Until now, no study in the field of cheater detection research has surpassed the greater-than-chance accuracy threshold via the use of still photographs. The successful detection of cheaters apparently requires a lot more information. Brown, Palameta, and Moore (2003) found a significant altruist detection effect in response to videotaped storytelling. However, this study did not examine whether information of a more limited nature (nonverbal images or stills, for instance) yields the same detection effect. In contrast, Frank, Gilovich, and Regan (1993) found that participants needed a 30-min “get-acquainted” meeting before they were able to predict individual behavior in a one-shot PDG with any degree of accuracy. Although it seems reasonable “to suppose that cooperators can identify one another, at least in a statistical sense” (p. 256), prediction accuracy did not significantly exceed chance when subjects had access to information of only a limited nature. The same conclusion was reached with regards to cheater detection.

Several arguments, however, cast doubt on this conclusion. Evolutionary studies in psychology document that individuals are not indifferent to cooperative behavior. Cooperation for mutual benefit is a pervasive feature of social living and has been of crucial importance to the evolution of hominids (Trivers, 1971). Evolutionarily inspired reasoning assumes the existence of adaptive skills for discovering when someone has cheated and for remembering the person who has done it. Studies have abundantly demonstrated that people are proficient when it comes to these abilities. The Wason Selection Task research shows that people are particularly apt at determining when a social contract rule has been violated (Cosmides, 1989, Cosmides & Tooby, 1992). Further research indicates that people are also able to memorize the face of a cheater more accurately than that of a noncheater (Chiappe & Brown, 2004, Mealy et al., 1996, Oda, 1997). We think, however, that evolutionary models of social cooperation offer the possibility of a cheater detection skill that is predictive as well. Humans may be equipped not only to discover and remember noncooperative individuals but also to predict whether an individual will be cooperative. Since noncooperative deals could have fatal consequences for deceived parties, the possession of a limited predictive cheater detection mechanism may be of vital importance in order to pass over or break off potentially defective deals. Consequently, it is not unreasonable to assume that evolution designed the human mind to scan others for information that might signal intentions to defect (for instance, by scrutinizing the presence or the absence of emotional cues that cannot be faked).

In this study, we examine the possibility of a predictive cheater detection module that functions within certain realistic limits. Our main objective is to precisely delineate these limits. On theoretical grounds, it is unlikely that predictive cheater detection mechanisms could rely on permanent facial features revealed by neutral facial expressions (Brown & Moore, 2002, Frank et al., 1993, Trivers, 1985). If differences between cooperators and cheaters were obvious, natural selection would move cheaters towards extinction. In a transparent world, cooperators who assortatively interact only with cooperators always get the highest payoffs. Consequently, cheaters only survive when they learn to fake cooperative intentions or to mimic expressions that might distinguish cooperators from defectors (or vice versa). Since these imitated expressions become unreliable for cooperators, evolution favors the selection of more subtle expressions and/or more sophisticated detection skills. Taking this ongoing oscillation between signal detection and signal deception into consideration, we do not believe that cues of a less subtle nature (such as permanent facial features or voluntarily produced expressions) demarcate defectors from cooperators. If humans are able to identify cheaters to a certain extent, this ability must be based upon the reception of more delicate signals.

On the other hand, we believe that a cheater detection mechanism can prove more powerful than previous results have suggested. We do not believe that 30 min of verbal interaction is necessary to predict whether someone will be cooperative. In Paleolithic times, it is quite plausible that quick nonverbal impressions mattered more than lengthy chats—that it was not conversation length or verbal coding but facial expression that was the decisive factor in the detection of cheaters during hominid interactions. Although a multitude of facial expressions can be gathered during a conversation (videotaped or real), we speculate that a single event-related picture may do the same job. If the event strongly resembles one that may invite defective behavior, accuracy rates ought to increase. If someone decides to cheat, facial cues related to expressions of anxiety, guilt, or greed are likely to reveal one's intentions and future behavior. One quick impression, laid down in one photograph, should suffice to stop the transaction or to break off the deal.

We suggest that subjects ought to be able to predict the behavior of unknown individuals in response to a single event-related picture. A similar study (Yamagishi, Tanida, Mashima, Shimoma, & Kanazawa, 2003) found that humans were better able to remember the faces of cheaters than the faces of cooperators, even though no information (no labels attached to the photographs) as to whether the subjects had cheated was given when the faces were first seen (or memorized). Since Yamagishi and Tanida (unpublished data) also found that participants were unable to identify either type of player above chance when gauging neutral-expression photographs, this result came across as somewhat puzzling. In fact, there seems to be more than accurate remembrance but less than accurate identification. Unfortunately, Yamagishi and Tanida never asked their participants to discriminate between event-related pictures of cheaters and cooperators. They used these pictures in memory tasks alone, not in identification tasks. Here we explore the possibility that this omission is the key to a more precise delineation of a predictive cheater detection module. By incorporating the experimental design of Yamagishi and Tanida, we were able to demonstrate that people can indeed identify unknown cheaters when confronted with a single event-related picture.

2. Stimuli

We asked participants to rate stimulus pictures that were obtained from target subjects at three distinct moments. After an initial neutral-expression picture was taken in front of a white wall, target subjects played a computer-mediated one-shot PDG for real money (€0>€1>€3>€5). Players were kept ignorant as to whom they were playing with. Recognition and acquaintance were prevented by using separate entrances for each player and by hanging sheets between computers. From their computer screens, target subjects could read self-paced instructions that explained the PDG. It was explicitly communicated that, although defection was the most rational choice, cooperation was the most socially minded choice. All players read the following lines (originally in Dutch):

“Some people maintain that you should always choose the Big Prize (€5). This seems the most rational choice. At worst, you always get something; at best, you get the large amount. You never go home empty handed. Choosing the Modest Reward (€3) seems a little foolish. You risk gaining nothing from the experiment. Although the other player might not even have wanted to share, he gets everything while you get nothing. Other people claim that you should always choose the Modest Reward (€3). This seems the most socially minded choice. Only in this way do you avoid the other player going home empty handed. Choosing the Big Prize (€5) hardly even seems moral. The other player risks gaining nothing from this experiment. Although the other player might have wanted to share, you receive everything. These are just opinions.”

They were then asked to complete a two-question multiple-choice quiz designed to assess their comprehension of the game. If both players completed the quiz, a signal sounded, followed by a countdown indicating that decisions were to be expected within 10 s. At the end of the countdown, two buttons appeared: red (defection) and green (cooperation). Decisions were made with a click of the mouse. Our software mutually communicated the decisions. With the help of a visible webcam in front of the monitor (Logitech Quickcam 8.0), pictures were taken at the very moment of the mouse click. Target subjects then received the money they earned and were asked to complete a postexperimental questionnaire.

In order to assess the impact of event-related expressions on cheater detection, we introduced an intermediary recording moment. A practice round, without payoffs, preceded the proper round. The practice round was categorically announced and served to familiarize subjects to this rather short game. Again, webcams took pictures at the moment of decision; decisions were mutually communicated. From each player, we thus acquired three pictures: one non-event-related neutral picture and two event-related pictures (one practice-round picture and one proper-round picture). Due to the absence of real payoffs in the practice round, we predicted that the accuracy rate for correctly identifying the practice-round pictures would stand about midway between the rate of the proper-round pictures and the rate of the neutral-expression pictures.

Pictures of 112 target subjects were taken. The average age was 21.18 years (S.D.=5.09). Five players were excluded because they believed that they were recognized by their opponents. Two other players were removed because postexperimental questionnaires revealed that they did not understand the PDG. In the practice round, 67 subjects (60%) decided to cooperate and 45 subjects (40%) opted to defect. In the proper round, 58 subjects (52%) ultimately cooperated and 54 subjects (48%) decided to defect.

3. Experiment 1

4. Experiment 2

5. General discussion

Evolutionary models of social exchange leave room for a predictive cheater detection mechanism that allows individuals to predict, to a certain extent, the social propensities of another. This mechanism processes facial information during the process of making a decision, before any potential cheating might begin. It differs substantially, therefore, from cheater detection skills documented to date (i.e., discovering and remembering defectors; Chiappe & Brown, 2004, Cosmides & Tooby, 1992). Although it may seem to parallel the dubious ancient art of physiognomy, we argue that face-reading skill makes sense from an evolutionary point of view. When noncooperative deals threaten social exchange in a dramatic manner, a predictive cheating detection mechanism offers the opportunity to pass over or break off potentially costly deals. In accord with Frank (1988) and Hirshleifer (1987), one can assume that the ability to scrutinize emotion-based facial cues might contribute to solving commitment problems. Commitment problems arise when information about potential partners' future cooperative behavior is lacking. Since the likelihood that potential partners will defect during future interactions cannot be dismissed, partners are reluctant to engage in risky transactions without sufficient guarantee of another's commitment to a strategy of cooperation. A predictive cheating detection mechanism that is sufficiently sensitive to signals of noncooperative intentions may help to overcome this information deficit.

In this study, we presumed the existence of such a predictive cheater detection mechanism. We asked participants in two experiments to rate the cooperativeness of unknown target subjects who had played a one-shot PDG earlier. We took photographs of these target subjects at three different moments: a neutral-expression picture prior to the game, and event-related pictures taken during decision-making moments of both a practice round and a proper round. We found that participants in both experiments were able to discriminate noncooperative from cooperative pictures with a 66% accuracy rate in response to pictures taken during the proper round. For the two remaining categories (neutral-expression pictures and practice-round pictures), identification rates did not exceed chance. In the case of identifying cooperators in proper-round pictures, the accuracy rate only proved significant in Experiment 1.

These findings demonstrate that predictive cheating detection contains a kernel of truth. To a certain degree, people reveal their noncooperative intentions. A single picture, taken at the moment of decision, can apparently give enough visual information to expose these intentions. This conclusion contradicts that of Frank et al. (1993). Cheater detection does not require a lengthy videotaped talk. One event-related picture taken at the moment of a defective decision can produce the same results. To our knowledge, this is the first time in the field of cheater detection research that the greater-than-chance accuracy rate has been successfully obtained from (nonmoving) photographic stimuli. We did not, however, confirm the kernel-of-truth hypothesis with regard to the less expressive pictures. Our study indicates that defective behavior cannot be predicted from facial appearances in which only permanent facial features are exhibited. In this respect, evolutionary psychology does not support physiognomy. In our view, faces are not untrustworthy in themselves; yet, when in provocative social settings, they can radiate cues of noncooperativeness. This view also explains the puzzling results in Yamagishi et al. (2003). Even unknown cheaters are better remembered because observers can separate them from cooperative players, but only when the pictures are taken in provocative social settings. The more expressive the picture, the more pronounced the cheater bias will be in recall and identification. To restate, aside from accurate remembrance, there is accurate identification as well, but only in response to event-related pictures.

From a theoretical perspective, these results are in line with Brown and Moore's (2002) conceptual analysis of the evolution of reliable altruism indicators (reliable signaling theory), which presumes a continuous oscillation between signal detection and signal deception in evolutionary time. Following Frank (1988) and Trivers (1985), the generation of reliable altruism signals (which allow cooperators to avoid cheaters) and their faked counterparts (which seduce altruists to cooperate with defectors) is subject to a predator–prey arms race. If it becomes too easy to mimic altruism signals, evolution favors cooperators who give off more subtle cues of cooperativeness that are more difficult to fake. The cheater bias replicated here suggests that our attention is focused towards noncooperative people and their absent or failed faking of reliable signals. However, alternative scenarios are conceivable. Instead of producing more sophisticated honesty cues, evolution might stimulate the selection of more fine-tuned detection skills. According to this alternative model, the prey–predator arms race between detection and deception improves the quality of receivers' cheater detection skills, whereas the range of expressions emitted by senders remains unchanged. While evolution does not affect the complexity of the signaling, it does generate detection mechanisms able to pick up more subtle expressive differences between cheaters and cooperators. Despite dissimilarities, both models nonetheless assume a trend towards more sophistication. This corresponds to our main finding that predictive cheater detection requires event-related cues rather than permanent facial features.

It should be noted that there is plenty of room for future studies to complement our findings. Firstly, we are aware that defection in a one-shot PDG differs from cheating in a repeated PDG. Game theorists characterize cooperation in a one-shot PDG as a strongly altruistic behavior (hardcore altruism), whereas cooperation in a repeated PDG requires only reciprocal altruism (soft core altruism). Accordingly, we may distinguish noncooperative behavior in a one-shot PDG (soft cheating) from noncooperative behavior in a repeated PDG (hard defection). For several reasons, we preferred to limit our experiments to a one-shot PDG design. Undoubtedly, since evidence for a soft cheating detection module a fortiori substantiates the case for a hard cheating detection module, this option does not undermine our conclusions. We, nevertheless, admit that a comparison with pictures taken during a repeated PDG would be welcome. Secondly, it might be objected that in our experimental setting, in contrast to real life, the photographed players had no incentive to conceal their facial expressions. This might facilitate the detection of noncooperative players. To increase the percentage of noncooperative players, we opted for an anonymous setting in which visual communication between the players was prevented. It might be assumed that open settings, in contrast, benefit defectors who are adept at hiding their intentions during direct interactions. It should be noted, however, that the players' gestures were recorded by two visible webcams, that a third video camera stood near the participants and overlooked the whole scene, and that an experimenter observed both players. Recent evolutionary psychological research has shown that even subtle cues suggesting the presence of others (e.g., eye spots) change our social choices, arguably because reputation might be at stake (Haley & Fessler, 2005). Although anonymity was assured with the players, our setting offered sufficient “eyes” to encourage players to conceal their defective intentions from possibly influential witnesses. Nevertheless, we again welcome experiments in which more incentives to hide facial expressions are introduced.

Three further issues can be highlighted from this study. A first concerns the possible differences between cooperators and cheaters. Which precise facial features are involved in the self-betrayal of (unsuccessful) defectors? Our study did not investigate the matter but, nevertheless, offers background for future research. Brown and Moore (2002), for instance, used smiling icons to show that posed smiles are observed differently from involuntary or spontaneous smiles (e.g., asymmetrical) and negatively influence reputations in cooperative contexts. It would be interesting to investigate whether manipulations of the subject's pose, causing expressions such as smile symmetry/asymmetry, could affect identification rates. Our identification set offers the chance to investigate a picture sample more realistic than that of the Brown and Moore experiment.

A second intriguing issue concerns the difference between unsuccessful and successful defectors. Why were some cheaters discovered while others were left unexposed? This may be due, of course, to extraneous elements. Our photographs were limited to decision-making moments alone and did not frame all of the subjects' expressions. Cheaters could have displayed facial cues at different points in time and thus remained unnoticed. Presumably, if more slices of visual information were presented, accuracy rates would increase. Since we also videotaped the games, an opportunity to investigate this hypothesis exists. We did not expect, however, that participants would be able to identify all of the cheaters. As already mentioned, predictive cheating detection is only possible within certain limits. When cues of noncooperativeness would prove too obvious, the logic of natural selection predicts that cheaters would disappear.

A last issue concerns the (neuro)psychological mechanism that underlies the predictive cheating detection mechanism. Postexperimental questionnaires revealed that participants did not really know how they were able to detect cheaters. In addition, participants answered that the detection of defectors was (“very”) difficult (Experiment 1: 86%; Experiment 2: 90%), and, although they actually performed quite well (“very”), bad performance was to be expected (Experiment 1: 89%; Experiment 2: 87%). These poor estimations suggest that predictive cheating detection is an implicit automatic processing skill. In a recent study, we confirmed this presumption using a dot-probe task, in which conscious shifts of attention were obviated. Results showed that participants unconsciously pay more attention to pictures of cheaters who decide to defect during the proper round (as opposed to players who decide to cooperate) (Vanneste, Verplaetse, & Braeckman, in press). In an explicit task, social impression assessment is more accurate, participants are more confident, and they are quicker in responding to noncooperative proper-round pictures. In an implicit task, more and longer attention is allocated to these stimuli.

These findings may help us to disentangle the neural underpinnings of a predictive cheater detection module. Noncooperative proper-round pictures appear to automatically trigger neural regions that are involved in assessing the trustworthiness of a given face. Recent functional imaging studies show that certain brain regions become more active when responding to faces judged untrustworthy (Adolphs, 1999, Adolphs, 2003). Until now, however, no fMRI study has made use of event-related pictures. Winston, Strange, O'Doherty, and Dolan (2002) found increased activation in the bilateral amygdalar and right insular cortices, yet used previously selected trustworthy/untrustworthy faces that originated from individuals whose social adjustment was of no importance. Singer, Kiebel, Winston, Dolan, and Frith (2004) found that faces of (known) cheaters and defectors who played a PDG activated distinct brain circuits, but event-related pictures were not administrated in this case, either. It may thus prove productive to extend this line of neuroimaging research to different stimulus sets and to investigate, for instance, whether the neural activity located in distinct brain circuits (including the amygdalar and/or insular cortices) gradually increases in response to noncooperative event-related pictures. If so, this knowledge might shed light on neural matrices that allow us to glean social information from another's face.

In conclusion, we have established here that humans can predict noncooperativeness from facial photographs within a limited degree of accuracy. We presume that people subconsciously pick up cues of noncooperativeness—cues that do not involve permanent facial features, but rather consist of facial expressions elicited by significant social decisions. These findings are in accord with the possible existence of a distinct cheater detection mechanism that deduces someone's willingness to cooperate from event-related visual materials.