Context effect and confirmation bias in criminal fact finding

Purpose. Fact finding is an important part of the job of criminal trial judges and juries. In the literature, several potential pitfalls hindering fact finding have been identified, such as context effects (i.e. an unintended effect of non-probative information on conviction) and confirmation bias (i.e. a skewed selection of and overreliance on guilt-confirming evidence and neglect of exonerating information). In the present study, the effect of irrelevant contextual information on conviction and subsequent confirmation bias was tested.

information that is in fact irrelevant. For example, the explicitness of the evidence (e.g. black-and-white vs. full colour crime scene photographs) may affect conviction, even though it should have no effect (Douglas, Lyon, & Ogloff, 1997). As another example, knowledge of the suspect's psychopathic personality may increase the conviction that he/ she is guilty of the crime, even though psychopathy should have no influence on conviction, but is merely relevant for the question of criminal responsibility and for the determination of the appropriate sanction (Aspinwall, Brown, & Tabery, 2012;Rassin, 2017). Conviction has even been found to be affected by the order in which the evidence is presented, in that a primacy and/or recency effect can occur (Englich, Mussweiler, & Strack, 2005;Kerstholt & Jackson, 1998). In addition, the interpretation of individual pieces of evidence might be affected by knowledge of other pieces of evidence (cf. an assimilation effect; Rassin, 2017).
Another pitfall that has been argued to hinder criminal fact finding is confirmation bias or tunnel vision (e.g. Findley & Scott, 2006). For starters, confirmation bias may lead investigators to select information that is incriminating for the primary suspect, while leaving out exonerating information (e.g. Kassin, Dror, & Kukucka, 2013;Kassin, Goldstein, & Savitsky, 2003;Schmittat & Englich, 2016). Ultimately, this biased selection may lead to a skewed view on the probability that the suspect is guilty. Similarly, confirmation bias dictates that incriminating evidence will be valued more than exonerating evidence (e.g. Ask & Granhag, 2005;Nickerson, 1998;Rassin, 2013). Notably, it is also possible to have confirmation bias after experiencing an initial impression that the suspect is innocent. In this instance, the decision maker will be prone to selectively seek exonerating evidence.
The context effect and confirmation bias discussed above are generally likely to be disadvantageous to the suspect, in that they tend to increase (rather than decrease) perceived strength of the evidence and thus conviction. Hence, they may fuel miscarriages of justice (see Saks & Koehler, 2005). The purpose of the present study was to test to what extent context effects and confirmation bias occur in the decisionmaking of professional criminal trial judges. While many studies on bias in legal decisionmaking relied on analogue samples, a handful of studies included professional judges and yielded findings suggesting that professionals, like laypeople, are indeed susceptible to various pitfalls. For example, Englich et al. (2005) demonstrated that professional judges are sensitive to the order in which the evidence is presented, in that if they are presented with incriminating evidence first, conviction rates increase, compared to when exonerating evidence is presented first. As another example, de Keijser and van Koppen (2007) found that professional judges can display a conviction paradox. That is, the judges needed less evidence to convict the suspect if the crime at hand was more severe. Hence, it can be concluded that professional judges are susceptible to some psychological pitfalls. In the present study, an elaborate version of a case file was employed that has previously been used to demonstrate confirmation bias in undergraduates (Rassin, Eerland & Kuijpers, 2010). It was hypothesized that professional judges, like lays, are indeed susceptible to context effects and confirmation bias.

Participants
One hundred and five Dutch professional criminal trial judges (63 women, 60%) participated in this study. The mean age in the sample was 40.90 years (SD = 9.16). The judges took part in a psychology conference that they were obliged to attend as a part of their continual professional education. At the start of this conference, they completed several assignments of which the current data were a part. Participants (N = 111) were asked permission to use the learning materials for scientific study. Those who agreed to participate (N = 105) handed in their assignments. The data were collected in the year 2018. Participants took part in small groups of approximately 15-20 individuals. The sample was divided into two groups, based on the precise content of the assignment. Participants were randomly assigned to one of the conditions. These two groups did not differ with regard to age, t(103) = 0.66, p = .514; Cohen's d = 0.13; BF 10 = 0.25, or gender, v 2 (1) = 1.43, p = .231; Likelihood ratio (LR) = 1.44; BF 10 = 0.49.

Measures and procedures
Participants were given a case file, consisting of 10 pages, loosely based on de Keijser and van Koppen (2007). The case was about a young man, Anton de Koning, who, one evening, walked in the street with his girlfriend Corine de Jong. They encounter three boys: Joesef Abdullah, Sjon Tegelaar, and Bas van Vliet, the man who will become suspect. The latter made a remark about Corine, and Anton swiftly gave a witty reply. They all went their own way. Soon afterwards, the three boys parted from each other. Soon after that, Anton is attacked and physically abused. The police think that the suspect (aged 27 years), after having said goodbye to Joesef and Sjon, has followed Anton and Corine, and attacked Anton from behind, out of revenge for the witty remark that made van Vliet look stupid. The case file consisted of short reports, each fitting on one page, made as realistic as possible given the limited amount of space. The following information was included: 1. A witness statement by Corine de Jong in which she testifies roughly about what had happened, 2. A report about a simultaneous photo-line-up in which Corine de Jong identifies van Vliet, 3. A witness statement by Anton de Koning, who hardly remembers anything, but does recognize van Vliet from his photograph, 4. A statement by Joesef Abdullah, who testifies that van Vliet acted aggressively on the pertinent evening and may well have followed the victim and his girlfriend after splitting from him and Sjon Tegelaar, 5. A similar statement by Sjon Tegelaar, 6. A report of the interrogation during which van Vliet confesses, 7. A medical report about de Koning's injuries, 8. A report about a simultaneous photo-line-up undergone by an eye witness, Alastair Offermans, who declares that he does not recognize any of the shown photographs as the perpetrator he saw, 9. A psychiatric report about possible mental illness and criminal responsibility on the side of the suspect, 10. The prior criminal record of the suspect.
Unknown to participants, there were two versions of the case file. In the 'suspectfriendly' condition (n = 50), the medical report stated that the victim suffered from a mild, temporary retrograde amnesia, the psychiatric report concluded that the suspect did not suffer from any psychiatric complaints, the suspect had only one prior crime in his record (i.e., damage to property at 11 years), the exonerating evidence was presented early in the file, and the photograph of the suspect, included in the prior record, was a computergenerated face adopted from Todorov, Pakrashi, and Oosterhof (2009), as a face that tends to elicit feelings of trustworthiness (see also Wilson & Rule, 2015).
In the 'suspect-unfriendly' condition (n = 55), the victim was said to suffer from permanent aphasia and amnesia, the suspect was concluded to suffer from psychopathy, and to be in need of prolonged therapy, there were four prior crimes in his record (i.e. the same property damage as in the suspect-friendly version, plus three physical abuses of which the most recent took place 3 years earlier), the exonerating evidence was presented later on in the file, and his photograph was an untrustworthy version adopted from Todorov et al. (2009). Note that all of these differences are, legally speaking, not relevant to the question of whether or not the suspect is guilty. The information is not evidence. However, due to the context effect discussed in the introduction, they may affect conviction nonetheless. Arguably, the length of the prior record is relevant to the determination of guilt, in that past crime predicts future crimes. However, for professional judges, the prior record should not be taken as evidence, despite its scientific predictive value.
After reading the case file, participants were asked whether or not the suspect should be convicted for abusing the victim, by circling no or yes.
Next, participants were instructed as follows. 'Imagine that you are the judge who will ultimately determine the suspect's guilt. It is possible for you to initiate additional investigations. Obviously, it is important to work thoroughly, but at the same time, the possibilities to keep on investigating are limited. Which of the following lines of investigation would you like to have carried out? You may circle as many investigations as you wish. Try to be realistic'. Participants were presented with a list of 18 lines of investigation (see Table 1). Half of these (i.e., the even numbered ones in Table 1) were shown in previous research to be considered guilt-confirming endeavours . The other half (the odd-numbered ones) were exonerating endeavours. Confirmation bias will manifest as a preference for incriminating investigations, even, or particularly, if the participant has already decided that the suspect should be convicted (cf. Snyder & Swann, 1978).

Results
The data were analysed with JASP (free Bayesian software available at www.jasp-stats. org). JASP allows for both inferential null hypothesis significant testing and Bayesian analysis. Both are reported below. Crucially, the latter analysis yields a Bayes factor which represents the LR for the fit of the data in the null and in the alternative hypothesis. BF 10 s smaller than 1 indicate that the data fit better in the null hypothesis than in the alternative hypothesis. BF 10 s larger than 1 suggest that the alternative hypothesis predicts the data better. BF 10 s larger than 3 can be interpreted as positive/substantial support for the alternative hypothesis. BF 10 s larger than 10 represent positive/strong support, and BF 10 s larger than 20 provide strong support for the alternative hypothesis (Jarosz & Wiley, 2014). In the current analyses, the prior odds were left undefined and thus set at 1.0.
In Table 1, the percentages of participants who selected the pertinent investigations are presented. On average, participants in both conditions chose a similar number of 14.
The police could be ordered to employ local media (e.g. billboards, newspaper, radio, and TV) to reach potential witnesses So far, data suggest that severe context information, as compared to mild context information, inflates conviction and confirmation. However, strictly speaking, it is only confirmation bias if confirmation attempts follow one's conclusion that the suspect is guilty. Therefore, additional analyses were run in which the independent variable was based on conviction. The 55 participants who had chosen to acquit the suspect selected on average 4.09 (SD = 1.89) investigations compared to 3.10 (SD = 1.56) investigations selected by the 50 convicting participants: t(103) = 2.92, p < .004; Cohen's d = 0.57; BF 10 = 8.46. Remarkably, the acquitting participants selected more exonerating investigations (1.75; SD = 1.42) than did the convicting participants (0.76; SD = 0.82): t (103) = 4.30, p < .001; Cohen's d = 0.88; BF 10 = 512.51. By contrast, the number of selected incriminating investigations was similar in both groups, that is, 2.35 (SD = 1.29) in the acquitters and 2.34 (SD = 1.27) in the group of convicters: t(103) = 0.02, p = .983, Cohen's d = 0.008; BF 10 = 0.21.

Discussion
The present study sought to explore the influence of irrelevant context information on legal decision-making, particularly conviction in criminal proceedings. By and large, the findings support the hypothesis. In the sample of professional judges, irrelevant contextual information (e.g., severity of the victim's injury and even facial appearance) affected conviction rate, and the relative preference for additional incriminating investigation endeavours (cf. confirmation bias). Thus, the data suggest that professional

15.
It could be investigated whether the interrogation of the suspect was conducted properly (e.g. was he informed about his rights and was there no illegitimate social pressure?) 56%* 33% 16. The suspect could be evaluated psychologically in order to estimate the recidivism risk 0% 6% 17. It could be investigated medically whether the physical damage suffered by Anton could also have been caused by some other agent than a beating by the suspect 4% 4% 18. The home of the suspect has not been searched. A warrant to conduct such a search could be produced 0%* 9% Note. *p < .029; BF 10 > 4.0.
judges are susceptible to bias. The danger of bias in judicial contexts has been studied before, but its occurrence in professional judges has been studied rarely (cf. Englich et al., 2005;de Keijser & van Koppen, 2007). A few limitations of the present study deserve attention. First, there were multiple pieces of irrelevant information in the stimulus materials (e.g., facial appearance of the suspect and criminal [ir]responsibility). This makes it impossible to determine which of these were crucial in affecting conviction. Previous studies have demonstrated that some of the non-probative information can indeed affect conviction. For example, knowledge of the suspect's psychopathic traits may increase conviction (Rassin, 2017). Likewise, the severity of the injuries suffered by the victim may increase willingness to convict (de Keijser & van Koppen, 2007). Facial appearance has been argued to affect the sentence rendered by professional judges (Wilson & Rule, 2015). Particularly, the effect of information in the prior record on conviction is interesting, because it has scientific predictive power, albeit not legal relevance. This is obviously a potential topic for future research. In addition, it also remains unclear how the non-probative information affected conviction. For example, judges in the suspect-unfriendly condition may have become more prejudiced against the suspect. Alternatively, the information may have elicited negative emotions that subsequently caused a shift in conviction (Feigenson & Park, 2006).
Second, the contextual information in the case file turned out to affect conviction and confirmation proneness. Notably, the participants selected interesting additional investigations, but they were not informed about the outcome of the investigations they selected. Thus, strictly speaking, the present data tell us that irrelevant context information can affect conviction and confirmation proneness, but they tell us nothing about the effect of confirmation bias on conviction. Likewise, the data are silent about the reasons for the confirmation bias. While confirmation bias can be construed as a robust phenomenon, simply naturally present in decision-making (see Nickerson, 1998), it is interesting to speculate about potential causes and moderators of confirmation bias. For example, a preference to confirm may be rooted in the desire to reduce cognitive dissonance (see Festinger, 1957). That is, by selectively exposing ourselves to information that confirms our initial decision, we prevent being confronted with disconfirming evidence increasing uncertainty. Alternatively, confirmation bias may simply be a result of our wish to be consistent in social interactions (see Cialdini, 2009). Yet another possibility is that confirmation bias is actually nothing more than positive testing, that is, the tendency to solve problems by searching for evidence in the direction implied in the problem (Klayman & Ha, 1987). For example, when finding out whether the suspect is guilty, one will automatically look for incriminating evidence. By contract, when finding out whether the suspect is innocent, one will prefer looking for exonerating information. As a final example, confirmation bias has been argued to be an artefact of primary error detecting and minimization (Friedrich, 1993). That is, judges who believe that the worst possible mistake is to acquit a dangerous criminal will be more prone to select incriminating evidence than judges who believe that the worst possible mistake is to convict an innocent suspect. In this line of reasoning, confirmation bias is actually not a bias, but a conscious strategy.
A final limitation is that two conditions were included in the study: a suspect-friendly and a suspect-unfriendly condition. There was no unbiased control condition. Given that the case file was fictitious, although the reports were realistic and resembled real ones as encountered by judges, it cannot be concluded whether convicting the suspect was ultimately biased or not. Nonetheless, the data at least suggest that professional judges are susceptible to context information and confirmation bias.
It has been argued that protection against bias can be found in 'alternative scenario thinking' (e.g. Rassin, 2010). In this line of reasoning, it must be appreciated that the strength of the evidence is not solely determined by its support for the suspect's guilt, but simultaneously by its fit in an alternative scenario in which the suspect is innocent. Notably, so far it seems that this approach has only limited effect (see Ask & Granhag, 2005;O'Brien, 2009). Rassin (2018b) argued that alternative scenario thinking may become more effective if the decision maker literally writes the two competing scenarios (i.e. the primary and the alternative scenario) on a piece of paper and subsequently writes down how each piece of evidence fits in both scenarios. An even more stringent way of considering alternative scenarios is to adopt a Bayesian approach to evidence. In this approach, each piece of evidence is attributed a LR, which is the quotient of the likelihood of finding the evidence under the primary scenario, divided by the likelihood of finding the evidence under the alternative hypothesis (see Royall, 1997). While theoretically strong, this approach has met some resistance, because, if for no other reasons, it requires actual calculations. 'Computing' the strength of evidence is unusual in legal settings (except for forensic technical evidence, perhaps) and hence difficult to understand (see Fenton, Neil, & Lagnado, 2013;de Keijser & Elffers, 2012). Finally, some authors recently proposed that a good remedy might be to simply automate disconfirming problem solving, for example by means of repeated practice or even by playing tailor-made video games in which disconfirmation and otherwise bias-free decisions lead to victory (see Shaw et al., 2018).
In sum, the present study provided evidence for the susceptibility to context effects, in a sample of professional criminal trial judges. Irrelevant contextual information not only affected conviction, but also subsequent confirmation bias. Hence, the findings suggest that professional judges need protection against bias. Finding out where such protection can best be found is an important mission for future study.