Agencies should develop policies and processes to minimize the negative effects of human factors that can undermine the accuracy of forensic evidence.
Comment:
a. Human factors. The term “human factors” refer generally to the application of psychological and physiological principles to workplace processes and systems, as discussed and defined in § 8.05, in the context of evidence collection generally. In the forensic evidence context in particular, cognitive biases, i.e., systematic errors in thinking that affect decisions and judgments, can negatively affect the work of scientific and technical experts. It is particularly important to ward off bias in areas in which decisionmaking criteria are not clear, and in which outside information can influence the decision. In the area of forensic science, cognitive biases can result in the failure to test evidence, and they can affect the conclusions reached during forensic analyses. For example, information about an investigator’s theory of the case can bias forensic analysts towards finding results consistent with that information. It is crucially important that agencies attend to the sources of bias and seek to minimize the impact of bias on casework.
b. Defining task-irrelevant and task-relevant information. Standard procedures should define what information a forensic expert should or may make use of when conducting analysis, and what information is not informative for such analysis. Forensic experts should not receive unnecessary information about a case. They generally do not need to know the suspect’s criminal history or what other evidence investigators have gathered in order to conduct a forensic examination. Conversely, they may need to know the type of surface from which evidence was collected or who may have come into contact with the evidence.
c. Sequential unmasking. Some information that is needed for one part of an analysis is not relevant for other stages of the analysis. To prevent information from biasing their work, agencies should give examiners only the information that they need and only at the time that they need it. Agencies should ensure that examiners review information sequentially so that they do not receive information that may affect their judgments, such as information about a suspect’s criminal record. Other procedures that order an examiner’s work can help reduce cognitive biases. For example, examiners can adopt a process by which they examine crime-scene evidence before they examine evidence from potential suspects. Such processes can also can avoid the danger of circular reasoning, in which persons may naturally tend to focus on similarities at the expense of differences when comparing evidence side by side.
d. Blinding. When feasible, a forensic expert, in verifying a conclusion reached by another forensic expert, should not know what conclusion the other expert reached. By requiring the use of a blinding process, agencies can ensure that an expert’s verification is independent. In addition, by randomly assigning cases to analysts, agencies can ensure that analysts do not seek out cases in a way that can bias outcomes.
Reporters’ Notes
It is now well understood that all forensic techniques can involve some degree of judgment and interpretation and therefore are vulnerable to cognitive bias. Exec. Office of the President, President’s Council of Advisors on Sci. and Tech. (PCAST), Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods 8 (2016) (“PCAST Report”) (“The findings of forensic science experts are vulnerable to cognitive and contextual bias.”). See, e.g., I.E. Dror, D. Charlton & A.E. Péron, Contextual Information Renders Experts Vulnerable to Making Erroneous Identifications, 156 Forensic Sci. Int’l74, 77 (2006) (“Our study shows that it is possible to alter identification decisions on the same fingerprint, solely by presenting it in a different context.”). PCAST noted that because so little had been done by the forensic-science community to study the bias issue, the full magnitude of the problem could not be known. PCAST Report, supra, at 9.
Studies have identified how common procedures in forensics can and do bias results. See, e.g.: Itiel E. Dror & Greg Hampikian, Subjectivity and Bias in Forensic DNA Mixture Interpretation, 51 Sci. & Just. 204, 204-208 (2011); Larry S. Miller, Procedural Bias in Forensic Examinations of Human Hair, 11 Law & Human Behav. 157, 158-159 (1987); Paul Bieber, Fire Investigation and Cognitive Bias, Wiley Encyclopedia of Forensic Science, 2014, onlinelibrary.wiley.com/doi/10.1002/9780470061589.fsa1119/abstract. Contextual bias can occur when experts receive information about the facts of a case that change the way they approach their work.
In a well-known example, several FBI analysts mistakenly attributed a fingerprint from a subway bombing in Madrid, Spain, to a lawyer in Portland, Oregon. The FBI’s subsequent inquiry concluded that circular reasoning and bias led to this error. PCAST Report, supra, at 46; U.S. Dep’t of Justice, Office of the Inspector Gen., A Review of the FBI’s Handling of the Brandon Mayfield Case (2006). This sort of problem occurs because of what is called “confirmation bias.” Human observers tend to focus on similarities rather than differences, leading them to confirm a hypothesis. For example, when observing a suspect and latent print side by side, observers may tend to focus on features that appear to match, and to discount differences.
Cognitive psychology has much to offer by way of methods that can be used to minimize the inevitable effects of biasing information on human observers. For example, one type of sequential approach is to examine the latent print from the crime scene first, before turning to a suspect. The PCAST Report recommends that approach: “Examiners should be required to complete and document their analysis of a latent fingerprint before looking at any known fingerprint and should separately document any additional data used during their comparison and evaluation.” PCAST Report, supra, at 10; see also Dan E. Krane et al., Sequential Unmasking: A Means of Minimizing Observer Effects in Forensic DNA Interpretation, 53 J. of Forensic Scis. 1006-1007 (2008).
Poor procedures also can make crime-scene analysts “vulnerable to cognitive and contextual bias.” Nat’l Research Council, Comm’n on Identifying the Needs of the Forensic Scis. Cmty., Strengthening Forensic Science in the United States: A Path Forward 4 (2009). When evaluating whether a latent print at a crime scene came from a particular suspect, for example, it would be inappropriate for the fingerprint examiner to be influenced by whether the suspect made incriminating statements or had a convincing alibi, or whether other forensic evidence implicated the suspect. Those are matters to be considered by police, prosecutors, and jurors. That kind of information is irrelevant to a scientific assessment of the latent print, and thus should not be allowed to influence the examiner’s assessment. Nat’l Comm’n on Forensic Sci., Ensuring that Forensic Analysis is Based Upon Task-Relevant Information 2 (2015), https://www.justice.gov/archives/ncfs/page/file/641676/download. Information is task-irrelevant if it is not necessary for drawing conclusions about the propositions in question, or if it assists only in drawing conclusions from something other than the physical evidence designated for testing or assists only in drawing conclusions by some means other than an appropriate analytic method. Id.
The decision to collect crime-scene evidence also often is influenced by the officer’s or technician’s personal perceptions as to the seriousness of the crime. See Joseph L. Peterson et al., Forensic Evidence and the Police: The Effects of Scientific Evidence on Criminal Investigations 97-98 (1984) (concluding from statistics on cities’ evidence collection that police investigators “will usually go to greater lengths collecting information to attempt to solve personal crimes than they will for property crimes”). Manuals typically offer guidance on how evidence should be collected, but not on which types should be collected. Evidence intake forms and initial communications with investigators also can result in forensics experts receiving unnecessary information that can skew their findings., Saul M. Kassin, Itiel E. Dror & Jeff Kukucka, The Forensic Confirmation Bias: Problems, Perspectives, and Proposed Solutions, 2 J. Applied Research in Memory & Cognition 42, 43 (2013). Managing the flow of information in a crime laboratory can help to address those problems. Evidence intake forms can be designed to ensure that only relevant information is provided. Communications between experts and investigators can be minimized or prohibited, as is done in some laboratories. Itiel E. Dror et al., Context Management Toolbox: A Linear Sequential Unmasking (LSU) Approach for Minimizing Cognitive Bias in Forensic Decision Making, 60 J. Forensic Sci. 1111 (2015). Specialization within an agency can help to minimize bias by tasking different people with receiving evidence, communicating with law enforcement, and deciding what tests should be performed—and then handing that evidence off to a person who performs the discrete testing task. In so doing, the expert who tests the evidence is “blind” to the other information about the pending criminal investigation. Itiel E. Dror, A Hierarchy of Expert Performance, 5 J. of Applied Research in Memory and Cognition 121-127 (2016).