Agencies should ensure, to the best of their abilities, that the methods used to collect, analyze, present, and preserve forensic evidence are sufficiently repeatable, reproducible, and accurate. As much as is practicable—to the extent permitted by legal rules and court orders—forensic-evidence work should be conducted independent of law enforcement, and the results should be made available to the prosecution and the defense on equal terms.
Comment:
a. Definition of “forensic evidence.” Forensic evidence can refer generally to any evidence used in court, but these Principles use the phrase “forensic evidence” to refer specifically to scientific or technical evidence for use in litigation. Other evidence is referred to simply as “evidence,” and is discussed in Chapter 8. The scientific or technical practices included in the field of forensic science span a broad range of disciplines and vary widely with regard to techniques, methodologies, reliability, level of error, research, general acceptability, and published material. Some of the disciplines are laboratory-based (e.g., nuclear and mitochondrial DNA analysis, toxicology, and drug analysis). Others largely are based on expert interpretation of observed patterns (e.g., fingerprints, writing samples, and toolmarks). Some activities require the skills and analytical expertise of individuals trained as scientists (e.g., chemists or biologists). Other activities are conducted by scientists as well as by individuals trained in law enforcement (e.g., crime-scene investigators, blood-spatter analysts, and crime-reconstruction specialists), medicine (e.g., forensic pathologists), or laboratory methods (e.g., technologists). Still other practices include digital technology, such as biometric databases, and algorithms designed by outside providers. The reference to “agencies” in this Chapter includes not only police departments, but any other organizations engaged in collecting and evaluating forensic evidence, including crime laboratories.
b. Scientific standards. Basic scientific standards should apply to the use of forensic evidence, whether it is scientific or technical evidence, or a combination. In general, scientific standards seek to ensure accuracy through the use of consistent procedures that yield repeatable and reproducible results. “Repeatable” means that, with known probability, an examiner will obtain the same result when analyzing samples from the same sources. “Reproducible” means that, with a known probability, different examiners obtain the same result when analyzing the same samples. “Accurate” means that, with known probabilities, an examiner obtains correct results both: (1) for samples from the same source (true positives); and (2) for samples from different sources (true negatives). When these Principles note that evidence must be “sufficiently” accurate, repeatable, and reproducible, they do not mean to set any particular threshold levels that agencies must adhere to. Indeed, those threshold levels may differ depending on the uses to which evidence is being put.
c. Uses of forensic evidence. These Principles use the terms “repeatability,” “reproducibility,” and “accuracy,” rather than a more general concept of reliability. The term “reliability” typically is understood in the legal system as a concern about whether evidence should be admissible in court. For example, in the case of admissibility of proffered expert testimony, Rule 702(c) of the Federal Rules of Evidence requires that expert testimony be based on “reliable principles and methods.” These Principles are not primarily concerned with the admissibility of evidence in court. These Principles eschew use of the term “reliable” because they are directed primarily to agencies, which should aim for the highest standards regarding collection, analysis, and retention of evidence, regardless of how courts define “reliability.” Courts often are unfamiliar with scientific concepts and assume (despite much evidence to the contrary) that the parties can ferret out inaccuracy through the adversarial process.
Repeatability, reproducibility, and accuracy should matter at all stages of an investigation, from the initial search for evidence, to evidence collection, analysis, and subsequent preservation of the evidence. For example, poor evidence collection may prevent any analysis at all, regardless of whether a court proceeding is anticipated. Similarly, the preservation of evidence in a way that avoids deterioration may be important well after the initial adjudicatory proceeding is complete. Further, the sufficiency of evidence, in terms of repeatability, reproducibility, and accuracy, will depend on what the forensic method is being used to accomplish. A forensic method may be quite accurate as a means of excluding potential suspects or of generating potential leads, but not accurate if it is used to identify a specific individual.
d. Evidence collection. The decision to collect crime-scene evidence and the manner in which it is collected traditionally have been influenced by an officer’s or evidence technician’s perceptions as to the seriousness of the crime. Those perceptions often can be quite personal. Such decisions involve policy choices and should be governed by written policy that is informed by scientific standards and research. Further, because evidence collection increasingly requires specialized crime-scene technicians—or oversight by trained scientists familiar with the types of forensic evidence that can be collected and how to collect it properly to avoid contamination or degradation—agencies require sufficient resources for quality crime-scene investigations, and should coordinate their work with scientists who may work in separate laboratories.
e. Analysis. The analysis of forensic evidence similarly must follow scientific standards. Traditionally, many forensic techniques were not tested empirically, which is to say that their repeatability, reproducibility, and accuracy had not been confirmed through empirical testing. Instead, forensic techniques were said to be reliable based on the experience of analysts using them over time. Experts could not necessarily explain how they conducted their analyses, and often they lacked objective criteria for doing so. They could explain their conclusions only by stating that their opinions were based on their experience and training. Different experts might reach different conclusions when examining the same evidence.
A range of forensic methods currently in use still take this form, based largely on the subjective judgment of the analyst. Those methods include firearms and toolmark comparisons. In contrast, modern forensic methods can and have been validated. Some of those methods still rely on the judgment of the examiner in individual cases, but the use of the methods have been subjected to empirical testing to confirm their value. Latent fingerprint comparison is one example; although the method depends on the judgment of the examiner in individual instances, two recent studies have documented the performance of groups of examiners under realistic conditions. Other forensic analyses, like nuclear DNA testing and drug testing, largely rely on automated processes that also have been validated empirically. Empirical validation should be required for every forensic method. Otherwise, there is the risk that the supposedly valuable technique is not actually so. The research community requires resources to conduct empirical testing to assess the foundational validity of forensic techniques.
f. Preservation. Following the collection and analysis of forensic evidence, agencies must preserve that evidence in case there is a need for later analysis, for example in cold cases or closed cases that are reopened. Preservation also is important because new methods for conducting forensic analysis may be developed, or new, independent analyses may produce different results. Agencies should set out policies for the preservation of evidence, prioritizing cases in which doing so is most important. Agencies require resources to preserve evidence carefully in criminal cases.
g. Independence and oversight. Ideally, individuals involved in collecting, analyzing, presenting, and preserving forensic evidence, typically in a crime laboratory, should conduct their work to the extent feasible independent of law enforcement, and subject to scientific oversight. Independence and scientific oversight are necessary to achieve the quality standards set out in these Principles. Some degree of evidence-collection work necessarily will be conducted by law enforcement, but that work should be supervised by professionals who possess a scientific background. Some amount of communication between law-enforcement and forensic-evidence professionals is also appropriate and inevitable. However, independence can preserve the benefits of those communications while avoiding bias through directive or task-irrelevant communications. Oversight, in the context of algorithmic evidence, biometric evidence, and uses of artificial intelligence, also should require that such technologies be made accessible to independent researchers in order to assess the reliability and fairness of such evidence. Further, there should be a strong presumption that any such technologies need to be fully interpretable and transparent, meaning that the processes are comprehensible and accessible to people, including with the assistance of experts who can examine the underlying technologies. If technologies are not fully interpretable and transparent, and cannot be vetted by independent experts or researchers, or evaluated by law enforcement, lawyers, the court, or factfinders, then the use of such technologies should be strongly discouraged in criminal cases. There is not strong evidence that non-interpretable technology performs sufficiently well such that its use is justified over fully interpretable models.
h. Equal access. Relatedly, the mechanics and results of forensic tools and laboratories should be accessible equally to law enforcement, the prosecution, and the defense. Forensics is not a weapon in the quiver of the police and prosecution, to be shared grudgingly with the defense. Rather, forensics are a tool to be utilized by all participants in a search for objective truth. To be sure, local discovery rules may not provide for equal access to all sides. Further, in some circumstances, a judge may order that specific evidence not be disclosed in discovery. These Principles reflect the view that as a matter of scientific and ethical principle, forensic professionals should strongly prefer to share information equally with all sides, absent some external legal constraint upon doing so.
i. Policy and practice. Agencies should promote these objectives through written policies, training, and supervision. Many agencies do not have written policies regarding the collection, analysis, disclosure, and preservation of forensic evidence; and among the agencies that do have policies, many still do not make their policies public. Similarly, although accredited crime laboratories must meet certain standards, those standards often have not met scientific criteria for assessing performance. Examiners and officers should abide by clearly documented standards and procedures. Compliance with written policies should not be optional; a failure to comply should result in consequences for the personnel involved, as well as disclosure to defense lawyers, prosecutors, and courts. Agencies should conduct appropriate reporting and review of the use of forensic techniques.
Reporters’ Notes
1. Definition of “forensic evidence.” Although “forensic evidence” can refer to a broad category of any evidence used in court, these Principles use the term to refer specifically to scientific or technical evidence so used. See Forensic Evidence, Black’s Law Dictionary (8th ed. 2004) (defining “forensic evidence” as “[e]vidence used in court; esp., evidence arrived at by scientific or technical means, such as ballistic or medical evidence”). The field of “forensic science” is “the application of scientific or technical practices to the recognition, collection, analysis, and interpretation of evidence for criminal and civil law or regulatory issues.” Exec. Off. of the President, President’s Council of Advisors on Sci. & Tech., Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods 21 (2016) (PCAST Report). Authorities often use “forensic evidence,” “scientific evidence,” and “forensic science” interchangeably to refer to evidence derived from the application of scientific or technical knowledge. Erin Murphy, The New Forensics: Criminal Justice, False Certainty, and the Second Generation of Scientific Evidence, 95 Cal. L. Rev. 721, 797 (2007).
2. Organizations involved in forensics. There are a variety of organizations that can be involved in the collection and analysis of forensic evidence. In the case of forensics, such organizations commonly include crime laboratories, which may assist with the collection, analysis, presentation, and preservation of evidence. In some jurisdictions, that laboratory function may be fairly independent, while in others, all or part of the function may be housed within a police department. Most crime laboratories operate as divisions of police departments. See Sandra Guerra Thompson, Cops in Lab Coats: Curbing Wrongful Convictions Through Independent Forensic Laboratories 181-183 (2015). Crime laboratories may operate at the local, regional, and state levels. Further, both policing agencies and crime laboratories may rely on individual consultants and outside vendors when they conduct forensic analyses. In addition, professional organizations, accrediting bodies, forensic-science commissions, and scientific organizations play a role in forensic science.
3. The importance of forensic evidence. Forensics provide an increasingly valuable and important source of evidence in criminal investigations. The U.S. Supreme Court acknowledged that forensic and DNA evidence has “the potential to significantly improve both the criminal justice system and police investigative practices . . . to exonerate the wrongly convicted and to identify the guilty.” Dist. Atty’s Off. for Third Jud. Dist. v. Osborne, 557 U.S. 52, 55 (2009). Given that forensic evidence can have such a significant impact on the outcome of a trial and the rights of criminal defendants, it is vital that the evidence be collected, analyzed, and preserved with care. As stated in the Oregon State Police Physical Evidence Manual, “the value of properly collected physical evidence followed by examination and interpretation by the forensic laboratory cannot be over-emphasized”; accordingly, it is important that evidence be “collected, handled, and stored in a way that will ensure integrity.” Or. State Police Forensic Servs. Div., Physical Evidence Manual (8th ed. 2013), http://www.crime-scene-investigator.net/Phys_Evid_Manual_OR.pdf.
4. Repeatability, reproducibility, and accuracy. These Principles do not focus on the term “reliable” as used in the requirement in Rule 702(c) of the Federal Rules of Evidence, but rather the terms “repeatability,” “reproducibility,” and “accuracy.” That is because these Principles are directed primarily to agencies, which should aim for the highest standards regarding collection, analysis, and retention of evidence. Scientific standards should offer guidance in assessing repeatability, reproducibility, and accuracy. Agencies should base their decision to employ forensic techniques on how consistently experts achieve the same results when using a particular technique. Agencies also should provide careful documentation of the forensic analyses, so that the results can be reproduced. Finally, agencies should report information regarding the accuracy of work done. A forensic examiner can report error rates established in studies, or through routine proficiency testing of experts designed to use realistic cases to measure performance. PCAST Report, supra, at 56. Without measures of the performance of experts, agencies do not have a way of knowing how accurate their work is. Similarly, without measures of the performance of a forensic method, agencies have no way of knowing how accurate it is. In the past, agencies often have lacked such information about the repeatability, reproducibility, and accuracy of many forensic methods.
Forensic evidence has not always been used in a repeatable, reproducible, or accurate fashion. As a consequence, poor forensics have played a role in a large number of wrongful convictions of innocent people. Nat’l Rsch. Council, Comm. on Identifying the Needs of the Forensic Sci. Cmty., Strengthening Forensic Science in the United States: A Path Forward 4 (2009) (2009 NRC Report). One study of cases in which DNA evidence exonerated persons post-conviction found that invalid forensic testimony was a contributing factor in 60 percent of the cases. Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 Va. L. Rev. 1, 14 (2009); see also Melendez-Diaz v. Massachusetts, 557 U.S. 305, 319 (2009). Exonerations involving false or flawed forensic science have attracted deserved public attention. See, e.g., Radley Balko & Tucker Carrington, The Cadaver King and the Country Dentist: A True Story of Injustice in the American South (2018).
Entire methods, such as the FBI’s bullet-lead comparison technique, have been discontinued due to lack of scientific validity. Nat’l Rsch. Council, Nat’l Acad. of Scis., Forensic Analysis: Weighing Bullet Lead Analysis (2004). Other methods have been found by leading scientific bodies to be scientifically unsupported, and yet they still are in use; bite-mark analysis and firearms comparisons are examples. PCAST Report, supra. Forensic experts often have made unsupported and false claims at trial, such as that there was a zero-error rate in conducting latent fingerprint comparisons. Simon Cole, More Than Zero: Accounting for Error in Latent Fingerprint Examinations, 95 J. Crim. L. Criminology 985 (2005). Or they have used statistics to support disciplines for which no statistical research has been conducted. Brandon L. Garrett, Constitutional Regulation of Forensic Evidence, 73 Wash. & Lee L. Rev. 1147, 1183 (2016). The problem is extremely serious. A large-scale FBI review of cases involving testimony concerning microscopic hair comparison concluded that 96 percent of 2,900 cases reviewed involved flawed testimony, including 33 of 35 death-penalty cases. FBI/DOJ Microscopic Hair Comparison Analysis Review, at https://www.fbi.gov/services/laboratory/scientific-analysis/fbidoj-microscopic-hair-comparison-analysis-review. Nor are DNA testing methods immune from error or entirely free from some element of subjective interpretation. Erin Murphy, Inside the Cell: The Dark Side of Forensic DNA (2015).
5. Use of forensic evidence. Whether forensic evidence is sufficiently reliable to be used may depend on whether it is used solely to generate leads, as opposed to identifying particular suspects. Microscopic hair comparison can be used to indicate that the culprit may have had dyed-blond hair or to definitely exclude suspects who do not have dyed or blond hair. However, it would be invalid to claim, based on microscopic examination, that a hair had come from a given suspect. See Garrett & Neufeld, supra. Another important concern is that a forensic technique not be used to generate leads in a way that might prevent subsequent and more accurate testing. For example, some field DNA-testing kits consume the evidence, making later laboratory testing impossible. Heather Murphy, Coming Soon to a Police Station Near You: The DNA “Magic Box,” N.Y. Times, Jan. 21, 2019. Agencies should not conduct testing in the field, or use untrained officers, if doing so would delay or prevent subsequent testing in more accurate and controlled laboratory conditions.
6. Constitutional regulation of forensic evidence. Constitutional criminal procedure provides very little guidance concerning the appropriate use of forensic evidence. Brandon L. Garrett, Constitutional Regulation of Forensic Evidence, 73 Wash. & Lee L. Rev. 1147, 1183 (2016). The U.S. Supreme Court repeatedly has held that the police and prosecutors, together, have anobligation under the Due Process Clause to provide the defense with exculpatory and impeachment evidence. Brady v. Maryland, 373 U.S. 83 (1963). That obligation is notably underdeveloped in the context of forensic evidence. Garrett, Constitutional Regulation, supra, at 1179. The rule provides an important statement of principle—that material exculpatory and impeachment evidence should be disclosed—but it provides little guidance to forensic professionals concerning what evidence should be documented or disclosed and how to best implement a system of disclosure that works in practice.
The Sixth Amendment’s Confrontation Clause also regulates the use of forensic evidence in court, but it similarly provides an inadequate safeguard of the rights of an accused insofar as forensic evidence is concerned. The right to confront a witness at trial has limited practical benefit, as most criminal cases are plea-bargained and few criminal cases proceed to a trial. When cases are tried, cross-examination may not uncover underlying flaws in the forensics. Garrett, Constitutional Regulation, supra, at 1150. Cross-examination provides inadequate protection against flawed forensics, particularly when the defense may not have adequate documentation concerning the methods used or the limitations of those methods, or access to its own independent experts. David Alan Sklansky, Hearsay’s Last Hurrah, 2009 Sup. Ct. Rev. 1, 73-74 (2009).
Finally, the U.S. Supreme Court has held that an agency’s failure to preserve evidence in a manner that permits forensic testing is a violation of due process only if done in bad faith. Arizona v. Youngblood, 488 U.S. 51, 56-58 (1988). The defendant must prove that the evidence possessed exculpatory value apparent before it was destroyed, and that the defendant would be unable to obtain comparable evidence by other, reasonably available means. California v. Trombetta, 467 U.S. 479, 489 (1984). Such “bad faith,” “reasonably available means,” and “materiality” showings are extremely difficult to make. Indeed, the defendant in Youngblood later was exonerated by DNA testing due to an advance in technology. Brandon L. Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong 196 (2011).
Sub-constitutional regulation of evidence reliability also often has been proven inadequate. The U.S. Supreme Court’s ruling in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and subsequent revisions to Federal Rule of Evidence 702 have deepened and expanded the trial judge’s responsibilities and control over scientific evidence. Daubert clearly stated that the burden of proof lay with the proponent of expert testimony. 1 Mod. Sci. Evidence § 1:9 (2017-2018). However, neither federal nor state courts have with any consistency or clarity acted to impose standards ensuring the application of scientifically valid reasoning and reliable methodology in criminal cases. That is not surprising. The Supreme Court itself described the Daubert standard as “flexible.” That means that, beyond questions of relevance, Daubert offers appellate courts no clear substantive standard by which to review decisions by trial courts. 2009 NRC Report, supra, at 11. It is extremely uncommon for courts to reject forensic evidence in criminal cases. 1 Mod. Sci. Evidence § 1:35 (2017-2018). When courts have done so, it often has been evidence introduced by the defense. Brandon L. Garrett & M. Chris Fabricant, The Myth of the Reliability Test, 86 Fordham L. Rev. 1559, 1564 (2018).
7. Need for research and standards. The legal system is poorly equipped to address fully the challenges posed by forensic evidence. There are many reasons for that. Those include the rules governing the admissibility of forensic evidence, the applicable standards governing appellate review of trial-court decisions, the limitations of the adversarial process, and the common lack of scientific expertise among judges and lawyers who must try to comprehend and evaluate forensic evidence. Judicial review, by itself, cannot solve the foundational problems with forensics, 2009 NRC Report, supra, at 53, though it certainly could play some role. Indeed, despite the shift in legal standards for admissibility, judges continue to permit evidence at trial regarding forensic techniques that have not been demonstrated to be reliable. Garrett & Fabricant, supra, at 1568. The National Research Council of the National Academy of Sciences’ Committee on Identifying the Needs of the Forensic Science Community concluded that apart from nuclear DNA analysis, “no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.” 2009 NRC Report, supra, at 7. That Report identified substantial problems with the existing system of crime laboratories and forensic sciences and called for research and standard-setting to place a range of forensic disciplines on a sound scientific footing.
There is a real need to develop valid scientific standards regarding the collection, analysis, and preservation of forensic evidence. Standards provide the foundation against which performance, reliability, and validity can be assessed. Adherence to standards reduces bias, improves consistency, and enhances the validity and reliability of results. 2009 NRC Report, supra.
Professional organizations have established sanctions for experts who violate their ethical standards, and in doing so have provided an additional source of regulation if particular experts make serious and willful errors. However, the sanctions available to such organizations generally are limited. The most extreme sanction is the loss of a license. Often the sanction is no more than exclusion from membership in the relevant organization. 1 Mod. Sci. Evidence § 2:14 (2017-2018). “[I]t is nearly impossible to ascertain the frequency with which organizations attempt to sanction members because of their expert testimony.” Id.; see also Int’l Ass’n for Identification, International Association for Identification Code of Ethics, https://theiai.org/docs/code_of_ethics.pdf; Ass’n of Firearm & Tool Mark Exam’rs, AFTE Code of Ethics, https://afte.org/about-us/code-of-ethics; Am. Nat’l Standards Inst., Guiding Principles of Professional Responsibility for Forensic Service Providers and Forensic Personnel, https://anab.qualtraxcloud.com/ShowDocument.aspx?ID=6732.
Agencies, as consumers of forensic science, should insist through policy and practice that only scientifically valid methods are used, and that any limitations of a method be included as part of any conclusions reported. There is a notable dearth of peer-reviewed, published studies establishing the scientific bases and validity of many forensic methods. 2009 NRC Report, supra, at 8. This is a serious problem. To be sure, few crime laboratories have an in-house research function. They cannot themselves be expected to conduct basic research on forensic methods. However, agencies can support research efforts by sharing materials and data with the scientific-research community. The National Institute of Standards and Technology (NIST) has supported a wide array of research into forensic techniques, as have forensic grants and work by scientific researchers. Those efforts have resulted in improved and more objective forensic methods that agencies can adopt. Agencies also can qualify the language they use to report conclusions. They can follow national standards adopted by organizations such as the Organization of Scientific Area Committees for Forensic Science (OSAC) convened by NIST, and the recommendations in scientific reports by groups such as the National Academy of Sciences and the American Association for the Advancement of Science (AAAS).
Individual agencies or jurisdictions also can create scientific advisory boards to provide oversight and guidance to policing agencies and crime laboratories. Several states have created such bodies, as have individual crime laboratories. See, e.g., Va. Dep’t of Forensic Sci., Scientific Advisory Committee, http://www.dfs.virginia.gov/about-dfs/scientific-advisory-committee/; D.C. Code § 5–1501.11 (West 2018) (creating science advisory board). For example, the Texas Forensic Science Commission makes recommendations to state laboratories, investigates the validity of forensic techniques in use, and investigates allegations of professional negligence or misconduct by forensic professionals. See Tex. Code Crim. Proc. Ann. art. 38.01; Juan Hinojosa & Lynn Garcia, Improving Forensic Science Through State Oversight: The Texas Model, 91 Tex. L. Rev. 19, 20 (2012); Brandi Grissom, Bill, Budget Expand Authority of Forensic Science Commission, Tex. Trib. (May 25, 2013), https://www.texastribune.org/2013/05/25/reforms-expand-forensic-science-commission-authori. Such bodies can play a role in approving crime-laboratory policies and procedures, as well as in conducting audits and ensuring quality control.
Forensic laboratories should move toward written standardization in their policies, including in their performance and ethics codes. In 2014, 75 percent of crime labs had written standards for performance expectations, up from 72 percent in 2009. Matthew R. Durose et al., Bureau of Just. Stats., U.S. Dep’t of Just., Census of Publicly Funded Forensic Crime Laboratories, 2009 2 (2012).An estimated 94 percent of crime labs had a written code of ethics in 2014. Today, over 20 states have forensic lab manuals published online. The FBI and standards organizations like NIST also make available to the public model policies and manuals. Nat’l Inst. of Just., U.S. Dep’t of Just., Crime Scene Investigation: A Guide for Law Enforcement (2000), https://archives.fbi.gov/archives/about-us/lab/forensic-science-communications/fsc/april2000/twgcsi.pdf; Nat’l Forensic Sci. Tech. Ctr., Crime Scene Investigation: A Guide for Law Enforcement (2013), https://www.nist.gov/sites/default/files/documents/forensics/Crime-Scene-Investigation.pdf. Those changes are an important improvement, but many crime laboratories and other agencies involved in forensics work do not make their policies available to the public. Sandra Guerra Thompson & Nicole Cásarez, Three Transformative Ideals to Build a Better Crime Lab, 34 Ga. St. U. L. Rev. 1007 (2018). Such standards should be shared with outside entities that review the work of a lab. 2009 NRC Report, supra, at 201.
Finally, it is important that national scientific bodies and academics conduct basic research into validity and reliability of forensics methods, as individual laboratories lack the resources to do this on their own. Matthew R. Durose et al., Bureau of Just. Stats., U.S. Dep’t of Just., Census of Publicly Funded Forensic Crime Laboratories, 2014 4 (2016); Jennifer L. Mnookin et al., The Need for a Research Culture in the Forensic Sciences, 58 UCLA L. Rev. 725 (2011); Jennifer L. Mnookin, The Uncertain Future of Forensic Science, 147 Daedalus 99 (Fall 2018). There has been a notable federal effort to fund such research in recent years, including through the National Institute of Justice (NIJ) and the National Institute of Standards and Technology (NIST), which supports the Organization of Scientific Area Committees for Forensic Science (OSAC). The OSAC is developing standards for forensic disciplines. NIST supports the Center for Statistics and Applications in Forensic Evidence (CSAFE), a research consortium extending across five universities. That research can and should inform the work that agencies conduct.
8. Independence. Leading scientific organizations, such as the National Academy of Sciences, have called for the independence from law enforcement of crime laboratories and forensic-evidence-related work. The National Research Council of the National Academy of Sciences’ Committee on Identifying the Needs of the Forensic Science Community, in its 2009 report, explained:
Scientific and medical assessment conducted in forensic investigations should be independent of law enforcement efforts either to prosecute criminal suspects or even to determine whether a criminal act has indeed been committed. Administratively, this means that forensic scientists should function independently of law enforcement administrators. The best science is conducted in a scientific setting as opposed to a law enforcement setting.
2009 NRC Report, supra, at 23. The goal is not just budgetary independence, but rather accountability for the scientific standards and quality of the work. Law-enforcement communication, priorities, and involvement can affect outcomes negatively in forensic work. Those contacts can cause forensic examiners to “face pressure to sacrifice appropriate methodology for the sake of expediency.” Id. at 24. The forensic evidence function should be in service of the broader goals set out in these Principles, and should be in service of law enforcement, prosecutors, courts, and the defense on equal terms.
A central feature of independence and accountability is that all forensic methods used are subject to scientific review. The need for vetting of methods, as well as quality controls, extends to all forensic evidence. For so-called “black box” methods, in which the technology, such as an artificial-intelligence-based system, is not transparent or interpretable without specialized knowledge or sometimes even with it, the need for oversight is particularly important. In the context of algorithmic evidence and uses of artificial intelligence, it is important that such technologies be, to the maximum extent feasible, accessible to independent researchers in order to assess the reliability and fairness of such evidence. Further, there should be a strong presumption that any such technologies be fully interpretable and transparent. Judges, defense lawyers, prosecutors, and law enforcement all need to appreciate fully the strengths, limitations, and operation of such technologies. They may not be in a position to evaluate such technologies without retaining experts, while independent researchers may have that expertise. In recent years, technologies ranging from field DNA kits to facial-recognition technology (FRT) have been adopted by law-enforcement agencies, without the assurances provided by independent review of reliability and fairness, or the interpretability and transparency that could permit legal actors to adequately understand the results.
Most crime laboratories are not independent. The most recent federal survey of approximately 300 crime laboratories revealed that 79 percent of all laboratories responding were located within law-enforcement/public-safety agencies, and 57 percent “would only examine evidence submitted by law enforcement officials.” Durose et al, supra. The National Academy of Sciences report highlighted that independence allows the lab director to have an equal voice in making critical decisions for the lab, set priorities regarding expenditures, and reduce cultural pressures to serve law enforcement. 2009 NRC Report, supra, at 184. The National Academy of Sciences recommended that federal funding assist state and local jurisdictions in removing forensic laboratories and facilities from the administrative control of law enforcement. Id. at 24.
Of course, even if a crime laboratory is independent, communication and cooperation between law enforcement and forensics professionals is appropriate and essential. However, care should be taken that such communications do not bias the forensic work. Procedures and protocols should be in place to minimize the role that potentially biasing communications can play. It is possible for there to be close cooperation between crime labs and law enforcement, while preserving the integrity of the science. Policies and procedures to create independence and minimize the role of bias are particularly important for forensic disciplines that involve more subjective exercise of judgment and use of case-specific information. There is evidence that large numbers of deaths— more than half of police killings in the United States— have been mislabeled and not counted as police-violence-related deaths, due to medical examiners’ conclusions in death certificates. As the New York Times noted in describing this research, “forensic pathologists regularly consult with detectives and prosecutors and in some jurisdictions they are directly employed by police agencies.” Tim Arango & Shaila Dewan, More Than Half of Police Killings Are Mislabeled, New Study Says, N.Y. Times, Sept. 30, 2021.
Functional and financial independence is no substitute for quality control and accountability of a crime laboratory. Instead, oversight through quality assurance and independence go hand in hand. Independence can facilitate the goal of scientific oversight, see § 9.04, by making clear that the mission is to produce high quality forensic evidence, and not solely to provide a service for law enforcement.