NACDL

In February 2009, the National Academy of Sciences (NAS), the most prestigious scientific organization in the country, issued a landmark report titled Strengthening Forensic Science in the United States: A Path Forward (NAS Report). After studying and evaluating a number of different forensic science disciplines, the NAS concluded, “With the exception of nuclear DNA analysis … no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.”¹ The report concluded that there was in fact little science in many of the forensic science disciplines. Moreover, “much forensic evidence — including, for example, bite marks and firearm and toolmark identifications — is introduced in criminal trials without any meaningful scientific validation, determination of error rates, or reliability testing to explain the limits of the discipline.”² This report is surely one of the most important developments in the realm of forensic science and a critical development for criminal defense lawyers. Not surprisingly, the NAS Report, sometimes referred to as the NRC Report of 2009,³ has not been embraced by the forensic science community.

Response to the Report

Shortly after its issuance, numerous forensic science organizations released statements criticizing the report, the composition of the NAS committee, and the process by which the committee came to its conclusions. Instead of seriously considering and legitimately responding to the deficiencies identified in the report, various organizations representing fingerprint examiners, and firearm and toolmark examiners, among others, very quickly sought to do superficial damage control. For example, regarding fingerprint analysis using the Analysis, Comparison, Evaluation, and Verification (ACE-V) technique, the NAS reported that “ACE-V provides a broadly stated framework for conducting friction ridge analyses. However, this framework is not specific enough to qualify as a validated method for this type of analysis. ACE-V does not guard against bias; is too broad to ensure repeatability and transparency; and does not guarantee that two analysts following it will obtain the same results. For these reasons, merely following the steps of ACE-V does not imply that one is proceeding in a scientific manner or producing reliable results.”⁴ The International Association for Identification (IAI) issued a response: “There is no research to suggest that properly trained and professionally guided examiners cannot reliably identify whole or partial fingerprint impressions to the person from whom they originated.”⁵ One might assert that this response effectively demonstrates the IAI members’ lack of understanding of some of the most basic tenets of science. It also seems to reveal that criticism intended to lead to improvements in the forensic sciences is unwelcome.

The response by the Association of Firearms and Toolmark Examiners (AFTE) was more nuanced, acknowledging deficiencies of some firearm and toolmark examiners who are “out of the mainstream,” but nevertheless asserting the “NAS painted an incomplete and inaccurate portrait of the field of firearm and toolmark identification using a very broad brush, and in doing so did not consider the appropriate scientific principles on which our discipline is founded.”⁶ 

During his tenure as Deputy Director of the Bureau of Alcohol, Tobacco, Firearms, and Explosives, Kenneth Melson issued what may be characterized by some as a scathing criticism of the NAS Report in which he challenged the recommendations made in the report, the NAS panel responsible for the report, and the process by which the panel came to its conclusions.⁷ Melson suggested that if the report is used during cross-examination, the expert should be able to “foil the effective use of the report” or alternatively provide the person who is conducting the examination with “an answer he or she does not like.”⁸ 

Relying on Courts to Force Change

Currently, efforts are being made through the Executive Office and the legislature to implement many of the recommendations made in the NAS Report. Progress is, however, slow and uncertain. Consequently, it is more likely that if there is to be any real reform, it must come from the courts as a result of challenges by lawyers. Relying on the courts to force change is not unprecedented. Challenges in the courts to DNA, handwriting, and fingerprint comparison evidence finally caused the forensic community to take notice that a growing number of courts would no longer simply accept the status quo.⁹ In United States v. Green, Judge Nancy Gertner, faced with a challenge to firearms evidence, stated it well: “The more courts admit this type of toolmark evidence without requiring documentation, proficiency testing, or evidence of reliability, the more sloppy practices will endure; we should require more.”¹⁰ While there have been few monumental decisions from the courts, there has clearly been a recognition that forensic science evidence must now be carefully scrutinized and no longer admitted without limitation.

Since the issuance of the NAS Report, defense attorneys in state and federal trials and postconviction cases have challenged forensic science evidence and, in particular, pattern impression evidence, raising many of the deficiencies described in the report. In some cases the evidence has been excluded entirely. Courts excluding this evidence have held that (1) the evidence did not meet the requirements of Daubert and/or Frye, or (2) there was insufficient documentation of the underlying process such that if the evidence were to be admitted, the defense would not be able to adequately cross-examine the examiner or a defense expert could not reliably assess the examiner’s conclusions.¹¹ In addition, courts have excluded forensic evidence based on a lack of experience on the part of the examiner.¹² 

In other cases, courts have ruled the examiner may only identify the matching characteristics without stating the significance or the weight to be attributed to these matching characteristics.¹³ Other courts have imposed a variety of limitations on what the examiner may testify to in court. Specifically, courts have held an examiner (1) may not state that the error rate for a discipline or method is zero;¹⁴ (2) may not state the evidence matches the suspect to the exclusion of all others; (3) may not state he is certain or that it is a practical impossibility that another would match;¹⁵ and (4) may not state the two match to a reasonable degree of scientific certainty.¹⁶ Indeed, one court limited the examiner to stating that the evidence “more likely than not”¹⁷ matched the suspect while another limited the examiner’s testimony to stating that the evidence matched the suspect “to a reasonable degree of ballistic or fingerprint certainty.”¹⁸ A number of courts have taken the position that these techniques are more of an art than a science and that the NAS criticisms are proper areas for cross-examination. Some courts have given instructions to the jury limiting the weight they should attribute to such testimony.¹⁹ Whether lay jurors will grasp the significance of some of these limitations without further explanation is something about which lawyers must be concerned.

One court issued a procedural order employing language that may be useful to lawyers seeking to raise these issues. “While the report does not speak to admissibility or inadmissibility in a given case, it raised profound questions that need to be carefully examined in every case prior to trial: (1) the extent to which a particular forensic discipline is founded on a reliable scientific methodology that gives it the capacity to accurately analyze evidence and report findings, and (2) the extent to which practitioners in a particular forensic discipline rely on human interpretation that could be tainted by error, the threat of bias, or the absence of sound operational procedures and robust performance standards.”²⁰ The order required all parties to (1) specify whether or not they sought to introduce trace evidence; (2) state whether or not either party sought a Daubert/Kumho hearing prior to trial; and (3) identify the witnesses required for the Daubert/Kumho hearing and the exhibits that the parties sought to admit. No later than two months before the pretrial conference, counsel also had to indicate (1) if counsel was appointed, whether expert funds were sought to deal with the trace evidence; and (2) whether all discovery obligations under the Local Rules had been met or whether additional discovery was required.²¹ 

Limiting Testimony

While wholesale exclusion of trace or pattern impression evidence has been rare, the U.S. District Court in United States v. Smallwood²² actually did just that.²³ Defendant filed a motion to exclude testimony that a particular knife associated with the defendant created toolmarks that matched the toolmarks found on the tires of a vehicle that had been vandalized.²⁴ The court held that “by AFTE’s own standards, there is no reliability in the instant case.”²⁵ The judge ruled the examiner did not possess the skill and training to reliably make the required subjective determination, and that the witness could not be subject to meaningful cross-examination because the photograph that purported to document the examiner’s comparison was not sufficiently detailed for that purpose. “Accordingly, the match determination was effectively insulated from any meaningful cross-examination by the inability to produce photographs representative of what an examiner sees under the actual comparison microscope.”²⁶ The court also found “simply no consensus that toolmark evidence is reliable” and insufficient studies available from which a statistical estimate of an error rate could be made.²⁷ For all of these reasons, the court excluded the testimony of the toolmark examiner.

Increasingly, the testimony of pattern impression examiners has been limited rather than wholly excluded by the courts. In United States v. Willock,²⁸ the court adopted the magistrate judge’s recommendation that the examiner not be permitted to testify that in his opinion it was a practical impossibility for any other firearm to have fired the cartridges. Further, the examiner would only be permitted to state his opinion and the basis of his opinion without any characterization of his degree of certainty.²⁹ Yet other courts have permitted examiners to testify to their opinions within a “reasonable degree of certainty” in their field of expertise,³⁰ but not state explicitly or imply that that the examiner’s opinion is a result of an infallible scientific ^process31 or the error rate for the method is zero. One court held the examiner could not testify that no two people have the same fingerprints or that there was an objective basis for the examiner’s opinion or that it was supported by scientific principles.³² It seems obvious that if such a ruling is likely to be the result of a challenge, the defense lawyer must be prepared to suggest appropriate limitations or language and cogently argue her position.

In denying motions to exclude testimony, some courts have ruled that the issues raised in the NAS Report are proper subjects for cross-examination.³³ “The absence of a known error rate, the lack of population studies, and the involvement of examiner judgment all raise important questions about the rigorousness of friction ridge analysis. To be sure, while further testing and study would likely enhance the precision and reviewability of fingerprint examiners’ work, the issues defendant raises concerning the ACE-V method are appropriate topics for cross-examination, not grounds for exclusion.”³⁴ 

One court gave the following charge to the jury: “The fingerprint examiner’s testimony is his opinion. It should not be considered by you as conclusive fact, but should be weighed along with all the evidence that you have heard in this case. His opinion should be treated the same as any other evidence, which means that you are free to give it the weight you believe it deserves. You may accept or disregard it in whole or in part. Fingerprint examiners may be of assistance to you. However, their skill is practical in nature, and despite anything you may have heard, it does not have demonstrable certainty.”35

Defense Strategy

Defense attorneys must familiarize themselves with the NAS Report and work to develop effective strategies for raising these issues in the trial courts. Challenges to the admissibility of this type of evidence should regularly be brought when prosecutors seek to introduce such evidence, as the report unequivocally concludes there is no scientific research demonstrating the validity of many of the pattern impression disciplines. Many, if not most, of the claims made by examiners in the fields of fingerprints, firearms, and toolmark and bite marks have never been empirically tested. The report boldly and thoroughly calls into question the scientific reliability of the claims made by these examiners and demonstrates there is no consensus in the scientific community at large regarding general acceptance of claims made by forensic examiners in these fields.

Challenges seeking complete exclusion of this type of evidence should first be made under Daubert³⁶ and Frye.³⁷ Additionally, the particular expert may lack the necessary qualifications, skill and/or training to be able to reliably make the subjective determination required. In other words, if the toolmark examiner has only previously examined and compared a small number of objects like the one in the defendant’s case, he may not have sufficient experience or skill to render an opinion that is sufficiently reliable. Thus, where appropriate, an admissibility challenge seeking exclusion of the evidence should be based on the lack of qualification of the expert.

The Confrontation Clause of the Sixth Amendment and the right to assistance of counsel may be implicated in situations in which inadequate documentation of procedures and poor record-keeping leave little record for independent review of what was done and insufficient detail to permit cross-examination. This should lead to another challenge seeking exclusion of the evidence. Pattern match evidence should be inadmissible when there is no sufficiently detailed documentation of the examination and comparison conducted by the examiner. The documentation must permit an independent examiner to determine exactly what steps the examiner followed, what characteristics he relied on, and on what basis or bases the examiner was able to conclude there was a match. Furthermore, the detailed documentation must be sufficient to permit meaningful cross-examination. It must provide a basis for the cross-examiner to inquire regarding points of dissimilarity that were disregarded, as well as the points and number of similarities the examiner relied on as the basis of his opinion. In other words, in addition to challenging the admissibility of the evidence on Frye/Daubert grounds, the evidence should be challenged and may be inadmissible under the Sixth Amendment based on violations of the Confrontation Clause and the right to assistance of counsel if the documentation lacks sufficient detail to permit review and cross-examination.

Finally, in a capital prosecution the Eighth Amendment should be invoked as a ground to exclude the evidence. Specifically, it is apparent from the NAS Report there is no scientific foundation for claims made that an examiner may match a latent print to a suspect to the exclusion of all others. What is also clear is that there will be researchers in the future who will be able to determine the probability that another person, gun, or tool could have made the pattern observed. Thus, in the future there will be studies that demonstrate the testimony of the examiner was simply false in its claim of exclusivity and that the jury was permitted to attribute more weight to the evidence than can be scientifically supported. However, in a capital case, the defendant may have already been executed by the time there is sufficient understanding of the weight that should have been attributed to the pattern match. Consequently, it would be a violation of the Eighth Amendment to execute someone based on this evidence. This provides a strong basis for asserting that the court has a duty to exclude this evidence or at least limit what may be said about the match.

If the court denies a defense attorney’s admissibility challenge, she should be prepared to move to limit the testimony that may be provided by the examiner. For example, the court should not permit the examiner to state the error rate for the method is either zero or so small as to be negligible.³⁸ Courts should not permit examiners to testify, for example, that a latent print matches a suspect print to the exclusion of all others or that the latent print and the suspect print are one and the same. Such testimony should be limited to either a statement that there is consistency in the characteristics observed between the latent and suspect print, or the suspect cannot be excluded as being among a group of individuals of unknown size who could have left the latent print. A court should also limit what the expert may state regarding his level of certainty in declaring that the two items match. While it is not entirely clear what may be stated, it is clear what may not be stated by the examiner, which is that there is a match to “a reasonable degree of scientific certainty.” As mentioned above, some courts have permitted examiners to express their opinions to a reasonable degree of “fingerprint certainty” or to a reasonable degree of certainty “in the field of fingerprints.” If the court does limit the testimony in this manner, then the court has implicitly accepted the argument that this is not science and the jury needs to be told what this means.

If it is not science, jurors must be instructed what it means to express an opinion to “a reasonable degree of ballistic/fingerprint certainty” and should be given guidance as to how they should view and give weight to the evidence. Thus, a court might give the following instruction: “A match to a reasonable degree of fingerprint certainty means that the defendant is among a pool of individuals, the size of which is unknown, whose fingerprint impression is indistinguishable from this latent print.” In a separate instruction, the jury should be told that (1) the opinion presented is not a scientific opinion; (2) the error rate for this comparison procedure is unknown; (3) no standard error of measurement has been established for this procedure; and (4) the research in the field is incomplete and ongoing and it is not understood how to limit the effects of cognitive bias on the procedure. It is incumbent upon judges to give jurors some guidance on how they are to view this type of evidence and what it means in the context of the case. If the court denies the admissibility challenge, counsel should request the expert testimony be limited and request the jury be properly instructed. Regardless of the rulings on these motions, the NAS Report provides a tremendous opportunity for cross-examination as is evident in the following excerpt from a case handled by the Los Angeles County Public Defender’s Office.

Defense Counsel: And in that particular treatise, it is determined by those scientists working collaboratively that, “with the exception of nuclear DNA analysis, no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.” Is that right?

Examiner: Before I answer that, could I qualify a little bit about this organization, the NAS, National Academy of Sciences. It’s a committee that is made up — it’s a nonprofit committee that’s made up of attorneys — no offense — of judges, of engineers, of statisticians, mathematicians, and maybe a couple of scientists, and maybe one scientist in the field of forensic science. And they meet every so often in Washington, D.C., and they pontificate about how things should be in a perfect world, and their last venture was to attack forensic science. And they had a 30-minute presentation from one of our presidents, AFTE President Pete Striupaitis, where he tried to cover years of training that we all go through as examiners in 30 minutes and pretty much presented the whole field of firearm identification. And within these 30 minutes this committee critiqued our profession and deemed that they didn’t approve of the way we conducted our analysis, and they compared to DNA. DNA analysis and firearm identification is [sic] two different disciplines altogether. They have statistics. They have probabilities. They have numbers they can attach to — when they identify DNA. We don’t. Our science does not in itself. And to attempt to lump us with another discipline that’s totally different in nature is not professional — and that’s why this committee in our field is really a nonissue. I mean, they produced a nice book, hard bound, but, basically, a committee. We have responded to this attack, and it’s really a nonissue. And it’s — like I said, they’re not our peers, just scientists that actually have probed into our field, and in order to justify maybe a position down the road to be as a watchdog in our profession. And we have our own watchdog, which is AFTE, the recognized organization and American Academy of Forensic Science.

Defense Counsel: So obviously, you disagree with the National Research Council of the National Academy of Sciences.

Examiner: Totally.³⁹ 

Conclusion

Cases that rest in whole or in part on forensic evidence must be carefully scrutinized. In addition to traditional investigation and obtaining expert assistance, challenging the admissibility of the evidence should be considered and evaluated. Moreover, one should seek to limit how the evidence is presented by the examiner commensurate with existing research. The NAS Report should be used when cross-examining the examiner, and jury instructions advising jurors how to view the evidence should be requested. Real reform must come from the courts, and effective representation by criminal defense lawyers will spark these reform efforts.

Notes

1. National Academy of Sciences, Strengthening Forensic Science in the United States: A Path Forward 7 (hereinafter NAS Report) (2009). Download the report at http://www.nap.edu/catalog.php?record_id=12589.
2. Id. at 107-108.
3. The National Research Council (NRC), the operating arm of the National Academies, has strived since 1916 to provide elected leaders, policy makers, and the public with expert advice based on sound scientific evidence.
4. NAS Report at 142.
5. IAI letter from President Robert Garrett to its membership (February 19, 2009); http://www.theiai.org/current_ affairs/nas_memo_20090219.pdf (last viewed Jan. 8, 2012).
6. Response of AFTE to the NAS Report (June 22, 2009).
7. Ken Melson, “Embracing the Path Forward: The Journey to Justice Continues,” 36 New Eng. J. on Crim. & Civ. Confinement 197 (2010).
8. Id. at 228.
9. Paul C. Giannelli, Daubert and Forensic Science: The Pitfalls of Law Enforcement Control of Scientific Research, 2011 U. Ill. L. Rev. 53 (2011).
10. Giannelli, note 9, supra, citing United States v. Green, 405 F. Supp. 2d 104, 109 (2005).
11. United States v.Smallwood, 2010 WL 4168823 (W.D. Ky. 2010).
12. Id.
13. United States v. Willock, 696 F. Supp. 2d 536 (D. Md. 2010).
14. Greenwood, Los Angeles Superior Court No. BA351185 (2010).
15. St. Gerard, U.S. Army Trial, Fifth Judicial Circuit, Germany (2010).
16. United States v. Faison, Superior Court, District of Columbia (2010).
17. United States v. Glynn, 578 F. Supp. 2d 567 (2008).
18. United States v. Taylor, 663 F. Supp. 2d 1170 (D. N.M. 2009).
19. Comm. v. Heang, 454 Mass. 1011, 908 N.E.2d 373 (Mass. 2009).
20. Procedural Order: Trace Evidence issued by District Court Judge Nancy Gertner (March 8, 2010).
21. Id. 
22. See Smallwood, note 11, supra.
23. Cf. State v. McGuire, —- A.3d ——, 2011 WL 890748 (N.J. Super. A.D. 2011).
24. Other forensic testimony not relevant to this discussion was also challenged, and the court made rulings primarily limiting the expert testimony.
25. See Smallwood, note 11, supra.
26. Id.
27. Id. 
28. Id. 
29. Id. at 9.
30. Taylor, note 18, supra; Faison, note 16, supra.
31. Greenwood, note 14, supra.
32. Comm. v. Heang, note 19, supra.
33. United States v. Aman, 2010 WL 4103157 (E.D. Va. 2010); Greenwood, note 14, supra. 
34. Aman, note 33, supra. 
35. Comm. v. Heang, note 19, supra.
36. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).
37. Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923).
38. “[A]lthough the precise error rates of these forensic tests are still unknown, comparison of their results with DNA testing in the same cases has revealed that some of these analyses, as currently performed, produce erroneous results.” NAS Report at 42. Furthermore, claims that these disciplines have an error rate of zero “are not scientifically plausible.” NAS Report at 142.
39. People v. Dewey, Los Angeles Superior Court, No. BA343051 (Deputy Public Defender Dylan Ford).

© Jennifer Friedman, 2012. All rights reserved.