FBI's Forensic Test Full of Holes
The science, known as comparative bullet-lead analysis, was first used after President John F. Kennedy's assassination in 1963. The technique used chemistry to link crime-scene bullets to ones possessed by suspects on the theory that each batch of lead had a unique elemental makeup.
In 2004, however, the nation's most prestigious scientific body concluded that variations in the manufacturing process rendered the FBI's testimony about the science "unreliable and potentially misleading." Specifically, the National Academy of Sciences said that decades of FBI statements to jurors linking a particular bullet to those found in a suspect's gun or cartridge box were so overstated that such testimony should be considered "misleading under federal rules of evidence."
A year later, the bureau abandoned the analysis.
But the FBI lab has never gone back to determine how many times its scientists misled jurors. Internal memos show that the bureau's managers were aware by 2004 that testimony had been overstated in a large number of trials. In a smaller number of cases, the experts had made false matches based on a faulty statistical analysis of the elements contained in different lead samples, documents show.
...
William Tobin, an FBI lab metallurgist for a quarter-century, won accolades working on cases such as the crash of TWA Flight 800, in which he helped prove that the plane was downed by an accidental fuel-tank explosion, not terrorism. Shortly before he retired, Tobin was approached by a woman who believed that the bullet-lead science used against her brother, a New Jersey murder defendant, was flawed. Still employed by the bureau, Tobin was not permitted to help.
But when he retired in 1998, he decided to look further. Bullet matching had always been done by the lab's chemists, and as a metallurgist, Tobin wondered about their assumptions. Soon he joined with Erik Randich, a metallurgist at Lawrence Livermore National Laboratory.
By 2001, the two had finished a study that challenged the key assumptions that the FBI had been making about bullet lead. They found that bullets made from the same batch did not always match, because subtle chemical changes occurred throughout the manufacturing process. Tobin bought bullets at several stores in Alaska and found that a large number of bullets with the same composition and manufacturing date were often sold in the same community, suggesting that it was wrong to assume that a bullet match could be narrowed to one suspect.
"It hadn't been based at all on science but, rather, had been based on subjective belief," Tobin said in an interview. "Courts, and even practitioners, had been seduced by the sophistication of the analytical instrumentation for over three decades."
Soon, Tobin began appearing as a witness for defendants challenging FBI bullet-lead matches. Courts began to take notice, too, and the FBI suddenly faced a barrage of questions about a science that had gone unchallenged for three decades.
Adams asked the National Academy of Sciences in 2002 to examine the FBI's work, temporarily halting new bullet-lead matches. Two years later, the academy's findings stunned the bureau.
The panel concluded that although the FBI had been taking accurate bullet-lead measurements in its lab, the statistical methods and its expert testimonies were flawed.
The science "does not . . . have the unique specificity of techniques such as DNA," and "available data does not support any statement that a crime bullet came from a particular box of ammunition," the panel concluded. All the FBI could say going forward was that bullets made from the same batch "are more likely" to match in chemical makeup than those made from different batches. Adams soon declared that such testimony was so general that it had no value to jurors, and he ended the technique.
