One of the challenges of writing about science is that important concepts are not always exciting, and it is no small feat to make a dull subject shine. Recent events, however, have highlighted a topic that is both deadly dull and deadly serious: instrument calibration. Calibration is the process of making sure an instrument is working accurately. Usually this involves testing against a known standard (or set of standards). Every scientist who works with an instrument learns to calibrate it; organizations that make many measurements, such as the U.S. Geological Survey, have protocols for ensuring that it is done regularly and accurately. Calibration services are a major part of the work of the National Institute of Standards and Technology, on which both industry and other federal agencies rely. But it is not something that you do once and forget about; instruments drift, so they have to be checked regularly. Without calibration, the measurements we make may be meaningless. And this, it turns out, is what has happened with a highly consequential instrument used every day across America: the Breathalyzer, used to determine whether a driver has had too much to drink. In 2017 a Massachusetts judge threw out thousands of drunk-driving convictions on the grounds that the kind of Breathalyzer used was not reliable. This ruling followed an earlier agreement among the state’s district attorneys and lawyers representing alleged drunk drivers that data from breath tests would not be used at trial (except for serious offenses), after evidence emerged that the results were questionable. By one account, the decision affected 35,000 outstanding cases from 2011 to 2017. Another account suggests that the total number of affected cases could exceed 58,000. Other states have also questioned convictions based on Breathalyzer results: in New Jersey more than 20,000 drunk-driving convictions have been called into question. Drunk driving is a huge problem. Every year more than a million Americans are arrested for it and, according to the Centers for Disease Control and Prevention, more than 10,000 deaths can be attributed to it. Society has a compelling interest in identifying drunk drivers and getting them off the road. But if the tools used to identify them are unreliable, then innocent people can lose their licenses and, in some cases, be wrongly convicted and incarcerated. The reverse is also true: with tens of thousands of cases where police had reason to suspect that the driver was impaired now thrown out, it is likely that many guilty parties will go free. Why did this happen? 0ne pervasive problem is instrument calibration. It turns out the Breathalyzer, like all scientific instruments, needs to be regularly calibrated, and police officers often are neither trained nor equipped to do this work. In some cases, it appears that police precincts did not even realize calibration was needed. Here is a pretty simple solution: police need to bring in technicians to check their instruments. As one company that offers instrument calibration puts it, just as motor vehicles require regular maintenance, so do the Breathalyzers used to test their drivers. It would add a bit to policing costs, of course, but surely that outcome is preferable to losing years of prosecutorial work or to sending innocent people to jail while guilty parties walk free. There is a deeper lesson here about science and technology: scientific instruments do not perform magic tricks. You cannot just blow air in a machine and get a good result; accurate data are the product of sustained attention. It takes good work to get good numbers. For decades science teachers have been admonished to teach not just facts but processes, including the process of doing scientific research. I recall my own children raising tomatoes, performing a census of marine life and constructing volcano simulations. But what state science standards include a unit on calibration? Maybe it is time they add one. The miscarriage of justice caused by the misuse of scientific instruments underscores why we need to understand not just the findings of science but also the processes by which scientific evidence is obtained, as boring as they seem.
Calibration is the process of making sure an instrument is working accurately. Usually this involves testing against a known standard (or set of standards). Every scientist who works with an instrument learns to calibrate it; organizations that make many measurements, such as the U.S. Geological Survey, have protocols for ensuring that it is done regularly and accurately. Calibration services are a major part of the work of the National Institute of Standards and Technology, on which both industry and other federal agencies rely.
But it is not something that you do once and forget about; instruments drift, so they have to be checked regularly. Without calibration, the measurements we make may be meaningless. And this, it turns out, is what has happened with a highly consequential instrument used every day across America: the Breathalyzer, used to determine whether a driver has had too much to drink.
In 2017 a Massachusetts judge threw out thousands of drunk-driving convictions on the grounds that the kind of Breathalyzer used was not reliable. This ruling followed an earlier agreement among the state’s district attorneys and lawyers representing alleged drunk drivers that data from breath tests would not be used at trial (except for serious offenses), after evidence emerged that the results were questionable. By one account, the decision affected 35,000 outstanding cases from 2011 to 2017. Another account suggests that the total number of affected cases could exceed 58,000. Other states have also questioned convictions based on Breathalyzer results: in New Jersey more than 20,000 drunk-driving convictions have been called into question.
Drunk driving is a huge problem. Every year more than a million Americans are arrested for it and, according to the Centers for Disease Control and Prevention, more than 10,000 deaths can be attributed to it. Society has a compelling interest in identifying drunk drivers and getting them off the road. But if the tools used to identify them are unreliable, then innocent people can lose their licenses and, in some cases, be wrongly convicted and incarcerated. The reverse is also true: with tens of thousands of cases where police had reason to suspect that the driver was impaired now thrown out, it is likely that many guilty parties will go free.
Why did this happen? 0ne pervasive problem is instrument calibration. It turns out the Breathalyzer, like all scientific instruments, needs to be regularly calibrated, and police officers often are neither trained nor equipped to do this work. In some cases, it appears that police precincts did not even realize calibration was needed. Here is a pretty simple solution: police need to bring in technicians to check their instruments. As one company that offers instrument calibration puts it, just as motor vehicles require regular maintenance, so do the Breathalyzers used to test their drivers. It would add a bit to policing costs, of course, but surely that outcome is preferable to losing years of prosecutorial work or to sending innocent people to jail while guilty parties walk free.
There is a deeper lesson here about science and technology: scientific instruments do not perform magic tricks. You cannot just blow air in a machine and get a good result; accurate data are the product of sustained attention. It takes good work to get good numbers. For decades science teachers have been admonished to teach not just facts but processes, including the process of doing scientific research. I recall my own children raising tomatoes, performing a census of marine life and constructing volcano simulations. But what state science standards include a unit on calibration? Maybe it is time they add one.
The miscarriage of justice caused by the misuse of scientific instruments underscores why we need to understand not just the findings of science but also the processes by which scientific evidence is obtained, as boring as they seem.
