Why We Don T See Lions Bombs And Breast Cancers

Let’s imagine that I suggest you look for lions while we are strolling through the park. For most people reading this, the local park will be an unlikely place to find a lion. If you take my request seriously, you might look around but you would not spend vast amounts of time on the search. If you saw something of a light brown color out of the corner of your eye, you would probably be reluctant to jump up and down, shouting, “I saw one.” Now let us suppose that we are on safari in Kenya, strolling through the Serengeti and I suggest that you look for lions. In this case, you might spend the whole day on this search and you might well jump up and down, shouting, “I saw one!” as that light brown, furry something disappeared from sight.

The difference between these two situations is “target prevalence.” Lions are extremely rare in your normal haunts and much more common on the Serengeti plain. Your behavior would be entirely reasonable. You really should not spend too much energy on a search for lions in Minneapolis or Munich.

Now suppose you are a checkpoint screener at the airport and you have been asked to search x-ray images of baggage for threats like guns, bombs, and knives. Or perhaps you are a radiologist, screening women for breast cancer. Threats at the airport and breast cancers in the normal population are also rare, “low-prevalence” events. Does the prevalence of targets matter in these socially important search tasks?

It certainly matters in the lab. For example, we did an experiment in which people looked for guns and knives in x-ray images of luggage. These were volunteers from the general public, not airport screeners. Indeed, that is why we only used guns and knives. Bombs are too difficult for untrained observers to spot. We compared 50 percent, high target prevalence (say, 20 threats in 40 bags) to 2 percent, low prevalence (20 threats in 1000 bags). The threat objects were the same in the two conditions, but our observers missed about 20 percent of the threats in the high-prevalence context and about twice that many, 40 percent, in the low-prevalence context. The frequency of targets made a big difference.

Of course, our observers knew they were in an experiment and they knew the stakes were pretty low. To test the effect in the real world, Karla Evans and I teamed up with Robyn Birdwell and the breast cancer screening service at Brigham and Women’s Hospital in Boston. We arranged to slip 100 cases into their normal workflow: 50 with cancer and 50 without. We did this very slowly – at a rate of 1 or 2 cases a week over 9 months. As a result, we did not change the low prevalence of cancer cases by very much. For comparison, we had a group of radiologists read all 100 cases in a single, high-prevalence session.

The results, which we have just presented at the Radiological Society of North America meeting in Chicago were quite striking. In a high-prevalence context, our radiologist missed 12 percent of the cancers. At low prevalence, in the clinic, they missed 30 percent. Now, the message here is not “Oh my goodness, that hospital misses a lot of cancers.” These were a set of cases chosen specifically for this study and not designed to measure overall performance. Studies in the medical literature suggest that the miss rate in breast cancer screening is around 20 or 30 percent. What our work suggests is that something like half of those missed cancers might be missed, not because the image was bad or because the radiologist wasn’t doing her job. They were missed because our brains were designed to respond to low-prevalence cancers as if we were looking for lions in the park. What might we do about this? We have surprisingly little voluntary control over this aspect of our behavior. Our internal search engine is, in the words of Jerry Fodor, “cognitively impenetrable.” You can’t just tell yourself, “Sure I am looking for something rare, but this is important.” However, there are some tricks that work in the lab. In one experiment, before people searched for low-prevalence targets, they did a much briefer search for high-prevalence targets with good feedback about how they were performing. The high-prevalence “booster shot” inoculated people against the low-prevalence effect, at least for a while. Perhaps that means that a radiologist could gain some benefit from a high-prevalence warm-up exercise before starting her low-prevalence work. We don’t know, but we hope to do the experiment soon.

The prevalence effect is just one of many examples of what can happen when a brain that evolved to deal with the conditions of the real world is faced with important but artificial tasks created by our civilization. (You brain was not built to text while driving, for example .) An important role for us as researchers is to identify the places where the interactions of brain and civilization may not be ideal. Then we can work to improve the relationship.

Are you a scientist who specializes in neuroscience, cognitive science, or psychology? And have you read a recent peer-reviewed paper that you would like to write about? Please send suggestions to Mind Matters editor Gareth Cook, a Pulitzer prize-winning journalist at the Boston Globe. He can be reached at garethideas AT gmail.com or Twitter @garethideas.