As an emergency physician, Megan Ranney is used to dealing with hard questions about injury and illness. But during the COVID-19 pandemic, she has found herself scrambling for answers. “I kept getting asked by people whether it was safe to do certain activities,” says Ranney. “‘Is it safe to go to the beach, or to an outdoor party or a restaurant?’” Millions have struggled with the same questions throughout 2020. Scientists and clinicians are still learning about the epidemiology and pathology of the coronavirus SARS-CoV-2 and, for the public, it can be difficult to understand how to maintain some semblance of normality while minimizing the risk of infection. As co-director of the Brown-Lifespan Center for Digital Health in Providence, Rhode Island, Ranney saw an opportunity to use her expertise in developing online tools for public health. Working with her Brown-Lifespan colleague Elizabeth Goldberg, she developed MyCOVIDRisk, a simple, questionnaire-based web app that gives users a sense of their infection risk on the basis of their planned activity in a specific place or situation. Filling the gaps Launched in October, MyCOVIDRisk is just one of several apps developed to communicate the risks of COVID-19 to the public (see ‘COVID risk calculators’). These tools are filling important gaps in public-health messaging, particularly in the United States, where an uneven response to the pandemic has exacerbated its severity. “This type of app is something that could have been developed very early in the pandemic by the [US] Centers for Disease Control and Prevention or another national entity,” says Ranney. “But unfortunately, that wasn’t done.”
Credit: Nature
Individual research teams have approached risk assessment from different angles. Perhaps the most straightforward approach is the web-based COVID-19 Event Risk Assessment Planning Tool developed by scientists at the Georgia Institute of Technology in Atlanta. The site estimates the probability (as a percentage) that a person will encounter someone with COVID-19 at a gathering, on the basis of the size of the group and where the event takes place. For example, as of 14 December, the tool predicted that at a gathering of 10 people in Boston, Massachusetts, there would be a 30% chance of at least one attendee having the virus. The 19 and Me calculator, developed by Mathematica, a policy-research company in Princeton, New Jersey, draws on demographic and health information as well as user behaviours such as hand washing and mask use to determine relative risk of exposure, infection and serious illness. And in December, a team led by biostatistician Nilanjan Chatterjee at Johns Hopkins University in Baltimore, Maryland, released the COVID-19 Mortality Risk Calculator, which estimates an individual’s relative risk of death from COVID-19 on the basis of their location, pre-existing conditions and general health status. MyCOVIDRisk takes a more situational approach, estimating the risks associated with specific errands or recreational activities. The estimate is based on the location and duration of a gathering, and the number of masked or unmasked people attending. This can help users to avoid activities that are likely to be high risk in a pandemic hotspot, such as spending an hour in an indoor gym, in favour of safer alternatives — a masked meet-up in the park, for instance. To make predictions, these tools require the latest data and models that accurately capture the current levels of infection and the likelihood of transmission in various scenarios. The New York Times runs a coronavirus database that has become a popular resource, and both MyCOVIDRisk and the Georgia Tech tool use it to assess local disease activity. These calculators are currently focused on the United States, although the Georgia Tech team has extended its tool to cover ten European countries. Researchers are still teasing out the factors that predispose some individuals to infection and severe disease, and they continue to study how distance, ventilation and masks mitigate the spread of virus-laden droplets. For 19 and Me, Xindi Hu — the data scientist at Mathematica who led the tool’s development — and her colleagues regularly comb through the latest peer-reviewed literature to refine their infection models, relying heavily on reports from the US Centers for Disease Control and Prevention (CDC). For MyCOVIDRisk, the team turned to an aerosol-transmission estimation model developed by atmospheric chemist Jose-Luis Jimenez and his colleagues at the University of Colorado Boulder, which simulates how far viral particles are likely to travel in various indoor and outdoor scenarios. The researchers then have to come up with ways to wrangle this information into a format that is easy for non-experts to understand, without it being intrusive or frustrating. To build a simple and engaging app that balances depth with usability, the Brown-Lifespan team relied on user testing and guidance from behavioural scientists. “In an ideal world, you’d ask how big of a room are you in and what’s the rate of ventilation,” says Ranney, “But if you asked me, I wouldn’t even know the answer.” This need for simplicity also motivated the Georgia Tech team. Its tool solicits only two pieces of information: a user’s location and the size of the expected crowd. Initially, this ranged from 10 to 10,000 people. “Those upper bounds were useful when political rallies were occurring over the summer,” says Andris. But as the holidays drew closer, the team decreased the upper bound to 5,000 people and increased the tool’s level of detail at the lower end of the scale to deliver estimates for relatively modest gatherings of 10, 15 or 20 people. To convey risk, MyCOVIDRisk opts for a simple, 5-tier continuum scale of green to red, whereas the 19 and Me team devised an abstract numerical readout ranging from 0 to 100, which Hu likens to a credit score. “That is a compromise we’ve made to keep the rigour of the calculation while making it into a number that is easily understood by the general public,” she says. “And we couple it with a text explanation and an interactive risk gauge.” This explanation summarizes the potential for exposure and infection, along with the likelihood of serious health consequences and hospitalization if infection occurs. However, developers caution that the apps provide estimates intended to assist in decision-making, and are not definitive indicators of the likelihood of contracting disease. Casting a wider net Quantifying and mapping COVID-19 risk has been challenging from the start, because scientists and physicians have struggled to understand the complex range of symptoms associated with the disease. This has been exacerbated by national and regional policies that have limited people’s access to testing on the basis of symptoms that might not fully reflect the prevalence of infection. Online surveys are now helping researchers to get a better handle on the clinical manifestations of the disease. As part of the Coronavirus Pandemic Epidemiology (COPE) consortium, Andrew Chan, an epidemiologist at Massachusetts General Hospital in Boston, and his US and UK colleagues launched the COVID Symptom Study. This effort initially recruited hundreds of thousands of volunteers in the United States and United Kingdom who were already participating in clinical studies; it has subsequently been opened up to the public in those two countries. In parallel, a team led by computational biologist Eran Segal at the Weizmann Institute of Science in Rehovot, Israel, has deployed its own online symptom questionnaire in that country. “We built an algorithm that basically allows self-assessment testing that integrates all the symptoms you report, and tells you the likelihood of testing COVID-positive,” says Segal. Such surveys are constrained by their dependence on willing users, and by access to the Internet. To bolster participation, Segal’s team used a phone survey in the Israeli cities that were hit hardest by COVID-19. The data have revealed valuable insights that can distinguish the disease from colds and other common maladies. “These surveys contributed a lot to identifying loss of taste and smell as a symptom that is the most distinctive for COVID-19,” says Segal, noting that Israel has now changed its testing practices to recognize the diagnostic importance of this symptom. Symptom surveys are also informing maps of the pandemic. Segal says that initial data from his questionnaire suggested there were two to three times as many active COVID-19 cases in Israel as were represented in official clinical reports at the time. Signals generated from these surveys could, therefore, offer a real-time red flag for emerging hotspots where testing and personal protective equipment are needed, giving public-health authorities a valuable head start in responding. In the United Kingdom, the National Health Service is supporting further development of the COVID Symptom Study app, which has now also been deployed in Sweden. Chan is enthusiastic about additional data in the United States, as well. “We have been interested in aligning what we’ve been finding on the app with publicly available information on testing results or hospitalization rates,” he says. But equally enthusiastic, if not more so, is the public. Ranney says that MyCOVIDRisk has been used more than one million times since it launched, and the Georgia Tech team says that its tool had delivered more than 40 million risk estimates as of late November, with a particular surge around the US Thanksgiving holiday. And despite several vaccines entering the clinic, the public is likely to remain hungry for these data well into 2021. “There are going to remain some very high-risk activities for months or even a year to come,” says Ranney. “So I think this kind of informational and behavioural change-motivating programme will unfortunately remain relevant.” This article is reproduced with permission and was first published on December 21 2020.
Millions have struggled with the same questions throughout 2020. Scientists and clinicians are still learning about the epidemiology and pathology of the coronavirus SARS-CoV-2 and, for the public, it can be difficult to understand how to maintain some semblance of normality while minimizing the risk of infection.
As co-director of the Brown-Lifespan Center for Digital Health in Providence, Rhode Island, Ranney saw an opportunity to use her expertise in developing online tools for public health. Working with her Brown-Lifespan colleague Elizabeth Goldberg, she developed MyCOVIDRisk, a simple, questionnaire-based web app that gives users a sense of their infection risk on the basis of their planned activity in a specific place or situation.
Filling the gaps
Launched in October, MyCOVIDRisk is just one of several apps developed to communicate the risks of COVID-19 to the public (see ‘COVID risk calculators’). These tools are filling important gaps in public-health messaging, particularly in the United States, where an uneven response to the pandemic has exacerbated its severity. “This type of app is something that could have been developed very early in the pandemic by the [US] Centers for Disease Control and Prevention or another national entity,” says Ranney. “But unfortunately, that wasn’t done.”
Individual research teams have approached risk assessment from different angles. Perhaps the most straightforward approach is the web-based COVID-19 Event Risk Assessment Planning Tool developed by scientists at the Georgia Institute of Technology in Atlanta. The site estimates the probability (as a percentage) that a person will encounter someone with COVID-19 at a gathering, on the basis of the size of the group and where the event takes place. For example, as of 14 December, the tool predicted that at a gathering of 10 people in Boston, Massachusetts, there would be a 30% chance of at least one attendee having the virus.
The 19 and Me calculator, developed by Mathematica, a policy-research company in Princeton, New Jersey, draws on demographic and health information as well as user behaviours such as hand washing and mask use to determine relative risk of exposure, infection and serious illness. And in December, a team led by biostatistician Nilanjan Chatterjee at Johns Hopkins University in Baltimore, Maryland, released the COVID-19 Mortality Risk Calculator, which estimates an individual’s relative risk of death from COVID-19 on the basis of their location, pre-existing conditions and general health status.
MyCOVIDRisk takes a more situational approach, estimating the risks associated with specific errands or recreational activities. The estimate is based on the location and duration of a gathering, and the number of masked or unmasked people attending. This can help users to avoid activities that are likely to be high risk in a pandemic hotspot, such as spending an hour in an indoor gym, in favour of safer alternatives — a masked meet-up in the park, for instance.
To make predictions, these tools require the latest data and models that accurately capture the current levels of infection and the likelihood of transmission in various scenarios. The New York Times runs a coronavirus database that has become a popular resource, and both MyCOVIDRisk and the Georgia Tech tool use it to assess local disease activity. These calculators are currently focused on the United States, although the Georgia Tech team has extended its tool to cover ten European countries.
Researchers are still teasing out the factors that predispose some individuals to infection and severe disease, and they continue to study how distance, ventilation and masks mitigate the spread of virus-laden droplets. For 19 and Me, Xindi Hu — the data scientist at Mathematica who led the tool’s development — and her colleagues regularly comb through the latest peer-reviewed literature to refine their infection models, relying heavily on reports from the US Centers for Disease Control and Prevention (CDC). For MyCOVIDRisk, the team turned to an aerosol-transmission estimation model developed by atmospheric chemist Jose-Luis Jimenez and his colleagues at the University of Colorado Boulder, which simulates how far viral particles are likely to travel in various indoor and outdoor scenarios.
The researchers then have to come up with ways to wrangle this information into a format that is easy for non-experts to understand, without it being intrusive or frustrating. To build a simple and engaging app that balances depth with usability, the Brown-Lifespan team relied on user testing and guidance from behavioural scientists. “In an ideal world, you’d ask how big of a room are you in and what’s the rate of ventilation,” says Ranney, “But if you asked me, I wouldn’t even know the answer.”
This need for simplicity also motivated the Georgia Tech team. Its tool solicits only two pieces of information: a user’s location and the size of the expected crowd. Initially, this ranged from 10 to 10,000 people. “Those upper bounds were useful when political rallies were occurring over the summer,” says Andris. But as the holidays drew closer, the team decreased the upper bound to 5,000 people and increased the tool’s level of detail at the lower end of the scale to deliver estimates for relatively modest gatherings of 10, 15 or 20 people.
To convey risk, MyCOVIDRisk opts for a simple, 5-tier continuum scale of green to red, whereas the 19 and Me team devised an abstract numerical readout ranging from 0 to 100, which Hu likens to a credit score. “That is a compromise we’ve made to keep the rigour of the calculation while making it into a number that is easily understood by the general public,” she says. “And we couple it with a text explanation and an interactive risk gauge.” This explanation summarizes the potential for exposure and infection, along with the likelihood of serious health consequences and hospitalization if infection occurs.
However, developers caution that the apps provide estimates intended to assist in decision-making, and are not definitive indicators of the likelihood of contracting disease.
Casting a wider net
Quantifying and mapping COVID-19 risk has been challenging from the start, because scientists and physicians have struggled to understand the complex range of symptoms associated with the disease. This has been exacerbated by national and regional policies that have limited people’s access to testing on the basis of symptoms that might not fully reflect the prevalence of infection. Online surveys are now helping researchers to get a better handle on the clinical manifestations of the disease.
As part of the Coronavirus Pandemic Epidemiology (COPE) consortium, Andrew Chan, an epidemiologist at Massachusetts General Hospital in Boston, and his US and UK colleagues launched the COVID Symptom Study. This effort initially recruited hundreds of thousands of volunteers in the United States and United Kingdom who were already participating in clinical studies; it has subsequently been opened up to the public in those two countries. In parallel, a team led by computational biologist Eran Segal at the Weizmann Institute of Science in Rehovot, Israel, has deployed its own online symptom questionnaire in that country. “We built an algorithm that basically allows self-assessment testing that integrates all the symptoms you report, and tells you the likelihood of testing COVID-positive,” says Segal.
Such surveys are constrained by their dependence on willing users, and by access to the Internet. To bolster participation, Segal’s team used a phone survey in the Israeli cities that were hit hardest by COVID-19. The data have revealed valuable insights that can distinguish the disease from colds and other common maladies. “These surveys contributed a lot to identifying loss of taste and smell as a symptom that is the most distinctive for COVID-19,” says Segal, noting that Israel has now changed its testing practices to recognize the diagnostic importance of this symptom.
Symptom surveys are also informing maps of the pandemic. Segal says that initial data from his questionnaire suggested there were two to three times as many active COVID-19 cases in Israel as were represented in official clinical reports at the time. Signals generated from these surveys could, therefore, offer a real-time red flag for emerging hotspots where testing and personal protective equipment are needed, giving public-health authorities a valuable head start in responding.
In the United Kingdom, the National Health Service is supporting further development of the COVID Symptom Study app, which has now also been deployed in Sweden. Chan is enthusiastic about additional data in the United States, as well. “We have been interested in aligning what we’ve been finding on the app with publicly available information on testing results or hospitalization rates,” he says.
But equally enthusiastic, if not more so, is the public. Ranney says that MyCOVIDRisk has been used more than one million times since it launched, and the Georgia Tech team says that its tool had delivered more than 40 million risk estimates as of late November, with a particular surge around the US Thanksgiving holiday. And despite several vaccines entering the clinic, the public is likely to remain hungry for these data well into 2021. “There are going to remain some very high-risk activities for months or even a year to come,” says Ranney. “So I think this kind of informational and behavioural change-motivating programme will unfortunately remain relevant.”
This article is reproduced with permission and was first published on December 21 2020.
