SHAPING UP SCIENCE As a professor emeritus of genetics who spent many long hours writing grant proposals, I agree with “Rethink Funding,” by John P. A. Ioannidis [State of the World’s Science 2018]. The system is biased in favor of “politically savvy managers.” Yet Ioannidis does not address the overhead funds that line the coffers of universities. With state funding constantly dwindling, they rely on overhead more than ever. This is why academia favors big grant getters over innovative research. Reducing bloated academic administrations would be one modest way to solve the conundrum, but who is going to do that? PAUL F. LURQUIN Washington State University There is a danger that new ideas will be held back if attention is directed too narrowly on the precision of scientific methods. Such ideas often arise from the use of imprecise approaches. For example, single case studies in medicine, surveys showing correlations in my own field of consumer behavior, and odd observations in astronomy can all lead to major advances because they pick up serendipitous findings that are hard to anticipate. The new ideas that are generated are usually tested by experiments, but such tests often provide limited stimulus for new thinking. ROBERT EAST Emeritus professor, Kingston University London IOANNIDIS REPLIES: Lurquin points out the problem of large overheads, which have grown. Eliminating them is not easy, because one needs to find other sources for sustaining the infrastructure of research institutions. Unnecessary bureaucracy could be trimmed, of course. East advocates the support of imprecise exploratory methods when they fuel new, exciting ideas. Such research is justifiable and necessary when we have no other better tools for initial discovery. But it needs to be recognized explicitly as being exploratory and thus often likely to be wrong and in need of careful subsequent validation with better methods. REPLICATION TROUBLE As an academic researcher, I was not too surprised to learn that a large fraction of results in even the best journals cannot be reproduced in “Make Research Reproducible,” by Shannon Palus [State of the World’s Science 2018]. As reported in both Palus’s and Ioannidis’s articles, researchers have many institutional pressures and personal motivations to publish flashy results and none to replicate those of others. We must explicitly acknowledge, fund and motivate reproduction. It would help if journals had a section or associated publication accepting studies by independent authors seeking to reproduce works previously published by those journals. Their referees would not judge originality or interest but would value methodological rigor, clarity and, possibly, improvement or extension of the results. JOSE M. SOLER Autonomous University of Madrid I think Palus’s note that the original work discussed “appeared in a topflight journal,” whereas “the replication effort can be found in a comparatively smaller one” is perhaps her most important observation. What struck me was the high-handed way that some of these journals don’t stand by their product. That cheapens the worth of the publication. If you publish a paper, the reputation of the publication is behind that study from a marketing POV. If the paper is later refuted by, or can’t be replicated in, another study, you have a duty to publish the latter paper as well. This could be encouraged by an independent organization that simply records the number of times a counter paper was published in a different journal because the original publication refused it. NEIL ROBERTSON El Cerrito, Calif. LIMITED DECISIONS It was fascinating to learn in “The Unsolvable Problem,” by Toby S. Cubitt, David Pérez-García and Michael Wolf, that certain important questions in theoretical physics are undecidable by computation. In discussing the primary example of such a question, the authors assert that determining the existence, or not, of an energy gap between the lowest energy state of a material and the next state up would depend on the material extending to infinity. Yet in that case, presumably the material itself will be forever unable to decide whether it is gapped or gapless because any causal influence between distant regions can travel only at the speed of light. TONY DURHAM Brighton, England THE AUTHORS REPLY: Strictly speaking, any undecidable problem must have an infinity somewhere. If you impose any limit, even the lifetime of the universe, then it is decidable, although in practice, that is not much better than if it were not. In the case of the spectral gap problem, for any reasonably large, finite lattice size, the systems we construct will either be gapped or have an energy spectrum that is so dense, it becomes indistinguishable from gapless. In principle, if you limit how large the lattice can get (say, it needs to fit in your lab!), then the problem is decidable. But the undecidability of an idealized infinite lattice implies there is no better way to figure it out than taking a sample of material the size of your lab; a smaller sample will tell you nothing about the lab-sized one. Worse still, even if you determine whether the lab-sized sample is gapped or gapless, this could change just by adding a single extra atom. It’s important to emphasize that no materials anyone has encountered in reality display this perverse behavior. But we can look for simpler systems that exhibit similar physics, and we have made some progress on doing so in a follow-up paper. Durham’s scenario is somewhat similar to what we describe: In principle, given infinite time, the speed of light is no obstacle. A time limit would be qualitatively similar to imposing a finite size limit, equal to time multiplied by the speed of light. DOWN UNDER DEVELOPMENT In “Body Balance” [Advances], Maya Miller reports that developmental biologist Alberto Roselló-Díez and his colleagues found that when they suppressed growth of a limb in a mouse fetus, the surrounding cells communicated with the placenta, which slowed down the growth of the other three limbs to keep them symmetrical. This mechanism for maintaining symmetry in development would, however, work only with placental mammals. How would marsupials manage this coordination? DAVID WEINTRAUB Edison, N.J. ROSELLÓ-DÍEZ REPLIES: It is worth noting that even though they lack a true placenta, marsupials do form a yolk-sac-derived placentalike structure. And whereas the most obvious mechanism we found involves the placenta, this does not mean it is the only one. It is possible that other organs with a key role in body growth, such as the liver, also participate in the systemic response triggered by a local injury. They could do so either in parallel to the placenta or at subsequent (postnatal) stages once the placenta is no longer present. ERRATUM “The Unsolvable Problem,” by Toby S. Cubitt, David Pérez-García and Michael Wolf, should have worded a mathematical statement about deriving the number 1 from any whole number in this way: “If you take any whole number and divide it by 2 if it’s even or multiply it by 3 and add 1 if it’s odd, and then repeat the process, you always eventually reach the number 1.”
As a professor emeritus of genetics who spent many long hours writing grant proposals, I agree with “Rethink Funding,” by John P. A. Ioannidis [State of the World’s Science 2018]. The system is biased in favor of “politically savvy managers.”
Yet Ioannidis does not address the overhead funds that line the coffers of universities. With state funding constantly dwindling, they rely on overhead more than ever. This is why academia favors big grant getters over innovative research. Reducing bloated academic administrations would be one modest way to solve the conundrum, but who is going to do that?
PAUL F. LURQUIN Washington State University
There is a danger that new ideas will be held back if attention is directed too narrowly on the precision of scientific methods. Such ideas often arise from the use of imprecise approaches. For example, single case studies in medicine, surveys showing correlations in my own field of consumer behavior, and odd observations in astronomy can all lead to major advances because they pick up serendipitous findings that are hard to anticipate. The new ideas that are generated are usually tested by experiments, but such tests often provide limited stimulus for new thinking.
ROBERT EAST Emeritus professor, Kingston University London
IOANNIDIS REPLIES: Lurquin points out the problem of large overheads, which have grown. Eliminating them is not easy, because one needs to find other sources for sustaining the infrastructure of research institutions. Unnecessary bureaucracy could be trimmed, of course.
East advocates the support of imprecise exploratory methods when they fuel new, exciting ideas. Such research is justifiable and necessary when we have no other better tools for initial discovery. But it needs to be recognized explicitly as being exploratory and thus often likely to be wrong and in need of careful subsequent validation with better methods.
REPLICATION TROUBLE
As an academic researcher, I was not too surprised to learn that a large fraction of results in even the best journals cannot be reproduced in “Make Research Reproducible,” by Shannon Palus [State of the World’s Science 2018]. As reported in both Palus’s and Ioannidis’s articles, researchers have many institutional pressures and personal motivations to publish flashy results and none to replicate those of others.
We must explicitly acknowledge, fund and motivate reproduction. It would help if journals had a section or associated publication accepting studies by independent authors seeking to reproduce works previously published by those journals. Their referees would not judge originality or interest but would value methodological rigor, clarity and, possibly, improvement or extension of the results.
JOSE M. SOLER Autonomous University of Madrid
I think Palus’s note that the original work discussed “appeared in a topflight journal,” whereas “the replication effort can be found in a comparatively smaller one” is perhaps her most important observation.
What struck me was the high-handed way that some of these journals don’t stand by their product. That cheapens the worth of the publication. If you publish a paper, the reputation of the publication is behind that study from a marketing POV. If the paper is later refuted by, or can’t be replicated in, another study, you have a duty to publish the latter paper as well. This could be encouraged by an independent organization that simply records the number of times a counter paper was published in a different journal because the original publication refused it.
NEIL ROBERTSON El Cerrito, Calif.
LIMITED DECISIONS
It was fascinating to learn in “The Unsolvable Problem,” by Toby S. Cubitt, David Pérez-García and Michael Wolf, that certain important questions in theoretical physics are undecidable by computation.
In discussing the primary example of such a question, the authors assert that determining the existence, or not, of an energy gap between the lowest energy state of a material and the next state up would depend on the material extending to infinity. Yet in that case, presumably the material itself will be forever unable to decide whether it is gapped or gapless because any causal influence between distant regions can travel only at the speed of light.
TONY DURHAM Brighton, England
THE AUTHORS REPLY: Strictly speaking, any undecidable problem must have an infinity somewhere. If you impose any limit, even the lifetime of the universe, then it is decidable, although in practice, that is not much better than if it were not.
In the case of the spectral gap problem, for any reasonably large, finite lattice size, the systems we construct will either be gapped or have an energy spectrum that is so dense, it becomes indistinguishable from gapless. In principle, if you limit how large the lattice can get (say, it needs to fit in your lab!), then the problem is decidable. But the undecidability of an idealized infinite lattice implies there is no better way to figure it out than taking a sample of material the size of your lab; a smaller sample will tell you nothing about the lab-sized one. Worse still, even if you determine whether the lab-sized sample is gapped or gapless, this could change just by adding a single extra atom.
It’s important to emphasize that no materials anyone has encountered in reality display this perverse behavior. But we can look for simpler systems that exhibit similar physics, and we have made some progress on doing so in a follow-up paper.
Durham’s scenario is somewhat similar to what we describe: In principle, given infinite time, the speed of light is no obstacle. A time limit would be qualitatively similar to imposing a finite size limit, equal to time multiplied by the speed of light.
DOWN UNDER DEVELOPMENT
In “Body Balance” [Advances], Maya Miller reports that developmental biologist Alberto Roselló-Díez and his colleagues found that when they suppressed growth of a limb in a mouse fetus, the surrounding cells communicated with the placenta, which slowed down the growth of the other three limbs to keep them symmetrical.
This mechanism for maintaining symmetry in development would, however, work only with placental mammals. How would marsupials manage this coordination?
DAVID WEINTRAUB Edison, N.J.
ROSELLÓ-DÍEZ REPLIES: It is worth noting that even though they lack a true placenta, marsupials do form a yolk-sac-derived placentalike structure. And whereas the most obvious mechanism we found involves the placenta, this does not mean it is the only one. It is possible that other organs with a key role in body growth, such as the liver, also participate in the systemic response triggered by a local injury. They could do so either in parallel to the placenta or at subsequent (postnatal) stages once the placenta is no longer present.
ERRATUM
“The Unsolvable Problem,” by Toby S. Cubitt, David Pérez-García and Michael Wolf, should have worded a mathematical statement about deriving the number 1 from any whole number in this way: “If you take any whole number and divide it by 2 if it’s even or multiply it by 3 and add 1 if it’s odd, and then repeat the process, you always eventually reach the number 1.”
