The half-life of facts

By R.D.A. | LOS ANGELES

IN PRIMARY school Babbage learned that there were nine planets in the solar system. None were known to exist outside it. Since then, astronomers have spotted over 800 planets around other stars (and thousands more "candidates") and demoted Pluto to a mere "dwarf planet". Even a cursory glance at other fields reveals similar patterns.

Samuel Arbesman, a mathematician at Harvard, calls this "The Half-life of Facts", the title of his new book. In it he explains that this churn of knowledge is like radioactive decay: you cannot predict which individual fact is going to succumb to it, but you can know how long it takes for half the facts in a discipline to become obsolete. Such quantitative analysis of science has become known as scientometrics. We talked to Dr Arbesman about how knowledge changes over time, and what this means for the way people consume information.

What is scientometrics?

Put simply, scientometrics is the science of science. It grew out of bibliometrics, the science of books and research papers. In bibliometrics the unit of measurement is a research paper, which are easy to study because you can quantify different aspects of it: who the authors are, who has co-authored papers with those authors, how often a paper is cited, by whom, and so on.

Librarians were some of the first people to do this. In the 1970s people started looking around and noticing that scientific knowledge was growing very rapidly, but papers had not been digitised yet, and libraries were finite in size and had finite resources. And so librarians had to grapple with the question what to carry on their shelves. They had to calculate which fields get overturned really rapidly, in other words, which papers and books people were unlikely to care about in the future.

But bibliometrics is only one subfield of scientometrics. There are all kinds of ways that you can quantify science: you can measure the number of discoveries that are occurring within a particular field, the number of elements in the periodic table, etc. Broadly, scientometrics is about quantifying and understanding how science occurs.

That includes both the social aspects of science and the relationship between science and technology. There is a tight interplay between the capacities of our tools and what we can actually discover. Technology is crucial to the story of science. Science of science is about all these different things. And my book is about how the facts of the world—the stuff we know—grow in number, and how they change.

What does it mean to say that a fact has a half-life?

When I say that a fact has a half-life, I am trying to illustrate how knowledge changes by making an analogy to radioactivity. With radioactivity, if you give me a single atom of uranium, I can tell you it will eventually decay. When it does, it will break down into specific bits and release a certain amount of energy. But I have no way of telling when it is going to decay. It could be in the next half-second or not for millions and millions of years.

But things change when you go from a single atom to lots of atoms. When you have a big chunk of uranium, you can graph out the decay; you can say it takes 4.47 billion years for half of the atoms in a chunk of uranium to break down. You aren't going to know which half, but you know the overall rate of the decay. And the same thing is true for science, and for knowledge in general. Even though I cannot predict what discovery is going to be made or what fact is going to be overturned, there are regularities in how knowledge grows and changes over time.

For example, in the area of medical science dealing with hepatitis and cirrhosis, two liver diseases, researchers actually measured how long it takes for half of the knowledge in these fields to be overturned. They gave a whole bunch of research papers from fifty years ago to a panel of experts and asked them which were still regarded as true and which had been refuted or no longer considered interesting. They plotted this on a graph. What they found is that there is a nice, smooth rate of decay; you can predict that every 45 years, half of this particular sort of knowledge gets outdated.

You can use these same methods with citations in newer papers. There, you look to see how long papers are cited in a field and then derive a half-life based on how long it takes for papers to receive half the citations they used to receive. Of course, some papers are no longer cited precisely because they are so influential. No one is citing Newton's Principia even though we still use a lot of his ideas. But by and large, the citation rate of papers is a good proxy for the half-life of knowledge.

What scientific fields decay the slowest—or the fastest—and what drives that difference?

Well it depends, because these rates tend to change over time. For example, when medicine transitioned from an art to a science, its half-life was much more rapid than it is now. That said, medicine still has a very short half-life; in fact it is one of the areas where knowledge changes the fastest. One of the slowest is mathematics, because when you prove something in mathematics it is pretty much a settled matter unless someone finds an error in one of your proofs.

One thing we have seen is that the social sciences have a much faster rate of decay than the physical sciences, because in the social sciences there is a lot more "noise" at the experimental level. For instance, in physics, if you want to understand the arc of a parabola, you shoot a cannon 100 times and see where the cannonballs land. And when you do that, you are likely to find a really nice cluster around a single location. But if you are making measurements that have to do with people, things are a lot messier, because people respond to a lot of different things, and that means the effect sizes are going to be smaller.

What is a "fact phase transition" and how does it make events like the first Moon landing predictable?

First, here is what I mean by a phase transition. An example in the natural world is when water goes from liquid to ice when it freezes. For most people that is pretty unremarkable. But it is actually really interesting when you look at it from a physics perspective. A continuous change—in this case, a change in temperature—is accompanied by a step-change is other properties: water going from being a liquid to a crystal. This is a good way to think of rapid changes in knowledge.

Some of these happen rapidly, but underneath there are these gradual changes. For example, with the moon landing was a pretty big change in human knowledge and human accomplishment. For all of human history, no one had ever set foot on the moon, and then one day in 1969 people had. But if you look carefully you will see that the moon landing was completely predictable. Look at the fastest speeds enabled by technology, for instance, and it turns out that they follow a regular curve. In the 1950s the American air force graphed this out and determined that if transportation speeds continued rising at the rate they were going, humans should be able to get into orbit, and then eventually land on the Moon, within a set number of years. And, sure enough, right on schedule, Sputnik happened, and a decade later humans landed on the moon. That was a fact phase transition, an abrupt change with slow incremental processes hiding beneath the surface.

One theoretical fact phase transition that you describe is "actuarial escape velocity", a concept borrowed from medical science.

Actuarial escape velocity is the idea that at some point average human lifetime will grow by more than a year each year. Right now the rate is only a fraction of a year (thanks to changes in medical science and hygiene) a year. If it exceeds one year per year, people will effectively live for ever, without having to solve the immortality problem. The reason I bring it up in my book is to illustrate that small changes in science can actually bring about big changes in other areas of knowledge, or elsewhere in the world.

For example, if an astronomer finds another planet outside solar system, unless it has certain properties, it will just be another piece of data. It is not going to alter the structure of people's ideas about planets. But if he discovers a planet that can harbour life, that is a game-changer. And actuarial escape velocity is similar, in the sense that these incremental changes in medical science and hygiene can eventually create a huge change in how we live our lives.

In your book, you make a convincing case that scientific breakthroughs are becoming more difficult to achieve with time. One gets the sense that the low-hanging fruit of empiricism have been picked. But you also argue that science as a human activity is growing, and getting better. How is that?

In some fields science is getting harder, but I would not say that science as a whole is becoming more difficult. We are still adding new scientists every year, but the rate of growth has slowed and science is increasingly being done by large teams. But there are many areas where we thought there is nothing left to explore, only for someone to come along and say that there is something there, after all.

In mathematics there was an extreme case of this in the 1990s, when two high-school students figured out a new way to prove one of Euclid's theorems, something that had not been done in a thousand years. So even though basic geometric proofs are not the frontier of mathematics, there are still things you can do. And even where things slow down in science, often that slowing forces scientists to be cleverer, both in finding ways to create new knowledge but also in finding new ways to combine disciplines. Plus nowadays new technology is a real driving force; the new computational tools have created the potential for a scientific revolution.

Reading your book it is difficult not to think about consilience, the term that Edward Wilson uses to describe an idealised unity of all scientific knowledge. Do you think scientometrics can get us to something like consilience faster than if science were merely left to its own devices?

There is a great deal of power in the idea of consilience, and in synthesising ideas. When it comes to understanding the march of knowledge, scientometrics can be very helpful. I don't think it is necessarily going to help us realise the complete synthesis of all knowledge, but if we have a better sense of how we know what we know, and how what we know changes, that will force a reckoning in how we think about how knowledge as a whole is organised. If you create a networked view of different scientific fields, you quickly realize how connected they are. There are surprisingly few steps from thinking about abstract mathematics to thinking about models of how population size changes in an ecosystem. As science grows and becomes more and more complicated, having people that can exist in these liminal spaces is going to be increasingly important.

It seems that one of your purposes in writing this book is to call attention to the human habit of becoming accustomed to whatever state of affairs is true when a situation is initially examined. By showing how knowledge about the world shifts systematically, you seem to be suggesting a renewed vigilance against growing complacency about knowledge of the world.

That is certainly one of my arguments. I want to show people how knowledge changes. But at the same time I want to say, now that you know how knowledge changes, you have to be on guard, so you are not shocked when your children coming home to tell you that dinosaurs have feathers. You have to look things up more often and recognise that most of the stuff you learned when you were younger is not at the cutting edge. We are coming a lot closer to a true understanding of the world; we know a lot more about the universe than we did even just a few decades ago. It is not the case that just because knowledge is constantly being overturned we do not know anything. But too often, we fail to acknowledge change.

Some fields are starting to recognise this. Medicine, for example, has got really good at encouraging its practitioners to stay current. A lot of medical students are taught that everything they learn is going to be obsolete soon after they graduate. There is even a website called "up to date" that constantly updates medical textbooks. In that sense we could all stand to learn from medicine; we constantly have to make an effort to explore the world anew—even if that means just looking at Wikipedia more often. And I am not just talking about dinosaurs and outer space. You see this same phenomenon with knowledge about nutrition or childcare—the stuff that has to do with how we live our lives.