Since 1945, the rate of disruptive scientific papers and patents is rapidly declining. This is the conclusion of a recent study, which analyzed 45 million papers and 3.9 million patents.

Does this mean that progress in science and innovation is slowing down?

Interestingly, while the volume of papers and patents is steadily increasing, the sheer number of disruptive contributions remained very stable over the decades. Indeed, the COVID-19 vaccines provided a stark example of how world-changing scientific developments can come true in record time. But the fraction of disruptive papers and patents is getting smaller and smaller.

“That suggests that we’re just paying for too much,” said Diana Hicks, a professor of public policy at Georgia Institute of Technology who was not involved in the study.

Standing on the shoulders of giants

Is disruption a positive trait? As co-author Erin Leahey from the University of Arizona explained, a paper is disruptive if subsequent researchers cite it, but ignore the foundational work on which it was built. “You stood on the shoulders of giants to produce that work, but now you’re contributing to their obsolescence,” she said.

Still, at the other end of the spectrum, highly consolidating papers are “deeply integrated and engaged in a knowledge stream,” she added. That is, work that cites that paper also cites the prior work that fed into it, which is not a bad thing, according to Leahey.

So, the distinction between disruption and consolidation is not about the amount of impact, but the nature of it.

Indeed, the declining rate of disruption is not due to the diminishing quality of the work, the data from the study shows, as disruption is decreasing even among papers published in top journals and among Nobel Prize-winning works.

Instead, a common criticism of modern science is the pressure to “publish or perish”, which many believe is contributing to the trend. But the evidence for this is still inconclusive.

Another possible factor could be the tendency of funding agencies to put out very targeted calls for research projects. “Is that steering not just the type of work or the topics that people are working on but, I don’t know, making more than the typical number of people kind of line up behind that area?” Leahey asked, adding that such a consequence might make a disruptive discovery less likely. Again, the data is not conclusive.

Narrower slices of knowledge

What the data does show is that the diversity of works cited has decreased over time. The volume of available research papers is going up, but scientists and inventors are using the same knowledge base over and over, and their scope is narrower than it used to be. As the authors wrote in the paper, “Relying on narrower slices of knowledge benefits individual careers, but not scientific progress more generally”.

Information overload may be a factor behind this trend, and Leahey points out that the change in the way we obtain that information might also be playing a role. “When I was in graduate school, I went to the library,” she recalls, and browsed the social science section as a whole.

But now, she, like many researchers, rely on feeds from search engines to peruse the literature, whose underlying algorithms sway results significantly. “It’s not just […] there’s so much out there that we don’t have enough time to read it all, it’s that maybe we are being slightly steered or the work that is brought to our attention is really within our niche,” she said.

Other studies have found that social media favors the formation of echo chambers, i.e., groups of like-minded users reinforcing a shared narrative. Could something similar be happening in research and innovation – and could social media be a factor? At this point, this question remains open-ended.

Can AI help?

In any case, if science and innovation are indeed slowing down, the consequences could be far-reaching. “Then the problem is that economic growth is going to slow down,” Hicks said. “The whole capitalist system is premised on economic growth [and] a large part of that over time has come from innovations, which, you know, come from technological and scientific advance.”

Can this trend be halted or even reverted?

Proposals are not lacking. As Hicks explained, the OECD ran a workshop in 2021 to explore whether artificial intelligence could help science become more productive. The premise was that, perhaps, technology could be instrumental to overcoming the increased complexity that challenges modern science.

But even more modest interventions might make a big difference. Leahey points out that the multiple administrative and other duties that surround research in academic institutions are probably getting in the way of researchers broadening their scope of works read.

“I just want to, you know, sit down on the couch and read a book, and when I do that, I feel like, lazy and guilty, like I’m not doing my job – I’m like no, this is very much my job!,” she said.

Reference: Michael Park et al., Papers and patents are becoming less disruptive over time, Nature (2023). DOI: 10.1038/s41586-022-05543-x