The Frontier of Scientific Plausibility

As part of my fellowship at IFP, I’ve been embarking on a longer-term research project that seeks to produce a retrospective of the development and deployment of mRNA vaccines during the COVID pandemic, and extract lessons for the future about accelerating scientific and technological progress. As this effort gets underway, I’ll be posting reflections, thoughts, and other research notes from the project to Macroscience. This is the first of these posts. 

I recently had the pleasure to interview Chris Snyder, the Joel Z. and Susan Hyatt Professor in the Economics Department at Dartmouth College. Chris served on the team of economists including Susan Athey, Michael Kremer, and Alex Tabarrok that were early advocates for advance market commitments around the COVID vaccine, and helped advise the White House on its ultimate rollout. 

The interview was fascinating for many reasons, but one anecdote in particular stood out on the backdrop of the thinking I’ve been developing here in Macroscience. 

To inform the team’s recommendations, Chris and his colleagues conducted meetings with various experts in industry and academia during the darkest days of the pandemic. These conversations focused in part on the various possible alternatives that might exist in developing a viable vaccine for COVID, and the relative probability these alternatives would actually succeed in practice. These assessments would help inform a model for how the US government should ultimately allocate its dollars among these possibilities. 

In our interview, Chris casually dropped the following statement: when it came to the then-untested idea of doing mRNA vaccines, multiple scientists and industry insiders asserted that any modeling would be wildly miscalibrated if the probability of success was anything more than zero. In other words, the very idea that would come to provide a way out of the crisis and vaccinate millions worldwide was considered ex ante by career scientists a dead end, a red herring, a fantasy. 

There were sensible reasons for this assessment. For instance, the logistical nightmare implied by a vaccine that spoiled if not kept intensely cool at all times, and a previous history of poor results in developing DNA vaccines. But still, the idea that the option of mRNA vaccines could be staring us in the face and that we might nonetheless have missed it seems to me an insanely lucky historical near-miss.

For Chris, the reason economists did not ultimately assign a zero chance of success in their modeling was because they held a deeply ingrained assumption of the elasticity of the world. Simply put, you stick more money in, and more things come out. It might not be exactly the amount you wanted to come out, but the frontier of progress always has some flexibility at the edges. 

This is not to say that economists always know better than biochemical experts. Some things may be genuinely impossible from a scientific standpoint. But this incident highlights a crucial, underdiscussed variable of scientific progress: the perceived viability of new, untested alternatives in a scientific or technological field. For the domain experts Chris consulted, the frontier of scientific plausibility lay dangerously short of the frontier of scientific possibility. 

How much progress do we leave on the table because the frontier of plausibility is set suboptimally in this way, not just in biotechnology but across scientific fields? 

The market psychology of the frontier of plausibility is complex and context-dependent. Breakthroughs might have spillover effects, causing experts within a domain to be suddenly more optimistic about nascent and untested possibilities than in the past. Overhyped projects that end in spectacular failure may deliver a crushing blow to researcher confidence in a field, constraining the frontier of plausibility dramatically for years or decades. 

These intangible expectations can produce real, compounding effects. As with deflationary spirals in the real economy, pessimism about future opportunities can lead to a failure to invest, leading to falling growth that justifies these fears and reinforces the cycle. In the economy of science, it seems to me that pessimism about new research directions and alternatives may itself create stagnation, slowing progress and working to further shrink the frontier of plausibility. 

Stewardship of the frontier of plausibility seems to me an important cornerstone for a mature theory of the governance of science, and our institutions should accept it as one of their key responsibilities. Implicit in that is the hard question of defining under what conditions is it appropriate for policy and funding interventions to influence this frontier, and why. I’ll be wrestling with this issue in the next edition of Macroscience.

---

Thanks as always to Santi Ruiz for his comments and feedback on earlier drafts of this piece, and to Chris Snyder for sitting down with me for the interview.