We were happy to see this inclusion, as their description of metascience units is similar to what we proposed in a response to an OSTP Request for Information in 2025.

As is often the case, though, the devil is in the details (and the implementation). Metascience units will only pay off if set up intelligently, pointed at an agency’s most important questions, and given the power to generate decision-relevant evidence.

So we wrote a piece for the IFP website explaining our vision for metascience units: why we should create them, what they should do, and how they should be organized. We have never cross-posted with the main IFP website before, but we think this article is particularly important given interest from the White House and beyond.

Summary

The US government is the largest funder of basic research in the US and the second largest funder of applied research after industry. Despite this predominant role, federal science agencies have limited information on how well that funding is being spent and how to spend it better.

Ideas about how to improve science funding abound, from shortening application timelines with fast grants to uplifting reviewers with AI tools. But most ideas lack strong evidence of efficacy, especially at scale or within the constraints of government. To find out how these ideas perform in practice, agencies need to be able to run experiments within their own walls, applying the scientific method to science itself.

Science agencies can accomplish this by creating metascience units: nimble teams of government staff and academic experts housed within agency leadership offices. Using the tools of metascience, these units would run institutional experiments and analyze the treasure troves of internal agency data to provide evidence about how to most effectively advance breakthroughs.

These experiments and analyses would allow agency leaders to identify and quickly scale up improvements to program design. Metascience units could lead to agencies funding higher quality projects, cutting review timelines, and closing gaps in the scientific ecosystem.

Subscribe now

Motivation

The federal government spends nearly $200 billion a year on research and development. If we could make the process of identifying, distributing, and managing those funds even 5% more effective, we would effectively unlock another $10 billion a year for innovations that benefit Americans. For context, that is slightly more than what we spend annually on NASA’s entire science budget or on the National Cancer Institute.

The federal scientific enterprise is vast and decentralized. This allows agencies to develop a diversity of approaches to funding and conducting science, from small grants and innovation prizes to ARPAs and National Labs. But knowing which approach will work best to further an agency’s mission is not a solved problem. There is much we do not know about where scientific progress comes from.

When agencies make decisions about how they structure, fund and conduct science, they typically lack high-quality evidence. What federal agencies do know about the efficacy and efficiency of their science investments comes from ad hoc external program evaluations, idiosyncratic internal analyses, or academic studies with limited, largely historical data. Often, these products are neither timely nor focused enough to influence program decisions. At the same time, they don’t make sufficient use of the agencies’ internal data, even though that data could provide some of the clearest signals about program strengths and weaknesses.

Inefficiencies in federally funded science are well-documented. These include slowness in funding grantees, an overwhelming volume of paperwork for academics, limited funding for early career scientists who might have the most disruptive ideas, risk aversion in peer review, and difficulty in coordinating within and across agencies.

We know the problems, but we need evidence on how to solve them. The best way to improve science investments would be for agencies themselves to conduct experiments, analyze their treasure troves of data, and scale approaches backed by their results. This will not happen organically. Existing agency staff do not have the mandate or bandwidth to do such research — especially with considerable staffing shortages in science agencies — and external researchers and consultants lack internal data and institutional knowledge. We need dedicated, empowered spaces within agencies to do this kind of work.

Subscribe now

Solution

Establish metascience units within federal science agencies

To generate the decision-relevant evidence and analysis they lack, science agencies should establish metascience units: proactive, nimble teams akin to internal think tanks. Units would analyze existing programs, conduct internal research, and develop better ways to fund and conduct science. Above all, the units should be anchored around answering the question: how should the agency invest its resources — whether budget, staff time, or the time of the reviewer community — to most effectively advance breakthroughs and further its mission?

While new to the US, the idea has found footing in the UK, which launched its own Metascience Unit in 2024 to conduct experiments and share insights across the country’s science ecosystem. Since then, the UK Unit has seen early successes, including a pilot that sped up grant application decisions by three months.

Since we advocated for the establishment of metascience units last year, the idea has also picked up steam across the federal government. The National Science Foundation (NSF)’s Fiscal Year 2027 Budget Request called for a metascience unit, and the White House Office of Science and Technology Policy (OSTP) Report Science: A New Golden Age made the same recommendation across science agencies more broadly. The National Institutes of Health (NIH) is in the process of launching the Office of Research Economics, Planning, and Analysis (OREPA), which will conduct economic research and facilitate replications.

Metascience units should generate evidence about improvements to program design

There are numerous possible research questions about how to structure, fund, and conduct science. But metascience units should start with questions they are best suited to answer with the levers they uniquely have: administrative data and prospective experimentation. This suggests the units should focus on post-hoc analyses, pilots, and controlled experimentation to improve funding mechanisms and award evaluation processes.

Problems should also be relevant to program design. They should therefore be sourced from divisions and programs that would actually implement them, and agency leadership can prioritize metascience experiments that provide the most decision-relevant information.

Improvements that a metascience unit could research include:1

• Fast grants, to move from application to funding in weeks rather than months (or years). The NSF effectively implemented fast grants during COVID-19, and other research funders have honed similar models. Metascience units should test how fast grants perform relative to conventional grants, using tools like cost-benefit analysis. They can gather evidence on when agencies should be using fast grants versus generally speeding up the conventional grant process.

• Multi-stage review, to address the challenge of applicants and reviewers spending substantial amounts of time on applications that will not be funded. Multi-stage review requires applicants to submit a brief concept note to start, and then only invited applicants submit full applications. This approach is already used in some agencies, but a 2025 National Academies of Sciences, Engineering, and Medicine report proposed expanding it government-wide. While this is a promising direction, metascience units should conduct experiments that compare multi-stage review to conventional processes to see how it impacts both efficiency and the quality of research.

• Golden Tickets, to address the problem of conventional review panels often rewarding consensus and punishing novel ideas. Golden tickets allow individual panelists to champion applications for funding even if other reviewers rated them poorly, potentially encouraging high-risk, high-reward projects. Metascience units should conduct experiments to compare golden tickets to more conventional panel review approaches. They can also use historical data to simulate what applications might have won if a panelist who was alone in giving a high score had gotten their way. A related approach would be testing the effectiveness of instructing reviewers to consider the benefits of higher-risk projects (as NIH has instructed panels to do) or encouraging reviewers to reward novelty.

• Person-based grants, to learn whether betting on researchers’ longer-term agendas rather than specific proposals yields better results. Science agencies already fund some person-based grants at research institutions, such as NIH’s R35 Investigator-Focused Awards. Evidence suggests that person-based grants can free scientists to conduct high-impact research rather than focusing on grant writing and reporting. Person-based grants should be compared against standard project-based grants.

• Longer grant durations or higher award values, to test whether giving researchers more time or money leads to more ambitious and creative work. New program types, like NSF’s recently-launched X-Labs program, can combine flexibility with award sizes that encourage scientists to take on ambitious projects that require sophisticated infrastructure and multidisciplinary teams.2 We think this will enable more high-impact research. Before expanding these pilots, metascience units should carefully analyze them to understand what’s working.

• New AI-enabled tools, to identify the most appropriate ways to leverage the technology for funding decisions. Metascience units are well-suited to measure whether AI input could improve reviewer accuracy through controlled “uplift” studies. Uplift studies are used to compare how people perform tasks with and without access to an AI tool, and they can help metascience units assess whether the tool provides meaningful assistance to agency staff. Promising applications include AI-assisted proposal scoring, automated extraction of proposal characteristics for portfolio analysis, and reviewer matching based on AI network analysis. UK Research and Innovation, where the UK Metascience Unit is housed, has already begun studying AI as a supplemental tool in grant proposal review. This would be best approached as a test environment where metascience units iterate on AI applications faster than an agency’s usual processes allow, building on the agency’s past data and grounding the applications in meaningful correlates of scientific impact.

These pilots and controlled experiments test alternatives directly. Alongside those activities, post-hoc analysis of how existing programs have actually performed are also useful. These efforts can reinforce each other. Retrospective analysis of existing data may be one of the cheapest ways to identify which questions are worth the cost of an experiment.

Subscribe now

Metascience units should make experimentation and internal data analysis easy

Metascience units can overcome the current difficulty of experimenting with new approaches and analyzing internal data. They can do this at a high level by making agency-level decisions about questions to tackle, cultivating talent in-house, shortening feedback loops between generating evidence and making decisions, and tapping into the institutional knowledge of agency staff in a way that external researchers or consultants could not.

But more importantly, metascience units should build the infrastructure for experimentation and internal data analysis as a core part of their mission.

To reduce the operational burden of running experiments, units can work with enterprise systems management and Chief Information Officers to build testing features into proposal management systems. Every improvement to this infrastructure lowers the cost of all following experiments.

Without careful attention, experiments could become burdensome for programs to partake in. Metascience units should build light-touch experimentation into default systems and provide program offices with incentives to participate. In some randomized controlled trials of new approaches, metascience units could provide additional funding for control groups, so programs don’t have to sacrifice portions of their budgets to generate evidence.

For internal data, the metascience unit should coordinate with system owners to capture and structure the data that makes evidence building possible. Much of the unit’s early value will be in organizing administrative data on grant applications and selection processes, especially non-winning applications, that is largely inaccessible to external researchers.

But the unit should also go beyond traditional output measures like citations and patents, tracking a wider range of outcomes: measures of scientific breakthroughs, grant turnaround times, staff time per award, and other process costs. Centralizing data on turnaround times and reviewer feedback quality in particular would give science agencies a clearer picture of how their processes perform over time.

Subscribe now

Implementation

Metascience units will only pay off if enabled to answer a science agency’s most important questions. Without sufficient staffing, support for cross-program experimentation, or the determination to scale the approaches that work, the promise of a metascience unit may end up unrealized.

Rather than being buried in bureaucracy, metascience units should be housed in an agency leadership office and entrusted with the authority to mandate pilots or experiments across agency programs. This ensures alignment with agency-wide goals, protects against capture by any one program, and provides a clear home for experimentation, rather than having the unit serve an advisory role.

To execute on the science agency’s metascience vision, leaders should prioritize recruiting talent to staff the offices and building strong relationships with stakeholders. These units should be staffed with a mix of civil servants and rotational experts like academic economists brought in via the Intergovernment Personnel Act (IPA). Career roles will hold institutional memory, while shorter rotational roles can bring in new ideas from the field.3 These staff must pay attention to the unit’s relationship with the metascience community and not assume it naturally maintains itself.

Metascience units should not function as traditional grantmaking or program evaluation offices

Academic metascientists have contributed strong research on some of the questions outlined here, and agencies should continue to support these researchers with external grants. But metascience units should not become grants or contracting offices for metascience. This would distract from their distinct comparative advantage: conducting internal analysis and supporting decision making.

For example, the programs on Science of Science & Innovation Policy (SciSIP) and Science of Science: Discovery, Communication and Impact (SoS:DCI) at NSF have funded foundational metascience research. But these sorts of programs do not focus on NSF’s own performance, nor should they. Tasking a metascience unit with external science-of-science grantmaking would distract from its goal, forcing the unit to develop extensive grant and contract management capabilities in addition to its core metascience capabilities. The metascience unit should maintain awareness of the external literature to inform the experiments it runs, but it should not be primarily responsible for the oversight of those awards.

To develop actionable research, agencies need proactive and nimble analysis directed at decision support. The evaluation offices that responded to the Evidence Act pushed agencies to make better use of their data, but did not produce the actionable evidence needed to make decisions. Evaluation offices often included a mix of managing traditional program evaluation activities (often conducted by third party contractors), reporting to Congress and regulatory agencies, and requesting and organizing data from offices across their agency. Unfortunately, in many cases they were overly burdened with compliance exercises and underdeveloped in key capability areas, which complicated their success. Metascience units must learn from those lessons. Agencies should resist the urge to combine metascience units with these and other program performance oversight functions. These units should be entrepreneurial and forward-looking, not compliance and backward-looking.

Subscribe now

Which agencies should have metascience units?

Not all science agencies need a metascience unit. To begin with, metascience units should be placed in the agencies that are the largest science funders. The NSF’s Fiscal Year 2027 Budget Request includes funding for a metascience unit, and the NIH is in the process of standing up OREPA to perform some functions of a metascience unit. The NIH has substantial internal capacity already, including a science of science scholars program, but due to the complexity of NIH’s 27 Institute and Center structure, OREPA could still unlock more effective cross-agency experimentation if given sufficient authority.

Additional agencies that should consider them include the Department of Energy (DOE), Department of War (DOW), and NASA. Given the prominence of Genesis Mission and massive ongoing infrastructure investments in the National Labs, DOE has unique metascience questions that won’t be answered by other agencies. DOW and NASA invest in diverse and complex R&D programs using a wide variety of innovative financing approaches, and both agencies could focus a metascience unit’s attention on industry-specific, translational questions.

Setting up metascience units for long-term success

Metascience units will likely produce decision-relevant evidence faster than external academics, and the experiments we suggested above could inform program design now. But the social return on scientific research is realized in long timelines (often over a decade or more), and some metascience unit projects will take time to pay off, likely crossing presidential administrations.

Those long-horizon payoffs require not treating these units as disposable, short-term efforts. Instead, metascience units should receive stable, sustained investment even as individual experiments resolve quickly. The White House and Congress have important roles to play to that end.

The White House has been supportive of metascience units, as exemplified by the Science: A New Golden Age report. To ensure the units’ success, they should support agency leadership in staffing and funding requests for metascience units, as well as providing flexibility and recognizing that the units may need to shift focus.

Metascience units provide benefits regardless of which party is in power by providing insights on science agency effectiveness and opportunities for improvement. Congress and the White House should therefore ensure that metascience units remain apolitical, truth-seeking offices and, to that end, support the sharing of findings publicly. A coordination body, perhaps convened by OSTP as a National Science and Technology Council subcommittee, could help in coordinating between metascience units, facilitating information sharing, and encouraging the external publication of results.

It’s also important to remember that metascience units themselves are a new institution in science — and are an experiment. As such, they too should be evaluated and improved based on evidence. And if they succeed in delivering more efficient, higher impact science, metascience units could be their own kind of breakthrough.

We just wrapped the third year of the Economics of Ideas, Science, and Innovation online short course. Targeted at economics PhD students, the course features many of the luminaries of innovation economics. This time around, we recorded the lectures and will be releasing one every other week to Macroscience subscribers (for free!), starting today. Regular Macroscience essays will continue on alternate weeks.

The reason we’ve held three iterations of this course and are publishing the recordings is that we believe more people should study the economics of innovation. Let me suggest five reasons.

Subscribe now

1. Impact

The best way to increase human flourishing in the long run is via technological progress, the speed of which is in part determined by social systems. The economics of innovation is the study of the social systems that drive technological progress — markets, governments, non-profits, universities, etc. There is no reason to think these systems already operate as well as they could, so better understanding could help us design better ones. Because ideas and discoveries are public goods and technological progress is cumulative, even small increases in the rate of discovery have enormous long-run effects on human welfare.

2. Government Demand

We need more people who understand this material. Scientific expertise is in short supply in some parts of government (a small share of legislators have STEM backgrounds), and this is especially true for social science research about science and innovation. While the US government has infrastructure for bringing in scientific expertise (nearly every major agency has a chief scientist and the American Association for the Advancement of Science places 250+ science policy fellows per year in government) and employs many economists, the intersection of these two sets is surprisingly small. In short, there are scientists and economists in the government, but few economists focused on science policy. Yet people with this kind of expertise are needed in science agencies, the White House Office of Science and Technology Policy, congressional offices, and in the Metascience Units that are emerging in the US and abroad.

3. Research Opportunity

For those interested in a research career, this is an exciting time. Scientific progress often accelerates when new tools for research come online, and AI seems poised to be just such a research tool. Modern AI is especially good at working with text, which means it’s very well suited to the economics of innovation, as scientific and technological progress generates a lot of text (scientific papers, patents, and software code).

Subscribe now

4. Creative Destruction

New AI tools make this an exciting time to study the economics of innovation for another reason. AI will likely change the way we do science, and it’s not that unlikely that we’ll need entirely new theories and empirics to design optimal policy. The results derived prior to AI may become like economic history, with questionable relevance to optimal policy design.

5. Inherent Mystery

Finally, the economics of innovation is, in itself, fascinating. Cumulative improvements in toolmaking — technological progress — is one of the things that sets humans apart from the rest of the animal kingdom, and this special feature of ours mostly stems from our social systems, rather than our biological capabilities. Species close to biologically modern humans existed for many thousands of years without strong technological progress. And we still don’t fully understand so much of how this system works. What a great thing to study!

If that intrigued you, I hope you’ll check out the Economics of Ideas, Science, and Innovation PhD short course. The course consists of 11 roughly hour-long lectures. Each is taught by a different expert — truly a who’s who of innovation economics:

1. Introduction to the Economics of Ideas, by Benjamin Jones

2. Idea-Based Models of Economic Growth, by Chad Jones

3. The Supply of Innovators, by Ina Ganguli

4. Open Science as an Economic Institution, by Pierre Azoulay

5. The Direction of Science, by Kyle Myers

6. Science and the Returns to R&D, by Matt Clancy (me)

7. AI and Innovation, by Kevin Bryan

8. Innovation Policy, by John Van Reenen

9. Immigration and Innovation, by Michael Clemens

10. Competition and Innovation, by Mitsuru Igami

11. Patent Policy, by Janet Freilich

Subscribe to Macroscience for an introduction to each course, along with recommended reading. If you find this topic interesting but aren’t yet looking for a PhD-level course, check out the Metascience 101 podcast, also produced by IFP.

Subscribe now

We’re launching the course with Benjamin Jones’ introduction to the Economics of Ideas. In this lecture, Jones discusses ideas as unique goods, with a particular focus on how market failures and spillovers can justify government support for the production of ideas.

Here are the readings Jones assigned to the PhD students (truly some of the foundational pieces on the economics of innovation):

• Arrow, Kenneth. “Economic Welfare and the Allocation of Resources for Invention.” In The Rate and Direction of Inventive Activity: Economic and Social Factors (1962). Princeton, NJ: Princeton University Press, 609-625.

• Jones, Benjamin F. and Summers, Lawrence H. “A Calculation of the Social Returns to Innovation.” In Innovation and Public Policy, University of Chicago Press (2021).

• Bloom, Nicholas, Mark Schankerman, and John Van Reenen. “Identifying Technology Spillovers and Product Market Rivalry.” Econometrica 81(4) (2013): 1347-1393.

• Jones, Benjamin F. “The Burden of Knowledge and the ‛Death of the Renaissance Man’: Is Innovation Getting Harder?” Review of Economic Studies 76(1) (2009): 283-317.

To paraphrase Jones in his presentation, if the social returns to R&D are high, the social returns to studying R&D are very high.

Go forth and produce some social returns!

A full transcript is available here.

Subscribe now

• I want first to discuss why we need a course specially about the economics of ideas.

• We’ll talk about the nature of ideas, which is part of the answer to that question. Ideas are special and unusual goods that have very different properties from other kinds of goods.

• I’m going to close by helping us think about spillovers and market failures.

This is an area where our free market welfare theorems tend to break down. That means that there’s going to be a distinctive role for policy in the innovation space — for particular institutions to try to overcome some of these market failures. In addition to innovation driving dynamics in the economy, this is also an area where there needs to be an enormous amount of policy. That gives you a whole different perspective, and there’s so much to be discovered still about what kinds of policies are effective.

Why a course on innovation?

The advance of ideas is a profound, propulsive force for rising standards of living, increasing health, and longer lives. This is showing rates of increases in income per capita, in real terms, going back to the 11th century.

In the 19th and 20th centuries, we see a much more rapid expansion. This course begins with the Industrial Revolution. AI is the latest of a series of general purpose technologies that have been part of this explosive, ahistorical — from a prior perspective — expansion in standards of living.

You can try to get at why we think ideas matter here, and you could get into macro residuals. My colleague at Northwestern, Joel Mokyr, who just won the Nobel Prize this fall, has a book, Lever of Riches, which I read during my PhD and recommend to you strongly. It’s a beautiful, technology-by-technology take about how advancing ideas, often embodied in physical capital equipment, have greatly advanced productivity. It’s a very direct take on how important ideas have been throughout this period and particularly in the Industrial Revolution.

More powerful arguments for why we should think about ideas come from thinking about a particular sector of the economy and how it’s changed.

• Transportation: In the middle of the 19th century — before trains, automobiles, and airplanes — if you wanted to get places, there was livestock; riding, or being pulled by, a horse. In the United States, if you wanted to go west you might be being pulled by oxen, who are going to be both your propulsion and possibly your food, because to get from Illinois, in the middle of the country, to California in the west, took something like six months. It was slow, dangerous, and you had to bring your own food. Today you can make that journey in less than six hours on a jet-powered airplane. Six months to six hours. How do we make that huge improvement in transportation efficiency? It has to do with a lot of new ideas that didn’t exist in the middle of the 19th century.

• Farming: We used to do a lot of hand labor. Go back to the late 19th century in the United States: 80-90% of all people were farmers. Now it’s something like 2%. We’ve had incredible improvements in productivity including mechanized agriculture: machines that can sow, harvest, and process grains.

• Computers: If you’ve seen the movie Hidden Figures, it’s about a team of women at NASA who were, as a job, called “computers,” because that’s what they did. They would be computing, by hand and with slide rules, orbital trajectories for launching satellites. They were replaced by what we think of today as computers, which are machines. At first they were replaced by an IBM mainframe. We know how much the advance of computing has done in terms of the breadth of effects across society.

We didn’t get to our standard of living today by saving and doing more of what we knew how to do in the 19th century. We’re not richer because we have more wagons per person. There’s something very different going on — we’re coming up with better ways to transport, harvest, and compute. Those are driving massive increases in output per hour, and advancing standards of living.

In those pictures we’re not seeing the ideas — we’re seeing implementations of those ideas: machines, for example.

Much of economics is built around the thing on the left — chairs. We think about production functions, prices, markets, and firms. But the thing we’re talking about in this course is on the right: it’s an idea. It’s the quadratic formula. One way to encapsulate that distinction is that there’s a knowledge production function that might be quite different from a production function for ordinary goods or services, like chairs.

To put it in formal terms, pretty much everything you’ll hear over the next 13 weeks is going to fit into this equation.

• What is the output on the left-hand side? A is some stock of knowledge, or maybe a measure of productivity or health, like how long you live, or survivability from a disease. We’re trying to improve A over time. The output of ideas is an improvement in the standard of living, through increases in productivity or health.

• On the right-hand side, in that function q, I have the natural inputs.

  • H is human capital. There’s a lot of human capital going into ideas production, which might be distinctive from the human capital that goes into ordinary production of a chair.

  • K is physical capital, like machinery, where you’re going to use a lot of specialized machinery like telescopes, or microscopes, or particle accelerators, or a lot of Stata or R doing statistics in economics.

  • Z would be the institutions that are going to shape incentives: rewards for creating ideas, which are very distinctive in science and R&D from other institutions in society.

  • Then there’s the current state of knowledge itself, A, which may have an implication for the ease with which we can continue to improve our understanding.

I’m not going to talk about AI today because I’m doing an introduction. But if you want to think about AI, AI is K — physical capital. A necessary input into idea production through history has been human minds — without that, you can’t make any progress. You can think about AI as shifting some of that to K. Maybe it’s really high quality K. To be determined. But that’s one way to think about AI in a coherent framework.

Thinking about supporting innovative activity, a wide range of institutions and policies are directed at this peculiar activity of idea production. You’ve got:

• The US Patent and Trademark Office: the patent system supporting private incentives to invest in research and development.

• Grant funders for science like the National Institutes of Health, or the European Research Council.

• Laboratories where things are actually happening: multinational collaborations like CERN for particle accelerators.

• On the bottom, that’s my university. I’m sitting along that lake right now. Lots of labs here doing things.

• Argonne is not a university but a US national energy laboratory.

• The Howard Hughes Medical Institute is a philanthropic, very high-end health research lab.

• Government funders for more solution-oriented process like DARPA, which is within the US Department of Defense, trying to come up with defense solutions to various problems.

• Venture capital funders like Bessemer.

• Philanthropies like Sloan.

• Private sector corporate labs like Abbott, which is trying to come up with new drugs for medical devices.

There’s a wide range of private, nonprofit and public institutions coming up with new ideas, funding them, or coloring the incentives with which we do things.

To encapsulate the first question for our course, why have an economics of ideas? First, because the advance of ideas informs central phenomena. It’s an incredibly powerful, first order force in understanding the path of economic prosperity, which would include income, health, and also how inequality evolves between labor and capital, and between different kinds of labor. It’s about the dynamics of how markets, industries, and international trade work. It’s about the role of specific institutions and policy that are suited and designed particularly for this peculiar space of idea production.

We talked about the chair versus the quadratic formula — there’s something about ideas that’s distinctive, and I’m going to try to clarify that in an introductory way today. It’s from those distinctive features that you can begin to build out this more complicated, rich understanding of idea production and all of its features. The related issue, which I’ll come to very quickly now, is that because of these special features of ideas, it’s natural to think there are very large market failures. The market is not going to produce ideas correctly, left to its own devices, and that’s why we need public policy and places like Institute for Progress to help think about how we can do that policy better.

I want to talk about the nature of ideas, and I’m going to come to several features: non-rivalry, excludability and cumulativeness. I’m not going to talk much about uncertainty in the interest of time, but I can mention it. These are features that are going to underpin the market failures associated with ideas, and in some sense this course.

Non-rivalry

This is a somewhat odd concept when you first hear it, and then it becomes very natural. Most goods are rival goods: chairs, cars, or computers. If you’re sitting in the chair, no one else can use the chair at the same time. You’re rivals. If someone is driving a car, you can’t be driving that same car somewhere else at the same time. If you’re getting a visit with a dentist, they can’t provide that service to somebody else’s teeth at the same moment. Ordinary goods and services are rival: if one person’s getting the service or using the good, nobody else can at the same time.

But with ideas, at the core of it all is this concept — like the quadratic formula. Let’s say I’m using the quadratic formula right now to solve some algebra problem. Andrew could be using it at the same time. So could Matt. So could all of you. There’s no sense in which my using the quadratic formula is at all influencing your ability to use and get value out of it. We’re not rivals. If I’m driving the car, you can’t drive the car. But if I’m using the quadratic formula — no effect on your ability to use it.

Ideas have this non-rival property: algebra, a biomedical concept like the germ theory of disease, the use of an assembly line. Once Henry Ford puts in an assembly line to make cars, every other car company’s like, “Maybe I could do that.” Or you’re making something else, scaffolds, or chairs — I could use an assembly line. A chemical process. Regression: if I run Ordinary Least Squares, you can too. Jennifer Doudna comes up with CRISPR. Now that might be patented and be a source of some debate, but when the patent expires, people can use it for free. There’s nothing fundamental about the idea that would say that your use of CRISPR would make it harder for me to use it.

That property, which is essential, is going to have very important implications. It’s going to mean that there’s a lot of value in ideas because value spills over. If one person creates it, it can spill over to other people. That’s going to lead to market failures. This course is going to suggest that markets are going to underinvest in new ideas. If the markets underinvest, we’re not getting enough of this incredibly important thing. So we’re going to want to care a lot about policy.

If it’s non-rival, let’s say one of you comes up with an idea, and you put it out in a paper and everyone’s like, “Wow, it’s a new method,” or insight, or fact. Other people will be like, “I’m going to start making use of that.” But because everyone else is getting value from it, that means that you, in creating it, are not getting the whole value of it to society. You’re getting the value you use it for yourself, and then its value is spilling over through non-rivalry to all these other people. So there are positive spillovers.

But where it becomes more problematic from a market investment point of view is as follows. The first time you come up with the idea — let’s say you’re going to write a book, or record a music album, or come up with a new pharmaceutical that fights a certain kind of cancer. Often the act of creation, like writing a book, is quite expensive. It takes a lot of time and effort. Coming up with a new pharmaceutical can cost a firm hundreds of millions of dollars to find it, show that it works, and that it doesn’t have bad side effects. So there’s this huge fixed cost to producing an idea. But if you show the pharmaceutical works, other people — if they could access the pharmaceutical — could make copies of the pill, not having to bear any of that fixed cost. If you make a book and everyone could make a copy and read it as a PDF. If you recorded an album at some expense and then everyone else can listen to the music and make copies and we all can listen to the same song at the same time for free. It’s non-rival. Then what happens? A market’s going to make it hard for this work.

Why? Here I’m showing you the units of ideas produced along the x-axis. But there was a fixed cost to come up with the idea. For the person who created the idea — they’ve got this fixed cost of coming up with a pharmaceutical and they’re going to make more copies of the pill, so their average costs are going to decline. But because of the fixed cost, their average cost is always going to be above the marginal cost of making a pill, because they still have to pay off this fixed cost of producing the drug in the first place.

Any entrant who wants to make copies of the pharmaceutical, or the song, doesn’t have to pay that fixed cost — they just make copies. Which means that if competitive entry happens, they’ll come in, and they can price just above their marginal cost and make money, but they’re going to be pricing below the average cost from the idea creator. Who’s going to win the market? Not the idea creator. They can’t charge that low because they have to pay themselves off for the fixed cost of creation. It’s the entrants who come in and copy with the non-rival ideas that can serve the market at the lowest price. So the person who actually did the invention goes out of business, from non-rivalry alone, if you let that be the dominating force.

In a context of non-rivalry, it’s very hard to invent. You might do it because we’re creative. I like to write a book; it makes me feel good. But you’re probably not going to marshal hundreds of millions of dollars to make a drug, because you’re going to lose all your money even if you’re successful. This is the first key market failure, and we need to solve it. To solve it, we have to get to a second concept, closely related to non-rivalry, but it’s more of a choice of society, which is excludability.

Excludability

Ideas themselves are not typically naturally excludable. They’re non-rival. If I heard about the quadratic formula, I’ll just use it. If I’m going to come in and use it for free — and I might undercut you who created the idea — in order to create a private decentralized incentive for someone to create that idea, we need to let them exclude others from using it. In other words, we need to give you something like a patent. If you did a pharmaceutical, and I could come in and undercut you and there was nothing you could do about it, you would have no incentive to invest. But if I give you a patent, I make you a monopolist — that gives you the right to exclude others from using your pharmaceutical unless you want to let them. You could charge a license or a royalty to let them do that. If it’s my pharmaceutical and I have a patent, I can decide who gets to produce it. Now I can charge a much higher price, and then pay off my fixed cost of investment. That’s a legal solution relying on creating intellectual property.

In general, whether you have excludability is going to depend on things like the policy institutional space in a country and the strength of the legal system. It can also depend on the technology itself. The patent system is the most obvious form of creating excludability. Think of it like this. Ideas are non-rival. But you can choose to make them excludable by allowing patents. Other intellectual property would be copyrights, trademarks, non-compete agreements, or trade secrets. There are other ways to do it which cover different kinds of creations. It’s up to a society to decide how much legal protection we want to give to certain ideas and creations, how long the patent right should be, how long you want a monopoly around these design questions.

The other way to go is not necessarily relying on intellectual property. A company or a person might get excludability in other ways. One thing is you can make it really hard to copy. The idea itself is non-rival, but you might encrypt it. An example would be satellite radio. If you’re driving around, the satellites are sending down the radio signals to all cars. Everyone is getting the signal, but you can’t listen to it unless you pay a subscription, because that is going to give your decoder in your car the decryption ability to listen to that channel. So there we don’t need patents. You could play a video game online peer-to-peer on some server and they don’t have to have a patent or intellectual property — because they can control your access to the server, and you can only play the game if you have access to the server. If you can control some necessary gatekeeping thing, or keep it secret, and people can’t figure out because it’s an internal process to your firm, maybe you can get excludability, despite the non-rival nature of things. This is why we’re going to have things like the patent system, because we need to create some incentive where the market without that property right would not succeed.

Uncertainty

Uncertainty is an absolutely fundamental feature of ideas. It’s not just that I’m going to flip a coin: is it heads or tails? I’m not sure. There’s that element where you’re not sure. If you start a drug program, 9 out of 10 pharmaceuticals that go into the first phase trial don’t make it through. They think they might work, but until they try it — they’re going to fail most of the time. Venture capital firms — 9 out of 10 bets are losers. They’re going to lose most of the time, and hope that some bets are really successful and on average carry the portfolio.

But it’s not just that you have a 1 in 10 chance. It seems to be that we can’t even foresee the reason an idea will be valuable or not. Before the internet, there’s NSFNET; before that, ARPANET. ARPANET was created by DARPA, a research agency in the Department of Defense. What were they doing? Let’s say we’re trying to communicate with each other across our operations, but someone knocks out one of our communications towers. We want to have a robust system of communication so we can still communicate. The idea was that you would divide information up into packets, and send it on many different routes through a network. If you lose one node of communications, information can still be rerouted across different nodes and people can still be in communication. The early idea of the internet for them was robust communications when you’re under attack. Were they thinking about Amazon, Uber, or e-tail? They were thinking none of those things. ARPANET was around for a while before anyone began to think, “Maybe this could be useful for something else.” So it’s very hard to know which ideas are going to take off.

Cumulativeness

The other feature of ideas that’s very important is cumulativeness. You create a spillover. It’s non-rival. If I come up with a song, other people can listen to it. If you read a book, other people can read it. But it’s not just that they can imitate or use the idea. It’s that that idea has an influence on future idea creation. Sometimes math achievements look like, “What’s the use of that?” But then later they prove important because they unleash further applications. An advance in physics might lead eventually to a new material, and that changes how we design airframes. So there’s an intertemporal spillover. Another version is you create a smartphone platform. Having created that, people can write apps for it. So there’s an intertemporal spillover on the ability to do software applications.

This intertemporal part seems very fundamental to knowledge. The most quoted line in all of innovation studies goes back to Isaac Newton, who said, “If I have seen further it is by standing on ye shoulders of giants.” He wrote this in a letter to a contemporary of his, a physicist, Hooke. He’s saying, “I can take a step forward because I have the benefit of all the knowledge that’s already been accumulated, from which I can step forward.”

If knowledge is cumulative, if there’s much more that is known now and that’s an input into future knowledge creation, does that make your lives as young economists easier or harder? Does it make it easier to do research today and come up with a big idea?

In one sense, there’s a lot more known and a lot of those things are tools. We can search the internet, look at Google Scholar, and read lots of papers. AI is a new tool. If you want to do econometric analysis, it’s easier doing it with modern computers than with punch cards on a mainframe, or before that trying to do it by hand, writing down the matrices and inverting them — that’s a pain. So certain tools make it more productive. Also the constituent matter that we can build on is vast; so we have lots of opportunities to combine things that are new.

On the other hand, and the thing I’m going to go in the direction of, there’s so much more known, so it’s harder to get to the frontier, because you have to learn something of what’s come before. Every year, there’s more that has been discovered. In some sense that’s an accumulating mass of knowledge. That’s what I call the “burden of knowledge,” which has the effect of making people narrower.

This is one of my earlier ideas in research. I hope the burden of knowledge is an evocative way to think about it. The question is, what happens if there is an advancing stock of knowledge, from the perspective of human capital accumulation and ultimately innovative potential? We’re born knowing nothing. You haven’t been taught anything. You know certain things instinctively and then you have to go learn. But you’re being born, with every passing year, into a place with so much more knowledge — biological, statistical, mathematical, knowledge of the known universe. How can you respond to that? You’re born knowing nothing and there’s a bigger mountain of knowledge that you’re confronting.

You have two options:

1. You could spend longer in school, spend more time training, as there’s more to know.

2. You could say, “I can’t possibly know everything, so I’ll become much narrower.” Maybe going back in Plato’s time, the inspiration may have been the same, and Aristotle could make all these contributions across the waterfront of knowledge at the time. But you can’t really be a leading biologist even anymore. Biology is so many sub-areas. A physicist, meaning what? You do cosmology, or quantum, or solid state physics and materials. These are very different math, and sets of ideas. So people get much narrower.

It may be that computers are making us more productive, and there’s lots of great ideas and they make us more creative and inspiration is quite the same in terms of a process. Yet if we spend more of our life cycle training, we have less time, other things equal, to innovate. We see this in the data. A bachelor’s degree used to be the ultimate degree in electrical engineering. Then back in the early 20th century there’s a master’s degree, then a PhD, now they do postdocs, so you spend much longer in your early years training.

More importantly, we’re narrower. If you’re narrower, that means the constituent matter that you can be inspired from is quite narrow. You’re doing some combinatoric search, based on things that you know — that are within your specialty. But also, your ideas are probably applicable to your specialty, and may not be as broad in their impact, because you yourself are narrow. So maybe we’re all branching into these niches. Our ideas are important in those niches, but there’s so many niches, and your collective impact — on productivity as a whole and on the economy as an individual researcher — is low.

You’re going to hear from Chad next week about idea-based models for economic growth. One of his main findings, early in his career, is that we seem to be putting a lot more effort into R&D — more people, more money — and getting less out in terms of Total Factor Productivity growth per person or per dollar spent. You could think of this narrowness effect of the burden of knowledge as one way to explain that phenomenon.

I had that idea when I was getting my PhD, and I went out using patent data — which is what we had — and looked at various patterns.

This upper picture is the age at which people get their first patent, and it looks like that’s going up. Another version of this in another paper is the age at which people do Nobel Prize winning work — not when they win the Nobel Prize, but when they do the research for which they would win it. Or people who are known technological inventors, like Thomas Edison and the light bulb, or Bell with the telephone — how old are they when they produce these signature contributions? You can see that the age at which people make those contributions is going up. If you unpack all this, you’re going to find that it’s going down in your 20s. You can still produce Nobel Prize-winning insights in your 20s. It’s just much less common now than it used to be, and people are typically peaking in their early 40s.

If you look at the lower rows, the one on the left is a measure of narrowness. It’s looking at an inventor and saying, let’s get two patents in a row that they do and ask if they’re likely to jump between one technology class on the patent to a new technology class. They’re less likely to jump with time, suggesting people’s ambit of research is more narrow.

On the right is one of our classic responses, which is — if everyone’s getting super narrow, what we might do is work in teams to pull together complementary forms of expertise. You see that too across all forms of patenting and science. Not only that, you see that teams are increasingly the source of the higher impact work. It used to be, in fields like math, if you wrote alone, you would do better than if you worked with a co-author or in a team. But now we see that reverse. Even in math today you’re going to have higher impact work if you work in a team than if you’re working alone.

Maybe AI is going to make you guys very productive. On the other hand, there is this training niche challenge that you have to overcome, and our classic solution to that is collaboration. Something you want to be thinking about, if you’re really good at econometrics but you aren’t a theorist, or vice versa, rather than bite your head against the wall of the other thing, can you collaborate with someone who brings those complementary skills together with you and produce a stronger paper more quickly.

Spillovers

The last piece I wanted to talk about was spillovers, to start bringing us around to the idea that this may be an incredibly high return activity and that we may greatly underinvest in it. That’s going to open up the policy and institutional space to us.

I had two papers I was going to talk about. One was Bloom, Schankerman, and Van Reenen, which asks whether firms’ R&D spills over to other firms, and what the returns are. A firm investing its own money in R&D is trying to come up with better products or services, or lower cost production methods, and they would capture the value of that themselves. But to the extent that they’re putting in place new discoveries, products, or services, other firms could learn about that and then be inspired for spillovers to do it themselves. That would not go to the private return of the firm. In fact, it might go against the private return of the firm because the other firms would now compete with them with imitative ideas. So this paper tries to pull apart key dimensions in those spillovers in a very clever and successful way. Given the time I’m going to go to the second paper, also because you’re going to hear from John Van Reenen directly later in the quarter.

Let me ask you a different question about spillovers. There are a number of spillovers in ideas:

• There’s non-rivalry. You produce some new idea and get some value out of it, but then I see it. I can use the idea if it’s not excluded through a patent, and then I get value out of it. Even if it was excluded through a patent, I could probably be inspired by your thing and come up with my own version, which doesn’t quite violate your patent, but still you inspired me and I got some of the value. That’s what we call an “imitative spillover.”

• There’s the intertemporal component. By producing a new mathematical idea, the next person can say, “That inspires me to something else,” and they’re going to capture the value in some intertemporal sense.

• If you make something new and you sell it in a market, people are willing to pay for it. They’re paying because they’re getting value in excess of what they’re paying. The consumer of your ideas is getting a consumer surplus benefit from your creation that is not going to be captured by the creator.

So there are several reasons to think that idea production would be undervalued. The private inventor, or the scientist, is getting less value personally than the social value that is created.

On the other hand, there are a couple of forces that might go the other way. One feature among private sector players is what we call business stealing. Let’s take Amazon. They’re going to come up with a new business model based on buying and selling things online and delivering them to you — originally books. They’re going to kill a lot of existing brick and mortar bookstores that find they’re not able to compete with Amazon. There’s going to be a huge shrinkage in the bookstore sector. If you think about the private returns to Amazon, they’re doing something that is valued by consumers and there’s some social gain. On the other hand, the private return of Amazon is also a transfer from the existing bookstores to Amazon selling them instead. That isn’t a social gain. It’s just a transfer. They are stealing business from the existing bookstores. That could mean that the return to the investors in Amazon is too high, because a lot of what they’re getting is stealing other people’s business but not really adding much that’s more efficient to society. So business stealing can be a negative spillover of innovation. It’s part of the creative destruction idea that you’d see in Philippe Aghion‘s work, who also won the Nobel in the fall, working with Peter Howitt.

Then the last one might be a racing phenomenon. If we’re all different research teams spending money and time in a race after a superconducting material, or a particular new drug, it’s like a tournament. One of us gets the patent, or the discovery, then the effort of all the other teams may be partly or fully wasted. In racing for certain opportunities, we may overinvest because we have a congestion or crowding externality on each other, or duplication externality from the perspective of the research spend.

In reality, there’s stories for positive and negative spillovers. It’s very much an empirical question, and papers like Bloom, Schankerman, and Van Reenen looking at business behavior net it out among the businesses in the sector and say, “It looks like on net a very positive spillover.” But even that is quite limited. What firms are doing is not really science. What about science like math or physics, and what about all that’s going on in universities? That’s a trickier question, because for a lot of things the spillovers are very diffuse.

Here are three examples.

• On the left, that equation is general relativity — that’s Einstein in 1915 and 1917. You might say, what is the benefit of general relativity? It’s a curiosity about the curvature of spacetime, and you might think it’s intriguing and neat, but not really useful. But it turns out that it’s incredibly useful. GPS satellites all work on atomic clocks that are very accurate. Those clocks have to be adjusted according to that equation, because they’re at lower gravity up in the air, and they’re moving at a speed compared to the surface of the earth. It turns out that if you don’t correct for general and special relativity, the whole GPS system doesn’t work. So you don’t get Uber — where we use GPS for drivers to find you and to navigate — unless you have the GPS system. You don’t have the GPS system unless you have relativity. That’s a hundred-year delay.

• That equation is an application of Riemannian geometry from the 1860s I think. Riemann came up with a geometry in math that seemed like it had no application. He was wondering about the geometry of spaces in more than three dimensions which have curvature. This non-Euclidean curving geometry in high dimensionality seems completely useless, because we live in three dimensions, and things aren’t curved. We can do everything in orthogonal planes or whatever. But it turned out that when it got to time and space there was a fourth dimension, and it turned out that mass was curving them, so the appropriate mathematics 60-70 years later for Einstein was Riemann. That’s going to lead to GPS. In other words, if I were to ask you the question, what’s the social value of geometry or general relativity, it’s a hard question to think about because of these really long lags. Obviously you need it. It’s necessary, but it’s not the only thing that’s necessary to the GPS system or Uber. So how would I allocate what the value of all that mobile internet I could put back on these equations? It’s very hard to think about.

• The one in the middle is Taq polymerase. That’s a protein that is the heart of Polymerase Chain Reaction, which is the heart of all modern biotech. It allows you to make millions of copies of the same strand of DNA, which we use for any kind of genetic engineering, or forensics, or PCR, as you probably know from COVID tests. That protein was discovered by a couple of biologists in a hot spring in Yellowstone. The organism it’s in is a bacteria that lives in really high temperatures, and at the time it was just a biological curiosity. That was in the 1960s. It turned out that that is going to unleash, a decade or two later, the entire biotech industry, and without that protein you can’t do it. What’s the social value of that research? This is now stuff that’s being funded by a government science agency.

• I put the iPhone up there because — try to imagine enumerating the social value, or cost, of the smartphone. It’s complicated, because there’s so many things it touches. How would I begin to think about these spillovers and all these different margins from that one device? So it’s very hard to do social returns in some adequate way. The more important the technology, the wider its use, probably the harder it is to measure, and we don’t want to miss those.

I want to close with one idea on this. There’s a big literature on how to calculate the social returns to R&D or to science. I’m going to give you a conceptual take that’s more macro, which I think will help motivate why what we’re doing in this course is so important and why we need policies. If we believe ideas are central to advancing prosperity — which I think there’s a huge literature coming from many angles telling us — without which we don’t really grow among advanced economies over the long run, could we harness that insight to think about the social returns from a macro perspective?

This goes to a paper I did a few years ago. The idea was to say — under the thought experiment of innovation and our modern understanding of it, as well as endogenous growth, growth theory, and macro in general — if you think of macro the variable is A, the state of knowledge, and if we don’t improve it we stay the same. If we just had all the ideas we have now, things would keep breaking and we’d have investment because I have to fix the computer I’m on, your car will break down, your house will need the windows repaired and the roof redone eventually. You’re going to do investment just to keep things going, but we wouldn’t be getting better at stuff. There’s diminishing returns to capital investment. So we can’t just rely on savings and investment. We need to improve the ideas. We didn’t get faster at transportation because of more wagons. We needed something much better than wagons.

What you can do as a thought experiment is say, “Imagine we just stopped innovating.” We would replace what breaks and stay where we are. You can use that insight to create an overall calculation of the social returns to R&D under some assumptions that you can then play with, that reveals an average return to that investment. I’ll show you the very simple baseline and then you can try to generalize it to confront a lot of other forces that you might think are important to deal with beyond the baseline. Then you could also think about how it relates to micro evidence, or other kinds of macro evidence.

Here’s the thought experiment. It’s got non-rivalry, temporal spillovers, imitation — it’s netting it all together.

Look at the upper row. Imagine you’re investing, on the green line, a certain amount of your total GDP into R&D — or innovative activity, however you want to measure that. R&D is 2-3% of GDP in most advanced economies. You’re following the green line. Then for one year, you’re going to go on to the black line, where everyone stops. We stop the course. You guys go work somewhere else for a while. The labs — we just put some sheets over the equipment. A year later, we come back, we pull the sheets off the equipment, and keep going on exactly the same projects that we were doing before. We just had a year-long delay. We don’t change the pathway of discovery. It’s the same projects and the same intertemporal set of projects. They’ll inspire the same things, but we just delay it a year. We save a bunch of money by not having to do it for a year. That’s what we invest in innovation. That’s the top line.

The next line is what would happen to growth in a simple model. If we don’t advance the state of knowledge, we stay at the same standard of living. So we’d be growing at — most advanced economies 2% per year — and then we wouldn’t grow for a year in the middle row, and then we just grow again on exactly the same path. To put this in income per capita terms, in the lowest row, you’d be growing at a log scale like 2% per year going up, and then for a year you wouldn’t grow at all. You wouldn’t get rich or poor, you just didn’t advance ideas, and then you would advance ideas again at the same rate as before.

What’s the cost and what’s the benefit?

f you invest 2-3% of your GDP in a given year, you would be 2% richer forever, because you would grow that year and you’d keep it, you don’t forget the ideas, they don’t depreciate, we still know algebra. So the value of that is this green wedge in the lowest row. In every future period, you’re going to be 2% richer if you invested in those ideas than if you didn’t. What’s the present value of being g% richer in every period? Its present value is g/r, where r is some discount rate. In other words, you invest a share x/y of your resources. That’s the cost in R&D for a year, and you get to be g/r in present value terms. That’s the thought experiment. There’s maybe a lot of imitation, there’s business stealing, all sorts of things are going on in a given year of R&D. Firms are destroying each other, some are winning, some are losing. Many drugs fail, some win, but on net, we get 1.8-2% richer per year. That gives us the net of all of that effort.

The present value of the benefits is g/r. The cost of investment is x/y from a share of resources. That’s the social benefit cost ratio. Now think about this intuitively. x/y is 2-3% of GDP, but you’re 2% richer next year — and the year after forever. That’s an amazing thing, because if you push up the level of ideas quite a bit today, given our R&D investment, and you get that benefit forever.

What do you get from that calculation?

g is something like 1.8%, 2% more accurately in the US. x/y is 2.7%. You can pick your discount rate, but I’m going to take a 5% discount rate. Look at that table below, you’ll see it says 13.3. That means, for every dollar we spend on R&D, we get $13 back in present value, which is incredibly high. It’s like a machine: you put in a dollar, you get all this money back. The logic that’s driving that is because you push up the level, and people generations from now are going to be smarter, because there’s Riemannian geometry. They’re able to figure out how to use it. The ideas don’t go away. You push things up and it stays there. And it turns out we don’t spend that much.

Maybe you think R&D formally measured is more than 2.7%. Maybe there’s a lot of innovation happening other ways and it’s badly measured. So this whole paper is about making this baseline calculation and then trying to kill it by making all sorts of different assumptions. You could put an adjustment factor I call β in front of that ratio.

Why would my $13 per dollar be too high?

• It might be because there’s lags: I invest now but I don’t get the growth right now, I get it for science sometime in the future.

• Maybe I have to embody the ideas in physical capital — they have to go into a smartphone, I have to spend on the smartphone.

• Maybe other sources of innovation are out there — not just people doing formal R&D.

On the other hand, there’s a bunch of reasons that that baseline might be too low.

• In macro we think that inflation is actually lower than we say because there’s a new goods bias if you follow that literature, so actually growth is higher than 1.8% in real terms. Maybe it’s more like 2.3% or 2.5%. That makes the returns to R&D even higher.

• We don’t actually value health directly in GDP. A lot of research is trying to make you healthier and live longer, not to make higher income per capita based out of productivity on the job.

• International spillovers. If the US does R&D, other countries can learn as well. So we’re a net exporter of ideas — the gain in standards of living can be far beyond your own borders.

You can balance these forces out. The conclusion is that it’s really hard to get the number of the benefit to be small compared to the cost, however you try to attack this. We actually said that on the very conservative end, you get $4-5 out per dollar. So this is a method where you’re trying to calculate the average return.

Where would that leave us? This is one version of the social returns. There’s many papers about the social returns to R&D doing different approaches, including a couple of very new ones. One of which, Fieldhouse and Mertens, is really about public investment in R&D alone and science, and it’s showing incredibly high returns. Not every paper agrees, but what you tend to see across a wide range of methods is repeatedly calculating very high social rates of return to idea creation. The paper I just showed you gives you some of that logic, because you push things up in a meaningful way that you benefit from in a future stream from your investments today.

In light of that, it seems like there are really high social returns. A lot of these papers are going to show higher returns on the margin. If we did another $1 billion of basic science, it’s going to be really valuable for society. So that means that both we’re going to have a bunch of institutions that are going to try to push on this, but also probably that they are currently way underfunded compared to their social benefit. Even conditional on their funding, do we have a good idea about the best way to design a science grant, or the patent system, or to run a university, or to create incentives for human capital like the tenure system, or how we give credit? All of these things are very open.

The questions in the bigger picture are:

• What are those policies?

• What are the right institutional structures?

• How should we design and fund these various systems?

There’s a big policy question. Then there are a lot of questions about understanding idea creation that are really interesting from a personal and a private sector perspective:

• Who captures value?

• How do you capture value from data?

• Where do you get excludability?

• Who gets credit?

• How does uncertainty affect our ability to produce and exchange ideas? There are some really interesting breakdowns in the market for ideas because of uncertainty between buyers and sellers. There are a lot of strategic interactions in markets in the production of ideas.

Then there are really big questions:

• Where do big ideas come from?

• Who produces them?

• How do they come up?

• What is one scholar doing versus another, versus an inventor, versus an entrepreneur?

• Does someone have a better process? What is that process?

• Are there better incentives? What are those incentive systems?

• What is the role of AI in all this? It’s coming at us very fast. Is it going to fundamentally change how we do things?

Let me close, trying to motivate you a little:

1. I tried to get across how important this subject is. The social returns to R&D are very high, because if you can make this system more efficient, the returns for society are vast.

2. It’s not an area where we’re just studying it because it’s curious and markets get it right. It needs interventions, and there are so many kinds. It’s a very complicated space, and there’s so much to be learned at a very micro level, to more industry sector levels, to macro levels, about how to do that well. That’s going to require great research.

3. This is a growing area with amazing data. When I started out doing this long ago, there was patent data. There was no data on papers and scientists. There was no real good data on entrepreneurs. There wasn’t really data on funders and who’s funding what. We didn’t have the full text of anything. Between the data sets that have emerged, and the tools that have emerged in parallel, both our classic econometric tools and compute that we have, but also now all these AI tools, in addition to our causal identification and structural tools, it is a vast opportunity, and there’s theoretical opportunities as well.

I have gotten to the point where every single data set I touch, I’m like, “I could spend the next 15 years on this data set,” and then I have to go over to something else. There’s so much data. Another way to say that is, our scarcity right now in the economics of innovation is not opportunity or data. It’s people. We need more people on that mountain figuring it out. I think there are great opportunities to find your own niche that is of great interest to the whole community on the mountain.

I hope this course, in addition to giving a taste of what people are doing at the frontier, and some of the core questions, can inspire you to realize how much great work there is to be done. I hope we can attract you into this research community.

The biggest challenge in intentionally driving breakthroughs through initiatives like the Cancer Moonshot or the Apollo Program is that, while we might want to focus on things that are important to society, it’s difficult to know what science is actually ready to be exploited. Kris and her co-authors developed a fascinating approach to predicting breakthroughs by using early signals in the volume of academic publications.

Kris is President and Founder of the Woodley Park Institute. This blog is cross-posted with Stuart Buck’s Good Science Project (subscribe here).

Picking winners or hedging bets?

Federal science agencies have long wrestled with the question of how to ensure their billions of dollars of grants and contracts result in maximum benefits for Americans.

On the surface, it might seem that a goal-oriented, interventionist style of management would be the best policy: prioritize our most pressing problems and distribute funding accordingly.This top-down strategy creates a unique hazard, though: bad choices can send resources in the wrong direction, resulting in slow progress that impedes the delivery of tangible benefits to citizens. Moreover, if decision makers pursue an applied advance when the underlying scientific principles remain poorly understood, they risk costly failures and a loss of support for the research enterprise at large. Examples include Nixon’s 1971 War on Cancer, undertaken on the premise that a cure could be achieved in less than a decade, the drive to build an operational commercial fusion plant by the mid-1990s, and the National Plan to Address Alzheimer’s Disease, which failed to produce the effective treatments and prevention strategies promised by 2025.

The contrasting laissez-faire approach is to hedge your bets. If the path to an advance is impossible to predict, then the wisest choice might be to distribute funding to as widely as possible. The main risk of this strategy is that spreading funding thinly may limit the resources available to the most promising research, resulting in the same negative outcomes as the interventionist approach: limited support for the most promising topics, slow progress, and delayed returns.

Subscribe now

In reality, at science agencies like the National Science Foundation and National Institutes of Health, program managers navigate between these two extremes, setting some funds aside to target high-priority areas and using the rest to cover the widest possible range of meritorious proposals. This middle path seems like the most rational way to proceed, but it still requires making choices. How much should be set aside for high priority topics? What should those topics be? How should we define merit? Expert opinion can provide a helpful guide, but leading experts can have remarkably different opinions about what matters and what should be prioritized. Moreover, experts and administrators alike can be risk averse, giving an edge to high-profile, well-established concepts.

What’s really needed is a data-driven framework to help guide decision-making, one that doesn’t simply repackage prestige or incumbency.

Recognizing breakthroughs years in advance

Many breakthroughs fail to attract funding or recognition at the earliest stages of development. Katalin Karikó, for example, spent years struggling to win grants for work on mRNA that would eventually win her and Drew Weissman a Nobel Prize. Similar stories can be told about Jim Allison’s groundbreaking studies leading to the development of cancer immunotherapy, Stan Pruisner’s demonstration that prions can self-replicate without DNA, or Robin Warren and Barry Marshall’s proof that Helicobacter pylori, not stress, causes ulcers.

Yet at some point, the scientific community recognizes and responds to transformative breakthroughs like these. How does that change happen? And what if science funders could identify the areas that are on the cusp of a breakthrough?

Those questions led my collaborators and me to study how scientists responded to past breakthroughs. Based on patterns in that data, we developed a means of predicting future breakthroughs, using biomedicine as a proof of concept.

It’s worth taking a moment to describe how our approach works. Briefly, our prediction process begins by grouping the 18 million or so peer-reviewed papers in PubMed, the authoritative database of biomedical research, into roughly 50,000 unique topics. A small subset of these topics are so widely acclaimed as breakthroughs that they have been recognized with a major prize like a Nobel or a Lasker Award. We theorized that studying the development of these outliers over time, especially before they gained acclaim, was the key to identifying future instances of them.

After defining topics, we chose 21 representative breakthroughs and asked what they looked like before they won prestigious prizes. To answer that question, we re-ran our topic-mapping algorithm repeatedly, winding back the clock one year each time, stringing together the years to follow the progress of each field as it grew and advanced.

We found that in each case, the years leading up to a discovery followed a pattern: a burst of papers on a new, rapidly evolving topic, many of which quickly became influential. These characteristics—the percent of papers that are new, the percentage of papers that are brought in from other topics, and the influence of each individual paper—can all be measured separately for any given topic. Together, they act as a predictive signal that is detectable an average of five years before a transformative discovery is made, and up to thirty years before the discovery receives a major prize. Scanning the current research landscape for topics that display this signal allows us to predict what work will likely produce a future breakthrough. Each year we analyzed includes four or five signals, a number that appears to have remained constant across a twenty year time frame.

In their classic work on the sociology of science, Bruno Latour and Steve Woolgar established that the desire to participate in a major discovery is an important motivator for scientists as they consider whether they should take up (or abandon) a research problem. Although further work is needed, the simplest explanation for our results is that this behavior is widespread enough to detect at scale. A foundational advance, or something like it, draws the attention of scientists and causes them to change the direction of their research. They vote with their feet, staking their own reputation and careers by publishing on and citing the new idea. That upswell of interest is the foundation of our breakthrough signal.

Subscribe now

What kind of research does our model identify as a breakthrough?

After building a model based on the patterns in our 21 preselected breakthroughs, we tested it by identifying all early signals of discovery from 1994 through 1997. Of the 18 signals we found in that time period, 17 can be traced forward in time to a breakthrough. Examples include super-resolution fluorescence microscopy (see the image above), the directed evolution of proteins and enzymes (Chemistry Nobel, 2018), and sequencing the human genome (National Medal of Science, 2008).

The discovery of the role of leptin signaling in obesity is a particularly interesting case study. Around 1950, scientists noticed the existence of a type of lab mouse that suffered from a unique inherited form of obesity. When given the freedom to choose their own diet, these mice ate more than normal and preferred food that was high in fat. Researchers had few clues as to why the animals overate; early experiments pointed to the existence of a soluble ob factor found circulating in the bloodstream that regulated appetite, but its identity was unknown.

In 1994, Jeffrey Friedman and colleagues identified and sequenced the ob gene, noting that the protein it encoded appeared to fit the profile of a circulating factor. A year later, a trio of high-profile publications demonstrated that ob was a hormone that regulated body weight and fat deposition by regulating appetite. One of these was co-authored by Friedman, who christened it leptin from the ancient Greek word for thin. Using our model, the data from 1996 make it clear this advance would become a breakthrough. Scientists had flocked to study the biology of the new hormone, generating the flurry of new papers and citations required to produce a signal. One of these demonstrated that serum leptin concentrations reflected the amount of adipose tissue in the human body, providing clinical validation of the earlier mouse studies. Before the identification of leptin, scientists were unable to point to a specific molecule that controlled appetite and adiposity, so that it was impossible to rule out non-physiological causes of obesity; its characterization fundamentally changed our concept of weight gain. In 2010, Friedman and fellow pioneer Douglas Coleman were awarded a Lasker for the discovery.

Our work also identifies breakthroughs in clinical practice and behavioral and social sciences, even though these areas are less likely to draw the attention of major prize committees. One example that falls into this category is the introduction of endovascular aneurysm repair, a minimally invasive surgical procedure for the treatment of abdominal aortic aneurysm, a leading cause of death among Americans over the age of 65. Relative to the previous standard of care, it reduces in-hospital mortality by almost four-fold. In 2017, Timothy Chuter received the American College of Surgeons’ prestigious Jacobsen Innovation award for his multiple refinements to the technique.

Another under-recognized breakthrough identified by our model is the development of standardized survey instruments to assess the quality of life for HIV⁺ patients. The antiretroviral cocktails introduced in the mid-1990s reduced AIDS mortality, but treatment came with serious side effects. The new evaluations showed that patients whose disease progressed had worse physical functioning than Americans with other chronic diseases, while those who were asymptomatic maintained a health-related quality of life on par with the overall US population. This evidence made it possible for patients and activists to argue that even in the absence of a total cure, and in spite of the side effects, alleviation of symptoms was a worthwhile priority for clinicians. My colleagues and I have failed to identify any significant recognition for the physicians and health policy experts who pioneered this advance, although in 2016, researchers called for the World Health Organization to set targets for good health-related quality of life as part of its framework to end the AIDS pandemic.

Subscribe now

From forecasting to funding

How should science funders use our predictions?

Any serious discussion of what we should do with this new capability needs to begin with an understanding of its limitations. We can’t (and shouldn’t) invest all of our resources in the small number of topics predicted to produce breakthroughs. Innovative new fields are born from existing ones; they rarely arise de novo. If funders don’t maintain a diverse portfolio, the breakthroughs of tomorrow will have no antecedents. Think of this as eating the seed corn, or killing the goose that lays the golden egg.

There are also multiple goals of funding beyond basic scientific discovery, including translating discoveries into clinical practice, developing new technologies, supporting economic development, and training new scientists. The investments needed to accomplish these goals are likely different from those required to support emerging breakthroughs.

Finally, we must be aware that we may not capture every instance of a breakthrough. The concept is fuzzy, and not every significant discovery is recognized with a major award. Further, breakthroughs are rare, meaning that our dataset is, by necessity, small. This makes formal estimates of accuracy challenging.

Although our overall success rate appears to be high, any one signal may turn out to be a false positive. We found one apparent example: the development of the third generation COX-2 inhibitors, Vioxx and Celebrex, which were much celebrated in the early 2000s as effective but non-addictive pain relievers, meets all the algorithmic criteria of a breakthrough. Five years after it received FDA approval, researchers demonstrated that Vioxx was associated with serious cardiovascular complications, and the manufacturer voluntarily pulled the drug from the market. Celebrex remains available, although like other NSAIDs, it carries an FDA warning for increased cardiovascular risk. Decision makers who were overly focused on the breakthrough potential of this research might have wasted substantial resources nurturing an area that ultimately proved to be a failure—but so would those who relied on expert opinion at the time.

That said, the potential benefits of funding the next mRNA vaccine or cancer immunotherapy years earlier than we might have are large. When we can’t assume the outcome of an action, it’s worth conducting an experiment. Can targeted support for areas that are poised to produce a breakthrough increase the return on scientific investment without cannibalizing future advances?

With the necessary administrative infrastructure in place, conducting this experiment would be straightforward. First, identify all the breakthrough signals between 2018 and 2023, select half at random, and commit to a decade of investment. For biomedicine, a six year window should yield about 30 topics with the potential to produce a breakthrough. Providing strong support to half of these might be accomplished for $300 million a year. That’s roughly half of the recent annual budget of the NIH Common Fund or around 20% of the fiscal year 2025 budget for ARPA-H. After five years, then again at ten, compare to see which group — the one that was actively managed, or the control — produced more breakthroughs, and on what timescale. Although we used biomedicine as an example, the same experiment could just as well be run for physics, materials science, or any other discipline.

Subscribe now

Any attempt to undertake such an experiment should keep three points top of mind. First, we can predict the topic of future breakthroughs, but the people and vision for how to move the work forward are still important. In practice, this means success requires active program managers with the expertise to develop the science and the authority to do so, including by recruiting investigators. Second, we would be wise to learn from the failures of past attempts to fund transformative research. High among these are funding modestly repackaged work, confusing the genuinely novel with the merely unfamiliar, and relying too much on incumbent investigators. All of these risks can be mitigated by thoroughly and intelligently integrating high quality data into portfolio management. Finally, we need to have patience. Even with strong funding support, producing a breakthrough takes time.

If supporting breakthroughs is the best way to nurture the birth of new fields, we also need to accurately describe and properly manage the other two stages, so that we invest wisely throughout the productive lifetime of ideas and divest once they reach the point of diminishing returns. We would benefit from more study of all of these phases, especially the transition from one to another. More research could answer questions like:

• When does a breakthrough become an established field?

• How do the availability of funding, the size and characteristics of the available workforce, and the accumulation of evidence that disagrees with prevailing models affect a field’s decline?

• Exactly how do old ideas give birth to new ones, and can we speed up the process?

The more we understand about the dynamics of scientific progress, the better we can maximize the odds that our curiosity leads to results. I’d call that finding a way to make our own luck.

Subscribe now

Two things can be true at once:

1. Science is the ultimate public good and has resulted in massive positive spillovers, making R&D one of the most important investments the federal government makes.1

2. The systems we use to fund, select, structure, and conduct that science have serious problems and could work much better.

The federal government spends nearly $200 billion a year on research and development. If we could make the process of identifying, distributing, and managing those funds even 5% more effective, we would effectively unlock another $10 billion a year for higher expected value science and technology.2 For context, that is slightly more than what we spend annually on NASA’s entire science budget, or on the National Cancer Institute.

This insight has driven a series of “science reform” movements over the years, some within existing science policy circles and some from other vantage points. Metascience is the study of the scientific enterprise, aimed at generating evidence about what increases the reliability, productivity, and impact of science. But different science reformers have distinct core hypotheses about what has gone wrong and what is most tractable to fix. How do the people building new Focused Research Organizations fit in with those working on the replication crisis? Are they all doing metascience? And how is metascience different from science policy in general?3

If policymakers and agency leaders are going to change how they spend billions of dollars of taxpayer money based on metascience advice, they should have a clear picture of the landscape of metascience and its various goals, perspectives, and terms.

This is our attempt to map the landscape. We’ve split the territory into six recognizable metascience ‘camps’. Each camp advances a distinct hypothesis about which parts of the scientific enterprise most need attention and how we should test what works.4

Subscribe now

A few caveats before we share the landscape.

• These camps are permeable. Many individuals and institutions move between two or three, and the borders are intellectual emphases rather than walls. Each is grappling with a different part of the same elephant.

• For this landscape, we are also limiting ourselves to metascience rather than the broader landscape of science policy. The line is debatable, but we think of science policy and metascience as overlapping circles: metascience asks how the scientific enterprise functions and how to make it work better, where evidence about outcomes is the primary arbiter. Science policy includes many metascience questions but also broader normative, political, diplomatic, and national security ones. Not all metascience is science policy, and not all science policy is metascience, but the two have plenty to say to each other.

• The methods we list for each hypothesis reflect dominant practice, not a logical constraint. In principle, most questions could be pursued with multiple methods.

With those caveats, here are the six camps and some examples of their associated research and initiatives.

Research Integrity

Hypothesis: Poor statistical practices and publication bias have often resulted in publishing inaccurate or overstated claims, leaving researchers to spend time following unproductive paths of inquiry. Science could be more reliable and current institutional incentives perpetuate the problem.

Theory of change: Fix methods and publication norms → fix the literature → more reliable science.

Policy relevance: This is the version of metascience that NIH Director Jay Bhattacharya has spoken about directly and launched initiatives to address. The “replication crisis” showed that in fields like behavioral psychology, a large fraction of published results don’t hold up when repeated (see examples below). A representative policy question: should we require replication studies as a condition of large grants?

Evidence and methods: Common methods include large-scale replication studies, meta-analyses, statistical power analyses, and adversarial collaborations to replicate empirical results. The inference that science needs stricter standards is supported by the pattern of failures rather than by randomized controlled trials (RCTs) testing the standards themselves.

Examples:

• “Why Most Published Research Findings Are False,” which argued that most published findings are likely false when studies are small, effect sizes are small, and many teams pursue the same question. This is the foundational text of the replication crisis literature.

• The Reproducibility Project: Psychology, which attempted to replicate 100 studies from three top psychology journals and found that only 36% of replications produced statistically significant results, even though 97% of the original studies had reported significant findings.

• The Many Labs project series, which has run a series of replications of classic and contemporary psychology findings with replication results varying widely (from 30% to 75%) across iterations (2014, 2018, 2016)

• Data Colada established the foundational case against “p-hacking” and has since surfaced evidence of data fabrication in high-profile research. P-hacking is manipulating data to artificially produce a statistically significant result.

• Registered Reports, a publishing format in which a study’s design and analysis plan undergo peer review and receive in-principle acceptance before any data are collected. This structure was designed to reduce publication bias and selective reporting of positive results.

Open Science

Hypothesis: A lack of transparency enables bad practices and slows progress. The case for openness is both that transparency is good in itself and that science advances faster when it’s open by default.

Theory of change: Increase transparency → increase accountability → faster, more reliable science.

Policy relevance: This perspective shows up in transparency requirements in agency grants and in open access mandates, such as the 2022 White House Office of Science and Technology Policy memorandum requiring publications (and underlying data) to be available without an embargo and Plan S, a mandate for immediate-publication access from a coalition of European research funders. It has produced practical infrastructure that many researchers now use regularly, such as preprints, preregistration, and registered reports. A representative policy question: should agencies mandate open data, preregistration, and open access for federally funded research?

Evidence and methods: Historically, their approach has been normative and values-based, with some empirical analysis. The movement’s strength comes from a combination of advocacy-driven policy reform, platform development, and community norm-setting, increasingly paired with empirical evaluation of outcomes.

Examples:

• The Open Science Framework, a free platform used by hundreds of thousands of researchers to host preregistrations, protocols, data, and study materials.

• The Public Library of Science (PLOS), founded in 2000 through an open letter signed by ~34,000 scientists who pledged to publish only in journals making articles freely available, and which has since grown into one of the largest open-access publishers in science and demonstrated that fully open-access publications are commercially viable.

• Preprint servers such as arXiv and bioRxiv, which allow researchers to share manuscripts before peer review and have dramatically accelerated dissemination in physics, mathematics, biology, and adjacent fields.

• Open data citation analyses, including a multivariate analysis that found papers making their data publicly available received more citations than comparable papers without publicly available data.

Subscribe now

Metrics of Science

Hypothesis: Science is a complex system with discoverable regularities. Large-scale data can reveal how breakthroughs happen, how careers unfold, and what predicts impact.

Theory of change: Understand patterns in how science works → more informed future program design → higher impact future research.

Policy relevance: This body of work provides evidence on optimal team size, career stage effects, and the novelty-versus-convention tradeoff. A representative policy question: should government programs fund more small teams, given that they are more likely to produce disruptive work than large teams?

Evidence and methods: Common methods are largely descriptive and include computational analysis of bibliometric data and machine learning on large scientific corpora. Some scholars track social dynamics (teams, citations, networks); others study materials used in research, arguing that new methods and tools are the primary engine of breakthroughs.

Examples:

• The “Science of Science” research program, which synthesized large-scale quantitative analyses of scientific careers, citation networks, team dynamics, and knowledge production, providing the canonical statement of the field’s research agenda and methodological toolkit.

• Disruption index research, which uses citation network patterns to distinguish papers and patents that consolidate fields from those that disrupt them and to measure whether disruptive work has been declining over recent decades.

• Hot streaks research, which identifies periods within scientific, artistic, and film careers where individuals produce their highest-impact works in concentrated bursts, with no accompanying change in productivity, often following a phase of exploration that transitions into focused work.

• Team size and innovation studies, which find that smaller teams are systematically more likely to produce disruptive work while larger teams develop and extend existing fields, suggesting that funding portfolios should support a diversity of team sizes.

• Atypical combinations research, which finds that the highest-impact papers tend to combine conventional foundations of prior knowledge with unconventional mixtures of ideas (e.g., drawing on research from two fields that rarely go together).

Innovation Economics

Hypothesis: Funding structures and incentive design causally shape the direction, quality, and quantity of science.

Theory of change: Understand incentives and causation →redesign programs → better science and translation.

Policy relevance: If a policymaker needs to know what might happen when you pull a specific lever, this is where to look. A representative policy question: do person-based grants produce more innovation than project-based grants and, if so, should more of the R&D portfolio be invested through person-based mechanisms?

Evidence and methods: Both causal and descriptive empirical methods are used, including RCTs, natural experiments, and other quasi-experimental analyses. This body of work relies on research designs that isolate cause and effect, which makes its evidence attractive for specific mechanism questions. It focuses on testing questions where credible counterfactuals exist.

Examples:

• Howard Hughes Medical Institute versus NIH comparisons, which find that scientists funded under HHMI’s “people, not projects” model, where grantees receive freedom to change direction and tolerance for early failures, produce more high-impact papers and explore more novel topics than comparable NIH-funded scientists.

• Intellectual Property (IP) protection and follow-on innovation studies, including evidence that a brief, non-patent form of IP held by the firm Celera over portions of the human genome reduced subsequent research and product development on those genes by 20–30%. Later research suggests that gene patents, by contrast, had little effect on follow-on innovation.

• Studies of expertise versus bias in peer review, including a quasi-experimental analysis of NIH grant evaluations suggesting that though reviewers are more biased about projects in their own subject area, the benefits of their expertise still weakly dominate the costs of bias. This implies that policies seeking to limit bias by using impartial evaluators may actually reduce decision quality.

• Small Business Innovation Research (SBIR) Program evidence, including a quasi-experimental study of Department of Energy SBIR applicants showing that an early-stage award roughly doubles the probability of subsequent venture capital investment and produces large positive effects on patenting and commercialization.

Subscribe now

R&D Management and Implementation

Hypothesis: Organizational design and management practices shape what science gets done. The same funding mechanism can succeed or fail depending on how the agency implements it. Beyond evidence about which mechanisms should work, you also need evidence about how to make those mechanisms succeed in practice.

Theory of change: Understand how R&D organizations work in practice→ design more practical reforms → more sustainable adoption.

Policy relevance: This approach answers the implementation questions generated by policy recommendations. A representative policy question: how should we design the organizational structure for a new agency metascience office, i.e. where should it sit, what authority does the director need, and how do we make sure it exists long enough to produce results?

Evidence and methods: Common methods include institutional analysis, comparative case studies, program evaluation, mixed-methods research, process tracing, and interviews with agency officials.

Examples:

• Comparative institutional analyses of NSF, NIH, DARPA, and ARPA-E, which document how organizational features such as program-director authority, hiring flexibility, and active project management distinguish ARPA-style agencies from traditional grantmakers.

• Studies of “open” versus “conventional” innovation procurement, including a quasi-experimental study comparing the outcomes of the awardees of Air Force SBIR “open” topics and conventional topics competitions. Winning an Open award increased subsequent non-SBIR Pentagon contracts, venture capital investment, and high-originality patents, whereas winning a conventional award increased the probability of winning a future SBIR contract.

• Analysis of ARPA-E’s active program management, where program directors retain authority to expand, contract, or cancel awards based on performance. Research found ARPA-E-funded cleantech startups filed patents at roughly twice the rate of comparable firms.

• NIH intramural technology transfer impact analysis, which traces how NIH-licensed inventions move from intramural labs to market. The study found that, in contrast to the simplistic linear “pipeline” model of innovation that agency policy often assumes, a chain-linked model in which research and product development continuously inform each other is a better fit to what NIH technology transfer offices actually do.

Institutional Entrepreneurs

Hypothesis: Existing institutions are too ossified to reform. The best path forward is to build new entities that demonstrate radically different models.

Theory of change: Generate hypotheses for new organizations → create existence proofs → competitive pressure and policy templates for incumbents.

Policy relevance: These experiments generate real-world evidence about what is possible. That evidence can be unusually persuasive to policymakers because it is demonstrated rather than hypothetical. A representative policy question: How much of a funder’s portfolio should fund new institutions?

Evidence and methods: Institutional entrepreneurs conduct real-world organizational experimentation, but these institutions are more proofs of concept than formalized experiments. The work resembles entrepreneurial iteration more than social science hypothesis testing. Its methods carry high persuasive power for demonstrating feasibility but make limited claims to generalizability.

Examples:

• “Reinventing Discovery: The New Era of Networked Science,” an early and influential argument that internet-enabled collaboration tools could fundamentally transform how scientific discovery happens, providing an intellectual foundation for many of the open and collaborative experiments that followed.

• Arc Institute, founded in 2021 with roughly $650 million in private funding, which provides investigators with eight years of unrestricted funding and lab support for up to 20 people in lieu of project-based grant applications.

• Convergent Research and the Focused Research Organization (FRO) model, founded in 2021, which structures mid-scale engineering projects as nonprofit, time-limited startups (typically $30–50 million over 5-6 years). FROs aim to build tools and datasets that fall in the gap between academic labs and industry.

• Arcadia Science, founded in 2021 as a for-profit research institute combining basic research, tool-building, and translational development under one roof, as well as experimenting with publishing outside traditional journals to release research products earlier and more openly.

Subscribe now

Charting the map

The camps in this landscape agree on a surprising amount! Federal R&D investment having a positive ROI overall is a longstanding consensus across metascience and science policy. In addition, metascience scholars and policy entrepreneurs largely agree that the current system of science funding is suboptimal, peer review can suppress risk-taking, administrative burden is excessive, career incentives distort behavior, and evidence should inform system improvements.

But those shared premises can lead to very different reform recommendations, depending on the theory of change, method, and intellectual tradition.5 This metascience landscape reflects the diverse priorities held by the metascience community. Some camps, particularly Innovation Economics, R&D Management, and Institutional Entrepreneurs, converge on a similar diagnosis of the funding system’s biggest problems. Others approach the same enterprise from different angles: Research Integrity worries about the reliability of published findings; Open Science prioritizes transparency. This intellectual diversity is healthy and is a sign that the field of metascience is maturing.

Why a map matters

We are in a critical window for R&D reform. The Trump administration’s proposed funding changes and reorganizations to federal science agencies create both pressure and opportunity to revisit the structure of federally funded research programs. Multiple kinds of metascience evidence can inform that work, and policymakers are recognizing it — expressing demand for economic evidence to drive system reform, R&D management knowledge to ensure reforms actually work, and examples of new institutions they can use in legislation and program design.

NSF’s recent launch of X-Labs is an indication of that appetite. X-labs will award institutional block funding to independent organizations with operational autonomy, with milestone-based awards to encourage team-based, infrastructure-heavy, and long-horizon approaches to research. This program structure was informed by metascience and is a new experiment in portfolio diversification.

We are also excited to see the evidence-generating and evidence-translating communities depicted in this map continue to grow. We believe the most durable reforms tend to draw from several perspectives at once. For example, a reform inspired by new institution existence proofs (institutional entrepreneurs), backed by causal evidence (innovation economics), embedded in an organizationally realistic design (R&D and innovation management) is more likely to survive first contact with reality than one informed by any single tradition alone. Building more connective tissue between metascience communities would help ensure that our perspectives and methods do not stay siloed and that our reforms are more robust.

Subscribe now

Building the connective tissue

Some funders have already prioritized evidence-building into their research programs, but this remains all too rare. Even the Congressional mandate to have an evidence-to-practice pipeline at Federal Agencies, including research agencies, under the Evidence Act did not produce a swell of metascience evidence.6

One of us (Jenn) experienced the dynamic of adapting evidence to program design firsthand while leading early-stage innovation programs at NASA. For program managers in the federal government, the demands of satisfying institutional requirements and culture while confronting resource limitations (people, IT, etc.) can make implementing changes to program design feel like hand to hand combat at best and insurmountable at worst. It helps when there is external evidence to point to, but that is often insufficient when proposing changes to institutional norms could introduce protest and oversight risk, create confusion for proposers and reviewers, and not pay off for many years.

In short, even if they want to, most program managers don’t have the time or the institutional cover to implement a routine evidence-to-practice pipeline that could in turn provide real-world experimental evidence.

Further, program managers often have their own hypotheses about program design and impact they’d love to evaluate. But they either are unaware of the metascience community eager to study these same questions or face significant barriers to working with the researchers (data access restrictions, timing of research findings not aligning with program reform windows, etc.).

Our map of the metascience landscape can help agencies recognize that they don’t need to embark on evidence production alone. For example, questions they’re already required to ask under the Evidence Act have intellectual homes — communities with methods, scholars, and accumulated evidence that could dramatically improve the quality of their answers.

At the same time, better coordination with the external metascience community cannot fully substitute for internal metascience capacity. Agencies’ experience with the last six years of learning agendas and evaluation plans developed under the Evidence Act demonstrates this: even when agencies have identified the right questions, they have lacked the in-house staff with metascience expertise to design and run high-quality evaluations, and the most policy-relevant data (proposals, reviewer scores, panel deliberations, and selection decisions) often cannot be shared with outside researchers for confidentiality and statutory reasons.

We believe that an important first step to drive more metascience understanding and improve the social ROI on R&D is to institutionalize metascience capacity inside the government through dedicated metascience units. In 2024, the UK established a Metascience Unit and the Administration’s FY 2027 budget request included a proposed Metascience Unit at NSF, which is a good start. These units should draw on research questions, methods and external expertise from across the metascience landscape.

Policymakers are hungry for evidence-informed ideas and answers. If we applied even a fraction of the funding we deploy for object-level science to studying what works and implementing those practices at scale, the payoffs could be enormous. Science needs more evidence about itself, and there is a broad metascience landscape poised and ready to help.

Subscribe now

As I said to the congressional staffers, I think there are meaningful differences between new scientific institutions, such as Focused Research Organizations and X-Labs, and the National Labs. That the Labs are massive, old, and federally-owned really is important when comparing them to institutions that are small, new, and free from regulatory and bureaucratic buildup. And it’s good to try new things; when the National Labs were set up in the middle of the 20th Century, they too were new scientific institutions.

I hope that some of the proponents of new institutions referenced in this piece (Caleb Watney and Ben Reinhardt) and anyone else with a take or a stake will respond. What can new institutions learn from the Labs? What can the Labs learn from new institutions? And how can metascience better integrate the Labs into our collective mental model of science?

Disclaimer from the author: This piece represents my personal opinions, and not the opinions of either of my employers.

Metascience proponents have produced a flurry of think pieces arguing that new institutions are needed to accelerate innovation in R&D. They contend that academia, government, and industry are “ill-equipped to solve” important challenges because they “don’t provide the engineering teamwork needed to produce the platforms, datasets, or tools” required.

I agree with metascience’s emphasis on continuously reevaluating whether we are maximizing scientific progress. But many of these think pieces fall prey to a common pitfall in academic R&D: novelty bias. While it’s understandable to be excited about novelty, this tendency can lead to reinventing the wheel, where we ultimately recognize we had the tools we needed all along.

I hesitate to pick on my hosts, but the X-Labs described by Caleb Watney sound a bit like the National Labs. To quote Watney, “X-Labs would fill a longstanding structural blind spot in the US research ecosystem: work that is infrastructure-intensive, team-based, exploratory, or oriented around critical bottlenecks.” They “would provide stable, full-time roles for staff scientists, engineers, and technicians, enabling institutional memory and technical depth more typical of industrial R&D.” Sounds familiar.

Though they differ in size, nimbleness, and specificity of mission, Focused Research Organizations (FROs) are pitched with similar features and an added dose of Bell Labs nostalgia.1 While we might want to recreate the creative spirit of Bell Labs in new institutions like FROs, the infrastructure and personnel structure actually survives within the National Labs.

Does the call for new institutions describe what the National Laboratories already do? As I discuss in this article, the answer is that sometimes metascience writers are describing what the Labs already do and sometimes they’re describing what the Labs aspire to (but don’t always achieve). It’s worth revisiting the role of the National Labs. The metascience discourse neglects them in favor of new models for understandable reasons, but we risk leaving massive value on the table if we fail to make National Labs a central component of efforts to accelerate progress in R&D.

Subscribe now

What are National Labs, exactly?

DOE National Labs are part of the broader category of Federally Funded Research and Development Centers (FFRDCs).2 Their mission is both to advance fundamental science and to occupy a missing middle: research that is too complex for a single principal investigator grant, but too risky or oriented on public goods for a startup or a large industrial incumbent to take on. National Labs shine precisely when the scale of a challenge demands approaches that are infrastructure-intensive, team-based, and motivated by critical bottlenecks, to borrow from Watney.

National Labs are a significant portion of the federal R&D budget. As of fiscal year 2024, the government invested over $31 billion in R&D at all FFRDCs, and DOE National Labs are a substantial portion of that investment. DOE contributes nearly $18 billion to the Labs. This figure is significantly larger than the budgets of other R&D funders, such as the intramural investments by the National Institutes of Health (~$5 billion) or the entire National Science Foundation (~$9 billion). Granted, ~40% of DOE’s overall spending is in the National Nuclear Security Administration (NNSA), whose programs in national security and non-proliferation makes the separation between “fundamental science” and “weapons maintenance” fuzzy. But NNSA directly manages three of the largest National Labs (Lawrence Livermore, Los Alamos, and Sandia), which house some of the flagship large R&D infrastructure and carry out plenty of fundamental science, so they cannot be dismissed simply because of their nuclear weapons maintenance function.

By design, National Labs house a lot of infrastructure. It is economically inefficient for every university or firm to have a particle accelerator, so that sort of equipment makes sense as a long-term federal investment. This practice holds true across fundamental–to-applied capabilities and scale, ranging from massive facilities to decentralized advanced instrumentation scattered across an institution’s buildings and campuses. The best-known examples of the former are the various X-ray light sources (synchrotrons) or the exascale computing facilities. On the NNSA side, they include the National Ignition Facility, which is important given renewed federal interest in fusion energy.

On the applied end of the basic-to-applied research spectrum, we find examples like facilities to accelerate manufacturing scale-up and nuclear test reactors. Many of these facilities have a mission to attract external users, which allows tens of thousands of scientists outside the National Laboratory complex to use them for their own research.3

Subscribe now

Complex, multi-year engineering loops require experienced, permanent staff who are not splitting time with coursework or dissertations. National Labs are explicitly built and staffed for this kind of long-term team science. Their emphasis on problems that span from basic to applied forces cross-disciplinary collaboration and removes silos that are typical of academic institutional structures. This structure builds multidisciplinary density. It co-locates, say, domain scientists, computational experts, and process engineers and organizes them into a stable, long-term team to solve large challenges. Staff permanence reduces loss of tacit knowledge inherent to the passing of generations of students.

While universities have departments that cover a vast constellation of topics, National Labs are more narrowly focused on energy technology (albeit broadly defined). And while universities have a dual mission of education and research, National Labs are primarily focused on research. As a result, universities have a high proportion of graduate students, who are on a roughly five-year clock, compared to the primacy of the permanent (not to be confused with tenured) staff scientist model of National Labs.

What does this structural design buy?

The structure of National Labs allows them to perform many of the functions associated with new institutions for science: taking on ambitious, focused initiatives; collaborating on pre-commercial technology; and crafting the future of AI for R&D.

National Labs are arguably the original FROs. There are many examples of “time-bound, technically ambitious efforts” at National Labs designed to unblock whole domains of technical progress (the Manhattan Project might be the most famous one). And although it’s not their primary focus, the National Labs played important roles in biomedical research: the Labs were crucial to the Human Genome Project, for instance, by hosting unique, large omics facilities that were accessible to users nationwide. In addition, the Protein Data Bank, the foundation on which AlphaFold was built, was founded at Brookhaven National Lab and contains data measured at the US synchrotrons.

Subscribe now

National Labs also offer an alternative to academia for scientists who need access to “equipment, skills or status” to advance their pre-commercial idea. In “Unbundling the University,” Ben Reinhardt writes that “it’s very hard for individuals and organizations to avoid interfacing with academia if they have an ambitious pre-commercial technology idea.” They can interface with the National Labs instead! Universities largely focus on basic generalizable principles and industry focuses on commercial deployment. National Labs are uniquely positioned to bridge this gap because they house both basic and applied R&D programs and can facilitate collaboration on pre-competitive research between academia and industry. Some of the applied programs at National Labs are explicitly tasked with taking a concept derived from a lab and piloting it, including through partnerships with industry.

The National Lab model is also uniquely suited to using AI and autonomous experimentation to accelerate science. Many of these facilities are centralized and operate year-round, so they can generate the massive, standardized datasets required to train foundational models, something a decentralized network of university labs cannot do. Moreover, the National Labs can host the repositories needed to curate large sets and make them widely accessible.They are also good hosts for large cloud labs that require millions of dollars in investment and upkeep. It is much harder for universities to serve this role because you cannot build a self-driving lab or a long-term AI-supported research loop if context is lost every time a student graduates. Institutions with staff permanence hold the substantial tacit knowledge needed for initiatives like the Genesis Mission, so it makes sense that the National Labs are central to their implementation.

So are National Labs the answer?

Although National Labs have many of the capabilities and staff needed for, say, an X-Lab, I must partially agree with the metascience writers: while the Labs can play the role that FROs and X-Labs are intended to, they often serve different functions and therefore require distinct strategies to maximize our return on R&D investment.

More specifically, the institutional dynamics of National Labs deviate from the model of X-Labs and FROs in four ways:

1. The current National Lab model may lead to mimicking the academic approach to research, structurally and culturally, rather than fulfilling the goal of complementing and augmenting academia. There is a tension between serving users at a large-scale facility carrying out their own R&D mission, competing for broad-based grant calls, and seeking scientific publications.4

2. Access to large facilities at National Labs is not as smooth as many in the private sector would want, and it can be expensive. But there are ideas to address barriers to using facilities.

3. Intellectual property agreements and security clearances create friction points to starting close partnerships with industry. But the Labs remain the only entity with the scale and neutrality to play this role, making the bureaucratic hurdles worth navigating.5

4. National Labs are subject to high overhead costs, which inspire questions about whether they are the most efficient use of tax dollars to achieve a desired R&D outcome.

Some of these issues are a result of institutional design. After all, DOE is a government agency that also uses the National Labs to manage nuclear weapons. It would be tough to get a National Lab to operate with the nimbleness and scrappiness of a start-up. Nevertheless, the Labs really do have many of the characteristics that proponents of new institutions say we need, and they have achieved impressive scientific feats. National Labs can be optimized, and metascience analysis and reform should consider how to refresh these institutions, rather than only angling to build new ones.

Subscribe now

Direct metascience analysis at the National Labs

For infrastructure-intensive, engineering-heavy, decadal science, there is no substitute for the National Labs. No FRO is ever going to build a synchrotron or anything like the portfolio of delocalized, cutting-edge instrumentation that a National Lab offers. Metascience’s reform portfolio misses an opportunity to put the impressive National Labs infrastructure to better use. Rather than ignoring the Labs, we should use metascience approaches to investigate the following questions (to start):

1. How can we further activate the significant potential of the National Labs? What more can we get out of the existing ~$30 billion per year National Lab network? We need a better answer than just “National Labs are too slow.” How do we make them fast(er)?

2. What falls outside the National Labs jurisdiction and institutional design? Those should be the target for new institutions (X-Labs, FROs, BBNs…).

3. How do university and National Labs’ rewards for performance differ? Are the incentive structures at National Labs sufficiently different from universities to meet their distinct roles in the R&D landscape?

Attention to these questions will allow us to more tactically target interventions to fill the gaps that National Labs fundamentally cannot close. The new institutions envisioned by metascientists may differ from National Labs in some aspects, but National Labs certainly belong in the toolkit. It’s time to consider how to use them better.

Subscribe now

I enjoy when Substackers share what they’ve been reading (Brian Potter and Lauren Gilbert publish roundups I like); it’s a good way to find pieces I might have missed and gives me a glimpse at what other authors are thinking about. Now I’m paying it forward.

Here are four articles and one new website that have influenced my thinking recently. These pieces span a few questions I’ve been considering: how science is globalizing, how it’s shifting with new technology, and how we can better measure its benefits.

1. The Pop-Up Journal Initiative

The Alfred P. Sloan Foundation and Coefficient Giving launched this initiative to “curate and synthesize evidence around specific, policy-relevant questions” with the goal of delivering actionable evidence to decision-makers. This is intended to be a series of journals, and the topic of their first — what is the social return on R&D? — is neglected. Our knowledge about the returns to R&D has grown in recent decades, but we still need more granular evidence of what is driving returns, and in which sectors and areas of the world.

I’m intrigued by the project’s format. Like focused research organizations (FROs) and Renaissance Philanthropy’s time-bound funds, this pop-up journal will dedicate bounded time and funding to producing policy-relevant evidence. By providing new research funds, Sloan and Coefficient Giving will grow the community of researchers studying the returns to R&D (and those researchers will likely continue on that work after the journal, uh, pops down).

The journal is accepting applications for research grants through April 30, 2026 (I realize the deadline is very soon — but perhaps you have a good proposal lying around or can work quickly). They’ll provide $250,000 for studies that will “provide key empirical insight into the social and economic returns to R&D investment.” Larger requests may be considered for uniquely ambitious projects. Apply here.

Subscribe now

2. “Are we ready for a multipolar world of research?” (Carlos H Brito Cruz in the London School of Economics Impact blog)

Brito Cruz points out a blindspot in the debate on whether the US or China is the most scientifically advanced country: in the meanwhile, low- and middle-income countries (LMICs) have been building their scientific capabilities. For the first time ever, the majority of authors in the SCOPUS Abstract and Citation database are from LMICs. In my view, this piece is a story about (1) China, of course (still technically middle-income), but also (2) developing countries investing in science, and (3) technology — from Zoom calls with co-authors to Google Translate — making it easier than ever for LMIC researchers to publish.

The piece ends, as more articles should, with a Bob Dylan quote and an exhortation to work harder:

“For the first time in history, there are more active researchers outside the richer countries…The impact this will have on the advancement of knowledge is only beginning to be grasped. Research institutions across the globe might do well to heed the words Bob Dylan wrote of the epochal changes of the 1960s, ‘you better start swimming, or you’ll sink like a stone.’”

3. “Does MAGA Actually Want American Science to Win?” (Ari Schulman in the New Atlantis)

What do rightwing science reformers actually want? What do they actually think would make American science stronger? Schulman argues that, while skeptics of the scientific status quo describe a destructive higher education reckoning as necessary, they don’t have a vision for what a better science would look like. Schulman writes:

“When asked how slashing support for science by about half, as the administration is proposing to do across its research funding agencies, will make American science stronger, the answer is always about how serious science’s ideological mistakes during Covid and the Great Awokening were, and how deserved and desirable is the correction — an obviously true claim that simply has nothing to do with the grave question being asked.”

Subscribe now

4. “There is no randomising a technological revolution” (Oliver Hanney in Oliver’s Substack)

This piece has some echoes of Tim Hwang’s Macroscience piece about the difficulty of measuring what works in science during a rapid technological transformation, but shifts the focus to international economic development. International development has been a testbed for randomized controlled trials (RCTs) and other innovations in economics (some of which have been an inspiration for metascience), and the field has leaned on RCTs to validate that what we think is working is actually working. But with AI increasing the speed of social and economic change in LMICs, RCTs on — for example — job training may end up measuring how things used to be rather than how things are.

I agree with Hanney that, in a time of rapid change, economists should neither limit themselves to analyzing things that can be readily measured, nor keep quiet when they have evidence-informed views — even if they aren’t 100% sure of them.1 As Hanney notes, an economist’s intuition about what will happen to developing country labor markets or redistribution will likely be quite good relative to someone who doesn’t have any background.

5. “How do scientists use Claude Code?” (Charles Yang in The Republic of Science)

We already know about common AI use cases (automating repetitive tasks, simplifying complex statistical projects, visualizing data, etc.). However, as Yang notes, we don’t actually know much about who is using advanced AI in academia, or how. But we should — AI has the potential to accelerate (and disrupt) research and as metascientists, and we need to understand the factors that influence scientific productivity.

Yang uses a clever approach of analyzing academics’ ORCID-linked GitHub profiles to study what kinds of scientists (seniority, location, etc.) are using Claude Code.

Around 2% of scientists with ORCID-linked GitHub profiles are using Claude Code. This is probably an undercount — there are likely plenty of scientists who are not connecting their Claude Code account to GitHub and their GitHub to ORCID.

I think we can go beyond Yang’s use of ORCID to see who is using Claude Code, and reach a larger, more representative sample of scientists. This is an area where good, old-fashioned survey research could be useful (the UK is doing something similar), to ask, for example, which AI tools US academics have heard of, which they use, and for what.

Subscribe now

In 2025, the federal science enterprise faced the most dramatic proposed budget cuts in modern history. The White House requested reductions of nearly 40% to the National Institutes of Health (NIH) and 57% to the National Science Foundation (NSF), alongside major cuts to other science agencies.1 These policy requests ran in opposition to public sentiment; in fact, new evidence shows that a large majority of Americans actually want science funding to grow.

Subscribe now

Fortunately, Congress ultimately rejected the President’s Budget Request, cutting the nondefense R&D budget by 5%, rather than the 21% requested by the White House. But while congressional appropriations were higher than many expected, scientists still have reason for concern: data shows that the government may not spend all of the money that Congress authorized for fiscal year 2026. Science spending for the year thus far has been slower than in previous years, and fewer new grants have been released than are usually out by this time of year. While that may be compounded by the fall 2025 government shutdown, the White House has also been slow in approving agency spending plans. If agencies don’t spend their appropriated funds by the end of the fiscal year, they risk losing them.

To meet the public demand to protect science spending, the Executive Branch must actually spend the money Congress appropriated.

New evidence shows that US citizens largely support federal science funding

When the Trump Administration came into office in 2025, it began freezing and terminating science grants and laying off staff at federal agencies. In response to these cuts, and with funding from the Science for Progress Initiative at J-PAL, Francesco Capozza, Krishna Srinivasan, and Mattie Toma surveyed 2,008 US citizens about how much the US should spend on R&D.2 They found that over 80% of Americans want to increase R&D spending, and that the median citizen would prefer R&D to receive 7% of the federal budget, compared to its current 3%.

What made this study unique, compared to most public opinion polls, is that the researchers gave respondents context on the federal budget and tested whether question phrasing would influence their stated preferences. Participants received a short primer on what R&D is and which domains federal R&D funding supports, and were then randomized to receive different framings of the question. In one version, respondents were asked to allocate the total federal budget across different categories, including R&D. In the other, they were first told the actual allocations to different spending categories before being prompted for their own distribution.

Across these groups, preferences remained consistent. No matter how the question was posed, respondents preferred allocating 6–10% of the federal budget to R&D, and most respondents favored growing R&D spending beyond current levels. There was some variation in opinion by demographics: respondents with higher levels of education and higher incomes were more likely to support increased R&D spending, while conservative respondents were a bit less likely to want increased spending than liberals.

The study also surveyed experts in science and innovation, including both researchers and civil servants. One notable finding is that experts tend to greatly underestimate citizens’ support for federal R&D: 76% of experts mistakenly believe that Americans want to either maintain or cut current R&D spending, predicting that the typical American would prefer 3% of the budget to be allocated to R&D (the amount currently allocated).

Let us caveat these findings: survey results are not always airtight and voters may not put their money where their mouths are. For example, poll respondents have consistently stated support for gun control measures that they then fail to support at similar levels in state referenda, and public opinion on healthcare reform has also been inconsistent. These may be thermostatic changes in public opinion — voters don’t become activated against a policy or politician until it is dominant. But while we should hesitate to take respondents literally, there’s no evidence that they would endorse the proposed budget cuts.

Subscribe now

How does this evidence compare to other polling on science funding?

Polling from the General Science Survey (GSS) shows less support for science funding, but methodological differences may explain the gap. In polling conducted between 2002 and 2024, 34–44% of Americans reported that they think the US spends too little on science — considerably lower than the 80% found in Capozza et al, 2025. But the 8–13% who said the US spent too much on science in the GSS more closely aligns with our focal study’s finding that 11.75% of Americans wanted to lower R&D spending.3

While we can’t draw clear conclusions across two distinct studies, the context on the federal budget provided to respondents in Capozza et al. may have helped participants make more informed decisions.4 And in fact, prior to learning that 3% of the budget goes to R&D, the median respondent thought it was around 10% — their preference for increased science spending might be a response to learning that it was lower than they’d thought.

Another notable finding is that while Americans are critical of universities, they value the research universities produce. It’s broadly known that the public is unsatisfied with higher education institutions: according to a 2025 Pew Research survey, 70% of Americans think that higher education is headed in the wrong direction. This view is shared by Republicans and Republican-leaning independents (77%) and Democrats and Democratic leaners (65%).

But despite their frustrations with rising tuition costs and inadequate pre-professional support, 55% of Americans hold a positive view of universities’ ability to advance research and innovation. In the same vein, 77% of US adults say they have a “great deal” or “a fair amount” of confidence in scientists acting in the public’s best interests.

Taken together, the new data we’ve presented, the GSS data, and survey data from Pew suggest a coherent picture. Americans support science funding. When told how low it is, they support more science funding. And while they have grave concerns about American universities, they’re supportive of the scientific function of higher education.

If you are a member of Congress, you can confidently appropriate funds for science, knowing that the winds of public opinion are at your back. And if you’re in the Executive Branch, you can be sure that Americans want you to spend appropriated funds. Given these facts, why has science spending been so slow to get out the door?

Subscribe now

What’s going on with spending?

Agency data visualized at Tracking Science Spending suggests that the executive branch is not spending appropriated funds at its typical pace, leading to concerns that some science funding may not be used at all. This trend has raised the question of whether the Executive Branch can simply decide to not spend appropriated funds.

Historically, this has not been a major concern. The Impoundment Control Act requires that the Executive Branch spend money appropriated by Congress unless lawmakers explicitly approve unspent funds via “rescissions.”5 However, this assumption has recently been called into question, and the head of the Office of Management and Budget (OMB) has stated that he believes the Act is unconstitutional. And while the Trump administration ultimately spent most appropriated science funds in 2025, it awarded fewer grants than usual and spent much of the money quickly at the end of the fiscal year after pressure from Senate Republicans.

While the underlying reason for the slow pace of spending is unclear, failure to spend appropriations introduces the risk that OMB will either request rescissions from Congress (as it did in 2025), attempt a “pocket rescission” where they simply do not spend money at the end of the fiscal year, or force agencies to spend quickly at the end of the fiscal year.6 While the last case is the best option, as we saw last year, this creates funding uncertainty.

Congress has heeded public demand by appropriating science funding. Now the Executive Branch should spend the appropriated funds. Delaying or withholding appropriated science dollars has clear consequences: fewer grants, disrupted research programs, lost talent, and ultimately slower progress.

Americans support science funding. Policymakers should act accordingly.

Subscribe now

Factory Settings

How to update the default settings of government, by former CHIPS Program Office leadership.

Factory Settings Editor’s Note:

In August 2025, Secretary of Commerce Howard Lutnick cut funding to Natcast, a multi-billion dollar semiconductor R&D initiative enabled by the CHIPS and Science Act (also referred to as the CHIPS Act). Lutnick claimed that Natcast violated the law and accused the initiative of cronyism.

Taking a closer look at the program makes clear that these were misplaced accusations. The structure was neither radical nor corrupt, but rather a serious attempt to achieve the program’s outcomes under an ambitious mandate. It seemed like Natcast was on track to succeed — sunsetting it puts years of deliberate development to waste.

Today we bring you a piece that explains Natcast’s design from one of the leaders who set it up, Donna Dubinsky.

Although much attention has been focused on the $39B allocated by the CHIPS Act to build fabs1, Congress also provided the Department of Commerce with $11B for research and development (R&D), the centerpiece of which was called the National Semiconductor Technology Center, or NSTC.2 The Act specifies that the NSTC should conduct research and prototyping of advanced semiconductor technology, grow the domestic semiconductor workforce, and establish an investment fund. Congress required that this effort be operated as a public private-sector consortium with participation from industry, academia, the Department of Defense, the Department of Energy, and the National Science Foundation.3

Implementing this vision raised difficult questions. What institutional structure could balance government oversight with private-sector commercial mindset and speed? How should research priorities be set, and by whom? Should the effort be short-term or long-term? Below, I describe the gaps the NSTC was designed to fill, the key decisions we made in standing it up, and the lessons future policymakers should draw from its creation — and its termination.

NSTC was established to fill gaps in US semiconductor R&D

While tens of billions of private dollars are spent on semiconductor research in the US each year, some clear gaps still remain.

• Shared research facilities: The cost of creating a modern semiconductor research fab or advanced packaging facility is in the multiple billions of dollars. Production fabs are optimized for volume, not experimental work. While big companies have dedicated research facilities, some research needs cannot be served in-house because of possible material contamination or disruption to the production workflow. Smaller companies and academics have constrained access to such facilities altogether. Other governments, including China, Japan, and the European Union, have made large public investments to establish shared research facilities which are used by companies from around the world. The US has nothing comparable, thus sending advanced research overseas.

• Pre-competitive research: Certain research will not yield products today but lays the groundwork for future development. Much of this work happens in universities or university-industry partnerships, while companies focus on near- to medium-term products. The NSTC would fund this longer-horizon research, bridging the gap between academic discovery and commercial application.

• Workforce development: Companies tend to invest limited capital in targeted programs to fill identifiable, near-term needs rather than uncertain, long-term needs. Government funds can be used to expand education to prepare next-generation workers for future jobs across the growing industry.

• High-risk investment: The cost and time from idea to market for new semiconductor technology has grown so large that most private investors cannot tolerate the risk. For example, the last major innovation of the semiconductor industry, the FinFET, took 17 years to mature from university research to production by Intel. Venture capital invested in chips has declined from almost 5% of total VC dollars in 2010 to just over 1% today. Government funds can stimulate private investment in high-potential American semiconductor start-ups.

The public-private consortium model was well suited to this effort: the NSTC vision required diverse private-sector skills (technical expertise, workforce training experience, venture investing knowledge), the capital to build shared facilities, and a direct collaboration to yield both national security and commercial benefits.

Subscribe now

Implementation decisions

After the CHIPS Act passed, Secretary Gina Raimondo convened a team to evaluate how best to execute the NSTC. We gathered input from all constituents — the private and academic sectors (particularly the Industrial Advisory Committee formed under the CHIPS Act), the other agencies named in the Act, and the White House Office of Science and Technology Policy.

1. Grant program or an institution?

One foundational question was whether the NSTC should operate as a series of time-limited grant programs or as a lasting institution managing long-term assets.

Grant programs could be launched quickly using established government vehicles, but in deciding between a cluster of programs and a new institution, we needed to consider congressional intent. Most significantly, the Act mandated activities that required long-term investment at a large scale, particularly prototyping capabilities and advanced packaging facilities. These activities would consume most of the appropriations. While they could be implemented with the help of partners, they would need funding beyond the CHIPS appropriation. An institution that could recruit members, offer fee-based services, and build private-sector financial support was required.

2. What is the right institutional structure?

Having determined that the NSTC vision required an institution rather than a series of funding programs, we studied existing public-private partnerships and legal precedents to understand best practices. Two models were especially instructive. SEMATECH, a US semiconductor research consortium formed in the late 1980s, initially succeeded but collapsed after it stopped accepting government funds and became dominated by a few large firms. By contrast, imec, an independent nonprofit founded in Belgium in 1984, has thrived for over 40 years by maintaining independence while retaining approximately 20% public funding. The table below summarizes the models we examined.

Drawing on these lessons, we evaluated governance structures against the criteria our research had established: sector-wide independence, ability to attract senior talent, operational speed, capacity to attract private capital, and ability to create a strong government partnership. We concluded that we needed a new, focused nonprofit — purpose-built to attract senior talent, partner with the government, and earn industry credibility.

To comply with the Government Corporation Control Act (the GCCA), which prohibits the government from creating and managing a corporation without statutory authority, the Department selected experienced independent citizens through an open federal application to appoint a board. The initial board of trustees selected by the citizen committee included accomplished retired semiconductor executives, leading academics, and experienced corporate directors (I left the government and joined this board). The new board incorporated a nonprofit eventually named Natcast and hired an executive team who negotiated a contract with Commerce to operate the NSTC. This governance structure is not uncommon: SRI, In-Q-Tel, and Natcast all operate through standard government contracts rather than government board control.

Subscribe now

3. What is the right balance between government and private sector involvement?

Structuring the government-Natcast relationship was complicated. Too much government power risked politically driven decisions on programs and difficulty recruiting talent. Too much private-sector power risked national needs going unmet and domination by the largest companies.

The solution separated agenda-setting from execution. The government, Natcast staff, and a technical advisory board (comprised of leading technologists from industry and academia) would propose research topics; the government would approve topics and funding levels; Natcast would then run operations — awarding contracts and selecting recipients on purely technical and operational criteria such as feasibility, potential impact, and team experience, free from political interference or industry dominance. Natcast remained fully accountable through rigorous contractual obligations, national security compliance, and extensive reporting to Commerce. The same model applied to other programs such as workforce development and the investment fund; the government and Natcast together set strategic priorities, while Natcast selected and managed execution decisions.

4. When should we spend our appropriations?

We engaged a consulting firm to build a long-term financial model, proposing to use the appropriated funding — which fortunately did not expire — over 10 years, with preliminary budgets, membership structures, and program costs.

A strategic tension persisted. Many felt the goal was full self-sustainability by year 10. Others, myself included, believed ongoing government funding was critical — particularly for pre-competitive research that industry wouldn’t fund. When the entity is fully self-sustaining, there is no obligation to fulfill national interests, only corporate near-term objectives. The SEMATECH and imec case studies support this conclusion, with SEMATECH collapsing after declining government funding and imec succeeding over many years with ongoing support from the EC and the Flemish government. Since such continuing national support couldn’t be guaranteed, the model presumed self-sustainability — but one could return to Congress in five to six years and use successful outcomes to make the case for ongoing funding. In effect, the issue was deferred.

What we built

Natcast was incorporated in October 2023 and set about developing its own strategic plan, operating plan, and financial strategies. Although there were multiple major programs to create, Natcast made progress on all fronts:

• Early research programs were announced and some grants awarded; several other calls for proposals were ready to release.

• A Technical Advisory Board was established and a research agenda articulated.

• Partners were selected to build the prototyping center and the advanced packaging facility, and the EUV center was under contract, ready to start implementation.

• The Investment Fund was fully specified and had attracted substantial private interest.

• Workforce development programs were underway with academic institutions across the country.

• A variety of other programs were designed such as shared digital resources and a multi-project wafer capability

• Over 200 companies and institutions joined as dues-paying members.

Less than two years after creation, Natcast was executing well and meeting its goals.

Subscribe now

What happened?

In late August 2025, Commerce announced its intention to discontinue Natcast as the NSTC operator. The contract was terminated on the pretext that Natcast’s creation violated the GCCA, despite extensive review on this question within Commerce’s legal department and a green light from the Department of Justice’s Office of Legal Counsel in the Biden Administration.

Commerce’s current plans are unclear. In September 2025, the Department announced a research grant program, but only one preliminary award has been made public. NIST suggested that the new approach “will mirror a more venture capital-style approach” and “strongly recommended that applicants include an approach to financial return on investment” for the government in their proposals such as granting the government equity, warrants, licensing, royalties or revenue sharing.

Beyond Natcast’s discontinuation (and the apparent termination of the NSTC itself), the Industrial Advisory Committee has been disbanded, the National Advanced Packaging Manufacturing Program is not active,4 the new semiconductor-focused Manufacturing USA Institute has been discontinued, and the Consortium Steering Committee has not met since the change of administration.5 As these activities are mandated by the CHIPS Act, it is not clear how Commerce intends to comply with the Act without substantially increasing staff — at odds with the administration’s push for smaller government. From the outside, the new CHIPS R&D vision appears more like a profit-driven investment program than a provider of core infrastructure benefiting all participants and prioritizing American national and economic security.

Lessons for future policymakers

Natcast arose out of careful study and deliberation with multiple stakeholders, fulfilling the ambitious mandate Congress passed in the CHIPS Act. Despite the substantial progress in realizing this vision, it was hastily abolished without a clear replacement plan.

While it might be impossible to legislate against this kind of reversal in the future, I believe that if Natcast had been six months further along, it is less likely that its work would have been halted because many more constituents would have been impacted. This underscores how critical it is to move quickly. The program was delayed, in part, by preferencing the incentives program for leadership attention, the need to work across multiple government agencies, and by the substantial delay needed to resolve legal issues to ensure compliance with the GCCA.

In thinking about future programs, policymakers should consider the following:

1. Match investment time frames with desired outcome time frames. To the extent that a policy has a long-term goal, it needs to be matched with a long-term investment structure. Had the R&D part of the CHIPS Act been viewed as a short-term need, say research programs and not facilities, it could have been clearly defined and easily implemented. But since the NSTC was envisioned as a long-term effort to become globally competitive, it needed the corresponding ability to execute over the long term. By contrast, the $39B fund for incentives was imagined and executed as a time-limited grant program.
   
   Two specific long-term aspects could have been enabled in the legislation. First, if the legislation had permitted the creation of a government corporation, that would have been a possible execution path. In that case, the effort would have been more insulated from political change because the governance structure could have had several trustees appointed by the current administration, enabling a new administration to impact the agenda without resorting to outright cancellation. Second, the question of sustainability could have been addressed through some notion of continued funding possibility, particularly for the research program, even if not an explicit commitment. A model that spelled out a funding renewal process (contingent upon program success) would have enabled planning that could presume ongoing government participation.
   

2. Designate clear responsibility for implementation. The CHIPS Act called for the Department of Commerce to lead implementation, giving a clear signal that commercial success was as important as national security. Congress provided separate funding to the Department of Defense to address national security needs. However, the Act also required Commerce to set up the NSTC “in collaboration with the Secretary of Defense” and it required participation from the Department of Energy and the National Science Foundation.
   
   While it is admirable to strive for agreement among multiple agencies, we found that the complexity of consulting so many agencies (as well as the White House) on so many programs caused unnecessary delay. For example, the Act could have said that Commerce would design the program in consultation with the other agencies, but not require active collaboration or participation.

   I estimate that, had we been able to create a government corporation and just consult with other agencies rather than fully including them in program design, the program could have been launched nine months to a year earlier — critical timing for an urgent program.
   

3. Establish a group of expert advisors. The CHIPS Act specified the creation of an Industrial Advisory Committee for the research effort. Appointing and convening this group was one of the earliest activities undertaken at Commerce. Although there were many other ways we received input, including RFIs, individual meetings, papers presented by constituents and industry colloquia, there is no doubt that the IAC was the most efficient means of getting high-quality input.

   An excellent group of technical advisors from industry, academia, and government was selected and their reviews and reports turned out to be invaluable for our work. Although bringing together a group of fiercely competitive companies risks inviting a battle of parochial needs, in this case, the individuals selected and the effective management of the IAC enabled a truly collaborative effort. Whether articulated in legislation or not, the thoughtful selection of key individuals from the private sector to advise the program design can surface relevant input efficiently.

Conclusion

The cancellation of Natcast’s contract dismantled a new institution poised to make a generational investment to address structural gaps in America’s semiconductor research ecosystem. The research agenda, the prototyping and advanced packaging facilities, the investment fund, the workforce pipeline, and the 200-member consortium cannot be replicated with a short-term grant program. What Commerce has proposed to replace the entire NSTC program is a grant program that demands equity stakes and revenue sharing from recipients, and is thus unlikely to attract broad industry participation. The damage will compound over time as researchers, start-ups, and industry partners redirect their efforts to better-supported ecosystems overseas.

The deeper lesson here is about policy continuity. Major semiconductor technology developments take decades and require collaboration between industry, government, and academia. China, Japan and the EC are executing semiconductor development plans well, with consistent funding and institutional support, often across leadership transitions. The United States will not succeed in this geopolitical competition if critical programs can be canceled every four years. The NSTC’s termination signals to industry, to allies, and to rival nations that American industrial policy commitments are provisional.

The creation and funding of the NSTC was a compelling, bipartisan effort to address clear gaps in our industrial ecosystem, and Natcast’s potential was promising. Future policymakers should strive to insulate such initiatives from political headwinds. NSTC and Natcast’s elimination is a missed opportunity for American leadership in the industries of the future and for our national security.

Donna Dubinsky has spent her career in Silicon Valley helping to advance technology. She is a serial entrepreneur and was CEO of Palm, Inc. and co-founder of Handspring and Numenta, a neuroscience-based AI company. She joined the Commerce Department in 2022, reporting to the Secretary of Commerce, to lead the Department’s implementation of the CHIPS Act. She left the government and subsequently served as an unpaid trustee at Natcast. These comments represent her personal opinions and not those of any organization or other individuals.

The advancements that Dan describes in this piece took decades of effort and investment. We didn’t gain our knowledge about the brain in a flash of insight. Like everything worthwhile, it took work. We should celebrate the thousands of scientists and technicians who have made these advancements (even if they don’t always celebrate themselves).

Earlier this month, Eon Systems took a victory lap around Twitter after making some (incremental) improvements on a long-standing body of neuroscience literature. They claimed to have demonstrated “the world’s first embodiment of a whole-brain emulation that produces multiple behaviors,” and they had a dazzling animation that captured people’s imaginations.1 I was deeply involved in the work they built on, and was a little peeved by how they overhyped what they had done. Anyone who looks at the decades of prior work they built on will see that the improvements they made were squarely in line with what the field was already working on, and that patient, sustained effort from scientists in the field will be necessary to unlock meaningful achievements in whole-brain emulation.

Let me cash in some of my hard-earned neuroscience street cred to set the record straight on the state of the art.2 The field has made some extraordinary advancements in the last decade that are worth championing.

Subscribe now

A primer on systems neuroscience

In the Matrix, the protagonist Neo plugs his brain into a computer that effectively installs new abilities and behaviors, including kung fu and flying a helicopter. Systems neuroscientists try to run that process in reverse. We identify a behavior — say, kung fu fighting — and try to figure out which parts of the brain are associated with that behavior and how those parts coordinate to pull it off. Successes in the field can lead to innovations like more efficient AI algorithms and more targeted interventions for mental health diseases, offering alternatives to the side-effect-heavy medications often used for today’s treatments.

To put it more technically, systems neuroscientists think about the brain as a combination of different circuits that drive different behaviors. Each circuit consists of a set of neurons and their connections. Systems neuroscientists try to link specific neural circuits to a behavior and explain how the circuit properties enable that behavior. The hope of the field is that by figuring out how the individual circuits work, we can build up a full picture of the brain.

But we ain’t there yet!

We’ve made a lot of progress in understanding individual circuits, but those circuits are mostly tied to very specialized behaviors, like how a fly determines which parts of its body to clean in what order. Our picture of how the brain works as an overall system remains very fuzzy, and we still have no idea about how something like consciousness emerges.

I was drawn to systems neuroscience by the promise of answering big questions like this, as were many physicists and engineers who became interested in the field around 2010. We were tantalized by a raft of new tools that offered the possibility of understanding the brain from first principles. Many of these folks went on to refine and expand these tools and to develop amazing ones of their own.

Subscribe now

Recent advancements

The innovations of the last decade or so have focused on attempts to reverse-engineer the brain. Reverse engineering an electrical circuit requires identifying all of the different electrical components — the transistors, resistors, capacitors, and so on — figuring out how they’re wired together, and then measuring their electrical activity to make sense of their role in the circuit. Neuroscience tools now enable similar investigation of the brain.

The electrical components of the brain are the neurons. When I started in the field as a physicist with little biological training, I thought all neurons were the same and that the magic of the brain was the result of how neurons are connected to each other. Not true! There are hundreds of different types of neurons, which scientists organize into groups called cell types, each with their own electrical properties. For over a century, we’ve had to identify different cell types one by one, somewhat randomly. But new tools allow us to identify and categorize cell types en masse by their molecular identity, anatomical structure, and/or connectivity.

We can now also determine how hundreds of thousands of neurons are connected to each other and produce connectivity diagrams known as connectomes. The first connectome mapped all 302 neurons in the brain of a worm back in the 1980s. It took us another three decades to scale up to the brain of a fly, which has over 100,000 neurons. We can now construct the connectome of neurons in up to one cubic millimeter of tissue and have completed reconstructions of small parts of mouse and human brains. Scientists continue to improve the technology and hope to complete a full connectome of the mouse brain in the next decade. A monkey brain will likely follow, and ultimately a human one.

Subscribe now

The final piece of the puzzle is to measure the electrical activity of lots of neurons at once. The traditional way of doing this is by sticking an electrical probe into a brain. Recent developments have led to smaller and increasingly sensitive probes that can measure the activity of thousands of neurons at once.

However, if you stick a probe in the brain, you can only measure the activity of the neurons that are next to the probe. So scientists came up with a mind-glowing way to measure the activity of neurons throughout the brain using optical probes. It’s a clever technique:

1. Find a bioluminescent jellyfish.

2. Figure out which proteins produce the glow and then determine which DNA sequence encodes that protein.

3. Modify the protein — and the DNA — so that it only glows when calcium ions are around. You do this by breaking the protein in half and inserting a calcium-binding domain in the middle. When calcium binds to that domain, it “heals” the protein, allowing it to glow again.

4. Modify the genome of a worm, fly, mouse, or monkey so that its neurons now make this special protein.

5. Make a window in an animal’s skull and look at the brain under a microscope while the animal performs a task.3 When neurons are silent, they have very few calcium ions and will thus be dark. When they are electrically active, they take in calcium ions, causing the proteins to glow.

This method now lets us see neurons in the brain blinking off and on when they are active. Truly wild sci-fi stuff.

Cell typing, connectomes, and electrical and optical probes now allow systems neuroscientists to figure out how electrical circuits in the brain work. But as amazing as new neuroscience tools are, there is still much we can’t measure effectively — including animal behavior, the electrical properties of neural cell types, and how those properties change in response to neuropeptides and neuromodulators.4

Given that the aim of systems neuroscience is to link neural circuits to behavior, classifying all of the different possible types of behaviors is essential. Since humans are (understandably) hesitant to let researchers stick electrodes into their brains or to modify their DNA to make their neurons glow, most fundamental systems neuroscience experiments are done on animals. But animal behavior is still largely a black box, and animals, challengingly, can’t tell us what they’re thinking.5 We can create Big Brother-style experiments to record an animal’s every moment in lab settings, or use drones and other techniques to track them in the wild, but even with AI, it’s hard to parse all of that video data to determine which exact movements count as behaviors.

The electrical input-output properties of single neurons can also be incredibly complicated, and we don’t yet have models for all of the cell types that we’re discovering. To return to the electrical circuit comparison, we don’t know which cell types act like transistors, which act like resistors, which act like capacitors, and so on. Developing these models will require substantial efforts to map the location of electrical components (such as ion channels) on each neuron, incorporate those components into single neuron simulations, and validate the simulations. A component’s electrical properties can also change in response to neuromodulators like dopamine, or to neuropeptides released by other neurons in a circuit. Understanding these changes is a whole other area of research that remains in its infancy and would benefit from further tool development.

Subscribe now

Calibrating the hype

All of these challenges require new tools developed by multidisciplinary teams. And those tools will generate big datasets that require advanced analysis techniques. While these types of projects can be done within academia, they are often better suited to team science efforts at dedicated non-profit research centers, like Focused Research Organizations (FROs), the Janelia Research Campus, which is hard at work trying to understand animal behaviors with a neuroscience lens, or the Allen Institute for Neural Dynamics, which has been trying to crack the “electrical properties of cell types” problem.

The brain is an incredibly complex organ, and figuring out its secrets will demand sustained effort and hard work from many researchers, not just a single big breakthrough. While popular culture likes to lionize lone “geniuses“ who make groundbreaking discoveries, scientific advancement is far more often the result of the cumulative work of countless, mostly anonymous scientists who have dedicated their lives to the cause. Metascience might criticize the prevalence of incremental research, but very hard problems sometimes require many incremental advances towards a unifying vision. This is why I advocate for more roadmapping, especially in biology: we need to identify the grand challenge, break it down into smaller problems, and recruit and support the best people to solve those problems.

Overall, I’m glad that there’s so much excitement and effort being directed toward systems neuroscience. It’s a fascinating area of research that brings out the best ideas of scientists, engineers, and philosophers alike, and has the real possibility to generate better treatments for mental health diseases. But the real breakthroughs will require intentional development and patience.

Subscribe now

What good is science anyway?

In metascience, the reflexive answer to this question relies on the by-now standard Progress Studies playbook. We flip open the history books and review the annals of innovations that lifted societies out of poverty, eliminated sources of illness and decay, and made previously fantastical feats ordinary and accessible to all. For Progress Studies, science is good because it is the fuel of progress, because it provides for the “relief of man’s estate,” to quote Francis Bacon.

These are doubtless powerful and persuasive narratives, but ones rooted fundamentally in material outcomes. The message is this: science seems to provide overwhelming benefits relative to the resources invested in it, so we should seek to reverse the decline in the speed of scientific progress.

But the utilitarian calculus is not the only way to justify the importance of science. What if we were not quite so pragmatic about what science has to offer society? What if we instead rooted our desire to promote science in the inherent good of science itself?

Call it “virtue metascience.” From this perspective, it is of course valuable that scientists and scientific institutions deliver breakthrough innovations that expand our health and wealth. But more important is the critical moral value of science itself. As a well-functioning and healthy practice, science will foster a dedication to the truth and a methodical, long-term commitment to resolving hard problems that grounds a virtuous society. In other words, the cultivation of good character that occurs in a healthy practice of science should be a core motivation for supporting, reforming, accelerating, and expanding scientific discovery.

Subscribe now

The divergence between today’s metascience and this alternative position roughly parallels the distinction between consequentialist and virtue ethics. The consequentialist roots their choices in an analysis of what produces the best outcomes, limiting the costs while maximizing the benefits. The virtue ethicist instead proceeds from asking what a virtuous person who embodies higher values — courage, justice, wisdom — would engage in regardless of the outcomes. They ground decisions in these virtues, pursuing action that encourages their moral development and flourishing.

In such a view, science is desirable because a society which pursues it is a good society. End of story. It is worth noting that this view is in many ways the historical conception of science: Aristotle defended contemplation of the truth and first principles — theoria — as the highest virtue. Thomas Aquinas proposed that diligent study — studiositas — was a moral good. The more consequentialist vision of science we are familiar with in the modern era descends from Francis Bacon and was more recently institutionalized through the postwar thought of Vannevar Bush. After all, Endless Frontier explicitly treats science as a necessary tool in the “war against disease,” and for the development of “new and improved weapons” and “new and better and cheaper products,” rather than for the cultivation of the person.

We suspect that “virtue metascience” thinking would be hard for a metascientist — so rooted in economic rationalization — to accept. There cannot be a “virtue surplus” created by science that would accrue to the public or to certain stakeholders. The virtuous benefits of science would be hard to measure, and perhaps impossible to run randomized controlled trials on. For a field that calls itself a “science of science,” reorienting science policy to virtuous objectives might seem to undermine the careful, methodical, and quantitative approach that has shaped the field and its government strategy.

But with the architecture of American science undergoing its most volatile period in decades, the established epistemological approach of metascience may itself be under threat. Technological change, dramatic funding shifts, and organizational convulsion may make it practically harder to conduct the kinds of careful, long-term studies that metascience has enshrined as the gold standard of its work. As public trust in scientists and scientific institutions continues to decline, science’s political defense may need to rely on more than empirical studies of its value, especially if results become increasingly difficult to acquire.

Locating the value of science in the cultivation of virtue offers an alternative framing for many of the existing issues in metascience. The consequentialist sees the endemic problems of reproducibility in the sciences and fears that these systemic issues will introduce frictions that slow progress. The virtue metascientist sees the reproducibility crisis as a moral problem, a product of the corrosion of the very virtues necessary to the practice of science. Both want to make progress on the issue for fundamentally different reasons and by radically divergent, and perhaps even opposing, means.

Similarly, the consequentialist is excited about focused research organizations (FROs) because they promise more efficient scientific discoveries than the bureaucracy-choked world of the conventional research university allows. The virtue metascientist is interested in FROs precisely because they cultivate virtues of the scientific endeavor which have been eroded by the institutional dynamics that turn modern scientists into managerial grant-writing machines. Both are excited about the prospect of more experimentation in science’s organizational forms.

Subscribe now

Virtue metascience would have distinct advantages and makes the case for supporting certain kinds of science more readily apparent. Under a consequentialist regime, exploratory, open-ended “basic science” often requires a great deal of justification. Pure curiosity does not guarantee practical results, and the link to real world applications can be diffuse — or even taken as an article of faith. In contrast, virtue metascience backs basic science by default: the pure inquiry and the virtues of its practitioners are valuable in themselves, regardless of the ultimate outcome. Katalin Karikó’s dogged pursuit of the opportunities in mRNA, heedless of the professional consequences, is in this sense a virtue scientist par excellence.

But virtue metascience would not be a mere shift in rhetoric. It would require substantive changes to the policy priorities and research agendas.

For one, virtue metascience may envision a significantly different role for the state in science. For the consequentialist, the state serves simply to fill the gaps of the market, helping to address the systematic failures of the private innovation system. For virtue metascience, the state might be tasked with affirming deeper societal commitments to objectivity and truth-seeking, and developing institutions to inculcate those character traits. Jay Bhattacharya, Director of the NIH and current Acting Director of the CDC, has proposals in this vein, advocating for NIH to play a role in creating a “set of metrics that track good scientific behavior,” rewarding prosocial behaviors, like investing in reproducibility work. The consequentialist can find such emphases on character concerning, as they are distractions from the “bigger picture” focus of whether or not state interventions are unlocking more investments, more discoveries, and more companies.

The privatization of science is another point of contention. The consequentialist is agnostic: if for-profit enterprises and private investors are able to produce the same innovations faster and cheaper, then we should obviously use them. Virtue metascience is less ecumenical: it may be that a cost-benefit, market-driven form of exploration does not similarly cultivate the character of its researcher-participants due to pervasive commercialization incentives. The virtue metascientist might also have qualms with automating science and building “self-driving labs”; does a fully automated lab provide a virtuous model for society? Which virtues does the cloud lab develop in the scientist? If all that is left is prompting, what does the practice of science mean?

Subscribe now

One final divergence: taking virtue metascience seriously may mean more tightly restricting the title of “scientist.” The commitment to be a scientist should be a sacred one. The privilege of a life of intellectual inquiry demands strict adherence to standards of thought, speech, and deed, a path that only a few may be able to commit to. While fewer “scientists” may mean changing publication patterns, those committed to a virtue metascience may be willing to sacrifice the number of participants for the sake of a stricter and higher standard for joining the scientific ranks. For the consequentialists, it may be that such a narrowly drawn community of science might produce better results as well.

Navigating these philosophical tensions may challenge the values of those who have rallied around metascience. But as American science appears poised to exit the coming decade radically different from how it entered it, policymakers and the broader public will ask to what ends are science — and government involvement in it — ultimately directed. If metascience wants to shape the future, it will have to offer its own answers.

Tao initially posted part of this piece as a tweet. This longer essay defends a strong form of technological determinism and suggests how we can still positively steer technological development during critical windows of opportunity. I learned a lot editing this piece, and it helped me to better define what I believe about progress. I hope it challenges and provokes you (and that you enjoy it — it’s a good read).

In defense of technological determinism

The human experience is defined by technology. People in New York, Beijing, Abu Dhabi, and Vienna all wake to their phone’s alarm, turn on their electrical lights, use a modern bathroom with running water, get dressed in similar clothes, commute to similar workplaces in cars or trains powered by combustion engines or electric motors, and so on. We consume similar foods, use similar apps on our phones, consume more content in a week than our ancestors might have in a lifetime, are born in similar hospitals, and die from similar diseases. For all the important differences between the West’s liberal democratic order and the authoritarian rule in China, Russia, and other countries, our lives in rich metropolises have much more in common with one another than with those of the feudal or pre-agricultural societies of the past, anywhere in the world. Why did our societies converge in the same idiosyncratic ways?

Subscribe now

Technological determinism is a two-pronged worldview that offers a plausible explanation. Its first axiom is that technology is a critical determinant of human experience, societal structures, and even culture. It posits technological development (or lack thereof) as a core driver of our lives becoming so similar across the “developed” world. This could also explain why, for example, societies that have been cut off from technological progress, like the Sentinelese, have ways of life similar to those of every other technologically primitive society around the world, be it contemporary or ancient.1

Its second axiom is that technological development follows an internal logic — a structure determined not by us, but by the underlying shape of the technology tree. The idea of a “tech tree” is itself techno-deterministic: the technologies that make up the trunk of the tree must precede its far branches. You can’t invent the smartphone without batteries, nor batteries without an understanding of chemistry, itself necessitating advances in metallurgy, glassmaking, mathematics, and more.

Examples abound. You can’t have microscopes without glass, no space travel without calculus, no nuclear power without atomic physics, no modern aviation without aluminum smelting, no industrialization without the coal-powered steam engine, and, arguably, no scientific revolution without the printing press and, later, no rapid scientific progress without industrialization itself.

We can call this acknowledgment of necessary dependencies “weak determinism”; most find it intuitive. A stronger determinism would further concede that — even when certain technologies or conditions aren’t strictly necessary for others to emerge — technological development still follows an internal logic ruled by a myriad of weak dependencies, market incentives, gains in efficiency, and the properties of technologies themselves. Strong technological determinism moves the human inventor out of the usual focus, and instead centers the broader economic, technological, scientific, and social conditions that enable technological progress.

To weigh the evidence for strong determinism, we can ask: what would history look like if we did live in a techno-determinist world?

Subscribe now

In an October 2025 post, the AI company Mechanize offered two arguments to defend technological determinism: first, that simultaneous and independent discovery of technologies is common in history.2 Examples include the Hall–Héroult process to smelt aluminum, the jet engine, and the telephone, as well as conceptual breakthroughs, like calculus in the 1670s, evolution by natural selection in 1858, and many, many more examples not mentioned in their post. This widespread simultaneous discovery is exactly what we would expect if technological development were largely deterministic, where each discovery follows a long string of dependencies. Once the prerequisites for a discovery are in place and incentives emerge, multiple groups will converge on it independently at a similar time.3 Conversely, if discovery flowed mostly from individuals’ stroke of genius or serendipity, we should expect simultaneous discovery to be coincidental and rare.

Their second defense is that isolated societies have consistently converged upon the same basic technologies: the wheel, intensive agriculture with terracing and irrigation, similar city layouts, cotton weaving, metallurgy, writing, and more.4 If we lived in a technologically contingent world, we should expect isolated societies to independently forge (or at least stumble upon) wildly different technology trees. And instead of trade and technology transfer leading to societal convergence, we should see different societies choose divergent technological paths, whether due to randomness or intent.5

I find these two points compelling. But Mechanize’s post goes beyond arguing for the kind of strong technological determinism I defend here, into what Luke Drago terms “Technocalvinism.” As he defines it, Technocalvinism is “the idea that technological development is preordained beyond human control, leaving you blameless for your actions.” The Mechanize authors write:

Whether we like it or not, humanity will develop roughly the same technologies, in roughly the same order, in roughly the same way, regardless of what choices we make now…

Rather than being like a ship captain, humanity is more like a roaring stream flowing into a valley, following the path of least resistance. People may try to steer the stream by putting barriers in the way, banning certain technologies, aggressively pursuing others, yet these actions will only delay the inevitable, not prevent us from reaching the valley floor.

I largely agree with the techno-determinist view they lay out. We are, as a civilization, running blindly along the branches of the technology tree, feeling our way forward as we go, following the path of least resistance. To say that we’re usually actively choosing which paths of the tech tree to go down would be, in my view, naïve. Technological progress flows instead mostly from much larger, impersonal forces over which individuals exercise little control. And, when there are overwhelming incentives to develop a certain technology, there is little that can get in these forces’ way.6

And yet I disagree with the conclusion that human agency cannot meaningfully shape technological development. Even on this strongly techno-deterministic view, humans have substantial agency over technological development. Exercising this agency is one of the most important things we can do. Later in this piece, I offer a list of R&D projects that we should accelerate to prepare the world for advanced AI and invite your own pitches.

In defense of human agency

Yes, the tech tree is largely discovered, not forged. And simultaneous and independent discovery are solid evidence for determinism on long time horizons. But on shorter time horizons, our active choices to alter default outcomes can have lasting consequences.

What could these choices look like? The image below illustrates a few possibilities. Positively shaping the course of technological development often means accelerating the R&D of beneficial technologies — be it because they mitigate the risks or harms of new technologies or environmental threats (a, b, and c below), or because the technology itself is immensely beneficial (d below), such as antibiotics, cancer cures, etc.

Exercising this agency makes all the difference when a game-changing technology, like nuclear weapons, is developed — especially when the technology’s impact on the world is largely mediated by how it is developed, and whether appropriate safeguards for its disruption are developed in a timely manner. Or when accelerating the development of a technology can save millions of lives (indeed, this is the reason Mechanize provides for wanting to accelerate labor automation: expediting the arrival of a world of abundance, longevity, and unprecedented wellbeing).

Earlier I asked, “what would history look like if we did live in a techno-deterministic world?” and provided what, to me, is compelling historical evidence for a strong form of techno-determinism.7 Now to the question we are actually interested in: what does history say about humanity’s ability to shape technological development?

Subscribe now

Historical examples

1. Steering the development of a strategically decisive technology has immense effects

Nuclear proliferation has certainly been much slower than it would’ve been without active effort from the US and other governments and an expansive nonproliferation apparatus. According to a Brookings report, from 1949 to 1964, “an overwhelming majority of classified and academic studies suggested that proliferation to more countries was inevitable.” In 1960, then-Senator John F. Kennedy warned that “there are indications because of new inventions, that 10, 15, or 20 nations will have a nuclear capacity, including Red China, by the end of the Presidential office in 1964.” In those four years, only two countries — France and China — developed nuclear weapons. And only four more have developed them since. This success in limiting proliferation would not have come by default — nowadays, building a nuclear weapon is actually not that hard for even moderately-developed countries, and the military advantages of a nuclear arsenal are immense.8 Limited proliferation was the result of decades of effort by American and allied statesmen to contain the most dangerous technology ever developed.9

Limiting nuclear proliferation to nine nations is a success we can measure. Harder to appreciate are the silently averted disasters, like what might’ve happened if permissive action links (PALs) had never been developed. PALs are mechanisms used in modern nuclear weapons to prevent them from being armed or detonated without the right authorization code; without them, a terrorist could steal a nuclear weapon, or a disgruntled or insane military general could attempt a launch without proper authorization.10 Maybe we would’ve seen unauthorized launches of nuclear weapons already. Maybe these launches would’ve triggered a full-scale nuclear war. But we’ll never know, because we actively pursued R&D to create something like PALs.

Take another example, also related to nuclear weapons.11 In 1960, the US, UK, and Soviet Union were negotiating limitations on nuclear testing. Because a state cannot be sure that its nuclear weapons work without testing them, a comprehensive and verifiable nuclear test ban is seen as an important pillar of any successful arms control negotiation. At the time of these negotiations, we lacked the technical ability to reliably detect underground nuclear tests, making a comprehensive treaty unverifiable. These limitations led to the Partial Nuclear Test Ban Treaty in 1963, which covered only detectable types of nuclear tests. Just two years later, researchers developed the Fast Fourier Transform algorithm, making it feasible to distinguish underground nuclear explosions from earthquakes with real-time seismic analysis. But the algorithm arrived late — the 1963 treaty had already excluded underground tests, so testing moved underground (see the graph below).12

It took almost three decades, an arms race that peaked at over 60,000 nuclear warheads, and over 1,300 underground tests before the Threshold Test Ban Treaty was ratified by the US and USSR in 1990.13 This illustrates two things: the importance of human agency in controlling a dangerous technological competition, and the immense costs of not having technological solutions ready before critical points are reached. Had we put as much effort into building treaty verification technology as we did into building the bomb, could we have avoided the nuclear arms race?

2. Altering the sequence in which important technologies are developed can have long-lasting effects

While the tech tree is less fixed on short time horizons, this fact wouldn’t matter much in isolation. Say you accelerate the development of one technology by two years, putting it ahead of another one. Does that just leave you where you would’ve been two years later anyway? Not always. Changing the sequence of development matters if an acute event — perhaps fueled by tech development itself — creates critical windows of vulnerability, during which an offensive technology is developed or a destabilizing situation arises, but no defensive/stabilizing counterpart has been developed yet.

For example: The claim “a COVID vaccine would’ve been developed eventually” is true, but it matters that it was developed ~10 months after the start of the pandemic, as opposed to 10–15 years later, as normal vaccine development timelines would indicate. Biotechnology companies would not have managed the fastest vaccine production in history without coordinated, intentional acceleration, in part through interventions like Operation Warp Speed. These efforts saved millions of lives.

Now for a hypothetical example. Newer scholarship suggests that nuclear weapons were likely not a defining factor in the Allied forces winning WWII in the Pacific Theater, but let’s consider a counterfactual: what if Nazi Germany had not already capitulated, and the Soviet Union had not threatened Japan with invasion?14 In that world, the Manhattan Project could’ve made the difference between a US-led world order and a fascist one.

Were nuclear weapons going to be developed at some point “anyway”? Yes, probably. But that’s not the point. The point is that the world is decisively different depending on whether a democracy or a fascist regime develops them first.

And were PALs going to be developed at some point? Yes, probably. But whether it happens soon after the development of nuclear weapons or decades later is significant. The difference may be millions of lives lost.

Technologies that arrive first also attract disproportionate follow-on R&D, compounding their lead into durable path-dependencies. This evolutionary dynamic means that the conditions under which a technology emerges can lock in for decades.

Before getting to AI — the actual motive for this piece — let me state the broad thesis I’m defending: tech trees have real, hard constraints. Beyond those constraints, almost all technological development happens in a decentralized, impersonal way, driven more by default incentives, market dynamics, soft-dependencies, and efficiency than by individual geniuses and inventors choosing the path forward. Observing from afar, it may seem like the only realistic choice is to abandon any hope of positively shaping technological development. But there are small windows of opportunity for exercising agency, in which efforts to alter the default outcomes can yield long-lasting results.

I think we are now in one of those moments in AI development, but our chance is slipping away fast.

Subscribe now

So what do we do about AI?

The Mechanize piece makes a straightforward argument: since full automation of labor will happen anyway, best to speed it along and get the benefits of AI sooner.

I sympathize with this reasoning. As I’ve written at length elsewhere, the potential upside of AI is immense. Mechanize seems to care about bringing these benefits about as fast as possible, as do I, because the world desperately needs them.15 The world is much better off thanks to progress, but it is still awful, and it can be much better yet.16

Virtually all of the large gains in human welfare have stemmed from economic growth and scientific and technological progress. If we had somehow delayed the Industrial Revolution by a century out of fear of change, that plausibly would have been the greatest mistake we’d ever made.17

It’s possible that we are on the verge of curing our deadliest, most debilitating, and dehumanizing diseases. It’s possible that our descendants will look back on the people of the early 21st century and pity us, just as we pity those who had to endure the Bubonic plague, measles, polio, tuberculosis, or who had to endure the death of a child to what today is but a minor infection.18 I hope they’ll look back and see us as the unlucky last few generations that still had to endure cancer, heart disease, obesity, depression, Alzheimer’s, and yes, even abject poverty and psychopathy.

You’ll have to forgive my perhaps naïve optimism about the future — based on our record of the recent past, I can’t help myself. The year 2025 alone produced enough medical breakthroughs and scientific advancements to inspire optimism for an entire generation.19

If delaying the march of progress in the past would’ve been a grave mistake, failing to accelerate it now would likewise prove to be so.

So if technological and scientific progress are so important, and advanced AI could accelerate it, why not just rush ahead?

One reason is that, if AI can do almost anything a human can, it will be dual-use by definition. It’ll be key for cyber-defense and cyber-offense, for vaccine or antibiotics discovery and gain-of-function pathogen research, for autonomous passenger vehicles and autonomous military drone swarms, for the discovery of miracle cures and for the creation of wonder weapons likewise beyond our comprehension.

Beyond blatant misuse, this AI had better be reliable and broadly aligned with our intent. I don’t want to live in a world where fully autonomous and unaccountable AI agents pursue their own objectives, make their own money to spend as they wish, direct thousands of other instances of AI agents toward working on their own goals, and have those objectives be in conflict with ours.

Even with aligned agents and robust protections against misuse, the broad automation of human labor could greatly decrease the power of the common man, potentially leading to extreme concentrations of power and a broad disempowerment of the population. The Black Death killing 30–50% of Europe is often credited with catalyzing the demise of serfdom in Western Europe: as the supply of labor fell, its importance grew, which allowed serfs to demand better working conditions. If AI inverts this dynamic by making labor abundant and cheaper than human wages, we might expect the opposite effect on bargaining power. Though I fully agree that automating certain parts of the economy generally leads to more and better employment elsewhere — as it has in the past — I don’t discard the possibility that future AI will be different.20 At the extreme, general-purpose AI agents and autonomous robots that (near-)perfectly substitute for human labor could make us as useless for most tasks as horses became for most transportation.21

I’m not an economist, and I won’t try and give a pronouncement here on whether AI will lead to the full automation of labor, and what that would mean for us. But economists themselves don’t seem to be sure — transformative technologies have an uncanny way of shaping the human experience well beyond their initial intent.22

Our liberal democratic institutions are a civilizational achievement that should not be taken for granted. They are as much a product of thinkers like those that led the French and American revolutions, as of the conditions of technological and economic development that enabled that thinking to become reality by empowering the general population. Just as easily as this broad empowerment can enable democracy, broad disempowerment could take it away.

Enough has been said about these risks already. The calculus is really quite simple: the development of non-human entities that can do everything a human can do with a computer is a really big deal, and it can go very well for us, or very poorly.

There is no fundamental reason why everything should go well; no law of the universe conspiring to make things better. Borrowing Peter Thiel’s taxonomy, we should be definite optimists about AI development. Instead of vaguely hoping that the technology tree and the powers that be deliver benefits while avoiding the risks, we should take charge and design a future that lives up to our optimism.

My point is that though the consequences of inventing new technologies are hard to predict, proactively mitigating the risks that new technologies create is not a lost cause. When you invent the car, you also invent the car crash. The answer shouldn’t be to throw our hands up and accept that progress requires traffic fatalities. Nor should it be to attempt to halt the creation of cars. The appropriate response is asking, “how fast can we also invent the seat belt?”

Subscribe now

Reaching for better branches

I hope I’ve convinced you that we have real agency to shape technological development, even in a broadly techno-deterministic world. In the short run, tech trees are highly contingent and shapeable. And in fact, it is because technology is one of the strongest determinants of human welfare that steering technological progress is one of the most important things we can do.

Whether AI progress leads us to a better or a worse future could be overdetermined by the inherent qualities of the technology. Perhaps it’s in the nature of these systems that they cannot be durably aligned with human values. Perhaps it’s in our own nature that our liberal democratic societies crumble when enough people become substitutable by machines. But neither is a priori a foregone conclusion.

The destination reached could well depend on contingencies, such as whether AI systems will be interpretable and steerable before they have broadly human-level intelligence. (Will we have the PAL-equivalent ready before we get the Bomb-equivalent?) Or it may depend once more on whether a liberal democracy or a dictatorship gets the technological advantage in a critical early period. This is the level of resolution of the tech tree that matters, and it’s also the level of resolution at which actors can effect lasting change.

We are embarking on a battle for stability despite rapid progress, for a progress that actually benefits people. That “battle” will be fought everywhere. It’ll be so ubiquitous that it’ll look from afar like the great techno-determinist machine advancing at its pre-determined pace in its pre-determined direction. But it’ll be full of small wins and losses that will likely determine humanity’s ultimate course.

The ideas we already have

Over the course of 2025, colleagues and I assembled The Launch Sequence: 16 concrete projects to accelerate AI’s benefits while building safeguards against its risks.

The core premise mirrors this essay’s argument: If technology is a major determinant of human welfare, then steering its development is among the most important things we can do. Assuming advanced AI will be developed, and that its capabilities will diffuse widely, what should we build to prepare?23

Some proposals target the “seat belt” problem directly — building defenses before vulnerabilities arise:

• Operation Patchlight and The Great Refactor would use AI to proactively harden our cyber infrastructure before AI-powered cyberattacks undermine it at scale.

• Hardware-Based Verification would enable faster AI diffusion while preventing misuse, and eventually serve as the basis for lightweight domestic policy or effective international treaties.

• Preventing AI Sleeper Agents would red-team American AI systems against adversarial tampering — the PALs-analogue for a technology that will soon be embedded everywhere.

• Scaling Pathogen Detection would build the biosurveillance infrastructure to catch the next pandemic early — the kind of system that could have saved countless lives in 2020.

Other proposals would accelerate the beneficial applications we desperately need:

• A Million-Peptide Database would generate the training data needed for AI to discover new antibiotics before resistance claims more lives than cancer.

• The Replication Engine would use AI agents to automatically verify scientific findings at publication, addressing the replication crisis that wastes billions in research dollars annually.

• Self-Driving Labs would build robotic systems to test AI-generated material discoveries, closing the gap between digital prediction and real-world validation.

Subscribe now

The ideas we’re looking for

If you’ve read this far, you likely have ideas of your own. We’ve reopened The Launch Sequence with a rolling request for proposals (RFP). We’re looking for concrete, ambitious projects to prepare the world for advanced AI — whether by accelerating critical safeguards, unlocking scientific and medical breakthroughs, or building the infrastructure for more resilient institutions.

The first stage is just to submit a short pitch (200–400 words). If your idea is promising, we’ll work with you to develop it further and connect you with funders ready to act. Authors will receive a $10,000 honorarium, and other bounties are available. This project is advised by George Church, Matt Clifford, Kathleen Fisher, Tom Kalil, and Wojciech Zaremba.

We have much to do and no time to waste. Do not surrender to the default tech tree — there is no guarantee that it will be merciful. If we are to reach better futures, we must build them ourselves.

Read the RFP and submit your pitch here → ifp.org/rfp-launch

Acknowledgements: I thank Luke Drago, Gaurav Sett, Adam Kuzee, and Jonah Weinbaum for useful feedback. All errors are mine.

The world of research has gone berserk/
Too much paperwork.
- Bob Dylan, from the song Nettie Moore (2006)

I don’t need to tell you that the bureaucracy involved in securing federal funding and complying with regulations hampers science. In addition to being slow, the federal science enterprise is very big — so big that a report of 53 options for improving science processes only scratches the surface of what needs to be done. In 2025, the National Academies presented that menu in their Simplifying Research Regulations and Policies: Optimizing American Science, selecting options for their likely impact, and taking into consideration the Trump administration’s interest in reducing regulation.

I’m a pragmatist when it comes to improving science.1 I know the recommended reforms will not revolutionize the American research enterprise; large-scale changes require political decisions rather than technocratic tweaks. But the National Academies’ recommendations can remove red tape and meaningfully simplify federally funded science. There’s value in tractable, tactical solutions. We should be ready for major windows of opportunity — new institutional structures, Manhattan Projects, moonshots — and seize them. But if we fail to improve coordination and cut back bureaucratic sludge, the government will ossify in the meanwhile.

Subscribe now

All 53 of the report’s prescriptions are worth considering, but a few stand out to me as most tractable and promising. My heuristic here was identifying the recommendations that don’t cost money (staff time is OK), broadly align with the current administration’s priorities for science, and don’t require congressional action.

Why these three filters?

1. Finding unobligated funding is difficult and we can’t count on new congressional appropriations.

2. You’re most likely to succeed by aligning policy advice with a sitting administration’s priorities. This is doubly important when regulatory reform is highly centralized in the White House, as it is now.2

3. Congress is slow in the best of times, and is particularly dysfunctional now. In the current climate, it’s more efficient to focus on reforms that don’t require legislative changes.

Selecting five options means that I didn’t discuss the other 48. Some of my assumptions may diverge from those of the report authors (e.g., I think congressional action is unlikely right now). I’ve also left out ideas I wasn’t dazzled by or that were hyperspecific or esoteric — there’s a lot of content on animal testing, which is important, but a bit niche.

Here are the five interventions I suspect would be most effective, ordered by appearance in the report (though the first is my top priority).3

Subscribe now

Problem: Federal grant processes are inconsistent and burdensome

Option 1.1: Introduce a federal-wide, two-stage pre-award process

In a typical research grant process, scientists submit a comprehensive application (sometimes upwards of 30 pages), which is then reviewed by a panel of external experts. The National Academies report recommends a two-stage process instead, which would allow scientists to receive feedback on their ideas without the effort of a full application.

In a two-stage pre-award process, applicants submit a brief letter of intent (LOI) and agencies invite top applicants to submit a full application. The report states that in this model, “subject-specific review panels would evaluate the LOIs to assess the merit of the proposed research.” This could be a group of agency subject matter experts, obviating the need for external experts’ time and effort. This approach both lowers the barrier to submitting applications (potentially giving a boost to early career scientists) and reduces the burden on external review panels.

Select grants and contracts, such as the Department of Energy’s Small Business programs and DARPA activities, already use two-stage review, but is uncommon across the government. Agencies’ reasons for hesitating to implement it are legitimate — I recently spoke with some agency science leaders who were concerned that two-stage review might lead to an explosion in the number of LOIs (since the bar to submitting would be lower), and that researchers might object to being rejected based on just a brief submission. Other barriers might include insufficient staffing or in-house expertise to modify existing processes or review LOIs.

I don’t want to downplay these potential challenges. However evidence suggests that, with adequate staffing, empowered program officers can effectively select projects for quality. And an explosion of LOIs may be a positive signal that more early-career scientists or researchers from non-elite institutions are applying.

There are ways to manage potential implementation challenges. Agencies should consider creative strategies to manage an influx of LOIs, such as by limiting the number of LOIs that each applicant can submit annually. As for complaints about rejections based on mere LOIs: while this advice is easier to give than to take, agency staff should not be afraid to offend applicants. The applicant will learn something important about their alignment with agency goals or find ways to improve their idea, and — because the barrier to submitting an LOI is low — they can always make changes and submit again.

Implementing two-stage review is likely the highest-impact reform proposed in the report. If implemented well, with empowered program officers and a clear process, two-stage review can make it easier to pitch a project to funding agencies and reduce the number of full applications that need to be written and reviewed.

Problem: Financial conflict of interest (FCOI) requirements are inconsistent

Option 3.1: Create uniform “conflict of interest in research” policy

Federal science agencies currently have varying disclosure policies for potential financial conflicts of interest (FCOI). Two of the most influential are the National Science Foundation (NSF) policy and the Public Health Service (PHS) policy (which NIH uses). The PHS policy is stricter than the NSF policy, requiring additional reporting and training, and sets a lower monetary threshold for conflicts of interest (i.e., the amount of money the applicant can have received from another entity with an interest in the outcomes of the research).

A consistent policy would simplify compliance while still ensuring the government is collecting the data it needs on potential conflicts; the report recommends that the government go with NSF’s. One potential hitch is that Congress wants substantial oversight on science funding; a 2020 report critiqued the NSF policy for being more permissive.

The National Academies report authors don’t think the stricter PHS regulations are more effective than the NSF rules. And a study showed that when PHS reduced the threshold for reporting from $10,000 to $5,000 in 2011, it increased compliance costs, but only surfaced 13% more potential FCOIs.4 At the very least, the government should conduct a cost-benefit analysis of the PHS policy to determine whether the additional data collected and potential FCOIs avoided is worth the costs of compliance.

Subscribe now

Problem: Research security protocols are overly complicated

Option 4.1: Implement the National Security Presidential Memorandum-33 (NSPM-33) common disclosure forms and disclosure table without deviation as the primary means to identify and address Conflicts of Commitment (COCs) and develop federal-wide FAQs via the interagency working group; in addition, use the Science Experts Network Curriculum Vitae (SciENcv) system, persistent identifiers (PIDs), and application programming interfaces (APIs) across research funding agencies

Long title, but a straightforward idea — make it easier for researchers to comply with Presidential Memorandum 33, which requires research institutions to report to the government how they secure R&D against foreign interference. Science agencies have been gradually adopting a common research security biographical information form since it was finalized by the White House National Science and Technology Council (NSTC) in 2023 (with NIH expected to adopt it in early 2026). However, agencies such as NASA have already started adding on their own requirements; hence the report’s “without deviation” language. One can imagine a situation where an agency needs specific information for research security purposes, and no doubt agency staff will be tempted to view their situation as unique. But NSTC developed the common forms based on practices agreed upon by the interagency, and — at least in the case of NASA — the departures from the standard form seem more administrative than security-related.

Cole Donovan, who was OSTP Assistant Director for Research Security in the Biden Administration, agrees that agencies tend to view their circumstances as unique, but notes that for sensitive research there are other rules designed to prevent data being disclosed to other governments. So in nearly all cases, going beyond the NSPM-33 common forms would be unnecessary. Donovan tells me, “At a minimum, disclosure and security program requirements should be calibrated to not exceed those of agencies whose primary mission involves providing direct support to the warfighter or intelligence community, absent extremely strong, evidence-based justification.” In other words, some civilian agencies are putting in place research security protocols more complicated than military or intelligence agencies’ protocols, and that doesn’t make any sense.

OSTP, which issues the guidance for form use, should encourage uptake of the common disclosure forms and discourage agencies from adding their own requirements. Research security is important, but it is burdensome (many of the regulations noted in the COGR chart above relate to research security) and the government should do what it can to reduce costly and unnecessary burdens.

Problem: Regulations for research involving biological agents are complex and overlapping

Option 5.2: Simplify and harmonize current NIH, USDA, CDC guidelines, and exempt low-risk activities.

Right now, there are overlapping regulations, guidelines, and policies from federal agencies on managing the use of biological agents and toxins in research. This is onerous for research institutions that work with multiple agencies and suboptimal for risk management. The complexity and decentralized nature of the system make it hard for the government to track and oversee use of biological agents and toxins, which the report cites as having “potential national security implications” or “significant societal consequences.”

The report suggests that the NIH review its guidance and identify opportunities for simplifying it and harmonizing with other agencies’ policies, with the optimal option being broad adoption of the NIH policy. The report goes on to recommend a seemingly easy win, by government standards: exempting low-risk biological research from onerous oversight.5

American scientists conduct some legitimately high-risk biological research (on anthrax, tuberculosis, etc.), and oversight should focus on that.

Subscribe now

Problem: Inadequate regulatory adaptation to evolving research methods and technologies

Option 6.14: Establish a cross-agency initiative to align and consolidate guidance on emerging research methods.

Human subjects research has changed substantially in recent years and will continue evolving with new technology (the report cites “decentralized trials, use of digital health tools, and AI-driven protocols”). Federal regulations struggle to keep up with the changes, but now is the time to bring human subjects regulations up to date. The report suggests that HHS lead this process given its extensive human subjects research record.

Consolidating guidance may be difficult since HHS recently canceled the Secretary’s Advisory Committee on Human Research Protections, which was the primary entity coordinating human subjects regulations. It is unclear how HHS is making decisions about human subjects review in the absence of this committee; OSTP should work with HHS to lead a cross-governmental effort to update guidance.

Notably, the National Academies recommendation doesn’t define an intended outcome, and “establishing a cross-agency initiative” is just a starting point. But with technology advancing at a blistering pace and scientists leveraging it in novel ways, the federal government can’t be asleep at the regulatory wheel.

Cutting bureaucracy sometimes requires more bureaucrats

The National Academies report is largely about improving coordination: 22 of the report’s 53 recommendations focus on centralizing processes or establishing clear ownership.6 Done well, centralization means fewer processes for scientists to navigate, cutting administrative burden and freeing them to focus on research.

But harmonizing requirements and processes requires accounting for different agency needs and mandates, and demands effort by agency staff without immediately obvious benefits. Harmonization efforts are labor-intensive and require sustained commitment from political appointees and career staff alike. But the government lost around 335,000 staff in 2025 and, in some agencies, sparse political leadership makes change even harder.

Hiring and empowering the right people is crucial for reforming science. While current leadership within individual agencies can advance some of the report’s reforms, more meaningful change will require the Trump Administration prioritizing staffing up departments and agencies like OSTP, HHS, and NSF to coordinate across the federal science enterprise. This will entail short-run investment, is but well worth it to de-sludge American science.

Subscribe now

The National Science Foundation recently announced the Tech Labs Initiative, a new grant program that innovates on how the government funds scientific research. As part of the announcement, the Technology, Innovation and Partnerships (TIP) Directorate launched a Request for Information (RFI) to solicit potential Tech Labs applicants and advice on the structure of Tech Labs. Comments are due by January 20, 2026 at 3:00 PM EST.

Tech Labs builds on a long lineage of experiments on scientific institutional models. The idea of independent, non-profit research organizations dates back to the turn of the 19th century, starting with the Pasteur Institute and the Max Planck Society. In recent decades, philanthropists have funded new institutions in pursuit of breakthroughs in biology and neurobiology (Janelia Research Campus), brain science (Allen Institute), and biomedical research (Arc Institute and Broad Institute); Convergent Research has incubated several focused research organizations (FROs) to produce high-impact public goods to unblock scientific progress. These new institutions give scientists and organizations flexible funding, allowing them to focus on research rather than chasing money or reporting on grants.

But philanthropic funding is dwarfed by the federal science enterprise, which is why NSF’s investment in independent research organizations is so exciting. In 2022, Ben Reinhardt argued that the newly founded NSF TIP Directorate should “fund a portfolio of independent research organizations instead of funding specific research initiatives.” With Tech Labs, TIP appears to be doing exactly that.

Subscribe now

The case for alternative funding models is clear. Research shows that flexible funding for scientific discovery can produce higher research impact than conventional grants. Tech Labs shares DNA with IFP’s X-Labs framework; both are visions to expand the government’s science portfolio beyond incremental, project-based grants.

We can see enthusiasm for funding independent, flexible research institutions across the political spectrum. White House Office of Science and Technology Policy Director Michael Kratsios celebrated the Tech Labs announcement, and Democratic Congressman Josh Harder (CA-9) and Republican Congressman Obernolte (CA-23) just cosponsored legislation to launch X-Labs at the National Institutes of Health.

Below, we answer some common questions about Tech Labs, including how the initiative compares to the X-Labs model. We obviously can’t speak for NSF; rather, these responses are based on a thorough review of all publicly available information on Tech Labs and our own perspectives.

1. Why are Tech Labs needed right now?

As TIP assistant director Erwin Gianchandani said in the Tech Labs announcement, “As scientific challenges have become more complex and dependent upon the work of cross-disciplinary teams of experts, our nation must expand its scientific funding toolkit to adapt.”

We agree. By focusing most of our federal research dollars on smaller, more incremental grants, the government misses out on higher-risk, higher-reward opportunities that can lead to breakthroughs. And as the structure of frontier scientific production continues to evolve, it’s important for the federal government to create and support new frontier scientific institutions.

The RFI states:

“The Tech Labs initiative…is designed to address systemic barriers in the innovation ecosystem, including the limited translation of emerging technology to impact and limited industry engagement in early-stage technology development…Many of the most pressing challenges in technology translation require coordinated, interdisciplinary teams working with urgency and purpose. These challenges often face market failures that deter private investment, despite their potential for transformative impact.”

Subscribe now

2. How will Tech Labs be structured?

RFI responses will inform how Tech Labs is ultimately implemented, but we can get a sense of the likely Tech Labs structure from the information and questions provided in the RFI. The RFI requests feedback on funding per institution, length of commitment, and stages of NSF financial support. It proposes $10-50 million in annual funding per institution, provided based on milestones, for up to 5 years.

From the announcement:

“The Tech Labs initiative will support full-time teams of researchers, scientists, and engineers who will enjoy operational autonomy and milestone-based funding as they pursue technical breakthroughs that have the potential to reshape or create entire technology sectors. Tech Labs teams will move beyond traditional research outputs (e.g., publications and datasets), with sufficient resources, financial runway, and independence to transition critical technology from early concept or prototypes to commercially viable platforms ready for private investment to scale and deploy.”

From the RFI:

“This initiative will bet on teams – not individual projects – by funding full-time, dedicated teams which may transcend existing institutional structures and limitations to provide technologists with the autonomy to pursue ambitious goals… The Tech Labs initiative will include a lightweight application process (90 days), a 9-month planning phase, and 24-month performance phases with the intention of renewing high-performing teams for an additional 24 months or more.“

“The planning phase would allow the applicant to work directly with the NSF to co-develop their project. This unique detail should allow for higher-quality team development and strategic planning than in a traditional grant process, where these details need to be worked out at the application stage.”

3. How much money will be allocated to Tech Labs?

Tech Labs will invest up to $1 billion over five years. That’s a lot of money, but only a small share of the NSF budget. If appropriations stay roughly constant, then this funding amounts to a 2-3% share of the NSF budget (which totals around $10 billion per year, or $50 billion over five years) for this experiment in funding new institutions.

4. How does Tech Labs relate to IFP’s X-Labs framework?

Tech Labs are similar to the X02 awards we defined in our X-Labs proposal. They would fund scientific entities dedicated to solving critical infrastructure, tooling, or data challenges.

Our X-Labs proposal describes four award types:

• X01 Awards that fund cutting-edge basic science institutions with flexible research environments. These institutions would focus on foundational scientific discovery with stable, long-term support. The core bet behind X01s is on people, not projects. The goal is to assemble the best team in the world to pursue open-ended scientific inquiry with minimal bureaucratic constraint.

• X02 Awards that fund scientific entities dedicated to solving critical infrastructure, tooling, or data challenges. These labs would be designed for time-limited, high-impact interventions and use multi-year block grants with milestone-based evaluations. The fundamental selection principle is the challenge, funding a talented group with a nimble organizational structure to execute against a clearly defined bottleneck in the scientific ecosystem.

• X03 Awards that fund portfolio-based regranting and incubation organizations, acting as alternative funding institutions outside of the traditional government grant selection process. The animating principle behind X03s is to empower scientific scouts: individuals or organizations with the insight, network, and conviction to identify high-potential ideas, talent, or research directions long before they become consensus picks.

• X04 Awards that provide seed funding to support the formation and planning of new scientific institutions, enabling teams to refine their vision, build key partnerships, and develop initial proof-of-concept work before applying for additional X-Labs funding.

Given TIP’s focus on applied and translational gaps, an X02-style program design fits well within the directorate’s remit. As Tech Labs generate evidence for this funding strategy, we hope science funders will take up other parts of this model in basic science domains as well.

Subscribe now

5. How are Tech Labs different from university center grants?

Tech Labs differ in size and structure from NSF’s other large-scale research grants, which include the Science and Technology Center: Partnership Grants and Engineering Research Centers (both with grants of up to $6 million per year), and Global Centers and AI Research Institutes (both with grants of up to $5 million per year).

We’ve identified five key differences between Tech Labs and existing NSF grant programs:

1. Even the largest grants (up to $6 million per year) are much smaller than the scale of the Tech Labs (up to $50 million per year).

2. Depending on the specific implementation details, Tech Labs would likely have more independence from NSF and potentially lighter reporting requirements than university center grants.

3. Tech Labs will have more budgetary flexibility and can avoid the (sometimes arbitrary) distinctions between direct and indirect costs. Ideally, Tech Labs will empower scientists to decide whether their marginal dollar should go toward additional grad students, more GPU training time, or an updated electron microscope.

4. Center grants are often spread across numerous institutions and approximate a research consortia model. This can add to administrative complexity and often means that scientists are spread across physical distance. By concentrating teams and leadership within one institution, Tech Labs will be better able to capture the efficiencies and knowledge spillovers of colocated science.

5. Tech Labs establish concrete milestones, measured by progress on technology readiness levels (TRL) and technology platforms rather than basic research.

Below is a table comparing Tech Labs and other NSF funding mechanisms.

6. How will Tech Labs involve universities?

Here’s what the RFI says about institutional independence (emphasis ours):

“Tech Labs program will offer sustained, multi-year support to innovative and institutionally independent organizational structures operating outside of existing academic, start-up, and industry constraints to fill a vital gap in the innovation ecosystem.”

We don’t yet know what this will mean for university involvement, but it’s likely that universities cannot apply to be a Tech Lab. It’s possible that the grantee can be a nonprofit that has a university affiliation (e.g., shares faculty, some facilities, equipment). The Arc Institute and Broad Institute are both university-affiliated (Stanford/Berkeley/UC San Francisco and MIT and Harvard, respectively), and we wouldn’t be surprised if Tech Labs allows this sort of relationship.

That said, there’s value in trying to deliberately seed new institutions of science. The US government has done this before, from the establishment of the land grant universities to the national laboratories. At a certain level of scale, spinning out research activities into adjacent institutions would allow for symbiotic work with universities with the added benefit of institutional autonomy.

RFI responses may influence which university affiliation structures Tech Labs allow. The Federation of American Scientists published a piece encouraging universities to respond to the RFI and help shape the relationship between higher education and the Tech Labs. We think that’s a good idea.

7. How can I comment on the NSF Tech Labs idea?

Respond to the RFI by emailing TechLabs@nsf.gov with the subject line “NSF Tech Labs RFI Response” by January 20, 2026 at 3:00 PM EST. No need to respond to every question in the RFI, but don’t miss the deadline!

Subscribe now

Shortly after returning to office in 2025, the Trump administration began terminating thousands of ongoing grants to US universities, totaling billions of dollars in cuts. Terminating in-progress research grants is unusual — no previous administration has canceled grants at this scale1. Predictably, universities and scientific associations raised objections, taking issue with the volume of cuts, their unpredictability, and their targeting of specific topic areas (climate change, DEI, etc.).

These cuts are unique in their abruptness. It’s normal for priorities to change between administrations, increasing money for some areas and decreasing it for others. But because in-progress grants are rarely canceled, the research community can typically adapt and re-orient their efforts toward these new priorities. This time, universities were caught by surprise, and rather than adapt to changing priorities, they had to take drastic measures to protect their financial health and rescue PhD students and research projects that lost funding.

Abrupt grant terminations have the potential to harm research quality if the cuts are driven by political considerations. Large scale and unpredictable cuts could degrade our scientific infrastructure and talent pipeline. These risks have left me with two questions: What happens if termination of in-progress grants becomes a norm across administrations? Are in-progress grant terminations a threat to the structure of science in the US?

I would argue that we should be concerned about the potential for damage to the research system, but I’m not convinced that mid-stream terminations will become the norm. Optimistically (maybe delusionally), this might be an opportunity to develop a more resilient science funding system, or for Congress to create guardrails to protect ongoing projects.

Subscribe now

The effects of mid-stream terminations

When a research grant is terminated unexpectedly, work stops or is interrupted. If a grantee receives a termination notice from their funding agency, they must stop using government funding to conduct research. They can search for alternative sources of funding, but in some cases, such as clinical trials, research cannot be paused while additional money is identified. In other cases, researchers will be unable to find money to restart their projects.

Grant terminations might lead to three consequences: a reduction in the volume and quality of research, a reduction in human capital, and weakened university financial health.

Social cost of less, and lower quality research

The most fundamental cost of midstream terminations is that society doesn’t benefit from the research. Research generates high social return on investment (SROI) through improvements to human wellbeing and health and economic growth. With basic research in particular, it’s unlikely that an incoming presidential administration will readily understand the quality of a given grant they terminate. While any grant might sound silly to a newly hired political appointee, some silly sounding research (on gila monster spit, bacteria in geysers, or the aerodynamic characteristics of bird beaks) can lead to major breakthroughs2. Moreover, while unspent federal funding returns to the US Treasury, some of the primary uses of funds returned to the Treasury, like servicing the national debt, likely have a lower SROI than R&D.

In addition to reducing the volume of research, abrupt terminations can reduce research quality by rewarding caution. If scientists perceive specific types of research as being more likely to be canceled, a decline in high-risk science might follow. If, for example, a scientist is conducting clean energy research and that research is terminated due to the topic, it may reduce the likelihood that scientists will pursue similar clean energy research and other types of energy research that might be controversial to either political party.

Reduction in human capital

Terminated grants can lead to layoffs of non-tenured faculty, administrative staff, postdocs, and PhDs. Of these, I’m most concerned about how grant terminations might affect the pipeline of PhD students. Some PhD students who are funded by terminated grants may not complete their work, though universities will likely work hard to fund them. Students may even hesitate to start grant-funded PhDs if they fear the grant will be terminated before they finish. A PhD is usually a 5-7 year commitment, and there are limited benefits to partial completion. Even a small risk of losing funding and having to drop out might discourage applicants. And if PhD advisors think they will have to hustle to find funding if a grant falls through, they’ll be more cautious about taking on new students.

Don’t we have enough PhDs, though? For decades, academics and policymakers have been concerned that we have a glut of PhDs, but I don’t think there’s good evidence that we have too many STEM PhDs. The NSF Survey of Earned Doctorates shows that the percentage of STEM doctorates who graduate without a professional commitment post-degree is the lowest it has been in over 20 years. This is despite the number of STEM PhDs produced annually rising by about 65% during that period. PhD-holders also make more money than people with bachelors or masters degrees. In addition, while it’s possible that there are too many low quality PhDs, that’s not who is being affected by grant terminations. When the Trump administration froze $2.2 billion in grants to Harvard University, it’s likely the STEM PhD students affected were pretty talented.

Weakened university financial health

Widespread grant terminations can harm university financial health. Universities have high fixed costs (infrastructure, buildings, equipment) and commitments to tenured faculty. In response to funding uncertainty, a potential reduction in overhead, and reductions in foreign students, universities have conducted layoffs and hiring freezes. During past financial crunches, universities have cut services provided to students and raised tuition. However, it’s also true that there are low performing and redundant centers and staff that budget cuts can give administrators permission to remove. Harm to university finances depends on the scale of grant terminations, and whether they are coupled with other shocks (e.g. reductions in overall science funding, caps to indirect costs, declines in recruitment of international students).

Some of these harms mirror the consequences of federal science budget cuts; others are specific to mid-grant terminations. Effects on the volume of research produced and university financial health are probably similar for budget cuts and mid-grant terminations. In terms of the academic pipeline, my guess is that, while cuts to science budgets would mechanically reduce the number of PhD students, unexpected terminations are more damaging because they introduce the risk of students losing their funding after they have begun their programs. The potential for a reduction in controversial or high risk research is the most unique. While an administration or Congress can shift funding priorities and slowly increase or decrease funding for research areas, scientists may try to avoid future terminations by conducting cautious research.

Subscribe now

Are mid-stream terminations the new norm?

The harm caused by mid-grant terminations is driven by the scale of terminations and whether these terminations become a norm. In a very hand-wavy sense, the likelihood of mid-grant terminations continuing is driven by decisions by the two political parties, either to escalate or de-escalate. What matters is whether the two parties engage in it-for-tat retaliation, so it’s important to look beyond the current administration and consider how switches in the political party in power might influence decisions.

The table below suggests how changes in the administration party post-Trump administration might impact grant terminations. In two of the three scenarios, mid-grant terminations become a norm, but I’m not sure which is the most likely scenario.

One area of uncertainty is the extent to which the Trump administration will fund research that Democrats do not like. There is obviously research that Democrats want to fund that Republicans dislike (e.g., climate change science or DEI-related research). It is unclear, however, if the converse is true. One could imagine the Trump administration funding research connecting vaccines to autism or another controversial issue that a Democratic administration would want to cancel, but that has not happened yet. The research priorities outlined in the President’s Fiscal Year 2027 R&D Priorities are fairly uncontroversial, perhaps with the exception of a proposed Golden Dome missile defense program. Similarly, the Department of Energy Genesis Mission announcement has enjoyed a largely positive reception.

Subscribe now

How can we build resilience in the science funding ecosystem?

What can entities with a stake in the health of the American science enterprise do to build resilience against politicized, mid-grant terminations? I’m not confident about how well these ideas would work, but wanted to share some proposals for what universities, philanthropy, states, and Congress could do.

What universities can do: Universities can create pools of “insurance” that rescue PhD students and projects. Some universities (like Michigan State University and UMass Amherst) announced programs to rescue at-risk research projects and graduate student funding in response to terminations in 2025. While not advertised as insurance, that’s essentially what this is: the university has a common fund that affected researchers can tap into. A downside is that this may become unpopular due to moral hazard; if the university is using its own money, non-risky research subsidizes scientists who conduct risky research. Another downside is that it will be difficult for many universities to make up for the scale of grant cuts. In the case of Michigan State University’s rescue fund, the volume of available funding ($5 million this year) is a small fraction of the $104 million in federal funding they lost.

What philanthropy can do: Philanthropies can create pools for at-risk research or PhDs. There have already been some foundations that have done this, for terminated USAID grants, Corporation for Public Broadcasting grants, NSF grants, and biomedical research grants.

Beyond this, philanthropists could expand their science giving. As of 2022, private philanthropy was funding around $16.7 billion toward research. By comparison, just the NIH spends nearly $48 billion on research each year; federal R&D expenditures are around $141 billion. While the private sector has greatly expanded its R&D expenditures, spending $580 billion on R&D, firms tend not to fund basic research — they’re appropriately focused on R&D that supports their bottom line.

Philanthropists have the flexibility to fund research that supports social good, with no expectation of profits. Some funders, like the Chan Zuckerberg Initiative, are moving more assertively into research (increasing their basic research giving to $10 billion), leveraging new advances in AI. But foundations and high net worth individuals can and should do more. As Ari Simon and Aaron Seybert suggested earlier this year, philanthropists should expand their science funding, with a particular focus on protecting scientific infrastructure and personnel - through stopgap funding for research projects, support for the PhD pipeline, and preservation and curation of important datasets.

What states can do: One option in response to federal research cuts (both budget cuts and in-progress grant terminations) is for states to expand their research funding, as the state of California has considered. Federal science funding is more efficient, however, due to economies of scale. The federal bureaucracy may be slow, but having 50 different science funders would be inefficient and lead to redundancy. However, states already do fund some research and could expand funding in areas that have experienced research cuts and are important to those states (e.g. wildfire research in California).

What Congress can do: Generally, Congress can try to make grant termination more difficult or painful for the government or make terminations matter less to the research ecosystem. I’m not especially confident in the feasibility of my ideas to make termination difficult, but am sharing them because I think they’re directionally useful:

1. Include appropriations language limiting termination authority: In trying to reduce the odds that the government will terminate in-progress grants, Congress could include appropriations language prohibiting the Office of Management and Budget and science funding agencies from issuing “termination for convenience” notices on Congressionally authorized and appropriated research grants. “Termination for convenience” allows the government to end a grant when it is no longer needed, without the government needing to provide a reason. An administration might find other ways to terminate those grants, but such appropriations instructions would help on the margin.

2. Require automatic payouts for terminations: A more novel approach would be for Congress to require an automatic payout if research grants are canceled (e.g., some percent of remaining grant funds are paid out in cash). This would go beyond the existing requirement for the government to cover closeout costs and would provide unrestricted funds to ease the pain of cancellation on the grantee and add a bit of pain for the grantor.

3. Make research grants more like contracts: Congress could specify that the government must come to settlement agreements when they terminate certain research grants, as is currently done for contracts, but not grants. If the government routed more research funding through Other Transactions3, which recent jurisprudence suggests may be legally closer to contracts than grants, that may allow researchers to recoup more funding if their awards were terminated.

To create resilience in the science ecosystem, Congress can diversify the federal research portfolio. If part of the Administration’s motivation for terminating university grants is a belief that universities have broken the social contract (or are just too woke), this would allow administrations to fund in other, innovative ways. Elsewhere, IFP has proposed X-labs as a way to expand and diversify the federal science portfolio. Recently, the NSF announced a similar initiative called Tech Labs that will fund non-university research institutions. While funding research non profits would not reduce the harm to universities, it would create more resilience in the system and allow for political parties to ratchet up or down research funding to their preferred type of grantees without weakening American science.

My take: in the case of medium- or long-term instability in federal science granting, the best approach is for Congress to produce legislative fixes. In addition, though philanthropy doesn’t have the scale of the federal government, it should step up and expand basic research giving. While university funding to backfill terminated grant funding is useful in the short term, it’s unrealistic for most universities in the long run. Similarly, state-level funding may be useful as a short-term fix, but unless the federal government divests from research on a large scale, it is inefficient for states to take on this role.

Final thoughts

Midstream grant terminations impose real costs: interrupted research, a weakened PhD pipeline, and chilling effects on high-risk science. If terminations become a norm that persists across administrations, the damage to American science could be substantial. The likelihood of this becoming a norm, though, is unclear.

In the near term, universities and philanthropies have stepped in with emergency funds. More durable solutions like automatic payouts for terminated grants, limits on termination authority, or a more diversified federal science portfolio require congressional action. Given the polarization that would enable tit-for-tat terminations, such fixes won’t come easily. But both parties benefit from a strong science ecosystem, and that shared interest offers some grounds for compromise.

Subscribe now

Before you hop into Ruxandra and Jack’s piece, I wanted to share an opportunity for current or recently graduated PhD students. IFP has partnered with some of the best economists of science and innovation to create a free online course called the Economics of Ideas, Science, and Innovation. Apply by January 9.

OK — let’s learn about clinical trials.

Introduction

Public debates about how to revive productivity in the biopharmaceutical industry tend to be dominated by two camps. Technological optimists usually argue that declining industry outputs relative to investment reflect gaps in biological knowledge, and that advances in basic science will eventually unlock a wave of new therapies. The second camp, which traces its intellectual lineage to libertarian economists, focuses on easing the burden of regulation. In their view, excessive FDA caution has slowed innovation. They propose solutions that largely target regulatory approval: either loosening evidentiary standards or narrowing the FDA’s mandate to focus solely on safety rather than on efficacy.

Both perspectives contain some truth. Yet by focusing on the two visible ends of the drug discovery pipeline, early discovery and final approval, both camps miss the crucial middle: clinical development, where scientific ideas are actually tested in people through clinical trials. This stage is extraordinarily expensive, operationally intricate, and crucially, generates the field’s most consequential evidence. We believe that systematic optimization of this middle stage offers significant untapped leverage and deserves far greater focus.

Subscribe now

The need for such a shift becomes clearer when we consider the growing divergence between scientific potential and clinical results. Over the past few decades, biomedical science has advanced at a staggering pace. Genetics has moved from single-gene studies to sequencing the first human genome, a 13-year, $3-billion project completed in 2003, to today’s ability to read an entire genome in a day for a few hundred dollars. Protein science has followed a similar trajectory: from the earliest crystallographic structures in the 1950s, to large-scale structural biology pipelines, to AI tools like AlphaFold that can now predict protein structures computationally.

Yet, paradoxically, drug discovery has become increasingly inefficient. This trend, first identified in 2012 by one of the authors of this piece and termed Eroom’s Law (the inverse of Moore’s Law), describes how the inflation-adjusted cost of developing a new drug had doubled approximately every nine years since the 1960s. More recent data suggests that drugs per billion dollars spent has plateaued, while financial returns have continued to decline as new drugs tend to be approved for use in smaller groups of patients.

Declining returns on R&D are no longer the sole concern; intensifying competition from China has amplified the urgency for change. As a Time magazine headline in May 2025 warned, “The US can’t afford to lose the biotech race with China.” Like many such commentaries, it calls for reform and renewed attention on the question of what the US can do to compete in the current international landscape.

But where along the drug development pathway do we have the greatest opportunity to steer things differently?

Drug discovery can be thought of as a funnel: broad at the start and progressively narrowing toward approval. Feeding into the funnel at one end stands basic science, which generates countless hypotheses. Only a fraction of these will survive preclinical validation and enter clinical development, where they are tested in humans for safety and efficacy to inform approval. Across all therapeutic areas, only about 8–12% of drugs that enter clinical trials eventually receive FDA approval.

In principle, we can improve productivity at any stage of this funnel: we can raise the quality of inputs through better science, relax regulatory barriers at the end, or accelerate the middle part of clinical development. Yet it’s striking how little public attention focuses on the practicalities of clinical development, despite its importance. This is the stage where theoretical promise is tested against reality: where we discover whether a biological idea can become a safe and effective therapy. It is also the most resource-intensive phase, accounting for roughly 60–70% of drug development costs and timelines.

Because clinical development is so important, it makes little sense to focus attention only on the top and bottom of the drug discovery funnel. Even as basic science and AI advance rapidly, it is unlikely that in the coming decades they will substitute for evidence gathered directly in humans. Clinical trials, with all the financial burden they bring and ethical questions they raise, will remain the critical bottleneck in translating biological insight into real therapies.

Likewise, loosening approval standards is insufficient. In fact, the FDA has already become more permissive in its approval over the past 30 years, increasingly accepting surrogate endpoints in place of demonstrated clinical benefit. But we don’t just want to approve more drugs — we want to identify and deliver drugs that actually work. Even if approval requirements were relaxed, the underlying need would remain: we still have to learn, through testing in humans, which treatments are effective and which are not.

The case for optimizing clinical trials becomes even clearer when we consider the underlying economics of drug discovery. Not only do most drug candidates fail to reach approval, but even among those that do, only about half ever generate meaningful revenue. And within this small subset of commercial successes, only a handful — the so-called blockbuster drugs, such as GLP-1 agonists or the anti-tumor necrosis factors (TNFs) —are truly transformative.

Drug discovery outcomes follow a heavy-tailed distribution: most efforts, even those deemed successful, produce modest results, while a few outliers account for a disproportionate share of both clinical and economic value. Given that success is rare, hard to predict, and potentially enormous, we need to maximize our shots on goal to increase our chances of success. More concretely, that means scaling the number of molecules tested in humans. Expanding the capacity for in-human testing broadens the exploration, increasing the likelihood of uncovering rare, high-impact breakthroughs that drive the most meaningful form of biomedical progress.

While we don’t really know how quickly better science will sharpen our therapeutic hypotheses, we have evidence that clinical development can be faster and less expensive. In Australia, Phase I trials are completed 40–50% faster and at significantly lower cost than in the US, despite comparable ethical and safety standards. In China, new drugs often reach first-in-human testing within ~18 months, versus multi-year timelines in the US. This clinical nimbleness has been often cited as a key factor behind China’s rapid progress in biotech, which is now threatening the long-standing supremacy of the US in the industry.

Subscribe now

Clinical Trial Abundance, a framework for scaling and accelerating human trials, stresses the importance of optimizing clinical development. We already have a menu of promising solutions. Increasing regulatory transparency, strengthening clinical trial infrastructure through targeted investment, applying the Australian Phase I model in the US, relaxing excessive Good Manufacturing Practices requirements for early-stage development, and enabling remote and decentralized trials are just a few examples. But many of these ideas remain underdeveloped: the specific policy mechanisms, implementation pathways, and operational models are underspecified and insufficiently advocated for.

For the US, the ideal strategy lies in combining world-class science with highly agile clinical development. Yet clinical development has long been overshadowed by basic research, largely because it is operational, less glamorous, and thus, poorly suited for study within academic frameworks. This persistent asymmetry in attention must be addressed.

More science won’t be enough

Techno-optimists think better basic science can reverse Eroom’s Law in two ways. It can boost predictive validity so we know much earlier which drugs are likely to work, and it can unlock new kinds of therapies that finally reach targets we couldn’t hit before. In this view, a more accurate understanding of biology, from comprehensive cell atlases to organoids and AI-driven disease models, will allow researchers to navigate drug discovery with far greater precision. At the same time, emerging modalities (or tools) such as gene editing, RNA therapeutics, and targeted protein degraders will widen the spectrum of actionable biology. Together, these advances point toward a future in which each drug discovery effort has a substantially higher chance of success.

We agree that improving our therapeutic tools and our understanding of biology will be important for reversing Eroom’s Law. But we remain skeptical that scientific progress alone will make extensive human trials unnecessary in the coming decades.

The promise and perils of maps

In his novel The Glass Bead Game, Herman Hesse describes Castalia, a scholarly world devoted to the refinement of symbolic systems. Its practitioners become so absorbed in the elegance of their constructions that they forget the symbols were ever meant to point to anything outside themselves. The map becomes the world.

Something similar can happen in the life sciences. Biology is messy, dynamic, and nonlinear, yet scientists naturally gravitate toward models that make it seem orderly and tractable. A clean mechanistic pathway or target-based narrative is deeply appealing: it suggests that cause and effect are simple, knowable, and controllable. As our analytical tools improve, these explanations become more elaborate.

Through multiple waves of technology, from computer-aided drug design, via high-throughput screening, recombinant proteins, and genomics, techno-optimists have overestimated the innovation yield of the hot new thing. Again and again, scientists have placed too much confidence in the power of “biological insights,” or pre-clinical mechanistic foresight. Attention naturally concentrates on the few drugs that succeed, so it is easy to construct post hoc narratives of deliberate design.

Moreover, the biotech ecosystem rewards storytelling. From venture capital pitch decks to internal R&D reviews, a compelling mechanistic narrative makes a program easier to fund and justify.

Yet the empirical record shows that mechanistic foresight provides, at best, rough guidance. Drug discovery is better seen as an iterative design-make-test loop, in which real-world human data repeatedly guide the next cycle of design. Progress may depend less on hitting the best therapeutic hypothesis from the start, and more on generating a broad range of plausible attempts and winnowing them quickly based on clinical feedback. What works survives; what does not is modified or abandoned.

Subscribe now

In previous work, we described this dynamic as clinical selection: a process in which the clinic, rather than preclinical mechanistic theory, supplies the decisive information about which interventions genuinely benefit patients. We contrasted this with the familiar “intelligent design” narrative, which imagines a linear march from target identification to rational design to cure.

Many of the most successful drugs did not emerge from deep mechanistic foresight, but from iterative, empirical exploration. The clinic functioned as an evolutionary engine. Anti-TNF drugs failed in their original indication before becoming foundational in autoimmune disease; statins survived only because physicians noticed striking patient responses after the field had largely moved on; and drugs like Avastin and Gleevec accumulated unexpected indications as human studies reshaped both their use and their mechanistic stories over time.

GLP-1 agonists offer a contemporary case in point. The earliest drugs in this class, such as exenatide, were developed for diabetes and aimed primarily at improving glycemic control. Later agents like liraglutide offered better pharmacological characteristics, making weight-loss applications more feasible. Even so, many experts thought that meaningful weight reduction was unattainable, because it required higher doses that caused unacceptable nausea. That side effect was overcome through clinical experimentation: gradual dose escalation markedly improved tolerability, enabling liraglutide’s approval for obesity in 2014.

Once a strong clinical signal existed, investment shifted back to refining the molecules themselves. Through extensive screening and chemical optimization of stability, potency, and half-life, Novo Nordisk developed semaglutide, a more durable agent suitable for weekly dosing. Clinical experimentation in patients without diabetes then delivered another surprise: patients lost far more weight than most experts predicted. At higher doses, semaglutide showed ~12.4% weight loss baseline body weight vs. placebo, a result that had previously been seen as out of reach for pharmaceutical interventions.

On the back of these results, semaglutide became one of the most commercially and clinically successful medicines of the modern era. Ongoing trials continue to reveal additional, unforeseen benefits of GLP-1 agonism, including reductions in cardiovascular events that appear independent of weight loss, as well as improvements in liver disease.

Only in the last couple of years have researchers started to stitch together a more complete mechanistic picture of GLP-1–driven weight loss, integrating evidence from animal studies, neuroimaging, gut–brain signalling, adipose-tissue biology, liver metabolism, and long-duration receptor pharmacology. Crucially, this understanding emerged after, not before, the clinical breakthroughs. And the mechanistic model remains incomplete, while the clinical outcomes are unambiguous, a clear case where human trials revealed the therapeutic potential before mechanistic biology could explain it.

The incredible arc of GLP-1 agonists also highlights how powerful the profit feedback loop can be when it is aligned with clinical success: strong therapeutic effect attracts investment, investment fuels optimization, and optimization yields even greater patient benefit. By contrast, when investment is decoupled from demonstrated clinical benefit, capital can flow into drugs that fail to meaningfully help patients.

A broad historical lens reinforces the point. As described in The Rise and Fall of Modern Medicine, the mid-1940s to 1970s were characterized by a tight feedback loop between new chemistry and human experimentation. Entirely new molecular classes, antibiotics, corticosteroids, and antihypertensives, were tested rapidly in patients. Clinical selection proved remarkably effective, despite our limited biological understanding. This period remains the most productive era in pharmaceutical innovation, earning the label “the Golden Era of drug discovery”.

Subscribe now

So far, our arguments have relied in large part on extrapolating from historical precedent. But what if this time really is different? AI is often cited as the reason it might be.

We share the optimism that AI will improve efficiency across many stages of drug development, from target discovery to trial design. However, it is unlikely (at least in the coming decades) to eliminate the need for empirical testing in humans. Indeed, some AI company CEOs have identified the inefficiency of clinical trials as a key regulatory barrier to allowing the benefits of AI to spill over into biomedicine.

Whether AI models can replace clinical testing ultimately depends on the data they are trained on. AI has achieved remarkable success in biology when applied to problems that are well-constrained and richly parameterized; where large, high-quality datasets exist, and the mapping between inputs and outputs is tight. AlphaFold’s success in protein structure prediction is a paradigmatic example, as it addressed a closed system with extensive labeled data and well-defined ground truth.

By contrast, the central challenge in drug development, the translation of molecular intervention into complex, organism-level therapeutic effects, remains underdetermined. For most disease areas, the relevant data landscape is sparse, heterogeneous, and observational. Perhaps the richest forms of relevant data are large-scale multi-omic datasets. These integrate multiple layers of biological information, such as genomic, transcriptomic, proteomic, metabolomic, and epigenomic profiles, to map how molecular systems interact across scales. Yet, despite their richness, they are taken at single timepoints and fail to capture the dynamic feedback loops, nonlinearities, and stochasticity that characterize living systems.

While such efforts are important for the long term, they are unlikely to replace empirical testing, unless dramatically novel modes of data generation that reflect human physiology are developed. This is where increasing the number of clinical trials could actually complement the predictive power of AI models. More experimental medicine in humans is not a substitute for AI, but its informational and economic complement: it would generate the data that would make AI more predictive over time.

Widen the funnel, instead of approving more low-quality drugs

Contrary to the libertarian view, loosening the standards for approval won’t fix the problem. Their logic seems appealing: if the regulatory bar is lowered, more therapies will reach patients sooner. And in some cases, we agree that approval standards should indeed be relaxed. But by focusing on the end of the pipeline rather than the process of experimentation itself, such reforms risk admitting more weakly effective or ineffective drugs, while doing little to expand the discovery of genuinely transformative ones.

Lowering the approval burden would, by definition, reduce clinical development costs: if fewer trials are required, sponsors face fewer expenses. Economist Alex Tabarrok has gone further, arguing that the FDA should only evaluate safety, leaving questions of efficacy to be resolved through post-approval clinical use. In such a system, the low initial hurdle would naturally encourage more drugs to be brought forward due to the lower up-front investment needed.

There is historical precedent for new use cases for drugs emerging from real-world use as opposed to randomized controlled trials. But there are many diseases, especially chronic ones with slow and noisy progression, where we do not envision how high-quality data on therapeutic effect can be collected without rigorous trials.

Moreover, even if the FDA were to approve all drugs that are merely safe, questions around insurance coverage and reimbursement would remain. In such a scenario, private entities may emerge to evaluate efficacy. But if clinical trials remained costly and time-consuming, easing regulatory requirements at the approval stage alone would do little to reduce the burden of generating credible evidence.

Over the past three decades, the FDA has become more permissive in what it accepts as evidence for drug approval. Between 1995 and 2017, pivotal trials grew less methodologically rigorous, with declines in randomization, blinding, the use of active comparators, and the measurement of real clinical outcomes. Most notably, trials have increasingly relied on surrogate endpoints that predict patient benefit rather than demonstrate it directly. This shift enables faster and cheaper studies, but it also raises concerns about whether approved drugs provide meaningful benefits for patients in the real world.

For a dramatic case in point, look at the approval of aducanumab for Alzheimer’s disease in 2021 through the accelerated approval pathway, based on beta-amyloid reduction as an endpoint. Despite two Phase III trials failing to show cognitive benefit, broad dissent within the FDA regarding its approval, and existing meta-analyses showing a poor correlation between beta-amyloid reduction and meaningful clinical benefit, the FDA granted accelerated approval. The result was a $56,000-per-year therapy with uncertain benefit, triggering significant backlash and eroding public trust. Congressional investigations and internal FDA reviews later concluded that the decision lacked scientific justification and damaged the agency’s credibility.

Support for easing approval standards is not limited to libertarian economists. Patient advocacy groups, especially in rare diseases, have increasingly pushed for similar reforms. These groups have not pushed as hard on inefficiencies in the clinical process in part because the process is opaque, while approval decisions are public, dramatic, and easy to mobilize around. But in clinical development, everything from details around trial design to regulatory correspondence remains largely hidden behind company walls.

Conclusion

The central paradox of modern biomedicine is that our ability to design interventions has advanced faster than our ability to test them. We are living through an era of unprecedented biological insight and tools, yet the process that determines which ideas translate into real therapies has become slower, costlier, and narrower.

Restoring exploratory capacity requires a rebalancing of the system: recognizing that clinical trials are not a bureaucratic middle step between discovery and approval, but the core engine of therapeutic learning. The Golden Era of Medicine illustrates what this looks like when it works.

The imbalance between the investment and return on clinical trials implies substantial low-hanging fruit: inefficiencies that persist not out of necessity, but neglect. But global differences in trial speed and cost, the demonstrated advantages of adaptive and platform trials, and emerging efforts to enhance regulatory transparency collectively show that safer, faster, and more informative clinical learning is well within reach.

Subscribe now

As Macroscience relaunches, a fundamental question has been on my mind. Namely, how must metascience adapt to understand the shape of science in the coming years?

Since we went on hiatus late last year, the Trump administration has fundamentally reconfigured the government’s involvement in American science and research. The norms that govern the degree to which political authorities intervene in the life of the academy have been reshaped. From cuts to research funding to the reworking of overhead rules, the underlying economics of the research university are being tinkered with in a way that they have not been since Vannevar Bush’s 1945 report The Endless Frontier.

Subscribe now

While a future administration might reverse some of these changes, the last twelve months will have enduring effects on the national research ecosystem. Most obviously, financial pressure may lead research institutions to close or significantly restructure their operations.

There are permanent human capital implications as well. Funding uncertainty and visa terminations will force some researchers to exit the formal research ecosystem entirely. Those people will not return, and their fields will be indelibly changed as a result.

Technological shifts are further agitating this instability. While the role that artificial intelligence might ultimately play in accelerating scientific progress writ large remains unclear, now-commonplace technologies like ChatGPT and Claude are already shaping the building blocks of research. Literature review, citations, and grant preparation have changed, perhaps forever.

These changes will affect the metascience community and how it conducts research. A vulnerability of the metascience research typically conducted by university economists is that it assumes a certain fixity to the specific configuration of institutions, funding, and research that has dominated post-war science. This assumption leaves academic metascience ill-suited to this uniquely volatile moment. Practically speaking, institutional instability makes it more challenging to conduct the careful interventions and longitudinal trials necessary to come to sound empirical conclusions. Institutions may not be able to durably invest in metascientific experimentation alongside their other priorities.

Moreover, where research happens may shift. Rather than staying in the US, foreign talent may return home or travel elsewhere to conduct their research. Even domestic talent may abandon the machinery of academic research in favor of private industry or philanthropically funded focused research organizations (FROs), where they could pursue their work more nimbly. In all these cases, research may become more fragmented and privatized, limiting the visibility that metascience has into how science happens.

Thus far, metascience research has depended on the legibility of how we conducted science in the late 20th century. To pick a granular example, the value of citations — a controversial metric even at the best of times — assumes certain stable institutional priors: that research is published at all by participants in a field, that humans exercise judgment in choosing what to cite, that large bodies of research do not disappear or become unavailable. These assumptions are all being challenged in the present moment. The scattering of researchers into private entities may weaken the uniformity of publication norms. Researchers might use artificial intelligence to programmatically surface related research and curate citations. And government funding cuts may render longstanding archives of papers and research material unavailable, or at least unmaintained.

This goes beyond the cliche that we live in times of great change. The institutional homes of science are under dramatic pressures that introduce a wave of methodological challenges. These changes will strain metascience’s established toolkit.

It is comforting to think that nothing ever happens, and that perhaps after a few years of gyration things will by and large return to past norms. But I’m not so sure. We need to retool metascience so that it can deliver insights in a radically changed scientific context.

There is a lot to talk about here, but it strikes me that any new mode of metascience must take the following three elements into account.

First, the metascience community must build partnerships with the new power centers of scientific research. If the weakening of the traditional research university pushes the scientific endeavor into new auspices, metascience needs to follow it there. How might private companies be persuaded to support metascientific research at scale? How should we study bibliometrics in a world where private companies have weaker incentives to publish results?

Second, there are increased benefits to directly observing science. Rapid institutional and technological shifts mean that quantified, large-scale datasets may be the lagging (or even misleading) indicator of how scientific progress is happening. “Shoe-leather” metascientific observation will be more valuable; seeing what’s happening on the ground may be the only way to gain insight into research practices and incentives.

Third, we need experimental designs that provide useful empirics at higher speed. Research that yields meaningful explanatory power about how science works requires long-term institutional buy in. Institutional volatility makes that difficult. Research designs that take advantage of this volatility, or can “get in and get out” rapidly with useful results will become more valuable. This is as much a matter of the research tools being used as it is the efficiency of the research teams conducting the work.

To lay claim to being a “science of science” means that metascience itself must be nimble. We must study science as it is being conducted, not as we wish it was being conducted. This requires us to ruthlessly question our tools. Doing so will ensure that the field can continue contributing meaningful insights, even as science changes radically in the coming years.

Subscribe now

If you’re receiving this email, you subscribed to Macroscience in its original iteration as a project of our Senior Fellow Tim Hwang. In 2023 and 2024, Tim published a series of articles discussing the principles grounding government’s role in shaping science. Tim’s idea was simple: We have macroeconomics, which helps us manage the money supply. But we don’t have anything like macroscience to help us organize science on a large scale. In addition to Tim’s newsletter, we hosted interviews with top metascience thinkers and the Metascience 101 podcast.

Subscribe now

American science needs metascience more than ever. Our scientific institutions, while still productive, are increasingly cautious and bureaucratic. They feel rickety. At the same time, political and technological change is happening rapidly. Science budgets are being cut, AI is taking off. Can our scientific institutions, built at the end of World War II, keep up with this change? Should they be redesigned? Rebuilt from the ground up?

Fortunately, our understanding of the structure of science has massively expanded. Building from a basis of metascience evidence, scientists and policy entrepreneurs are experimenting with new models for funding and doing science. In the political and technological tumult, there are new scientific and policy ideas waiting to be tested and opportunities ready to be seized.

We are bringing that experimental ethos to Macroscience. We are relaunching Macroscience with a broader focus and community of writers, including the scientists and policy entrepreneurs shaping the future of American science. Macroscience will explore ideas for how to improve science and policy, and it’ll feature writers who challenge assumptions and start friendly arguments. And it’ll provide, I hope, an optimistic but plausible vision for the future of scientific progress.

While Tim will still write on a regular basis, my aim is to create a larger community of science and technology writers that’ll contribute regularly to Macroscience. I’m Andrew Gerard, and I care about doing science better because of its incredible potential to improve people’s lives. I’m a social scientist, and spent most of my career working in international science policy. Before coming to IFP, I was Deputy Director of the Research Division at the US Agency for International Development (USAID), and I’ve also worked in higher education and non-profits.

Here are some of the questions that we want to explore in the coming months:

• What will science look like as the government radically reforms its relationship to the American research university?

• The sociology of metascience: who are in the different camps shaping the metascience discussion?

• Can we simplify federal science policy to speed up science?

• Should the US government take an equity stake in scientific innovations?

• Can there be negative marginal returns to science funding?

• What’s stopping us from getting better drugs, faster?

• Should Americans care about international science?

It’s going to be great — look for more to come soon. Don’t hesitate to email me with questions, feedback, pitches, or ideas.

Subscribe now

We’re publishing the first of these — from on peptides and antibiotic resistance — today. Enjoy!

Introduction

For all of human history until the past 100 years, infectious diseases have been our deadliest foe. Even during the roaring 1920s, nearly one in a hundred Americans would die of an infectious disease every year. To put that into context, the US infectious disease death rate was 10x lower during the height of the COVID-19 pandemic in 2021. The glorious relief we enjoy from the ancient specter of deadly disease is due in large part to development of antibiotic treatments like penicillin.

But this relief may soon be coming to an end. If nothing is done, antibiotic resistance promises a return to the historical norm of frequent death from infectious disease. As humans use more antibiotics, we are inadvertently running the world's largest selective breeding program for bacteria which can survive our onslaught of drugs. Already by the late 1960s, 80% of cases of Staphylococcus aureus, a common and notorious bacterial infection agent, were resistant to penicillin. Since then, we have discovered many more powerful antibiotic drugs, but our use of the drugs is growing rapidly, while our discovery rate is stagnating at best.

As a result, antibiotic resistance is spreading. Today, certain forms of Staphylococcus aureus, like MRSA, are resistant to even our most powerful antibiotics, and the disease results in 20 thousand deaths every year in the US.

The most promising solution to antibiotic resistance comes from dragon blood.

Komodo dragons, native to a few small islands in Indonesia, are the world’s largest lizards. They eat carrion and live in swamps, and their saliva hosts many of the world’s most stubborn and infectious bacteria. But Komodos almost never get infected. Even when they have open wounds, Komodo dragons can trudge happily along through rotting corpses and mud without a worry.

Their resilience is due to an arsenal of chemicals in their blood called antimicrobial peptides. These peptides are short sequences of amino acids, the building blocks of proteins. These chemical chains glom onto negatively charged bacteria (but not neutrally charged animal cells) and force open holes in the membrane, killing the infectious bacterium. Humans have peptides too, and we use them for everything from regulating blood sugar with insulin to fighting infections.

Peptides are especially promising candidates for antibiotic-resistant pathogens for two reasons. One is that they are easily programmable and synthesized. Their properties and structure are the result of chaining amino acids together in a line, so it’s easy to work with them computationally and apply machine learning and bioinformatics. The second reason is that peptides are resistant to resistance. Researchers can use them to target much more fundamental properties of bacteria, whereas antibiotics target particular molecular pathways that are often closed by a single, small mutation. For example, bacterial membranes are almost universally negatively charged; it is a feature of their physiology which is not easily mutated away. Therefore, peptides which use this negative charge to seek out and destroy invading bacteria are difficult to avoid, even after those bacteria evolve through generations of intensive selective breeding as a result of being targeted.

Even though peptides are short, usually less than 50 amino acids, the combinatorial space of peptide sequences is vast. It’s difficult to search through this space for peptides that are effective against the resistant superbugs which threaten to return us to the medieval world of deadly infections. However, searching for these peptides is a well-defined problem with easy-to-measure inputs and outputs. The fundamental research problem is perfectly poised to benefit from rapid advances in computation. The cutting edge of research in this field involves building machine learning models to predict which sequences of amino acids will be bio-active against certain pathogens, similar to Deepmind’s AlphaFold, then developing those peptides and testing the model’s predictions.

But progress in this field is slower than we need it to be to meet the challenge of antibiotic resistance. This isn’t just due to inherent difficulties in the science, though of course those do exist. Progress towards antimicrobial peptides is slowed by scattered, poorly maintained, and small datasets of peptide sequences paired with experimentally verified properties. Machine learning thrives on big data, but the largest database of peptides only has a few thousand experimentally validated sequences and only tracks three or four chemical properties, like antimicrobial activity and host toxicity. These properties are often difficult to compare to other sources.

Most importantly, there is almost zero negative data in these sources. Scientists test hundreds or thousands of peptides to find one which is active against some pathogen, and then they publish a paper about the one which succeeded. That success might go into the database, but all of the preceding failures are kept in the file drawer, even though they are, at current margins, far more valuable for machine learning models than one more success data point.

Making a better dataset is feasible and desirable, but no actor in science today has the incentives to do it. Open data sets are a public good, so private research organizations will tend to underinvest. The non-pecuniary rewards in academia like publications and prestige are pointed towards splashy results in big journals, not a foundational piece of infrastructure, like a dataset.

This problem is solvable with an investment in public data production. A massive, standardized, and detailed dataset of one million peptide sequences and their antimicrobial properties (or lack thereof) would accelerate progress towards new drugs that can kill antibiotic-resistant pathogens. This would replicate the success of datasets like the Protein Structure Initiative and the Human Genome Project and put us on track to defeat these drug-resistant diseases, before they roll back the clock on the medical progress of the past century.

What Are Peptides, and How Do They Work?

Proteins are the machinery of biology: they constitute the motors, factories, and control surfaces of cellular life. Some proteins are incredibly complex, like this motor protein made of thousands of amino acids.

Peptides are a particular kind of protein. They are short and simple without many moving parts. Instead of using intricate and specialized binding sites like larger proteins, peptides just use thousands of copies of themselves and preferential chemical attractions to perform various tasks in the human body, like regulating blood sugar or pain sensitivity.

Antimicrobial peptides are peptides whose specific purpose is killing pathogens that are invading the body. These are subjects of active research in microbiology. Our body employs lots of antimicrobial peptides naturally. Peptides like Defensin or LL-37 are most frequently found on our skin or in our mouths and noses as the first line of defense against all of the pathogens we come into contact with.

Much is still unknown about exactly how peptides work and how to target them, but antimicrobial peptides tend to have a positive charge and two different surfaces along their structure that either attract or repel water. This attracts them to pathogenic bacteria, which have negatively charged membranes. Then, the hydro-phobic and -philic surfaces of the peptide interact with the membrane to drill holes in it, and the cell collapses and dies. Lower concentrations of peptide may not kill the invading pathogens, but they will slow down their metabolic processes, giving a head start to the rest of our immune system.

Eukaryotic membranes, which normal human cells are made of, have different fats on their membranes, which means they are much closer to neutrally charged and aren’t as vulnerable to the attacks that peptides make on cell membranes. Peptides can also target gram-positive vs gram-negative bacteria; they can preferentially attract to bacteria with thin, single-layer membranes or thick, multilayered ones. This specificity is important because it can help preserve non-pathogenic, beneficial bacteria while still attacking invaders.

None of this targeting is perfect. Peptides are sent out millions at a time and, since they get stronger as the concentration on a cell increases, small differences in chemical preference lead to big differences in activity. Some of our cells will bump into these peptides by chance and potentially be affected, but hundreds of times more peptides will be reliably attracted to targets like negative charge and particular chemicals on the cell walls of bacteria. This is similar to how traditional antibiotics work: There is some degree of targeting, but a heavy dose of antibiotics will still harm beneficial bacteria and human cells. That tradeoff is often worth it to fight off a deadly disease.

Peptides have two big advantages over antibiotics. The first advantage is resistance to resistance. Antibiotics often target very narrow biochemical reaction pathways into a bacteria’s metabolism or particular proteins found in the cytoplasm of pathogens, whereas peptides target general properties of a bacteria’s entire membrane, like charge or lipid composition. This gives antibiotics a slight advantage in specificity, but it also makes antibiotics easy to resist. Changing one residue in a target protein is a lot easier than changing the electric charge over the entire bacterial surface. This general targeting has allowed antimicrobial peptides to be effective first defenses against pathogens for millions of years without changing much.

The second advantage of peptides is that they are easy to synthesize and mass manufacture. Biology has done most of the heavy lifting for us here. Proteins are so versatile and fundamental to so many biological processes that nearly every cell has completely general purpose protein factories. We can take single-celled organisms that are simple and easy to grow, like yeast, insert the right DNA instructions, add sugar, and the yeast will start pumping out copies of the desired protein. There are dozens of companies that will synthesize custom proteins on demand for reasonable prices. By rapidly synthesizing and testing hundreds of different peptides, you can screen for effective and non-toxic treatments and scale them up in six or seven days. This is a stark contrast to small molecule antibiotic manufacturing, where figuring out how to synthesize a particular chemical can take years of trial and error, and making that synthesis efficient can take even longer.

The broad-spectrum chemical warfare and mass manufacturing ease of antimicrobial peptides makes them a promising avenue for combating antibiotic-resistant pathogens. Their ability to disrupt fundamental properties of bacterial cells, rather than specific molecular pathways, suggests that peptide-based treatments could remain effective over longer periods compared to traditional antibiotics, and the ease of synthesis means that new treatments can be made in weeks instead of years when the need does arise.

The Frontier of Research

Peptides have verified effects on the toughest antibiotic-resistant infections including MRSA, on viral infections like HIV, on fungal infections, and even on cancer. But they still aren’t common on pharmacy shelves or in hospital treatment. Some current clinical trials will change this, but the main barrier is still in the fundamental research.

Peptides are chains of chemicals where each link is chosen from 1 of 20 amino acids. Thus, the combinatorial space of possible peptides is incomprehensibly massive. We have mapped a tiny fraction of this space. Only a few thousand peptides are registered in databases, and there are even fewer with all the important information on not only antimicrobial activity, but also specific targeting and host cell toxicity. Much of the research on peptides has started by indexing naturally occurring peptides which takes advantage of evolution’s exploration of this combinatorial space over billions of years, but it’s still nowhere close to comprehensive.

The frontier of research in this field uses machine learning to explore the vast space of possible peptides and filter them down to the most promising candidates, similar to Google’s AlphaFold, which used machine learning algorithms to improve the prediction of a protein’s 3D structure based on the sequence of amino acids that make it up. Machine learning models of peptides also try to improve predictions based on the amino acid sequence of a protein, but they more directly target the medical properties of the peptides, rather than just trying to predict their 3D structure. Machine learning prediction on peptides may also be more tractable than AlphaFold because peptides are so much shorter than most proteins.

Based on a database of a few thousand peptide sequences, researchers have used machine learning techniques to predict brand new peptides that are active against MRSA, HIV, or cancer, and often at higher rates than naturally occurring analogs. One way they did this is by splicing, shuffling, and combining some of the existing sequences into new ones. Other approaches apply successive filters to the database and then combine the properties of those filtered sequences into a new peptide. Both of these approaches created peptides with high degrees of activity against multi-drug-resistant infections like Staphylococcus aureus.

All of this research is very promising, but it’s still moving slow because of one main constraint: data.

The Problem

Machine learning needs data. Google’s AlphaGo trained on 30 million moves from human games and orders of magnitude more from games it played against itself. The largest language models are trained on at least 60 terabytes of text. AlphaFold was trained on just over 100,000 3D protein structures from the Protein Data Bank.

The data available for antimicrobial peptides is nowhere near these benchmarks. Some databases contain a few thousand peptides each, but they are scattered, unstandardized, incomplete, and often duplicative. Data on a few thousand peptide sequences and a scattershot view of their biological properties is simply not sufficient to get accurate machine learning predictions for a system as complex as protein-chemical reactions. For example, the APD3 database is small, with just under 4,000 sequences, but is among the most tightly curated and detailed. However, most of the sequences available are from frogs or amphibians due to path-dependent discovery of peptides in that taxon. Another database, CAMPR4 has on the order of 20,000 sequences, but around half are “predicted” or synthetic peptides that may not have experimental validation, and contain less info about source and activity. The formatting of each of these sources is different, so it’s not easy to put all the sequences into one model. More inconsistencies and idiosyncrasies stack up for the dozens of other datasets available.

There is even less negative training data; that is, data on all the amino-acid sequences without interesting publishable properties. In current machine learning research, labs will test dozens or even hundreds of peptide sequences for activity against certain pathogens, but they usually only publish and upload the sequences that worked. Training a model without this data makes it extremely difficult to avoid false positive predictions. Since most data currently available is “positive” — i.e, peptides that do have antimicrobial properties — negative data is especially valuable.

Expanding the dataset of peptides and including negative observations is feasible and desirable, but no one in science has the incentive to do it. Open data sets are a public good: anyone can costlessly copy-paste a dataset, so it is difficult and often socially wasteful to put it behind a paywall. Therefore, we can’t rely on private pharmaceutical companies to invest sufficiently in this kind of open data infrastructure. Even if they did, they would fight hard to keep this data a trade secret. This would help firms recoup their investment, but it would prevent other firms and scientists from using the data, undercutting the reason it was so valuable in the first place.

Non-monetary rewards like publications and prestige are pointed towards splashy results in big journals, not toward foundational infrastructure like an open dataset. Scientists are often altruistic with open datasets and tools that they’ve developed for personal use. In the field of antimicrobial peptides, researchers host open peptide databases and prediction tools free for anyone to use. They are motivated by a genuine desire to see progress in this field, but genuine desire doesn’t pay for all of the equipment and labor required to scale up these databases to ML-efficient size.

The most common funding mechanisms for researchers in this field reinforce the shortfall in data infrastructure investment. Project-based grants, like the NIH’s R01, are focused on specific research questions or outcomes. These grants usually have relatively short timelines (e.g., 3-5 years) and emphasize novel findings and publications as key metrics of success.

This emphasis on short-term project-based grants stems from a desire for measurable outcomes, accountability, and novelty. University tenure committees and academics themselves heavily weigh high-impact publications and grant funding. Building infrastructure, while valuable to the scientific community, typically generates fewer publications, is often seen as less prestigious or less interesting, and has more spillover benefits that aren’t credited. NIH program officers also want clear metrics of their impact, and the higher-ups need to convince Congress that they aren’t wasting billions of dollars by enforcing accountability of their funding decisions to those metrics. Accountability is easier with smaller projects that have a shorter gap between investment and return. Mistakes are less damaging when the funding amounts are small and more of the responsibility for funding decisions lies outside of the NIH, in expert external review panels. Another important metric targeted by the NIH is novelty. The NIH and its remit from Congress explicitly prizes novelty of research and its results. Internal and external calls for the NIH to pursue more “high-risk, high-reward” research reinforce this desire for discrete projects with novel designs over and above expansions of already established scientific techniques.

The million-peptide database project is not a high-risk high-reward experiment, or a counterintuitive result that can turn into a highly cited paper or patent. Instead, it’s a massive scale-up of established procedures for synthesizing and testing peptides that will be more expensive and time-consuming than a project-based grant and have a less legible connection to the metric of success tracked by academics, the NIH, and Congress.

The Solution: A Million-Peptide Database

The data problem facing peptide research is solvable with targeted investments in data infrastructure. We can make a million-peptide database

There are no significant scientific barriers to generating a 1,000x or 10,000x larger peptide dataset. Several high-throughput testing methods have been successfully demonstrated, with some screening as many as 800,000 peptide sequences and nearly doubling the number of unique antimicrobial peptides reported in publicly available databases. These methods will need to be scaled up, not only by testing more peptides, but also by testing them against different bacteria, checking for human toxicity, and testing other chemical properties, but scaling is an infrastructure problem, not a scientific one.

This strategy of targeted data infrastructure investments has three successful precedents: PubChem, the Human Genome Project, and ProteinDB.

The NIH’s PubChem is a database of 118 million small molecule chemical compounds that contains nearly 300 million biological tests of their activity, e.g. their toxicity or activity against bacteria. This project began in the early 2000s and was first released in 2004. More than the peptide database proposed here, PubChem is about aggregation and standardization rather than direct data creation. It combined existing databases, and invited academics to add new molecules to the collection. This was still incredibly useful to the chemistry research community. With a budget of $3 million a year, PubChem exceeded the size of the leading private molecule database from Advanced Chemistry Development by around 10,000x and made the data free. PubChem is credited with supporting a renaissance in machine learning for chemistry.

Another success is the Human Genome Project. This 13-year effort began in the early 1990s and cost about $3.8 billion. Unlike PubChem, the Human Genome Project couldn’t rely on collating existing data, and had to industrialize DNA sequencing to get through the 3 billion base pairs of human DNA in time. Over the course of the project, the per-base cost of DNA sequencing plummeted by ~100,000-fold. By 2011, sequencing machines could read about 250 billion bases in a week, compared to 25,000 in 1990 and 5 million in 2000. Before the HGP, gene therapies were less than 1% of clinical trials; today they comprise more than 16%, all building off the data infrastructure foundation laid by the project.

Perhaps the closest analog to the million-peptide database proposal is ProteinDB, a database of around 150,000 complex proteins and their 3D structure. This open data base began as a project of the Department of Energy’s Brookhaven laboratory in the early ‘70s and has evolved into an international scientific collaboration. ProteinDB is like PubChem, in that it has become the primary depository for protein structure discoveries, but it is also like the Human Genome Project in that it was paired with a large data generation program: the Protein Structure Initiative (PSI). The Protein Structure Initiative was a $764 million project funded by the U.S. National Institute of General Medical Sciences between 2000 and 2015. The PSI developed high-throughput methods for protein structure determination and contributed thousands of unique protein structures to the database. By 2006, PSI centers were responsible for about two-thirds of worldwide structural genomics output. The hundreds of thousands of detailed 3D protein structures in the databank were the essential training data behind the success of AlphaFold.

These projects cut against the NIH’s structural incentives for smaller, shorter, investigator-led grants, but they still succeeded. PubChem was housed within the National Library of Medicine, which already had a mandate for data infrastructure, and received dedicated funding through the NIH Common Fund rather than competing with R01s. It also managed some of the drawbacks of data infrastructure projects in legibility and credit assignment by creating clear metrics of success around database usage, downloads, and a formal citation mechanism for database entries. Similarly, the Protein Structure Initiative was funded through the National Center for Research Resources, another NIH division with an explicit focus on research infrastructure.

The Human Genome Project overcame its barriers through a strong presidential endorsement and dedicated Congressional funding that bypassed normal NIH processes. It sustained this political momentum by developing clear technical milestones, like cost per base pair, that could be evaluated without relying on traditional academic metrics.

Here’s how a scientific funder like the NIH can adapt the success of ProteinDB, the Protein Structure Initiative, PubChem, and the Human Genome Project to create a million-peptide database:

Like PubChem, start by merging and standardizing existing peptide datasets, and open them to all. This alone would be a big help for machine learning in peptide research. A researcher today who wants to use all available peptide data in their model has to collect dozens of files, interpret poorly documented variables, and filter everything into a standardized format. Hundreds of researchers are currently duplicating all of this work for their projects. Thousands of hours of their time could be saved if the NIH or NSF paid to organize this data once and for all and opened the results to all interested researchers. Setting a Schelling point for all future data additions would also help keep the data standardized as the dataset grows.

Collecting existing data won’t be nearly enough to get to a million-peptide database. The next step, like the Protein Structure Initiative and the Human Genome Project, is to industrialize peptide testing. Mass-produced protein synthesis and testing are already well-established techniques in the field, so this project won’t need any 100,000x advances in technology to succeed like the HGP did. A scientific funding organization like the NIH only needs to support scaling up these existing techniques. Researchers can already test tens or hundreds of thousands of peptides simultaneously.

Industrializing peptide testing is more complicated than the demonstrations in individual research papers, because we need to screen for lots of variables in addition to a single measure of anti-microbial activity as the above research projects are doing. We want to know about the peptide’s activity against a broad range of bacteria, viruses, fungi, and cancer cells, we want to know about the peptide’s effects on benign human cells or beneficial bacteria so it doesn’t do too much collateral damage, and we want to know about the peptides that failed to have any interesting effects so our machine learning models know what to avoid. For peptide testing to match the scale needed by machine learning models, it needs to be funded beyond the resources available for a single paper.

This effort requires a purpose-made grant from a scientific funding agency like the NIH or the NSF, not a standard PI-led research project. The focus here should not be papers, citations, or prestige; just data. With a grant like this, a million-peptide database is achievable well below the budget and timeline standard set by the Protein Structure Initiative and the Human Genome Project.

Retail custom proteins cost $5-$10 per amino acid. At an average peptide length of 20 amino acids, that’s around $200 per peptide. That cost is just for the synthesis, not all of the time and labor required for testing, so a reasonable upper bound on the cost of a million-peptide database is $350 million. Even this large upper bound cost is likely justified by the potential impact of antimicrobial peptides. The direct treatment costs for just six drug-resistant infections is around $4.6 billion annually in the US, with a far greater cost coming from the excess mortality and damaged health.

The actual cost is likely considerably less than this $350 million upper bound. Performing protein synthesis in house and in-bulk rather than buying retail can greatly reduce costs. Additionally, these synthesis costs are for the highest-quality resin synthesis. High throughput methods, like SPOT synthesis, can be less than 1% of the cost per peptide, and allow researchers to synthesize thousands of peptides at once. Clinical use of the tested peptides would probably require retesting them with more expensive, higher purity methods, but you’d only need to retest the few most promising candidates. For the purpose of supplying millions of data points to a machine learning model, the purity of this high throughput method is more than sufficient.

Other methods use mass-produced DNA plasmids to induce bacteria like E. Coli to produce peptides on long chains attached to their membrane which, if they’re antimicrobial, end up killing the host cell. Researchers can then blend up all of the E. Coli and check which of the DNA plasmids copied themselves and which did not. The plasmids that didn’t reproduce are the ones which encoded antimicrobial peptides and prevented their host bacteria from multiplying. This method allowed University of Texas researchers to test 800,000 peptides at once, at a cost significantly lower than any other high throughput testing method. The downside is that you never get to isolate the actual peptide from the bacterial culture, which limits the types of tests you can run. But scaling up this process could easily generate hundreds of thousands of peptide candidates with some verified anti-microbial activity that can then move on to more detailed tests.

The time required to build a million-peptide database is also reasonable, perhaps less than five years. A single researcher can synthesize 400 peptides on a 20×20 cm cellulose sheet in 6 days using SPOT synthesis and can probably perform tests for antimicrobial activity, human toxicity, and other traits in another week. With an automated pipetting machine the yield increases to 6-8 thousand peptides in the same six days. A rate of 8,000 peptides synthesized and tested every two weeks would get to a million peptides in 1,800 days, just under five years. Most importantly, almost all of these processes are highly parallelizable, so scaling up the number of peptides you want to test doesn’t necessarily increase the amount of time it takes if you can set up another researcher or pipetting machine working in parallel.

The failure of standard scientific incentives to fund the creation of the peptide database is solvable. A single concentrated effort over several years would lay a foundation for a machine learning renaissance in antimicrobial peptide research, as PubChem, the HGP, and ProteinDB did for their respective fields.

Conclusion

The specter of infectious disease that haunted humanity for millennia is threatening to return. Our century-long respite from the constant threat of deadly infections is at risk as antibiotic resistance spreads. Already, antibiotic-resistant infections claim over 1.2 million lives annually worldwide. Peptides, in dragon blood and human spit, have been nature’s first line of defense against these infections for millions of years. We can learn from and improve upon nature’s example, making new effective treatments for some of the world’s deadliest and intransigent diseases.

More than simply preserving the 20th century safety that antibiotics created, peptides can exceed the effectiveness and versatility of antibiotics. Peptides are just short proteins and proteins are the machinery of all living things. Peptides can thus help prevent not only bacterial infections, like antibiotics, but also viruses, fungal infections, and cancer. Peptides are also programmable and easy to manufacture. Once we figure out how the properties of a peptide change as we substitute different amino acid building blocks, we will be able to design, test, and mass manufacture new treatments within weeks, rather than the decades it takes for new antibiotics to come to market.

The path towards this future is clear. Machine learning prediction on the sequence of amino acids is a promising and tractable way to advance our understanding and control over the properties of antimicrobial peptides. The most difficult scientific bottlenecks with this strategy have been crossed; all we need now is scale.

That means we need data. The existing data infrastructure for antimicrobial peptides is tiny and scattered: a few thousand sequences with a couple of useful biological assays scattered across dozens of data providers. No one in science today has the incentives to create this data. Pharma companies can’t make money from it and researchers can’t get any splashy publications. This means researchers are duplicating expensive legwork collating and cleaning all of this data and are not getting optimal results as it’s simply not enough information to fully take advantage of the machine learning approach.

Scientific funding organizations like the NIH or the NSF can fix this problem. The scientific knowledge required to massively scale the data we have on antimicrobial peptides is well-established and ready to go. It wouldn’t be too expensive or take too long to get a clean dataset of a million peptides or more with detailed information on their activity against the most important resistant pathogens and its toxicity to human cells. This is well within the scale of successful projects that these organizations have funded in the past like PubChem, the HGP, and ProteinDB.

We can meet this challenge and solve it quickly if we target our resources towards building open data infrastructure that thousands of research projects will use. Let’s not wait while antibiotic-resistant pathogens get stronger.

IN THIS EPISODE: Professor Heidi Williams, Professor Paul Niehaus, and Matt Clancy walk through academic, non-profit and private sector paths to research, the importance of your surroundings, and how you can find good use-inspired questions.

“Metascience 101” is a nine-episode set of interviews that doubles as a crash course in the debates, issues, and ideas driving the modern metascience movement. We investigate why building a genuine “science of science” matters, and how research in metascience is translating into real-world policy changes. 

Episode Transcript

(Note: Episode transcripts have been lightly edited for clarity)

Caleb Watney: Welcome. This is the final episode of our Metascience 101 podcast series where we’ll turn to how you can get involved in metascience research. Professor Heidi Williams discusses career paths with innovation economist Matt Clancy and Professor Paul Niehaus. This episode touches on academic, non-profit, and private sector paths to research, the importance of your surroundings, and how you can find good, use-inspired questions.

Heidi Williams: Great. Our goal with this discussion is to give some advice to students and young people who might have listened to some of this series and are excited to get involved but are not exactly sure what that might look like, from a practical perspective. 

On paper, the three of us here look similar in the sense that we all pursued PhDs in economics. I would guess that we each saw some value in the toolkit that the field of economics provides, to help us make progress on problems that we care about. Paul's and my career trajectories, on paper, look even more similar since we both finished our PhDs and went straight into academic jobs. In practice, however, each of the three of us actually took different paths that shaped how we thought about making progress on problems that we care about. 

We all value interacting with people with a wide variety of skill sets. And we wanted to bring our perspectives to this discussion on these issues for young people.

Let’s start off with careers in government. Matt, after you finished your economics PhD, your first job was at the U.S. Department of Agriculture, USDA. Tell us about opportunities to improve science from a public service perspective.

Matt Clancy: Sure. I worked for the Department of Agriculture, in the Economic Research Service (ERS) there. I was a research economist, and my government agency was unusual in that it was more like an academic department than many research departments in other agencies. 

In other agencies, often you’re focusing on solving a problem for your agency’s stakeholders. For instance, if you work for the Environmental Protection Agency (EPA), you might literally be doing cost benefit analysis-type stuff. We were still trying to publish in academic journals, and, in that sense, that made us more similar to you in your careers.

But there are differences with academia. There, you’re aiming to publish research that you think is interesting, and that you hope your peers are going to find interesting from a pure knowledge standpoint. Our end goal was to help policymakers craft policy, and we tried to anticipate their information needs, because research takes years to play out. There was an entrepreneurial element where we needed to forecast out three to five years, “What are going to be the issues that are going to be important in agriculture?” We have to start researching and gathering data on those things now, so that we’ll be able to inform policy down the road.

In making policy decisions, sometimes we can't identify something very well, the data is not very good, or there's not a nice clean experiment. Yet a decision still has to be made. Some number has to be used to guide it, or else it's just based on intuition. That mindset gave me a different framework about the value of my research from the one I had when I started my PhD. I started asking, “What's the end point of doing this?” At USDA, we knew that somebody needed to make a decision, and we wanted to inform that decision with better information.

Heidi Williams: There are a lot of different agencies that people don't think about as intersecting with science policy, but they actually have very important inputs into a lot of the topics that were covered in this series of episodes. 

To give one example, the Congressional Budget Office needs to tabulate the budget implications of basically every piece of legislation that comes through. They are asking questions like: What's the research and development investment budget, and how do we think about those implications? Or: What are the productivity implications of changes in high skilled immigration policy? 

Those are questions that economists themselves research. When you're at a government agency, you might be tackling a very similar question, but with a specific consumer in mind. You’re thinking, “We expect that a given set of people in Congress is going to have these types of questions, and we're going to need to pull together research and synthesize the best available answer to that question.”

That's much more the motivation than simply the need to come up with curiosity-driven research questions. In agency work, you know what the research questions are, and you have a very direct connection to the consumer of your work. Is that one way that you would describe it?

Matt Clancy: At the Department of Agriculture, and at Census and at the U.S. Patent and Trademark Office, the box of acceptable research questions is definitely smaller than when you're at a university. 

I went to university after this, and there you can do whatever you want. You don't have anybody over your shoulder checking in on you quarterly to see what research projects you're working on. 

But within this still large box of research that we thought would be relevant to policymakers, we had a lot of scope to do research that interested us. At the end of the day, the American taxpayer is paying you, and they are trying to get something for it. That’s the ethos in these agencies rather than just seeking knowledge for knowledge’s sake. You do have some autonomy, however. 

Here’s a concrete example: on my first day, at the Department of Agriculture, they told me, “We need to know about the implications of restrictions on antibiotic use in agriculture. We think using them so much may cause antimicrobial resistance, and it could be a problem. There are going to be new restrictions on antibiotics.”

That's going to have knock-on effects on agriculture, because they don't use antibiotics just for fun but for a reason. They actually help the animals grow more quickly. If we're not going to let them use them for that purpose anymore, can we incentivize the drug agencies to develop other drugs that will have the same effect without this antimicrobial resistance? That kicked off a two-year project to understand the whole sector and the incentives, and to be able to give advice about what we should do.

I was given an objective with what we need to do. It still interested me as this new problem that I could sink my teeth into. The other half of the job evolved from hearing, “Matt, you just sort of have to figure out what to do.” At that time I thought, “There's all this patent data that we're not using to study innovation in agriculture; let's build a data set and start exploring questions about that.”

Paul Niehaus: This is dynamic, and new opportunities like this are opening up. They're in places like USDA ERS that are very long-established, and where it's understandable what a job there looks like. There are places like USAID, which is most related to what I do, where they've had a chief economist for a long time, but not really a chief economist office and team. Now, Dean Karlan is trying to build a culture of evidence-based decision making, and that may open up new opportunities as well. Those are some of the most exciting, new opportunities for stuff like this.

Heidi Williams: I agree. It seems like students thinking about careers in government can choose an agency whose mission you find really inspiring, as in “I'm really inspired by Sasha Gallant, and I want to go work at Development Innovation Ventures at USAID.” They do have entry-level jobs that can become an on-ramp to further work. 

Or you can also match through fellowship programs that try to hit people at certain career stages and on ramp through them to more exposure to government. One that's very natural for PhDs is the American Association for the Advancement of Science (AAAS) Fellowships, which gives you a direct placement in government, and there's often support for you to see more than one office. For people just out of undergrad, the Horizon Fellowship is another program that is very good about helping you find a placement, even if you're not currently in government. These fellowship programs can provide a natural way to on-ramp people into government and to find a good placement for their particular skill set.

The second category I want to talk about is what you can think of as academia-adjacent research jobs. There is economic research that is done outside of academia, and that is often, in the way that Matt was describing, more closely related to real-world problems. Think tanks are one natural place. Some private philanthropies like Open Philanthropy are doing research in a very directed way. 

I would also put journalism tackling social problems in this category. I think of this as very closely adjacent to research. Something like the Vox Future Perfect Fellowships or public writing that's not necessarily attached to a given outlet, both are engaging in research on questions that you think are really important. 

I'm curious if you each could each share an example of someone you've seen in that position. What are the pluses and minuses of that career track for people as a means of exposure to other career opportunities?

Paul Niehaus: The first thing that comes to mind is the global development space in the NGO world. There are certainly positions in the World Bank, which is a well established track, and some of the other big multilateral development banks. Many of the bigger NGOs, especially the ones that are more evidence focused, have a research function internally. The IRC has a great research team. At GiveDirectly, we have a research team. There are people there with PhDs who are doing great economics research that is very focused on the needs and questions of the NGO that they work at.

I typically see people go there a little bit later in their careers, after having done some academic work and reached a decision that they would like to shift the balance. They might think, “I want to be doing things that are going to have an immediate tangible impact, and where I'm confident that the questions I'm looking at are important questions, because they're coming to me from the rest of the team in the organization.” That's a great route.

Matt Clancy: Where I work now, Open Philanthropy, has a number of different people engaged in basically pure research positions. I'm actually a research fellow, although a portion of my duties is grantmaking. There are some people who do pure research. Though again here, it’s not purely curiosity driven. 

There’s an instrumental objective, like say, “We're thinking of launching maybe a new program. There's an academic study that shows that the program was really effective. Can we dig into that study, replicate it, and make sure it's effective?” Other things are more open-ended, like learning about potential areas to fund. Sometimes it’s researching if there are tractable ways to make progress on those, if the problems are important, and if there is a valuable marginal dollar or whether that space is already saturated.

What you mentioned earlier about writing in public is an interesting, new path. The internet is a prominent way to network that we didn’t really have twenty years ago. It used to be that to network with people and find opportunities, you had to move to DC and meet the government policymakers at happy hours or different functions.

You can advertise what you're interested in on the internet as well. Writing a high-quality blog credibly signals, “This is what I'm interested in, and you can see my quality.” This may all break down with ChatGPT in the future. But that’s how I changed my career trajectory. I was in academia, working on the New Things Under The Sun project. That caught the attention of Caleb and Alec in the think-tank world, and that's how I began my collaboration with them.

Brian Potter was a construction engineer who was writing a super high-quality analysis of construction and asking why productivity in construction was not going up like other industries. Now, he’s joined the Institute for Progress too. I can think of other examples too. So if you're not in the job you want to be in, you're not in government, you don't work for a think-tank, one possible way to get attention is through the internet. 

Paul Niehaus: I have seen that work also in the opposite direction. There was a remarkable civil servant in India, who had blogged about the latest research papers that were coming out. We all wondered, “Who is this gem of a human being?” Then we started talking to him to figure out what research we should be doing, because we really valued his opinion. He's gone on to work at Global Innovation Fund, funding research, among other things. 

Matt Clancy: It can be a new kind of credential too, because for the right open-minded person, you can point to a voluminous documentation of your interest and expertise in the topic. 

Heidi Williams: I really encourage students to spend time in government at some point, whether right out of undergrad, or while they're doing graduate work, because you can often get a much better sense of the relevant constraints and objectives of the institutions that you study, from spending time physically working in them.

But I know, from people that similarly spent even a short time in private sector firms, that they learn a lot too. “Wow, the way that I conceptualize how firms make decisions, what they see as the regulatory constraints, or how they think about their path for getting ideas out to have an impact on the world is very different from how I thought.” That then brings them back to research a different set of questions. 

Do you think too few people see time in the private sector as something that they should do? Do people assume the private sector is a place that you go there to stay, and not to rotate in and out? 

Paul Niehaus: That sort of rotation, once you've committed to an academic path, can be a little tricky, because if you're really full-time, what do you do? You ditch your co-authorship relationships and tell the editors that you're not going to do referee reports. It’s hard to unwind the web of commitments and obligations that you make in any one path, and really commit to another one. 

But yes, 110%, there's incredible value in spending some time and exposure in the private sector. My own experiences, starting two companies and having to build things from the ground up, led to all kinds of painful, lived experiences and lessons learned that way.

The example that I give to my students, which I really love, is from Paul Oyer, your colleague at the business school at Stanford. Paul has this great job market paper, which shows that sales spike at the end of the fiscal year because salespeople want to make their quota. 

This paper came about because he was sitting in grad school, and they were looking at some data on seasonality and sales, and there was a spike and, and everybody said, “Well, that's weird.” Then they just moved on.Paul said, “Well, that makes sense, because it's the salespeople making their quota,” because he had worked in sales right before going to grad school. Everybody said “No, no, no, that wouldn't make any sense.” He said, “I'm pretty sure that's what it is.” 

So he wrote a great job market paper and got a great job out of it. It’s knowing how to interpret the things you're looking at, what sorts of things to look for, and not dismissing offhand things that seemed to not make sense from one mindset, because you've actually been out there in the world.

Heidi Williams: All three of us decided to go pursue a PhD in economics. If you were going to advise students on who should think about that path, what are the things that people often miss in thinking about this option as a path for having social impact with their work?

Paul Niehaus: The single biggest thing that I think people don't understand is that having a PhD is so flexible. Having a PhD and an academic job is such a platform, and people do such different things with it. Heidi, you're one of them, Matt, you were one of those people. I've done a whole diverse mix of things, including some research, but also starting a multinational NGO, and a couple of companies and lots of other things.

There are always trade-offs. But the first thing I want everybody to know is that a fundamental feature of the job is that you get to decide what to do with your time. If you want to get tenure, if you want to publish a lot of papers, that adds constraints. You have to think about how to do that, and what people are going to be responsive to. But that just gives you enormous freedom, right?

It also gives you a degree of security. When I'm doing entrepreneurial stuff, while I have this academic job, there's some risk here, but I know I can afford to take risks. If I want to express an unpopular opinion to a policymaker, I feel the freedom to do that, because I know that it's not going to cost me my job.

I think there is so much value to the platform aspect of it. But the key thing is that you need to envision it that way, not everybody is going to teach you to think of it that way.

Matt Clancy: I could imagine somebody who thinks, “I love research. I think I want to dig into these problems, but I don't want the academic life where I have to move all the time, and I have to do an extensive predoc, and then I have to jump through all these hoops, and then I'm racing to get tenure.”

That's one path, but that's not the path you have to necessarily take, like Paul said. 

I went to Iowa State University, and I'm doing fine in my life. Most of the people in my cohort are also doing fine, teaching at small liberal arts colleges or in government. We didn't have to run through all the postdoc stuff. If the predoc and this tenuous life are not what you want, the key thing is to ask, do you actually still want to do a PhD? Do you want to learn all these skills? Do you want to spend years digging into a problem and trying to get to the bottom of it? 

Paul Niehaus: Another thing that was really useful to me when I was deciding whether to do a PhD was a conversation where I was trying to decide whether to get into more of the “thinking" or the “doing” side of global development work. Somebody said to me, “It's a lot easier to get from the ‘thinking’ into the ‘doing’ than the other way around.” There's a lot of option value to that path. 

That really bore out in my life, because I ended up getting into a bunch of “doing” opportunities based on things I was seeing in the research. I realized, “Oh, the research says this is a good idea, and no one's doing it. So I guess I'm going to do that.” I think it’s still broadly true that there are more options by getting the PhD first.

Heidi Williams: To echo this idea that came up, people often look at the average path of somebody that takes this route and decide that’s not what they would want, and I think that is not the right way to think about this. Just because the average person who's doing a PhD in economics is really stressed out about this unidimensional measure of success and has one career track in mind that would equal happiness — actually, a big feature of getting a PhD is that you get to choose what path you want.

If you see economics as a toolkit that would let you make an impact on the social problems that you want to study, I completely agree with Paul that the world is your oyster. You can choose the problem that you work on, you can choose the institution through which you work on that problem, and you can bring a really rigorous set of tools that might not otherwise be applied to that. I agree that it’s a very flexible platform. 

Matt Clancy: Although I was saying with Iowa State University that I didn't do a predoc and take all this time, Heidi, you had a really good experience with your predoc. I’m not saying that you should avoid them.

Heidi Williams: Oftentimes the structures that a profession has sometimes get formalized as requirements, and then people do them because they're requirements. It would be better to think, “What would be something that I can do as an investment that would give me more information about whether this is a career path I want, and also give me more certainty about what area I want to go work in, if I do get a PhD?”

Straight out of undergrad, I was really lucky. I got a job with Michael Kremer, who's an amazing economist. I was working on a problem that was motivated by the very policy-relevant question, “How do we develop vaccines that are needed in low-income countries where it's not profitable for private firms to want to come develop them? But how do we bring the tools of economic theory to have contract theory papers written on the right contract that could actually incentivize private firms to do research on these problems that are socially important?” 

There is this term that gets thrown around sometimes in the sciences called Pasteur’s quadrant. It’s use-inspired research. We know what the problem is that we need to solve, but you actually need to do the basic theory research in order to come up with the right solution. 

My predoc was an incredibly rewarding experience. It made me think, “Oh, I absolutely want to go get a PhD.” It really honed my view of the area of research I wanted to work in.

But somehow the lesson comes out this way: “Oh, someone had a job like that, and then they got into graduate school. I need a job like that to go to grad school,” and then it becomes this box to check. When you're looking for these experiences, one important thing to think about is, “What am I getting out of this for my own development as a person, rather than thinking of it as a credentialing mechanism?”

It’s also really important to think about the impact that you can have by advising and teaching students. When you are an academic, you do your own research on problems that you think are important. But through your advising and teaching, you can also guide students towards working on those questions and support their work on those questions.

I don't know if either of you would like to share an example of that. For me, one of the main reasons why I have found it rewarding to stay in academia is providing this important source of value.

Paul Niehaus: The challenge of being individually productive is always interesting. You still find new problems to work on. But the challenge of creating a community around you — to have people who are collectively productive and creative and find good problems to work on — is so much more motivating.

Leadership in the academic sector looks different than leadership in the private sector. There you might get promoted through the ranks and at some point be doing strategy and bigger picture stuff. There isn't an obvious analog to that within the academy, but the kinds of mentorship and soft leadership that you can have by creating paths for younger researchers are exciting and rewarding as well.

Heidi Williams: Matt, Paul, and Tyler, who's here with us, each provide templates of mentorship. You can carve out ways to support people in academic research that I think are really great.

Paul Niehaus: Maybe this is segueing into things to think about if you do decide to do a PhD. But one thing that I do find very different in my academic versus non-academic experiences is that the non-academic experiences are intrinsically team efforts. You join a team, you're doing something together, everybody's all in. For many people, if it's a good team, and if the purpose you're working towards is something you care about, then that can be an incredibly fulfilling experience.

In the academy, that doesn't happen on its own. You have to be very intentional about finding the right people and putting those teams together and deciding what level of commitment you're ready to make to each other. 

For people who came and ultimately left, the key factor for them was just not having found that team, and experiencing a very solitary exercise. They were sitting alone in their room with a whiteboard or with their laptop, and that was not what they were looking for professionally. 

So if you choose this route, have this awareness that you're going to have to be much more intentional to have that experience of doing something important together.

Heidi Williams: If you do get a PhD, not because you want to be famous and publish papers in prestigious journals, but because you see this as a toolkit for making progress on problems that you care about — one thing that students may struggle with is that that's not the average reason why your peers are there. It may not be easy to find an advisor who empathizes with that being the reason that you're there. 

Paul, you're one of the people that I think of as most thoughtful on this. How do you structure support for retaining your center of focus on what's most important to you, as opposed to what's most important to the institution and people around you?

Paul Niehaus: Within economics, I do think there has been a big shift in recent years. There was a time when a lot of people would feel very uncomfortable talking to advisors about any sort of “non-traditional” career path, say about a non-academic job that they might be interested in. There’s fairly broad acceptance that that is not good, and that departments should create a culture where you can talk about anything that you want to do and be supported. In many places, I think that is also increasingly the reality. We're not all the way there, but I feel optimistic about that.

It’s important to be intentional about creating and finding a community of people who are like-minded and supportive. Sometimes I feel like there’s this invisible divide between people who are there mainly because they are curious and they like to satisfy their curiosity, and people who are there because they believe that if they're thoughtful, they might be able to have a big impact on the world through what they do.

They're all wonderful people, and I don't dislike curious people, but the second group is my tribe. Finding those people and spending time with them is super fun and life-giving, and it also helps me when I have to make decisions about what I am going to prioritize. I know that within that community, certain things are respected and valued, even if they don't necessarily maximize the number of lines on your CV. 

With my co-authors, we are very explicit and open with each other that what we're hoping to do is to improve anti-poverty policy in India, and that we're all comfortable with the fact that that may mean we don't publish as many papers, and that’s okay.

Heidi Williams: I want to talk about the fact that for many people, the institution where they spend time can have quite a substantive impact on what they value. Where you work can impact the way you think about what parts of your work are socially valuable, even in subtle ways. 

If you get a PhD in economics, you can end up teaching in a business school, a public policy school, an economics department, or a public health school. There are lots of different academic jobs that you could have. People often think of it as, “Well, I'm going to take the best job that I get, in the microcolony of environments that’s most attractive to me.” But in my experience, those institutions can offer very different incentives for what kinds of things you work on.

Many economists who study innovation teach at business schools, and they end up teaching courses for MBAs. Many of the problems that they end up getting exposed to are problems relevant to private sector firms that are doing innovation. 

There's also some alternative state of the world where all of the public policy schools recognize that innovation policy is a really important area, and everyone with my background is teaching masters of public policy students. And are then asking, “What do we need to train the next generation of policymakers that are going to really affect science and innovation policy?” 

For some reason, that split happened, and most people like me teach at a business school, and in my view, that probably had a really large impact on what kinds of questions people study. 

Matt, you can comment on some broader institutional differences across research in different environments. But even within academia, this is an issue that can really matter.

Matt Clancy: For much of my career, I don't think I appreciated how important your social environment is. When I applied to college, I got into University of Chicago and Iowa State University. I went to Iowa State University, because I thought, “Well, it's cheaper, and it's all physics.” I was going to major in physics. That's the end of my thinking about that. I didn't think about who my peers would be.

That probably would have made a difference, because in my subsequent experience, who my peers were did influence me quite a lot, such as working with USDA doing use-focused research. That deviated me from what I had thought of as the most valuable research when I was doing my PhD. When I came back to work at Iowa State University, somewhat by accident, I was given two office choices, and one was the Department of Economics and the other was the Agricultural Entrepreneurship Initiative Center, where I ended up taking an office because they were the ones who I thought it was good for me to be in the same building with.

I think the subsequent years were really different. I was surrounded by entrepreneurs and people who weren’t interested in talking about what my research was. But, they were trying to encourage students to start businesses and talking about these kinds of things. That affected what I viewed as a useful contribution that I could make as an academic, and I started New Things Under The Sun, a living literature review project, to try to make academic literature accessible to not only other academics, but also policymakers and these entrepreneurs are trying to start businesses.

The entrepreneurs often think that academic papers are too disconnected and irrelevant to their needs, because they're just lots of equations and 60 pages long. But I thought that there was a lot of value in the literature and started that project. I probably wouldn't have started it if I had been just across the street in the other department and been talking about my research projects all the time.

Then, I went to work for the Institute for Progress. Again, that was policy focused and policy relevant. Now, at Open Philanthropy, once again the most important research is viewed differently from academia. 

I don't know for sure how easy it is to select into the right environment until you've tried it, maybe there's nothing better you can do than just sample. Be aware that the values of your peers is not the only way things can be. Do you guys have any thoughts on that?

Paul Niehaus: Over time, one thing I look for is the people to hang out with. Insights come along, and sometimes an insight is a great policy idea, or sometimes it's actually a better business idea, sometimes it's a better research idea. So I just love being with people who are flexible about that and happy to consider any of those possibilities, as opposed to people who are always looking for just one of those things. That flexibility in your intellectual peers is worth looking for.

Heidi Williams: It's also very rewarding to look for institutions that support that broad approach too. If you come up with a problem and think, “This would be socially valuable to do,” some institutions may say, “Well, we can't really support that, or that's not the work that we do.” But sometimes you match with an institution that says, “We agree that's high social value. Just find a way to make that happen.” 

Or you meet people that have that mindset. Paul, you're a great example. You've been in academia, you've started a non-profit, you've started a for-profit. You think very flexibly about how to get a socially valuable idea out there. You don’t think, “You know, there's one specific tool that I have, and if this isn't there, you know, that's just an idea that's lost in history.” 

The three of us have talked about institutions that are a little more narrowly focused. For instance, maybe for-profit spin outs could happen, but if it's an idea that's not profitable, it's discarded and never brought up. Or an organization like Open Philanthropy is trying to do a very focused research on one question, but along the way, they come upon other questions that they would like to know the answer to, but those aren't the current priority that they’re working on.

It would be great to find better ways of connecting those use-inspired questions that arise along the way. An individual institution might not have the time or interest to pursue them, but can we more publicly raise those as questions that will be useful for people that are in academia or in more flexible settings to pursue? I'm curious if you have any examples of that, that you would flag as productive case studies.

Paul Niehaus: I think it's about mindset and connective tissue. Organizational specialization is a good thing, and it’s important and productive. But I see this issue when I talk to my colleagues about the policy impact of their work. 

One of my colleagues said to me, for example, “In Europe, there's this very well-established process where committees consume the latest research and it feeds into EU policymaking, but in the U.S., I just don't know what I'm doing.” Then, I call Heidi and say, “We need to get this guy connected to some people at Brookings, because we need connective tissue between the university and the sort of places in DC that are doing the hard work of translating this to make it legible to policymakers.” 

To me, that was an example of one person working seamlessly with one set of institutions, and in another case, none of that connective tissue had been built and was clearly needed. 

Matt Clancy: You said connective tissue. I've studied a lot of the economics of innovation. In the hard sciences, they have a direct connection to industry, because industry is building new technologies out of academic discoveries that are made, whether it's mRNA vaccines or rockets. In the social sciences, we haven't often had that. We haven't had organizations reading the latest social science research to figure out how to set up new products.

That's feedback that we have been missing, that is really healthy for the field, to just hear how your ideas play out in the real world. How theories work or don't work, replication, and validation of what we're doing. If it doesn't work, then that becomes generative of figuring out why. From a purely self-interested academic perspective, it’s really useful to create more of this circulation among different groups.

How do we find this connective tissue? You can take a sabbatical at a government agency, or you can sit on some of these joint advisory committees or something. But, the other thing you can do is try to find people like yourself, Heidi, the people who are in academia and engaging with a world that's wider.

Paul Niehaus: This happens but in a too idiosyncratic way. I make an effort to do this. I was talking to Heidi's colleague, Al Roth, a Nobel Laureate for market design. He has a weekly tea with his students, and occasionally an entrepreneur will write to him saying “Oh, I have this market design question related to this market I'm trying to build in my startup.” Al will say, “Come to tea and hang out with us.” Then one of the students may end up picking up that problem to work on it. These things happen, but now it's driven by individual people who make an effort to bridge the gap. The hope in the longer term is that we're going to see it more institutionalized. 

Matt Clancy: There’s another virtue to online public writing. It’s accessible to people outside your audience. We've got a great system for communicating academic ideas to each other on the seminar circuits, conferences, these journals, within academia. But what if you want to reach people outside that bubble? 

I hear from people all the time who read New Things Under The Sun who are practitioners or policymakers. We just meet for virtual coffee or Zoom to talk about some problem they're facing. They want to know if the academic literature has anything of value to say. I imagine that if more people had the time and space to communicate about their field in a way that is discoverable by people not in the field, then that'd be another way to build these connections.

Heidi Williams: As we wrap up, could you give a sales pitch to people who are motivated to use research to make progress on an important social problem? What's the case for them to go down that road, as opposed to doing something that is more direct service and less of this longer term path?

Matt Clancy: I'll take a shot. The best case is that you can think of knowledge creation and research as a lever that can have very long run impacts. If we can discover a marginally better way to do something, such as how we fund science or how we run peer review, then this evidence-based knowledge can spill out over the whole world.

One of the main values of knowledge is that it can be applied by everyone and it’s not trapped in any specific context. It’s non-rival. You can be exceptional at your role in direct service, but it's hard to extend your reach far. Research can go really far if it's done well and if it targets problems that matter. That's my pitch.

Paul Niehaus: I love it. I don't want to speak for science broadly — what do I know about many of the sciences? But Matt's point is that if there are things that are well-remunerated by the world, with the way the world is currently structured, there are going to be plenty of people to work on those problems. The problems that are going to be neglected, and therefore the ones where you're going to be able to have an outsized impact, are the ones that are creating these public goods. Knowledge that you can't capture all of the return and profit from it yourself — that's where the huge returns are going to be.

I think of this as a useful heuristic for myself. I try to find things that are not going to benefit me privately, precisely because that means they're likely undervalued and high impact. For economics in particular, I went into economics because to address the pressing issues of our time, I need to be able to think about human behavior quantitatively. I think that was a great heuristic and still is.

Matt Clancy: Compared to other social sciences, economics has a disproportionate policy impact. There are statistics about how often economists testify before Congress, and it's about twice the other social sciences all added together. So if you are going to pick a field where your goal is to have policy impact, economics is empirically really strong. 

What about you, Heidi? What are your opinions about why people should do an economics PhD, if they want to have a positive impact?

Heidi Williams: It’s related to things that both of you touched on. 

When you think about, “What do I want the scale of impact for my work to be?” I think it's really hard to think of that only in a direct sense. I’m somebody who really values my teaching and my direct advising. But at the end of the day, I want some substantial part of my work to be feeding into making systemic change. I want to be doing that in a way that's not just based on my own ideology and theory about what we should do as a society, but rather based on research that informs and gives me confidence that we can do something better than what we're doing right now.

Research plays such a unique role in honest advocacy for progress in a very directed way, and it’s a very rewarding life path. Academia is a place where you can have direct service of teaching, advising, and having individual relationships with students, and at the same time, you’re able to scale your impact through research that informs broader, more systematic change. I find this really rewarding.

Paul Niehaus: Just as a data point — although we're pretty clear-eyed about the constraints and limitations you sometimes face in academia, I would also say that we're having a blast.

Heidi Williams: Yes. On the right day, you will get me to tell you about how horrible academia is. But at the end of the day, I feel very happy with my job.

Matt Clancy: I would say you can get a PhD and do research and actually not be in academia. It is also possible. 

Heidi Williams: You can be very happy.

Matt Clancy: Yes. That's right.

Heidi Williams: So good. I think that's a good note to wrap up on.

Matt Clancy: Thank you.

Caleb Watney: The Metascience 101 podcast series has come to a close, but our colleague Tim Hwang will continue releasing fascinating interviews about metascience on this podcast feed. So stay tuned! 

You can find more information about the Macroscience newsletter at macroscience.org. You can learn more about the Institute for Progress and our metascience work at ifp.org, and if you have any questions about this series you can find our contact info there.

A special thanks to our colleagues Matt Esche, Santi Ruiz, and Tim Hwang for their help in producing this series. Thanks to all of our amazing experts who joined us for the workshop. Thanks to Stripe for hosting. Thanks to Prom Creative for editing. Thanks to you, the listener, for joining us for this Metascience 101 series.