In April, I wrote my first roundup of articles I’d been reading and it seemed like readers enjoyed it. So I’m doing it again, but with a twist! We have a smart, diverse, and growing metascience team and I’d love for you to see what interests them. In addition to my review of two pieces focused on the social sciences (how they are funded and how they benefit society), we have:
Metascience team lead Jenn Gustetic (formerly of NASA) and Senior Metascience Fellow Dan Turner-Evans (most recently of the US Senate) sharing pieces on the impact of AI on the talent pipeline (possibly bad) and on scientific research (probably good),
Metascience Fellow Matt Esche, who has the longest tenure of anyone on the metascience team at just over three years, discussing the potential for new cancer vaccines, and
Brand new Associate Metascience Fellow Hunter Wieman (coming from graduate school at Princeton University) reviewing an article on the challenges of conducting randomized controlled trials on AI.
Social Science at the NSF (Matt Clancy, Abundance and Growth Blog).
In early 2026, when the National Science Foundation (NSF) published a budget request to Congress that would eliminate funding for the Social, Behavioral, and Economic Sciences (SBE) Directorate, I was surprised. In addition to funding important social sciences research, SBE manages important national surveys such as the General Social Survey (GSS).
While the motivation for eliminating SBE isn’t totally clear (the budget request said that NSF would continue grants that align with Administration priorities — perhaps the implication is that other grants do not), Clancy’s piece is helpful in showing that this isn’t the first time a president has targeted SBE. The Reagan Administration cut social science funding substantially, and Congress approved a rescission package of a third of NSF’s social science budget.1 However, skepticism about whether NSF should fund the social sciences goes back to its beginnings. As Clancy notes, some drafts of the founding documents for NSF included a ban on funding social sciences, though that language did not ultimately end up in the legislation.
Something I wish Clancy’s piece had delved further into is: why did the Reagan Administration and Congress cut social science funding in the 1980s? Based on what I’ve been able to find, even at the time it wasn’t totally clear.
A Science piece from April 1981 sounds like it could have been written in 2026:
Rumors have been rife, and many people believe that the Office of Management and Budget went over the entire federal budget with magnifying glasses looking for social research to stomp on. The proposed cuts for social and behavioral sciences at the National Science Foundation have been detailed in unprecedented fashion, and have done more than anything else to arouse paranoia among social researchers.
OMB’s 1981 explanation for proposed cuts was that “support for the behavioral social and economic sciences is being reduced significantly because much of the support of these sciences is considered of relatively lesser importance to the economy than support of the natural science.” As with now, it is perhaps a mix of a perceived liberal slant in social science and a belief that, if cuts to science are needed, social science is less important to the economy.
What is the status of SBE now? Contra reporting released in April 2026, SBE has not been shut down preemptively, and its grantmaking picked up in June. However, as of August 16, SBE has only spent about half of what it would have in a typical year.2
The Mystery of Short Indian Children (Wilson King, In Development)
Here’s a mystery: why, despite rapid economic growth in India, have children remained short? This has not been the case in many other countries as they’ve developed economically (see the chart on economic growth and height in Africa). This is important because height, while not itself essential, correlates with health generally as well as cognitive development. As Wilson’s article states, the low height of Indians in India is not genetic; if their families move to wealthy countries, they will grow substantially taller.
Per Wilson, there are multiple potential reasons for this:
First, in India, there is a preference for sons, which might mean that daughters are malnourished. Second, there might be a preference for first children over subsequent children. Finally, India has lagged behind other countries in sanitation and hygiene despite its growth. That could cause children to remain sick, stunting their growth.
It’s likely a combination of these, and — while tackling something like a son preference is particularly difficult — improving sanitation is within the government’s power. Knowing this allows India to make policy decisions and financial investments that could improve the lives of millions of people and, through making them more productive, further enhance economic growth.
Why am I talking about height disparities in a science policy blog? It illustrates how social science, including all the papers discussed in Wilson’s article and linked above, can answer societally important questions.
I recently attended a workshop on the future of NSF’s SBE program, hosted by Arizona State University’s Consortium for Science, Policy & Outcomes. One of the discussions focused on whether there should be a dedicated unit at NSF that funds social science, or whether it should be spread out across the agency.3
I’m skeptical of breaking up a social science unit and using it to improve multidisciplinary projects because the primary benefit of social science research is not through interdisciplinary work with the physical and natural sciences. It’s through answering society’s most pressing questions. This piece on children’s height is a great example of this. Many of the questions that currently vex Americans can be answered, or at least better understood, through social science research:
What is the impact of AI on the labor market?
How did the COVID-19 pandemic affect children’s development?
Why are people getting married at lower rates, and why are they having fewer children?
Why do young people seem to be struggling with mental health issues? Is it all the screens?
What are the impacts of immigration on economic growth and labor markets?
Not to mention questions about how to make science more efficient and increase the likelihood of breakthroughs (important metascience papers like this and this came out of SBE funding). You’re not going to answer these questions by sprinkling social scientists around geology and quantum computing programs.
Now whether social science actually does tackle problems important to most Americans is worth considering. Parts of the social sciences have gotten bogged down in poststructuralist approaches rather than focusing on empirical, socially important questions. But SBE doesn’t tend to fund those things anyway.4
Let’s keep funding social science at NSF.
The Tragedy of the Cognitive Commons: How AI Could Disrupt the Regeneration of Professional Expertise (Nolan Lovett, preprint for Human Resource Development Review)
From Jenn Gustetic: AI is eroding the need for humans to conduct entry-level research tasks, and this paper suggests that might soon cause problems. I found the paper interesting because of its ties to the R&D management and implementation camp of metascience. It includes testable hypotheses about how AI is reshaping professional expertise. This may be particularly relevant to the R&D enterprise because declining STEM PhD admissions could compound the effects of AI.
Lovett proposes that by increasingly using AI for tasks that early-career employees used to do, our human capital across professions is experiencing a “tragedy of the cognitive commons.” While organizations make near term efficiency gains replacing entry-level research tasks with AI, the expertise depletes over time for the profession overall. The paper suggests two mechanisms that could weaken the future pipeline of employees with domain expertise (who can verify and refine AI outputs). The first mechanism is position elimination: fewer entry level hires. The second is reduced internalization of domain knowledge, through less cognitive struggle by entry-level employees in creating research products.
The paper posits that human capital systems need both “internalized mastery” and “distributed mastery” in the era of AI. Internalized mastery is the deep domain knowledge that is developed through sustained cognitive struggle and increasingly complex professional experience; it provides the foundation to do substantive validation. Distributed mastery is the ability to “orchestrate” AI systems to produce high quality outputs within a specific domain. Without internalized mastery you only get surface level validation, which decreases the quality of outputs over time.
Training and credentialing should create opportunities for both “tracks” within professions. If the stock of internalized mastery falls, substantive validation degrades, and validation error rates rise, then the quality of those professions decline. The paper proposes some policy ideas but suggests empirically testing the approaches. For example, governments might provide training subsidies in AI-exposed occupations, tax credits for firms that maintain developmental pipelines, or competitive grants to professional associations to experiment with governance approaches. We could also consider expanding federal training grants to the research community.
Perhaps these could be good metascience experiments.
AI for science needs reasoning, not just data (Eric Schmidt and Suhas Mahesh, MIT Technology Review) and Conjecture Machines: AI agents and the new validation bottleneck in science (Don Wallace, Conor Griffin, Sean O’Neill, Thang Luong, Owen Larter; Google DeepMind website)
From Dan Turner-Evans: AI agents are becoming increasingly good at doing science, and current and former Googlers are taking notice. Eric Schmidt, the former CEO of Google, and Suhas Mahesh at Schmidt Sciences argue that agents, rather than custom models like AlphaFold, will ultimately be the form of AI that leads to the greatest acceleration of science. While most existing scientific datasets have already been used to train new AI models, agents can be continuously applied to any field of science. They can automatically log every condition and decision — thereby increasing the likelihood of reproducing any finding while also increasing institutional memory — as well as rapidly testing out new ideas in simulation or against any paper ever published.
Meanwhile, the policy team at Google DeepMind explains why agents have gotten so good at doing science, how they are being incorporated into the research process, and the policy implications of these changes. They argue that better models, stronger scaffolding around these models, and improved customizability have made agents immediately useful for scientists for mundane tasks, coding, and idea generation and validation. But they caution that (1) graduate training needs to be preserved in an age of AI, (2) not everyone has access to agents, (3) not all data is ready for agents, (4) AI-generated ideas must be validated, and (5) peer review is under increasing strain.
AI agents are already proving to be incredible tools for math and computer science, but I am not yet convinced that they can drive breakthroughs in lab-based disciplines that are limited by data and experiments rather than logic. However, I do find the idea of agents helping with reproducibility compelling, and I know many scientists across disciplines who are experiencing noticeable productivity gains from AI. Still, the DeepMind team is right that more work is needed to validate their outputs. I’m still trying to figure out what graduate training should look like in this new era, but I plan to share more thoughts on this soon (and welcome your ideas!).
How to build a cancer vaccine, and whether they will work this time (Abishaike Mahajan, Owl Posting)
From Matt Esche: It’s been an outstanding summer for breakthrough cancer treatments. Pancreatic cancer, with its RAS mutation long thought to be ‘undruggable,’ saw the new drug daraxonrasib approved by the FDA. And with the positive clinical results from Merck and Moderna’s cancer vaccine for melanoma this August, abundant personalized vaccines for cancer treatments look a bit closer on the horizon. These results make me excited to think of all the therapies being developed now for previously untreatable diseases.
Mahajan’s blog post, which was written before the Merck/Moderna vaccine trial results, tempers my expectations a bit. Walking through decades of cancer vaccine approaches, the post shows how long it’s taken scientists to tease out the right approach, with a surplus of failures along the way. Even when an approach does work, the same design doesn’t simply translate to the next cancer, and we still don’t know why certain approaches in cancer immunology have failed.
To get there, we’ll need much more evidence and clinical results. There will be many more Phase 1 trials before we see widely-available personalized cancer vaccines. That’s why I’m glad to see movement on improving clinical trial policy, such as new pilots to improve the speed and quality of Phase 1 trials. Here’s to continued biomedical breakthroughs alongside clinical trial abundance bringing new therapies to market soon.
AI RCTs are booming - to be useful they must evolve (Tim Ohlenburg, Oliver Hanney, Sharif Kazemi, Joseph Levine, and Shahrukh Wani; VoxDev)
From Hunter Wieman: As AI is poised to transform the economy, economists and funders are rushing to study its effects. Last month, Anthropic’s Economic Futures Research Fund announced a $200 million commitment to “support ambitious external research on interventions to prepare society for the economic impacts of AI.” Much of this will go to randomized controlled trials (RCTs) that study the effects of AI initiatives and AI adoption. The authors of this piece warn that the default outcome could be a glut of economic studies that fail to provide meaningful insight.
Designing economic RCTs that generalize is already difficult, and the authors believe that AI will exacerbate this issue. Two mechanisms stood out to me in particular:
Testing Obsolete Models: Already, many studies of AI’s societal impacts use models that are practically obsolete by the time the study is completed; this could increase as AI progress accelerates.
Contaminated Control Groups: Due to rapid AI adoption, control group participants may start using AI tools as the study progresses.5 And even if the control group avoids using AI at all, the post-RCT environment will be one where a large fraction of the population is adopting AI.
As a result, researchers conducting RCTs on the economic impacts of AI should ensure they design and test programs whose effects are not specific to a particular AI model, and whose results will remain relevant as AI improves and diffuses through the economy.
This is very hard, and I don’t have any silver bullets. But if I had to design an RCT on the social or economic effects of AI, I would focus on two questions:
Can this RCT help us identify mechanisms that are likely to persist as AI progresses? In a rapidly changing world, theory-agnostic estimates of treatment effects become less durable. Researchers should instead design RCTs around interrogating theories about the mechanisms linking interventions to outcomes (e.g., what broad characteristics of AI models allow them to augment human workers instead of replacing them). Understanding how and why something works can generalize to new environments in ways that isolated estimates of treatment effects do not. With AI, this understanding might help us anticipate which newer models will produce similar outcomes to current ones and which will have qualitatively different effects.
For each plausible trajectory of AI progress and diffusion, what questions do we want answers to? A study need not apply to every possible future to be a good bet. In a world with more uncertainty around the future, we should be focusing on high value bets that give important information in some specific futures instead of ones that will robustly give some information in all futures. For example, we could design some studies under the assumption that AI will replace significant swaths of human labor and others under the assumption that it will instead augment and empower workers. The ambition should be less to future-proof every RCT than to build portfolios of studies that collectively ask high leverage questions across different paths of AI capabilities and adoption.
Economists interested in metascience (those in the “innovation economics” camp) often talk about applying experimentation and econometric methods to science. This piece underscores just how difficult and important it is to design programs and experiments well so that these methods actually answer important questions.
Some advocates were concerned that the Trump Administration would send Congress a rescission package for science funding in fiscal year 2025 — as they had for foreign aid funding — but the Executive Branch ended up spending all appropriated funds.
Though it’s just a bit behind its spending in FY2025, in which NSF spent money very slowly.
The discussion is summarized in this helpful workshop report.
Notices of funding for anthropology and sociology, for example, emphasize that findings must be falsifiable via empirical discovery.
Or may refuse to participate in studies if they can’t use AI.
