It’s not every day that philosophy of science ends up as the main character of a social media beef. But, then again, you don’t always have a high-profile tiff between Elon Musk and irascible neural networks “founding father” Yann LeCun.
In the midst of a lot of insults and trolling, Yann dropped this striking bit in a debate over whether or not what Musk does is “science”:
Drama aside, it’s worth taking a moment to mull over this quite doctrinaire interpretation of the scientific enterprise. LeCun presents a formalist account of what makes something science: Science must be (a) correct, in the sense of receiving peer review and approval, and (b) reproducible, in the sense of being “widely available” and “sufficiently interesting." Both attributes require publication describing methods in sufficient detail. If your work doesn’t meet these criteria, “it’s not Science” — end of story.
This claim is at least a little odd. After all, machine learning — the very field LeCun has presided over for decades — has violated these tenets in a big way. Consider:
Much of the activity in the field of machine learning has taken place on arXiv, where review of any kind is wildly light-touch and rarely delves into any form of “checking for correctness."
While open publication on platforms like arXiv has been a defining feature of the field, it has been by no means mandatory. Many “secret sauce” methods of model pretraining and fine-tuning remain closely held trade secrets, and it is widely assumed that leading research groups within OpenAI and Anthropic routinely hold things back.
Finally, there has been little or no centralized control over the degree of “sufficient detail” needed to qualify as reproducible and valid within the field. This has led to well-covered claims that machine learning is “alchemy” and faces its own reproducibility crisis.
And yet. Machine learning has succeeded as a field of scientific endeavor, in spite of all of these deviations from LeCun’s tidy picture of what is and is not Science. Arguably, it is precisely because of wide-open, unrestricted publishing norms that machine learning has quickly advanced. There are low barriers to participation, and critical knowledge has been widely accessible. That’s led to a lot of productive experimentation and iteration.
Has there been a lot of garbage published, and have a lot of dubious results been promoted? Yes. But it is hard to argue that these failures have cumulatively halted the progress of machine learning, or called the entire endeavor into question. The jury is still out on whether or not the proponents of this way of running the field of machine learning will “die bitter and forgotten," as LeCun suggests. But I’d submit that we know enough by now that these violations of the tenets of LeCun’s Science have not proven to be a fatal flaw.
So what good is LeCun’s Science, anyways? For what it’s worth, I don’t think LeCun’s vision is fundamentally flawed. Insistence on careful peer review, quality control, and reproducibility is tremendously valuable in the right context. This is particularly the case where a field must absolutely have one result built on the other in order to progress.
But sometimes context demands that we do not insist on these features of science! There are other goods we may want to optimize for within a field. In fields where there is more present value in exploration than in exploitation, the mechanisms LeCun touts can be a drag on scientific progress. Machine learning is arguably in this phase of its development: It gains more from unrestricted exploration of its vast opportunities, even though the lower bar on quality may occasionally result in embarrassing mistakes or outright frauds. The day may come where LeCun’s Science becomes the more valuable model for the field of machine learning, but the case is far from clear cut today.
In the end, debates over the Platonic ideal of Science are a red herring. One might accept the progress machine learning has made in recent years and still assert that all this activity is not Science, but mere “commercialization” or “product engineering.” Fine. But to what end? The value of any governing structure or practice of science should be weighed pragmatically against the benefits it provides and the costs it imposes on a field. The goal should be the actual production of knowledge, not clinging to a set of canons about scientific process.
The ideal research team structures, publication norms, and funding strategies will not be universal, and should bend to the needs of the thing being explored. This will vary from domain to domain. In situations where a set of common methods can be applied across many different kinds of problems to great effect (as in machine learning), looser frameworks may be the optimal design to maximize exploration and iteration. In situations where research results must be tightly interlinked and validated to make progress (as in pure math, or physical chemistry), stricter frameworks may ensure the necessary trust to build extensively on the work of others.
The debate should be not over what is or is not Science, but over what kinds of organizational arrangements will get us the best results out of a given field. We need an approach that sees our ideal Science as one capacious and flexible enough to meet the vast and varying scope of the scientific exploration itself.
> One might accept the progress machine learning has made in recent years and still assert that all this activity is not Science, but mere “commercialization” or “product engineering.” Fine. But to what end? The value of any governing structure or practice of science should be weighed pragmatically against the benefits it provides and the costs it imposes on a field. The goal should be the actual production of knowledge, not clinging to a set of canons about scientific process.
This reminds me of a section of Thomas J. Allen's _Managing the Flow of Technology_:
> The scientist's principal goal is a published paper. The technologist's goal is to produce some physical change in the world.
It strikes me that this aligns with the end of your quote above; the goal *of science* should be the actual production of knowledge. But for all of the progress made in AI over the past decade (or whatever), I'm not totally certain how much actual knowledge we've ended up with. Instead, I think we've produced changes in the world, and utility, and value.
This is all to say that we can criticize AI companies for not producing science while at the same time admiring them for the value they've created. Engineering is good! If I point at the Space Shuttle and say "that isn't science," that statement can (and to some extent should) be seen as a neutral semantic distinction, not a criticism of the space shuttle or any of the engineering that went into it.