Summary of Key Points
This article uses the metaphor of a "coacher's mindset" to illustrate that the traditional academic paper paradigm of the AI era—emphasizing execution and positive outcomes—is as outdated as an old horse-drawn carriage. We need to shift towards a new paradigm centered on the ability to ask meaningful questions, akin to filing reports. This new approach addresses issues such as the devaluation of execution brought about by AI, as well as problems associated with fraud and the "file drawer effect" (where many failed experiments are shelved). The allocation of research funds is the key lever that can drive this transformation in the academic system.
I. Why Does AI Make Traditional Papers Ineffective? — The Crisis of Detours in Knowledge Generation
We can divide knowledge into four levels (the DIKW pyramid):
- Data: Fragmented materials (e.g., a collection of exam scores);
- Information: Processed content with meaning (e.g., "The average score in this class is 80");
- Knowledge: Systems that can explain patterns (e.g., "Why does this class have a high average score? Because the teacher used an interactive teaching method");
- Wisdom: Judgments that guide action (e.g., "Other classes should also use the interactive teaching method").
AI can easily handle the conversion from data to information and even simulate the generation of knowledge. However, when students use AI to write papers, they often skip the process of thinking about why things are the way they are—just as a coach may not realize that there is no horse to pull the carriage. As a result, traditional papers, which were designed to assess knowledge acquisition and critical thinking skills, are undermined by AI.
II. The Dilemma of Traditional Papers: Misaligned Incentives Lead to Fraud
Traditional paper evaluation combines question formulation with execution verification, focusing primarily on the appearance of the data. This leads to three problems:
1. Fraud becomes inevitable: To get published, some people fabricate data or create stories to justify their findings (e.g., if data shows a correlation between A and B, they invent a story explaining how A causes B).
2. Failed experiments are ignored: Only positive results are published, while numerous failed experiments are shelved (the "file drawer effect").
3. A pointless arms race: Students use AI to write papers; schools use AI to check them; students then use AI to counter-check the checks. Everyone is caught in this old framework, with no one questioning what should be verified in the AI era.
This is like a coach who treats the horse as the essence of transportation rather than just a tool; traditional papers treat attractive data as the core of academia rather than a means to achieve a goal.
III. The New Paradigm: Filing Reports — Putting Questions First
The new paradigm, akin to filing reports, places the emphasis on asking questions. In the AI era, this is represented by the "car": the focus of evaluation shifts forward:
- First stage: Submit the research question and the proposed solution; peer review focuses solely on whether the question is meaningful and the logic of the solution is sound. If approved, the paper is generally accepted.
- Second stage: Execute the plan, and whether the result is positive or negative, it should be reported truthfully for publication.
The benefits are clear:
- Fraud becomes useless: The outcome no longer affects publication, so why bother with fraud?
- Failed experiments are valuable: They help prevent others from making mistakes, promoting faster scientific progress.
- Return to the essence of academia: The focus is on asking good questions—something AI cannot do.
IV. The Lever of Change: Reforming Fund Allocation
The academic system is like a chain of linked steps (publication → funding → training → ranking). To break the old paradigm, changing how research funds are allocated is crucial. The evaluation criteria for these funds should be revised in four areas:
1. Focus of applications: From "I have preliminary evidence that X causes Y" to "I have a question worth investigating, and here is my proposed solution."
2. Review standards: From "Will this hypothesis succeed?" to "Is this question important? Can the solution be thoroughly tested?"
3. Project requirements: From "Publish positive papers" to "Complete the project according to the plan and make the data public."
4. Funding models:
- Pull model: The community votes on the most worthy research topics; researchers propose solutions, and funding is provided if approved.
- Push model: For important unclaimed topics, funds are offered with incentives (increased funding or extended time) to attract researchers.
AI can also help identify unexplored areas, such as untested hypotheses in citation networks or gaps in systematic reviews.
V. Criticisms of the Old Paradigm: The New Paradigm Is Supported by Hard Evidence
Some may ask, "The new paradigm has a high reproducibility rate, but will it lack groundbreaking discoveries? Will young researchers feel motivated?":
- Regarding breakthroughs: 70% of so-called breakthroughs in the old paradigm are unrepeatable (e.g., studies from Open Science Collaboration show that only 30% of psychology papers are reproducible). The new paradigm aims to systematically reduce the unknown, making it more reliable.
- Motivation for young researchers: Honors are shaped by the paradigm. In the past, martial arts tournaments were prestigious; now, Nobel Prizes are. Under the new paradigm, asking important questions will be recognized as a significant achievement (e.g., the first person to ask a certain question may have their name highlighted in textbooks).
In five years, the new paradigm will be proven effective through tangible metrics: higher reproducibility, better use of funds, and more follow-up research driven by failed experiments. The old paradigm will naturally be phased out—like horse-drawn carriages that may still exist but are no longer the mainstream.
Conclusion: What Is the Core Value of Academia in the AI Era?
We should no longer ask how much AI can help us write papers, but rather what kind of papers AI should produce. The value of academia lies not in execution (which AI can handle) but in asking questions (something AI cannot do), not in finding answers but in defining problems, and not in producing positive outcomes but in systematically reducing the unknown. Filing reports represents the starting point of this new paradigm, and funding allocation is the key to driving this change. Recognizing this shifts us beyond the "coacher's mindset."