Innovation
Technology
December 03, 2024
How artificial intelligence surpasses human experts in neuroscience
LLMs can act as “intelligent partners” for scientists, processing large volumes of data and integrating information

A recent study published in Nature Human Behaviour demonstrated that advanced language models (Large Language Models, LLMs) can outperform human experts in scientific predictions in certain situations. The research, conducted by Luo et al. (2024), highlighted the potential of artificial intelligence (AI) to not only match human performance but also to offer significant advantages in analyses and predictions in the field of neuroscience.
Researchers created an innovative benchmark called BrainBench, designed to assess whether advanced language models can predict experimental outcomes in neuroscience more effectively than human experts. BrainBench is based on an intriguing task: participants (both humans and AI models) needed to analyze two similar scientific abstracts, one with the actual study results and another with altered results that maintained logical and scientific coherence. The challenge was to identify which abstract contained the correct results.
For this, language models such as BrainGPT, which was specifically fine-tuned with scientific publications in the field of neuroscience, as well as other more general models, were used. This approach not only tested the LLMs’ ability to process complex and noisy information, but also explored their skill in integrating data and identifying patterns in the methods and results described in the studies.
Result
The numbers are truly impressive and demonstrate the transformative potential of intelligence artificial in science. Language models achieved an average accuracy of 81% in identifying summaries with real results, significantly outperforming human experts, who achieved an average of 63%. Among the evaluated models, BrainGPT, which was specifically fine-tuned with neuroscience scientific publication data, stood out with even higher performance.
A particularly interesting point was the confidence calibration presented by the models. When the LLMs indicated high certainty in their answers, the probability of being correct was also higher, an indication that these tools can provide reliable predictions and well-founded ones. This characteristic is especially relevant in a scenario of collaboration between humans and machines, in which understanding the confidence in predictions is essential for scientific decisions.
Smart partners
Modern science faces a fundamental challenge: the volume of available information grows at an exponential rate, making it impossible for any human to keep up with all relevant publications in their field. Important studies and discoveries can go unnoticed amidst the vast amount of data generated daily. In this context, tools like advanced language models can act as “intelligent partners” for scientists, processing large volumes of data and integrating information in a way that humans alone cannot.
Furthermore, the study revealed that, while human experts often focus on isolated aspects of a problem, LLMs can integrate information holistically, identifying subtle patterns and correlations between methods and results. This accelerates the process of scientific analysis and can open doors to new hypotheses and discoveries by highlighting trends or relationships that might otherwise go unnoticed.
Risks
Although the advances are impressive, the use of language models in the scientific context also raises important concerns. One of them is the risk of excessive dependence on AI. If scientists begin to avoid exploring hypotheses that contradict the predictions made by the models, the field may lose opportunities to make groundbreaking discoveries, especially when these predictions reflect prevalent patterns in the literature, but not necessarily truths.
Another critical point is the possibility of bias in decisions. Language models trained on large volumes of data can inherit biases present in scientific literature, such as publication trends or knowledge gaps, which can compromise the quality of predictions. Furthermore, blindly trusting the AI’s suggestions, without critical analysis, can lead to significant errors, especially in areas where human nuances or interpretations are crucial.
Finally, there is an ethical and scientific debate about the balance between using AI as an auxiliary tool and the risk of it replacing analytical human skills, essential for interpreting and contextualizing discoveries. To mitigate these risks, it is crucial that predictions generated by models are accompanied by clear indicators of confidence and limitations, promoting a balanced collaboration between humans and machines.
The future
Artificial intelligence is shaping the future of science, with LLMs leading this revolution. These tools have the potential to transform scientific practice, making it more agile, efficient, and data-driven. As LLMs continue to evolve, they can be constantly updated with new scientific data, staying aligned with the latest discoveries and advances in the most diverse areas of knowledge.
In the case of models like BrainGPT, it is possible to imagine systems that not only assist in the analysis of existing studies, but also help scientists to plan more strategic experiments, providing opinions on probable methods and results or indicating gaps in the literature that deserve investigation. This type of application is already beginning to be seen in areas such as biomedicine and neuroscience, where the integration of AI accelerates the identification of new treatments and interventions.
Furthermore, tools of this type can be integrated into research pipelines , suggesting initial interpretations for results or assisting in prioritizing projects with a higher probability of success. This could reduce the time and resources needed to conduct research, accelerating the rate of innovation and bringing direct benefits to society.
However, the future will also depend on collaboration between humans and machines, leveraging the best of the computational and analytical capabilities of LLMs and the creativity and critical thinking of scientists. AI, thus, would be more of a partner than a substitute, enhancing human capacity for discovery and expanding scientific knowledge. With this partnership, the horizon of science seems broader than ever, ready to be explored.
| To access the references of this text click here |
Who wrote this column
José Erasmo Silva








