Stanford University recently published a paper titled ‘Can LLMs Generate Novel Research Ideas?’ The study found that ideas generated by LLMs (large language models) were rated significantly more novel than those from human experts.
To reach this conclusion, over 100 NLP researchers were asked to come up with new ideas and review both LLM-generated and human-generated ideas without getting to know their source. The results showed that LLM ideas were considered more innovative (with statistical significance, p < 0.05), although they were rated slightly lower in terms of feasibility.
The approach is similar to Japanese AI startup Sakana AI’s AI Scientist, which automates the entire research lifecycle. It generates novel research ideas, writes necessary code, executes experiments, summarises results, visualises data, and presents its findings in a complete scientific manuscript.
Interestingly, the startup claimed that each idea is implemented and developed into a full paper at approximately $15 per paper.
Generating new ideas is relatively easy for LLMs, thanks to their extensive training on …