Human wrote it, AI thought of it? New tool detects who came up with ideas in scientific papers

Artificial intelligence tools can now produce essays, research summaries and scientific writing that closely resemble human work. But as universities and researchers grapple with the growing use of AI, a more difficult question is emerging: even if a human writes the final paper, who came up with the original idea?

A new tool called IdeaLens aims to answer that question by detecting whether the ideas behind a document originated with a human or an AI system, rather than simply analysing the words used to write it.

The development, reported by Nature on October 14, could add a new dimension to the debate over AI-generated academic work, where conventional detection tools often focus on identifying machine-written text. IdeaLens instead attempts to identify the origin of a document's underlying concepts, potentially helping institutions assess how AI contributed to research and writing.

The research, titled IdeaLens: Detecting AI Ideas in Long-form Writing, was published as a preprint on arXiv on October 5 by Rishanth Rajendhran, Minjoon Choi and colleagues.

How IdeaLens works 

Most AI-detection tools examine features of the final text to estimate whether it was generated by a machine. IdeaLens takes a different approach: it attempts to separate the ideas in a document from the language used to express them.

The researchers convert documents into outlines, with each section represented by its role in the discussion and a brief description of its content. This process reduces the overlap between the original writing and the representation analysed by the detector.

The system is then trained to distinguish between human-originated and AI-originated ideas. The researchers used a dataset of one million documents from FineWeb, a large web-text collection, with labels supplied by another AI-text detection system.

The underlying premise is that a document's conceptual structure may reveal clues about its origin even when its sentences have been rewritten, edited or polished by a human.

Human writing does not necessarily mean human ideas 

One of the study's key findings concerns the difference between writing a document and developing its ideas.

In a controlled experiment, the researchers found that IdeaLens's AI-flag rate fell from 95% to 7% as AI models were given increasingly detailed plans created by humans. By comparison, another detector, Pangram 4, continued to flag 92% of the documents as AI-generated.

The pattern reversed when the plans themselves originated from AI. In that setting, IdeaLens continued to flag more than 96% of the documents.

The researchers also tested a separate dataset of 50 stories written by humans using AI-generated plans. IdeaLens classified 68% of these stories as AI-generated, compared with 8% for Pangram 4.

These results suggest that the tool may be able to detect AI involvement at the planning stage, even when people write the final text themselves.

However, an AI flag does not establish misconduct or prove that a person deliberately misrepresented the work. It indicates a model's assessment of the likely origin of ideas, not independent proof of authorship or intent.

Could this change academic integrity checks? 

The distinction could matter as universities, journals and research institutions develop rules for using generative AI.

A researcher might use AI to organise an argument, develop a research outline or suggest a conceptual framework before writing the paper independently. A conventional detector focused on the final wording may struggle to identify that contribution if the text is substantially rewritten.

A tool that examines the provenance of ideas could, in principle, help institutions investigate such cases more closely. It could also contribute to discussions about when AI assistance should be disclosed and how intellectual contributions should be credited.

But detection is only one part of the challenge. Institutions would still need clear policies on acceptable AI use, transparent disclosure requirements and fair procedures for reviewing suspected violations.

Bigger question: Who owns an idea in the AI era? 

IdeaLens remains a research-stage tool, and its results should not be treated as definitive proof that an idea belongs to a human or an AI system.

The study reports evaluations across 19 existing detection benchmarks and examines 90,000 predictions to identify differences between human and AI ideation. The researchers report strong detection performance at low false-positive rates, but performance on benchmark datasets does not automatically establish reliability in every real-world setting.

The system also relies on labels generated by another detector during training. This means the quality and limitations of those labels are relevant to how its results should be interpreted.

More broadly, human and AI contributions are not always neatly separable. A person can substantially reshape an AI-generated idea, while an AI system can help develop a concept that originated with a researcher. A binary classification may not fully capture that collaboration.

The preprint's findings therefore need further scrutiny, including independent evaluation and testing across different research disciplines and writing contexts, before the tool can be relied on for consequential academic decisions.