TUHH Open Research
 

Promises and risks in using LLMs in scientific review

Citation
24th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, IEEE PerCom Workshops 2026
Contribution to Conference
24th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, IEEE PerCom Workshops 2026  
Publisher DOI
10.1109/PerComWorkshops68308.2026.11585353
Scopus ID
2-s2.0-105044391125
Publisher
IEEE
ISBN of container
979-8-3315-7615-8
979-8-3315-7616-5
The current scientific peer review process faces increasing strain and is becoming difficult to sustain. While large language models (LLMs) can help reviewers by summarizing papers, assessing clarity, and generating timely feedback, the research community remains cautious about relying on LLM-based reviews. In this paper, we explore how LLMs could support and potentially improve parts of the peer review process. We examine both the benefits and limitations by analysing LLM-generated reviews of our own previously published papers, comparing them with human reviewer feedback, identifying points of agreement and disagreement, and considering requirements for the responsible and ethical use of LLMs in peer review.
Subjects
AI
ChatGPT
Ethics
LLM
Peer-review
DDC Class
006.31: Machine Learning
006.3: Artificial Intelligence
170: Ethics (Moral Philosophy)