Advanced pharmaceutical bulletin. 14(3):498.
doi: 10.34172/apb.2024.053
Letter to Editor
Navigating Scientific Peer Review with ChatGPT: Ally or Adversary?
ArunSundar MohanaSundaram 1, * 
Author information:
1School of Pharmacy, Sathyabama Institute of Science and Technology, Chennai 600119, Tamilnadu, India.
Copyright and License Information
©2024 The Author (s).
This is an Open Access article distributed under the terms of the Creative Commons Attribution (CC BY), which permits unrestricted use, distribution, and reproduction in any medium, as long as the original authors and source are cited. No permission is required from the authors or the publishers.
Dear Editor,
ChatGPT (Generative Pre-trained Transformer), a breakthrough innovation by OpenAI) is being touted as a revolutionary tool with immense potential in an array of medical and pharmaceutical research and scientific peer review (SPR) processes.1,2 Studies revealed that ChatGPT might act as a complementary tool to the human SPR and aid in expediting the process, reduce reviewer fatigue, and shorten publication timelines.3,4 Of note, ChatGPT displayed remarkable competence in providing shrewd feedback, detecting methodological defects, and measuring the article’s impact on the advancement of the respective field, all with a fair inter-rater agreement.5
On the other side, a recent article by Liang et al raised an alarm regarding the perils of using ChatGPT in the peer review process. The study found remarkable alteration using ChatGPT in nearly 17% of the peer-review reports.6 The researchers have analyzed about 146 000 peer reviews submitted to the AI conferences (pre- and post-launch of ChatGPT) and found a remarkable upsurge in the use of certain buzzword adjectives like versatile, meticulous, intricate, etc. (the telltale signs of ChatGPT-written text) in the review reports.
ChatGPT has limited utility in the SPR due to a lack of transparency in training data and decision-making process, issues with the reproducibility of review reports, inability to justify the recommendations, potential biases, and AI hallucinations (generation of fake/non-existing references for writing and reviews). Besides, lack of contextual expertise, missing human connect (iterative fine-tuning, personal interaction and collaboration between reviewers and authors, and nuanced, context-based considerations), lack of accountability and incapability in image interpretation (in free-version; ChatGPT 3.5) are additional challenges.5 Hence, performing fully automated ChatGPT-based SPR is far from practical implementation.3,4 Besides, there is a possibility of unprecedented repercussions in defining and shaping the scholarly communities when using ChatGPT for the SPR. However, regular training with appropriate, unbiased datasets, periodical audits and mitigation of model biases might improve the capabilities of ChatGPT.
Taken together, ChatGPT is a quintessential tool for advancing the SPR process. While it is good to leverage this technology in the AI-driven SPR process, continuous improvement, cautious implementation, and constrained, human-supervised processing is obligatory for disseminating high-quality and ethical scientific research.
Competing Interests
None.
Ethical Approval
Not applicable.
References
- Pradhan T, Gupta O, Chawla G. The future of ChatGPT in medicinal chemistry: harnessing AI for accelerated drug discovery. ChemistrySelect 2024; 9(13):e202304359. doi: 10.1002/slct.202304359 [Crossref] [ Google Scholar]
- Gomes WJ, Evora PR, Guizilini S. Artificial intelligence is irreversibly bound to academic publishing - ChatGPT is cleared for scientific writing and peer review. Braz J Cardiovasc Surg 2023; 38(4):e20230963. doi: 10.21470/1678-9741-2023-0963 [Crossref] [ Google Scholar]
- Saad A, Jenko N, Ariyaratne S, Birch N, Iyengar KP, Davies AM. Exploring the potential of ChatGPT in the peer review process: an observational study. Diabetes MetabSyndr 2024; 18(2):102946. doi: 10.1016/j.dsx.2024.102946 [Crossref] [ Google Scholar]
- Hosseini M, Horbach SP. Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review. Res Integr Peer Rev 2023; 8(1):4. doi: 10.1186/s41073-023-00133-5 [Crossref] [ Google Scholar]
- Biswas S, Dobaria D, Cohen HL. ChatGPT and the future of journal reviews: a feasibility study. Yale J Biol Med 2023; 96(3):415-20. doi: 10.59249/skdh9286 [Crossref] [ Google Scholar]
- Liang W, Izzo Z, Zhang Y, Lepp H, Cao H, Zhao X, et al. Monitoring AI-modified content at scale: a case study on the impact of ChatGPT on ai conference peer reviews. ArXiv [Preprint]. March 11, 2024. Available from: https://arxiv.org/abs/2403.07183.