Discover how AI tools are transforming systematic reviews in healthcare research. In this seminar, you'll learn how AI can be, and is already being, used in each stage of the review process. Gain insights into how these cutting-edge technologies can save you time
Available for NSLHD staff through the library intranet
RAISE recommendations
The Responsible use of AI in evidence SynthEsis (RAISE) recommendations provides a framework for ensuring responsible use of AI and automation across all roles within the evidence synthesis ecosystem. This statement has been endorsed by all major evidence synthesis review platforms.
Thomas J, Flemyng E, Noel-Storr, A. et al. Responsible use of AI in evidence SynthEsis (RAISE): recommendations for practice (version 2.2; updated 7 November2025). In: Open Science Framework [https://osf.io/], Washington DC: Center for Open Science. DOI 10.17605/OSF.IO/FWAUD
Flemyng, E., Noel-Storr, A., Macura, B., Gartlehner, G., Thomas, J., Meerpohl, J. J., Jordan, Z., Minx, J., Eisele-Metzger, A., Hamel, C., Jemioło, P., Porritt, K., & Grainger, M. (2025). Position statement on artificial intelligence (AI) use in evidence synthesis across Cochrane, the Campbell Collaboration, JBI, and the Collaboration for Environmental Evidence 2025. JBI evidence synthesis, 23(11), 2162–2166. https://doi.org/10.11124/JBIES-25-00480
Use of automation
Clark, J., et al. (2020). A full systematic review was completed in 2 weeks using automation tools: A case study. Journal of Clinical Epidemiology, 121: 81-90. https://doi.org/10.1016/j.jclinepi.2020.01.008
Use of AI
Pearson, H. AI slashes time to produce gold-standard medical reviews-but sceptics urge caution. Nature. https://doi.org/10.1038/d41586-025-01942-y
Some people use GenerativeAI to help them think through and structure their question
Luo, X., Chen, F., Zhu, D., Wang, L., Wang, Z., Liu, H., Lyu, M., Wang, Y., Wang, Q., & Chen, Y. (2024). Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses. J Med Internet Res, 26, e56780. https://doi.org/10.2196/56780
Priority Screening using AI
Title/abstract screening is a time-consuming process. When done manually, it can take dozens of hours per person.
Some review manager software platforms offer algorithm-assisted screening. This is called Priority screening, wherein machine learning algorithms learn from user behaviour to rank papers by relevance. The platform will then present the user with the most relevant papers first.
Further Reading
Priority screening in Covidence
When selecting "Sort by: Most Relevant" during screening, this feature begins identifying patterns in screening behaviour. Once the first 25 studies are screened, the algorithm begins automatically refreshing after each study screened, allowing the system to determine and display studies that are most likely to be included first. This reduces the number of records needing to be screened by up to 80%.
Read more.
Rayyan Prediction Classifier
After 50 screening decisions with a minimum of 5 included and excluded studies each, Rayyan's classifier begins identifying patterns based on screening behaviour. It calculates a confidence score for each unscreened title, providing a rating on whether the article should be included or excluded. Articles to be screened can be sorted by Most Relevant, Least Relevant, and Swap.
Read more.
PICO Portal utilises an AI-assisted priority screening algorithm, learning from the user's screening behaviours to prioritise and present the most relevant citations first.
DistillerSR uses machine learning algorithms to offer AI-powered screening to continually re-orders papers based on relevance, pushing the most relevant papers for review to the top.
Generative AI in Critical Appraisal
There is some literature to suggest that large language model generative AI tools perform comparably to humans in conducting critical appraisal. The tools are observed to be considerably faster than humans at critical appraisal and were found be consistent in their appraisal of articles. However, these tools were also found to hallucinate data and make mistakes at points.
Furthermore, as AI tools change rapidly, it is impossible to conduct critical appraisal using an generative AI tool in a way that would meet the standards for replicability required of a systematic review.
Lastly, uploading full-text articles into a generative AI tool would constitute copyright infringement.
Given the above, NSLHD Libraries would instead recommend the use of a specialised systematic review tools which utilise AI features, such as PICO Portal and RobotReviewer.
Further reading:
Below are some articles on the use of Generative AI tools in critical appraisal
PICO Portal
Pico Portal is an AI-powered literature review platform. Its features include:
Built on Cochrane's Risk of Bias 2 Tool AI-assisted quality assessment detects and highlights the relevant section of a paper to streamline for the reviewer to find quickly. While this does not replace human judgement in critical appraisal, it automates and streamlines the process.
NSLHD does not provide access to PICO Portal.
RobotReviewer
RobotReviewer is a machine learning system aimed at supporting evidence synthesis.
Using Cochraine's Risk of Bias 1 Tool, the platform allows users to upload Randomized Controlled Trial articles, and its machine learning system automatically determines information concerning the trial conduct, including:
RobotReviewer is not a substitute for human critical appraisal, instead it assists researchers by semi-automating the critical appraisal process.
NSLHD does not provide access to RobotReviewer.