Artificial Intelligence (AI) and the literature review process: Scoping and protocol development

The Student AI Guidelines recognize that you can use an AI tool in the preliminary stages of your assignment process, for instance to brainstorm the topic. You will need to check the accuracy and reliability of the information provided. ChatGPT has been successfully used for scoping and protocol development (Diaz et al., 2023; Dowling and Lucey, 2023;  Gupta et al., 2023b;  Rahman et al., 2023). These chat agents have similar capabilities: 

CopilotChatGPTClaude.aiGemini

Example prompts

Prompt (following Gupta et al., 2023):

Give me 10 novel systematic review ideas about cosmetic plastic surgery.

ChatGPT (listing the first 3 of 10):

1. Comparative Effectiveness of Facial Rejuvenation Procedures: A Systematic Review

Compare the effectiveness of various facial rejuvenation procedures, such as facelifts, dermal fillers, and laser treatments, in terms of outcomes, longevity, and patient satisfaction.

2. Psychosocial Impact of Cosmetic Surgery: A Comprehensive Review

Explore the psychological and social impacts of cosmetic plastic surgery on patients, including changes in self-esteem, body image, and overall quality of life.

3. Long-Term Safety and Efficacy of Breast Augmentation Techniques: A Systematic Analysis

Investigate the long-term safety and efficacy of different breast augmentation techniques, including implant types, placement methods, and associated complications.

Evidence

Dowling and Lucey (2023) found that ChatGPT's output of 100 words for the research idea in the field of cryptocurrency was highly rated by 32 experienced reviewers in the subject of finance. Their third version of the research prompted ChatGPT to improve its suggested answers usually by asking it to be more specific in particular sections.. Their conclusion was that "ChatGPT can generate, even in its basic state, plausible-seeming research studies for well-ranked journals. With the addition of private data and researcher expertise iterations to improve output, the results are, frankly, very impressive" (Downing and Lucey, 2023: 5).

ChatGPT was successfully able to generate novel  systematic review ideas in four unique plastic surgery topics (Gupta et al., 2023a).  It was prompted to give 10 general systematic review ideas and 10 systematic review topics focusing on 2 specific areas within four separate topics. When evaluated, eight novel systematic review ideas had no previously published literature regarding that topic and 49 of the 80 systematic review ideas that were derived from ChatGPT were novel.

Diaz et al. (2023) found three key benefits in using ChatGPT for the scoping process for doctoral students in design. 

• Saves you time:  ChatGPT can provide quick and intuitive answers to scoping questions, reducing the time and effort required for traditional literature reviews.
• Saves you money: ChatGPT can serve as a cost-effective alternative to traditional methods for scoping, reducing the resources required for research. 
• Improves access to the literature: the chat-based interface and natural language interaction can make the scoping process more accessible for students, particularly those not experts in the field.

They did report three caveats: the risk of bias and errors which are noted in a separate page; the need for support from supervisors that using AI tools is allowed and worthwhile; and the need in design science to use an inquiry framework to guide and support the scoping process.

AI tools have been used in the academic literature to:

• Show you the connections between academic papers.
• Help you visualise the literature on your topic.
• Overcoming the limitations of having to know all the relevant keywords and their synonyms for the search.

These AI tools enable you to find papers that you would not necessarily find from a direct keyword search because they use the citations to produce a visualisation.

Connected Papers

Connected Papers (Smolyansky, 2020) is based on the Semantic Scholar database. Its premise is that two papers that have highly overlapping citations and references are presumed to have a higher chance of treating a related subject matter.

The graphs that are produced are designed to highlight the most important and relevant papers immediately.

Limits: You will only get 5 graphs per month for free.

Connected Papers

Using Connected Papers

Enter the title details in Connected Papers.

Choose the correct paper from a list: This will then allow you to produce a graph of similar papers.

Screen the titles, abstracts, and full texts of all the connected papers to identify other relevant studies.

Clicking on Prior works shows you papers that were most commonly cited by the papers in the graph. This usually means that they are important works in this field so it would be useful for you to be familiar with them. Selecting a prior work will highlight all graph papers referencing it. Selecting a graph paper will highlight all referenced prior work.

Clicking on Derivate works will show you the papers that cited many of the papers in the graph. This usually means that they are either surveys of the literature or recent relevant works which have cited as evidence many papers in the graph.

Evidence

Connected Papers has been used in the academic literature in literature reviews in several disciplines to identify other relevant studies. These include understanding the identity of lived experience researchers and providers (Gupta et al., 2023), positive parenting interventions and body weight (Kong et al., 2023), components and characteristics of effective interventions involving men and boys in family planning in low- and middle-income countries (Aventin et al., 2023) and diagnostic windows in non-neoplastic diseases (Whitfield et al., 2023).

Research Rabbit

Research Rabbit also uses Semantic Scholar to search for papers but also uses PubMed to find biomedical and life sciences papers. It is committed to remaining free to researchers.

ResearchRabbit

Using Research Rabbit

For Research Rabbit, the prompt is to provide details of a seed paper in which you are interested.

Research Rabbit uses PubMed records in the biological and life sciences and uses Semantic Scholar for all other subject areas. It provides you with a list from which you select the correct paper.

Directly add papers to your collection with the Add Papers button or use the Add By Search button to find papers through PubMed.

Once you have added your papers, Research Rabbit will start generating recommendations for you in the buttons at the bottom of the panel. As you add more papers, Research Rabbit better understands your interests and generates better recommendations.

Research Rabbit allows you to:

• Explore papers (either similar work or listing the references or the articles citing it)
• Explore the authors or suggested authors;
• Search for similar results in the included articles.

Clicking on All Citations under the Explore Papers section will open a new panel showing the connections between the original paper and others. The panels are like a breadcrumb trail showing each step you take along a research rabbit hole. Each hop opens a new panel to the right, so you can see the path you’ve taken.

Create new collections and drag papers to the appropriate collection. Papers can be in more than one collection. All collections a paper is in will be shown as yellow tags. There is no institutional access to papers. Research Rabbit is relying on papers to be freely available to provide you with any copies. The FAQ page for Research Rabbit shows you how to import and export references to reference management software.

Evidence

Research Rabbit has been used in literature reviews to find papers similar to the original work you supply. These include community mental healthcare for people with severe mental illness (van Genk et al., 2023), the circular economy, urban design and urban planning (Bortolotti et al., 2023), reviewing imaging techniques for measuring the small intestine (Chacon and Wilson, 2023) and ice detection sensors (DiLorenzo ad Yu, 2023).

LitMaps

LitMaps is also based on the Semantic Scholar database. As with the above, it uses the citation network to construct graphs to visualise the research landscape of your topic. Its Discover tool enables you to find gaps in your own literature reviews and to upload existing literature reviews to find not only more recent papers but also papers that the reviews may have missed.

LitMaps

Using LitMaps

LitMaps allows you to either add articles from your searches or from reference management software or allows you to use an existing Litmaps collection.

Litmaps Seed allows you to quickly find the most relevant papers by taking an initial starting paper and visualising those papers with the most connections to your article:

• Go to Litmaps Seed and select a seed paper by either searching for a paper in the search bar or look for a specific paper you know.
• When you are viewing a list of articles, click the tick icon to select one as your seed.
• Once you've got a seed article, press "Continue". Click "Generate Seed Map".

Full details are given in their find relevant articles with seed maps. Keep track of relevant papers by adding them to a collection.

Litmaps Discover provides paper recommendations based on a given set of articles. By examining the connections between papers via the citation network, Discover can suggest the most relevant papers for you:

• Go to Litmaps Discover.
• Click “New Search”.
• Add articles from one of your existing Litmaps collections, your reference manager, or by searching for papers directly.
• Once you’ve added all the articles you’d like, click "Algorithm". The choice of algorithms are:
  • Top Connected: the default search which finds the most relevant papers based on citation network. 
  • Co-Author Search: finds the key authors among your papers and other works they've co-authored together.
  • Semantic search: Finds relevant papers based on title and abstract similarity.
• Click “Configure Filters” to adjust the Filters. You can limit results to certain keywords or date ranges.
• Click “Run Search” to see your search results. The display will show you articles which are highly connected to your input articles.

Review the results that LitMaps has found by scrolling through the list in the side panel, or click through the results in the outer circle of the visualisation. See the advice on how to expand your research collection.

As you find papers which are relevant, click on them. Use the Expand Search button to update the Discover Search to use the papers you've checked to find even more relevant literature. Your results will become more and more relevant as you continue to expand your search and Litmaps uses its AI features to understand your research interests.

Evidence

Ediansyah et al. (2023) used LitMaps to review research on medical tourism and to recommend areas for further research based on the keywords that were extracted from the 86 papers that were included in the final analysis.

Kaur et al. (2022) created a map using LitMaps using the phrase “Open Educational Resources” (OER). This found 36 relevant papers related to the search. The first 20 papers and 16 suggested papers were selected for visualization. Each node on the map represents a paper on OER, giving information about the author’s name, year of publication and paper title while the links between the authors denote its references and citation.

Sarkar and Shukla (2023) uploaded papers retrieved from Google Scholar on the subject of behavioural analysis of cybercrime into LitMaps to identify and include any relevant additonal papers.

Inciteful.xyz

Inciteful.xyz has pulled information and/or inspiration from Semantic Scholar but also three other data sources: OpenAlex, CrossRef and OpenCitations. It has used these sources to help researchers get up to speed on a new topic, to find the latest literature or to work out how two ideas are connected (Weishuhn, 2024).

There are two tools that are under active development:

• Paper Discovery builds a network of papers from citations, analyses the network. It allows you to get up to speed on a topic. by finding the most similar papers, important papers as well as prolific authors and institutions.
• Literature Connector allows you to enter two papers and it will give you an interactive visualization show you how they are connected by the literature.

Inciteful.xyz

Using Inciteful.xyz

Enter the details of a seed paper in the Paper Discovery search box on the left of the screen in Inciteful.xyz.

Explore the results by viewing the tables for each paper:

• The first table covers the most important papers, as measured by the PageRank algorithm. PageRank values papers which are cited by other important papers. As it takes time to build up citations, these papers tend to be the seminal papers in the field.
• Similar Papers uses an algorithm to show you the most similar papers based on who these papers cite: If the seed paper cites many of the same papers that another paper does, then they are considered similar. These papers tend to be more recent papers in the field.
• The Other Data section highlights other interesting information about the graph such as most important authors, institutions, and journals. It helps to get an idea of where the work in this field is being done, by whom, and in what journals.

Further details about the use of filters is given in the Quick Start guide.

Alternatively, you can use the Literature Connector search to enter the details of two papers to find relevant papers which connect your two ideas which you subsequently add to the Paper Discovery tool.

Evidence

Ortega et al. (2023) searched several databases including Scopus, Taylor & Francis and Science Direct to review the literature on the use of the common giant reed in a biorefinery context. They also used Inciteful.xyz to produce a chart showing the relationships between various studies on the use of Arundo in biorefineries. The tool provides the most important papers in the field, refining the search by the number of citations and closeness to the papers used as seed papers. The interconnections were found by refining the search to show papers on Arundo which allowed them to select the 50 most relevant papers.

Multidisciplinary databases such as Scopus and Web of Science contain analysis tools that help you to justify why yours is an important research area, provide evidence of when the study of your topic emerged in the literature and back up assertions you make in your literature review about leading researchers and the subject area in which your topic is applied.

They have also included AI elements into their search. Scopus AI is an AI-driven tool which is one of the search options on its landing page. It has been designed to help you explore the Scopus database, understand new topics more efficiently, offers succinct academic overviews and provides a concept map of the subject to help you visualise all aspects of the topic.

Apps such as VosViewer or Bibliometrix (in R) take these further and help you to spot research gaps in the literature, provide an insight into academic networks (by visual means) identifying leading researchers and key research topics in the specific field, identify important terminology related to a specific topic and identify thematic trends over time. These apps are not included in this guide as they are not based on artificial intelligence.