Skip to main content Skip to main menu Skip to section menu

Artificial Intelligence (AI) and the literature review process: Data extraction

Application of AI tools such as ChatGPT to searching and all aspects of the literature review process

Data extraction tools automatically identify data within a paper and save it into a table or spreadsheet. However, whereas most natural language processing research has examined the screening phase of systematic literature reviews, far fewer have investigated the data extraction phase. There were just 26 articles relating to this topic from January 2000 to January 2015 (Jonnalagadda et al., 2015). Pdf readers can be helpful in giving you a quick summary of an article in pdf. Tools such as Google Scholar Pdf Reader can potentially save you a considerable amount of time. However, you may want to specify what data you would like to be extracted. The systematic literature review by Clark et al. (2025) found that generative AI tools perform well in extracting data such as publication year, country or numerical data such as participant numbers, but they perform less effectively when extracting more complex data such as outcome data or descriptions of interventions.

The NICE Position Statement on the use of AI in evidence generation from August 2024 recognizes in clause 19 that LLMs could be used to automate data extraction from published quantitative and qualitative studies by inputting prompts into the AI tool to generate the preferred output. However, it acknowledges that this is less well established than the use of AI tools in other aspects of the review process.

AI tools for data extraction

pdf summaries

There are plenty of free AI tools on the market which take the pdf of an article and supply you with a summary. However, do bear in mind that many free AI tools a) only allow you to analyse a certain number of articles per day - there is a maximum number; and b) many free AI tools will only allow you to ask a certain number of questions per day - there is a maximum number of allowed questions. In their December 2023 survey,  Castillo-Segura et al. (2024) found only PdfGear provided an unlimited number of documents to analyse and an unlimited number of questions to ask. Since then, you can download Google Scholar Pdf Reader as an extension for most browsers. This AI tool is free, working within Google Scholar to provide you with the key points from any pdf article, although it currently lacks a feature to highlight important text.

Evidence

Generative AI tools such as ChatGPT can provide contextually-relevant and personalized responses from clinical studies but struggles to extract detail or key information (Liu et al., 2023). In addition, ChatGPT tends to miss important attributes in the summary, such as failing to refer to short-term or long-term outcomes that often have varying risks (Peng et al., 2023).

Other AI tools have therefore been used to save you time when extracting data from journal articles. Claude 2, for example, was impressive in its extraction performance when supplied with articles in pdf, achieving 96% accuracy across 160 data elements, missing key data in only a few cases. It also required less prompt engineering than anticipated (Gartlehner et al., 2024). Elicit performed best in terms of the quality of the extracts in an analysis of 33 papers (Spillias et al., 2025), significantly outperforming versions of ChatGPT with extractions closer to the quality considered acceptable by human reviewers. However, they concluded that "the AI tools that we tested were not reliable enough to be trusted to perform the task without human involvement" (Spillias et al., 2025: 14).

Elicit

Elicit is currently seen as "as a supplement to traditional library database searching for advanced searchers" (Whitfield and Hofmann, 2023: 207). Tools such as Elicit and SciSpace offer substantial assistance to the SLR process by extracting key insights from a large number of papers very quickly, reducing the risk of omissions and errors by researchers and saving them time (Dukic et al., 2024).

The comparative analysis of AI tools by Gobin et al. (2025) did not include Elicit among the AI tools considered. Their evaluation of the performance of each AI tool across 9 different articles found that ChatGPT and ELISE (an in-house tool developed by the authors) were most effective when extracting data with a minimal variation across articles. In contrast, Epsilon, Humata  and SciSpace/Typeset, other AI tools which can be used for extraction, a) were not able to extract all the data, with scores ranging from 60-70% accuracy, and b) were not consistent in their data extraction, missing standard bibliographic information. They concluded that "human oversight remains indispensable in validating AI-generated content, ensuring accuracy, compliance, and contextual relevance" (Gobin et al., 2025: 18).

Elicit

Elicit will extract data from pdfs you upload, saving you time and allowing you to then synthesise the information. Its own user survey found that 10% of respondents said that Elicit saves them 5 or more hours each week and that, in pilot projects, Elicit saved research groups 50% in costs and more than 50% in time by automating data extraction work they previously did manually (Elicit, 2023).

The free basic version allows you to extract data from papers and upload your own papers. However, only priced versions of the product will give you summaries of papers and allow you to extract the information into csv and bib formats.

 

Elicit: AI research assistant

Using Elicit

Elicit has been designed as an AI research assistant to assist with the literature review workflow. Elicit extracts papers into an organized table.

There are 24 different types of information that you can extract and view in columns in a table, including:

  • The findings
  • Details of participants
  • Location/country
  • Outcomes measured.

Viewing the information in a table makes it easier for you to synthesize the articles and increases your own understanding of the literature.

Elicit saves you time by doing this aspect of the review process for you.

In addition, by providing one sentence summaries of the abstracts, Elicit allows you to decide whether to include the paper in your review or whether you need additional information about the paper.

Elicit works best with the prompt: What are the effects of ___ on ___? You do need to include a question mark in the search.  It works less well for identifying facts.

Elicit's limitations are that:

  • it takes information from the Semantic Scholar Academic Graph dataset and relies on open access content via the Unpaywall plugin. This means that full-text content that cannot be found via Unpaywall may only have limited bibliographic information. You may therefore need to copy and paste the title into Google Scholar to identify whether the paper is a journal article, conference paper, book chapter or thesis.
  • it may replace some of the more routine tasks in reviewing the literature such as data extraction but "Elicit is not able to perform high-level cognitive functions that are required to create an understanding and synthesize the literature" ((Whitfield and Hofmann, 2023: 204).
  • "around 90% of the information you see in Elicit is accurate...  it’s very important for you to check the work in Elicit closely" (Elicit, 2023). Elicit tries to make this easier for you by identifying all of the sources of the information generated with language models. But it does mean that around 10% of the information you see is incorrect.
  • Elicit does not currently answer questions or surface information that is not written about in an academic paper. It tends to work less well for identifying facts and works less well in theoretical or non-empirical domains.
  • Elicit tends to include unnecessary extra information, in addition to the relevant material, in its extractions. This means that further human work is still required to sift out the irrelevant information (Spillias et al., 2025).

Was this page useful?