Paywall: "Evaluating the Performance of ChatGPT and Perplexity AI in Business Reference"


The Thomas Mahaffey Jr. Business Library conducted a study to assess the performance of two competing generative AI products, ChatGPT and Perplexity AI, in answering business reference questions. The study used a data set consisting of a sample of anonymized reference questions submitted through the library’s ServiceNow ticketing system between January 2018 and May 2022. The questions were input as prompts to each competing AI. . . . Results showed similar and underwhelming performance between each AI at the composite level. Analysis of scores in each individual scoring dimension showed greater variance in the score distributions between the competing AI. Through the evaluation process, key strengths, weaknesses, and trends emerged between each AI.

https://doi.org/10.1080/08963568.2024.2317534

| Research Data Curation and Management Works |
| Digital Curation and Digital Preservation Works |
| Open Access Works |
| Digital Scholarship |

Avatar photo

Author: Charles W. Bailey, Jr.

Charles W. Bailey, Jr.