Study compares ChatGPT’s and Google’s search performance and user experience


Chatbots like ChatGPT are already complementing and, for some, replacing the role of traditional search engines. A recent study tried to find out more about how they compare in terms of efficiency and quality.

A recent study compares the search performance and user experience of ChatGPT and Google search. The study, conducted by researchers from the US and Hong Kong, shows that ChatGPT users spend less time on tasks and rate the quality of information they receive higher. The study also points out ChatGPT’s weaknesses in fact-checking tasks.

Although the study meets scientific standards, its validity is limited by the relatively small number of participants (95). Participants were randomly assigned to one of two groups, ChatGPT or Google, and then completed a questionnaire on ease of use, usefulness, enjoyment, and satisfaction with the tool. The survey was conducted entirely online.

  • In Task 1, participants were asked to find the name and age of the first woman in space.
  • In Task 2, participants were asked to list five URLs that could be used to book a flight between Phoenix and Cincinnati in the United States.
  • In Task 3, participants were asked to read an excerpt from a news article and check three highlighted statements.

The ChatGPT users group took an average of 11:21 minutes to complete the three tasks, while the Google searchers group took significantly longer at 18:45 minutes. Participants self-reported their time per task.


The researchers attribute this difference to the fact that Google Search required users to formulate their queries multiple times. It was a trial-and-error process to get the results. ChatGPT, on the other hand, allows users to ask a question in natural language. The summarized answer eliminates the need for further reading.

In contrast, participants in both groups required a similar number of inputs for the three tasks, but the inputs were longer for ChatGPT. ChatGPT had the largest speed advantage in the first task (name and age of the first woman in space), which involved finding specific information.

In terms of search performance, ie the correctness of the answers, the researchers awarded up to ten points per task. Here, ChatGPT with 8.55, and Google with 8.77 are not far apart, so the difference is statistically negligible. But this also means that Google users take significantly longer to achieve similar quality.

There were sometimes significant differences from task to task. For example, the researchers found it noteworthy that in Task 1, all participants scored full points with ChatGPT, indicating that ChatGPT is very effective in finding facts. Google users made several mistakes here, with an average score of only 8.19.

For the second task (flight booking sites), both groups scored close to the maximum. Google seemed to be slightly more helpful, directing users to pages for flights between Cincinnati and Phoenix, while ChatGPT only directed the group to general booking pages.


Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top