Google’s medical language model “Med-PaLM 2” enters pilot phase with first customers



summary
Summary

Google is rolling out Med-PaLM 2 on a limited basis for initial testing.

Update April 14, 2023:

Google Cloud announces that Med-PaLM 2 will be rolled out to select Google Cloud customers for a “limited test” in the coming weeks. The goal, the company says, is to explore safe, responsible and meaningful use scenarios.

The medical language model could “facilitate rich, informative discussions, answer complex medical questions, and find insights in complicated and unstructured medical texts,” according to Google. It can also generate short and long answers to medical questions and create summaries from internal documentation and data sets, as well as from scientific sources.

According to Google, Med-PaLM 2 is the first language model to achieve expert-level performance on US Medical Licensing Examination (USMLE)-style questions with more than 85 percent accuracy. In the MedMCQA dataset, which includes questions from India’s AIIMS and NEET medical exams, it achieved a “pass rate” of 72.3 percent.

ad

Original article, March 18, 2023:

Google’s medical language model Med-PaLM 2 passes exam questions

Med-PaLM is Google’s variant of the PaLM language model optimized for medical questions. The latest version is designed to answer medical questions reliably at an expert level.

Last December, Google unveiled Med-PaLM, a version of Google’s giant PaLM (Pathways Language Model) language model optimized for answering medical questions. Med-PaLM was developed using a special soft prompting method combined with responses to medical prompts written by four clinicians.

Med-PaLM performed at the level of medical professionals in most of the benchmarks tested. Potentially harmful responses were generated 5.9 percent of the time, compared with 5.7 percent for human experts, the research team said.

Med-PaLM was also the first AI model to potentially pass the US Medical Licensing Examination (67.2 percent correct when tested with “licensing-style questions,” 60 percent required), correctly answering multiple-choice and open-ended questions and reasoning about its answers.

Recommendation

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top