Readability, Understandability, and Quality of Online Education Materials and Large Language Models for Retrograde Cricopharyngeal Muscle Dysfunction

Abstract

ObjectiveThis study aims to evaluate online patient education materials on retrograde cricopharyngeal dysfunction (RCPD) by comparing the readability, understandability, and quality of content generated by large language models (LLM).MethodA web search in December 2024 evaluated 51 online resources and four LLMs (ChatGPT 4.0, Gemini 1.5 Flash, Perplexity GPT-3.5, DeepSeek-V2.5). Readability was analyzed using Readable.io, understandability actionability was assessed using PEMAT, and information quality was assessed using DISCERN.ResultsThe average readability level of the online material and the LLM responses was at the 11th-12th grade level. The Flesch Reading Ease score was lowest for the LLMs, especially for the DeepSeek-V2.5 model (24.21). While PEMAT understandability scores were adequate for online (82%) and LLMs (79%), actionability was low across all groups (25-37%). DISCERN analyses showed that both sources of information were of limited quality in supporting treatment decisions.ConclusionThis study revealed that both online and LLM-generated materials on RCPD exceeded the recommended readability levels. Although the materials demonstrated acceptable understandability, they exhibited low actionability and inadequate overall quality, emphasizing the need for more patient-centered digital health communication.