Foot and Ankle Patient Education Materials and Artificial Intelligence Chatbots: A Comparative Analysis
Abstract
Background
The purpose of this study was to perform a comparative analysis of foot and ankle patient education material generated by the AI chatbots, as they compare to the American Orthopaedic Foot and Ankle Society (AOFAS)-recommended patient education website, FootCareMD.org.
Methods
ChatGPT, Google Bard, and Bing AI were used to generate patient educational materials on 10 of the most common foot and ankle conditions. The content from these AI language model platforms was analyzed and compared with that in FootCareMD.org for accuracy of included information. Accuracy was determined for each of the 10 conditions on a basis of included information regarding background, symptoms, causes, diagnosis, treatments, surgical options, recovery procedures, and risks or preventions.
Results
When compared to the reference standard of the AOFAS website FootCareMD.org, the AI language model platforms consistently scored below 60% in accuracy rates in all categories of the articles analyzed. ChatGPT was found to contain an average of 46.2% of key content across all included conditions when compared to FootCareMD.org. Comparatively, Google Bard and Bing AI contained 36.5% and 28.0% of information included on FootCareMD.org, respectively (P < .005).
Conclusion
Patient education regarding common foot and ankle conditions generated by AI language models provides limited content accuracy across all 3 AI chatbot platforms.
Level of Evidence:
Level IV
Get full access to this article
View all access and purchase options for this article.
References
1. The social life of health information. Pew Research Center: Internet, Science & Tech. Published June 11, 2009. Accessed November 13, 2023. https://www.pewresearch.org/internet/2009/06/11/the-social-life-of-health-information/
2. Battineni G, Baldoni S, Chintalapudi N, et al. Factors affecting the quality and reliability of online health information. Digit Health. 2020;6:2055207620948996.
3. Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res. 2015;17(5):e112.
4. Reddy SC, Porter DA. The importance of increasing awareness of FootCareMD. Foot Ankle Int. 2021;42(10):1362-1365.
5. Introducing ChatGPT. Accessed November 13, 2023. https://openai.com/blog/chatgpt
6. Crook BS, Park CN, Hurley ET, Richard MJ, Pidgeon TS. Evaluation of online artificial intelligence-generated information on common hand procedures. J Hand Surg Am. 2023;48(11):1122-1127.
7. Dubin JA, Bains SS, Chen Z, et al. Using a Google web search analysis to assess the utility of ChatGPT in total joint arthroplasty. J Arthroplasty. 2023;38(7):1195-1202.
8. Kirchner GJ, Kim RY, Weddle JB, Bible JE. Can artificial intelligence improve the readability of patient education materials? Clin Orthop Relat Res. 2023;481(11):2260-2267.
9. Kuroiwa T, Sarcon A, Ibara T, et al. The potential of ChatGPT as a self-diagnostic tool in common orthopedic diseases: exploratory study. J Med Internet Res. 2023;25:e47621.
10. Bluman EM, Foley RP, Chiodo CP. Readability of the patient education section of the AOFAS website. Foot Ankle Int. 2009;30(4):287-291.
11. Hartnett DA, Philips AP, Daniels AH, Blankenhorn BD. Readability of online foot and ankle surgery patient education materials. Foot Ankle Spec. 2025;18(1):9-18.
12. Irwin SC, Lennon DT, Stanley CP, Sheridan GA, Walsh JC. Ankle conFUSION: the quality and readability of information on the internet relating to ankle arthrodesis. Surgeon. 2021;19(6):e507-e511.
13. Noback PC, Trupia EP, Dziesinski LK, Sarpong NO, Trofa DP, Vosseller JT. Ankle fractures: the current state of online patient information. Foot Ankle Spec. 2021;14(4):324-333.
14. Schwarz GM, Lisy M, Hajdu S, Windhager R, Willegger M. Quality and readability of online resources on chronic ankle instability. Foot Ankle Surg. 2022;28(3):384-389.
15. Badarudeen S, Sabharwal S. Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites. J Bone Joint Surg Am. 2008;90(1):199-204.
16. Eltorai AEM, Sharma P, Wang J, Daniels AH. Most American Academy of Orthopaedic Surgeons’ online patient education material exceeds average patient reading level. Clin Orthop Relat Res. 2015;473(4):1181-1186.
17. Frosolini A, Gennaro P, Cascino F, Gabriele G. In reference to “role of Chat GPT in public health,” to highlight the AI’s incorrect reference generation. Ann Biomed Eng. 2023;51(10):2120-2122.
18. Májovský M, Černý M, Kasal M, Komarc M, Netuka D. Artificial intelligence can generate fraudulent but authentic-looking scientific medical articles: Pandora’s box has been opened. J Med Internet Res. 2023;25:e46924.
19. Cheong RCT, Unadkat S, Mcneillis V, et al. Artificial intelligence chatbots as sources of patient education material for obstructive sleep apnoea: ChatGPT versus Google Bard. Eur Arch Otorhinolaryngol. 2024;281(2):985-993.
20. ChatGPT sets record for fastest—growing user base—analyst note. Reuters. Published February 2, 2023. Accessed November 13, 2023. https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/
Cite
Cite
Cite
OR
Download to reference manager
If you have citation software installed, you can download citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: March 19, 2024
Issue published: February 2026
Keywords
Article versions
You are viewing the most recent version of this article. To download and view previous versions, see below:
PubMed: 38504411
Authors
Metrics and citations
Metrics
Publication usage*
Total views and downloads: 402
*Publication usage tracking started in December 2016
Publications citing this one
Receive email alerts when this publication is cited
Web of Science: 9 view articles Opens in new tab
Crossref: 13
- Evaluation of ChatGPT responses to common patient questions on ankle fusion
- Paradigm Shift in Online Health Information Search in the Era of Generative AI ? – A Bibliometric Literature Survey and Sentiment Analysis
- Retrospective Analysis of Malpractice Claims Following Achilles Tendon Rupture
- The Emergence of Applied Artificial Intelligence in the Realm of Value Based Musculoskeletal Care
- Large Language Models Applied to Health Care Tasks May Improve Clinical Efficiency, Value of Care Rendered, Research, and Medical Education
- ChatGPT Achieves Only Fair Agreement with ACFAS Expert Panelist Clinical Consensus Statements
- ChatGPT-3.5 and -4.0 Do Not Reliably Create Readable Patient Education Materials for Common Orthopaedic Upper- and Lower-Extremity Conditions
- Currently Available Large Language Models Do Not Provide Musculoskeletal Treatment Recommendations That Are Concordant With Evidence-Based Clinical Practice Guidelines
- Integrating domain-specific resources: Advancing AI for foot and ankle surgery
- Assessing ChatGPT responses to common patient questions regarding total ankle arthroplasty
- View More
Figures and tables
Figures & Media
Tables
View Options
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
