[Journal Article] Search engines cannot diagnose through symptom searching – only 14% accuracy

Ever enter your symptoms into a search engine (as Google) to find what was the cause? And bring the results to your health care provider?  Believe that search engines can correctly diagnose your symptoms?
A  medical researcher not only was thinking along these  lines. He also set up a system to see if search engines could diagnose symptoms accurately.

The results were published in a scientific paper.**

Here are some remarks from medical librarians at their discussion group.

• They don’t address the problem with these search engines of bias: Google,
  Bing, track what you’ve searched on, they aren’t ‘anonymous’ engines, thus
  biasing the results. A different computer, with different previous user
  will give different results with these search engines. Flawed article,
  in my opinion. Too bad, it is interesting.

• This article is very interesting. While it is about validating the instrument for analyzing the webpages, they found that only 14% of the website gave a correct diagnosis. Seventy percent came up with the diagnosis as part of a differential. It sort of scares me that many medical students and other healthcare students might use search engines to find differentials. One implication is that patients who bring in webpages may actually hold the appropriate differential in their internet printout. Physicians might consider that information. The article is NOT an open access journal. The abstract does not discuss the findings of accuracy since they were testing the scoring system.

My thoughts? Familydoctor.org (American Academy of Family Physicians) has great advice
Our symptom checker flowcharts allow you to easily track your symptoms and come to a possible diagnosis.
Remember,  be sure to consult with you doctor if you feel you have a serious medical problem.

As a medical librarian, we counsel people to use any information they find as a resource when consulting with their health care provider. Information on the internet may be outdated, flawed, and sometimes even wrong.
Also, the health care providers views you as a whole person, not just a narrow set of symptoms.  They use not only your symptoms, but other factors as health history, current and past treatments, and environmental factors to work toward a treatment plan.

Related Resources

   Online symptom checkers (Standford Health System)

**  Abstract from PubMed.
Full text of article not available online for free.
Might be available for free or low cost at a local public, medical, or academic libary.
Call ahead and ask for a reference librarian.
Many medical and academic libraries offer some help to the public.

Int J Med Inform. 2014 Feb;83(2):131-9. doi: 10.1016/j.ijmedinf.2013.11.002. Epub 2013 Nov 19.

The accuracy of Internet search engines to predict diagnoses from symptoms can be assessed with a validated scoring system.

Shenker BS.

Author information

• Rutgers Robert Wood Johnson Medical School and Rutgers Robert Wood Johnson Family Medicine Residency at CentraState, United States. Electronic address: bshenker@centrastate.com.

Abstract

PURPOSE:To validate a scoring system that evaluates the ability of Internet search engines to correctly predict diagnoses when symptoms are used as search terms.METHODS:We developed a five point scoring system to evaluate the diagnostic accuracy of Internet search engines. We identified twenty diagnoses common to a primary care setting to validate the scoring system. One investigator entered the symptoms for each diagnosis into three Internet search engines (Google, Bing, and Ask) and saved the first five webpages from each search. Other investigators reviewed the webpages and assigned a diagnostic accuracy score. They rescored a random sample of webpages two weeks later. To validate the five point scoring system, we calculated convergent validity and test-retest reliability using Kendall’s W and Spearman’s rho, respectively. We used the Kruskal-Wallis test to look for differences in accuracy scores for the three Internet search engines.RESULTS:A total of 600 webpages were reviewed. Kendall’s W for the raters was 0.71 (p<0.0001). Spearman’s rho for test-retest reliability was 0.72 (p<0.0001). There was no difference in scores based on Internet search engine. We found a significant difference in scores based on the webpage’s order on the Internet search engine webpage (p=0.007). Pairwise comparisons revealed higher scores in the first webpages vs. the fourth (corr p=0.009) and fifth (corr p=0.017). However, this significance was lost when creating composite scores.CONCLUSIONS:The five point scoring system to assess diagnostic accuracy of Internet search engines is a valid and reliable instrument. The scoring system may be used in future Internet research.Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.