[Reblog] My talk to Knight Science Journalism Fellowships at MIT Medical Evidence Boot Camp 2013

From the 8 December 2013 post at HealthNewsReview.org

Last week in Cambridge, I spoke again (5th time?) at this event.  Always an honor.   Always a smart audience.

Thanks to Phil Hilts for the invitation and the opportunity to share our work with a new group of journalists.

Here are my slides.

Some excerpts

Related articles

• The Mammogram Myth, Alive and Well on “Good Morning America” (psychologytoday.com)
• Health Studies: Fact or False Hope? (kstp.com)

[HealthNewsReview.Org Weekly Digest item] – Scientific Conference News Caveat

Those of you who read my posts regularly know one of my themes is to read scientific and medical news items carefully with an eye to interpreting them objectively. *

Here are some excerpts from the 12 November 2012 HealthNewsReview.org weekly update

…the cheerleaders of health care journalism often rush to publish news from scientific conferences that is not ready for prime time – at least not without caveats and context. So that explains our post this week:

• Be still, my heart, and let’s wait on medical conference abstract news

An important reminder – for journalists and for the general public – appears in the American Heart Association journal Circulation.  The paper is entitled, “The Conversion of Cardiovascular Conference Abstracts to Publications,” and it is published right about the same time as the American Heart Association’s own huge Scientific Sessions wrapped up for 2012.   We wrote earlier about some of our concerns about news coverage from this meeting.

The new journal article provides sound reasons for why our concerns about conference news coverage are sound. The researchers analyzed abstracts presented at the American Heart Association (AHA), American College of Cardiology (ACC), and European Society of Cardiology (ESC) Scientific Sessions from 2006-2008. Then they checked how many were published.

Drum roll, please….

• Less than one-third of the presented abstracts were published within two years of the conference presentation.
• After five years, the rates rose a bit to 49.7% for AHA, 42.6% for ACC, and 37.6% for ESC .

So while some journalists are glued to coverage of these meetings, publishing daily for general news audiences that may not understand the nuances of the limitations of drawing conclusions from talks presented at scientific meetings, most of this stuff isn’t even published in the medical literature for at least 5 years – if ever!

Why does this matter?

Such news coverage creates a rose-colored view of progress in research.  It may not be inaccurate but it most certainly misleads and lacks important context if it doesn’t present caveats about the limitations.

Don’t forget the important JAMA paper, “Media Coverage of Scientific Meetings: Too Much, Too Soon?” which concluded:

“Results are frequently presented to the public as scientifically sound evidence rather than as preliminary findings with still uncertain validity. With some effort on the part of meeting organizers, journalists, and scientists, it will be possible to better serve the public.”..

Related articles at HealthNewsReview.org

• Pumping out heart news from the big Heart Association conference
•  …much botched news coverage of a study showing a statistical association between statin use and fewer deaths from cancer.  The emphasis we added was deliberate and important.  That’s all it showed.  A statistical association – not a causal link. Some stories didn’t get it.
• In the face of this string of negative reviews, we were delighted to shine a light on excellence found – in all places – in a local television news story.  A five-and-a-half-minute TV news story – in a sweeps/ratings period – thatchose to do a profile of Dr. Tim Wilt, who serves on the US Preventive Services Task Force.  What some TV news directors may have viewed as a “yawner” topic was hardly that.  If more journalists got to know USPSTF members, maybe we’d have less “death panel” rhetoric about their work.***Evaluating Health Information (Health Resources for All edited by Janice Flahiff)

Grasping and even celebrating uncertainty ( How Journalists Can Aid Critical Thinking in Healthcare Decisions) With Resources By Yours Truly

As you regular followers of this blog realize, I champion critical thinking and hope at least some of these blog posts have fostered this approach to selecting what is best for one’s health.
Many of my posts caution against quick fixes, be it fad diets, supplement dependence, or use of potentially harmful complementary medicine substances. To be fair, I have also posted items questioning “Western medicine” practices as when robotic surgery is appropriate.

Gary Schwitzer at HealthNewsReview.org has posted yet another item on how journalists can help us all in healthcare decisions..
Excerpts

Marya Zilberberg posted, “Fast science: No time for uncertainty.”  Excerpt:

“…my anxiety about how we do clinical science overall is not new; this blog is overrun with it. However, the new branch of that anxiety relates to something I have termed “fast science.” Like fast food it fills us up, but the calories are at best empty and at worst detrimental. What I mean is that science is a process more than it is a result, and this process cannot and should not be microwaved….

So, let’s celebrate uncertainty. Let’s take time to question, answer and question again. Slow down, take a deep breath, cook a slow meal and think.”

That’s similar to how I ended my talk at the University of Wisconsin’s event, “Science Writing in the Age of Denial” this week.  I said that:

“Journalists could help people grasp uncertainty and help them apply critical thinking to health care decision-making issues…rather than promote false certainty, shibboleths and non-evidence-based, cheerleading advocacy.”

My talk to University of Wisconsin event, “Science Writing in Age of Denial”

View more PowerPoint from Gary Schwitzer

Related Resources (from my Health/Medical  News & Resources Web site)

• The Penn State Medical Center Library has a great guide to evaluate health information on the Internet.

  The tips include

  • Remember, anyone can publish information on the internet!
  • If something sounds too good to be true, it probably is.
    If the Web site is primarily about selling a product, the information may be worth checking from another source.
  • Look for who is publishing the information and their education, credentials, and if they are connected with a trusted coporation, university or agency.
  • Check to see how current the information is.
  • Check for accuracy. Does the Web site refer to specific studies or organizations?
• The Family Caregiver Alliance has a Web page entitled Evaluating Medical Research Findings and Clinical Trials

Topics include

• General Guidelines for Evaluating Medical Research
• Getting Information from the Web
• Talking with your Health Care Provider

• Consumer’s Guide to Taking Charge of Health Information (Harvard Center for Risk Analysis)
• How to Evaluate Health Information on the Internet (US National Cancer Institute)9iiu9
• Quackwatch (a private corporation operated by Stephen Barrett, MD)

…And a Rumor Control site of Note (in addition to Quackwatch)

National Council Against Health Fraud

National Council Against Health Fraud is a nonprofit health agency fousing on health misinformation, fruad, and quackery as public health problems. Links to publications, position papers and more.

What the ‘limits of DNA’ story reveals about the challenges of science journalism in the ‘big data’ age

This journalist ends her post by acknowledging that the medical news readership often bases their decisions on medical and science news articles. She and others strive to report based on sound science, despite current obstacles.

Excerpts from the 6 April 2012 blog post at The Last Word on Nothing

As a science journalist, I sympathize with book reviewers who wrestle with the question of whether to write negative reviews. It seems a waste of time to write about a dog of a book when there are so many other worthy ones; but readers deserve to know if Oprah is touting a real stinker.
On 2 April, Science Translational Medicine published a study on DNA’s shortcomings in predicting disease. My editors and I had decided not to cover the study last week after we saw it in the journal’s embargoed press packet, because my sources offered heavy critiques of its methods..

…I ended up writing about the paper anyway after it made a huge media splash that prompted fury among geneticists. In a thoughtful post at the Knight Science Journalism tracker, Paul Raeburn asked yesterday why other reporters didn’t notice the problems with the study that I wrote about. Having been burned by my own share of splashy papers that go bust, I think the “limits of DNA “ story underscores a few broader issues for our work as science journalists:
1. Science consists of more and more “big data” studies whose findings depend on statistical methods that few of us reporters can understand on our own. I never would have detected the statistical problems with the Vogelstein paper by myself. We can look for certain red flags that a study might not be up to snuff, such as small sample sizes or weak clinical trial designs, but it’s a lot harder to sniff out potential problems with complicated statistical methods.

2. Challenges in the news business are ratcheting up pressure on all of us. Reporters are doing much more work in much less time than we have in the past as we compete with an expanded universe of news providers who have sped up the news cycle. Yet it still takes time and effort to make sense of the developments we cover. It took me about three days to report my piece on the Vogelstein paper while I was simultaneously working on other assignments. That’s probably longer than most reporters can spend on a piece like this…

The article goes on to point out other challenges as

4. It’s becoming more difficult to trust traditional scientific authorities.

5. Beware the deceptively simple storyline.

6. Getting the story right matters more than ever.

Related Resources

• How to Read a Research Paper (and also Understand Health News Research Items) (jflahiff.wordpress.com)
• “Summaries for Patients” and other plain language summaries help patients and others understand medical studies and guidelines (jflahiff.wordpress.com)
• HealthNewsReview.org – Independent Expert Reviews of News Stories(jflahiff.wordpress.com)
  • HealthNewsReview.org  includes reviews of health articles in the news.
    It also has a toolkit with a number of tipsheets, primers, links and other resources to help journalists and consumers do a better job of evaluating claims about health care interventions.

Related articles

• Science journalism blows it, dolphin rape edition. (behavecology.wordpress.com)

• Science and Steamrollers: How Research Stories Can Go Off the Rails(CancerResearch101)

• Do scientists understand science journalism? (whyevolutionistrue.wordpress.com)
• DNA has limits, but so does study questioning its value, geneticists say (blogs.nature.com)
• Science journalists should be asking questions and deflating exaggeration (guardian.co.uk)

FCC seeks to change regulation of corporate interests disclosures on TV news (including local hospital segments on the news)

Those health news segments on the local news might not be as unbiased as they appear!

From the 3 January 2012 Washington Post article by Paul Farhi

V newscasts are increasingly seeded with corporate advertising masquerading as news — and the federal government wants to do something about it.

Concerned that subtle “pay-for-play” marketing ploys are seeping into the news, the Federal Communications Commission has proposed a regulation that would require the nation’s 1,500 commercial TV stations to disclose online the corporate interests behind the news….

…

“Unless you stick around for the end credits, you’re unlikely to know it’s payola,” said Corie Wright, senior policy counsel for Free Press, a media watchdog group backing the FCC proposal. “If broadcasters were required to put it online, you could check to see if it was actually sponsored or not.”

The proposed regulation is aimed at news programs that appear to viewers to be the work of independent journalists, but in fact sponsors have shaped or even dictated the coverage.

A common form of advertiser-supplied content, documented in a recent Washington Post article, is a live interview segment in which a seemingly neutral reviewer recommends a series of products that the “reviewer” has been paid by sponsors to mention. Stations across the country have also brokered “exclusive” relationships with local hospitals in which the hospitals pay the station to be featured in health stories.  [my emphasis] Other stations have aired “news” programs that feature interviews with sponsors who’ve paid for the privilege.

According to an FCC report, many stations also use “video news releases,” footage produced by a sponsor or corporate interest that looks like it was shot by the station.

Under current law, such arrangements aren’t illegal,

 

Medical journal news releases shouldn’t just try to make news but to make news reporting better

Medical news press releases do strongly affect health/medical news items in the “popular press”.
All the more reason, as this article points out, that these press releases need to be as complete and accurate as possible!

From the 31 Janauary 2012 HealthNewsReview.org*** article

Not to be missed:  last week’s BMJ published an analysis by a team at Dartmouth Medical School led by Steven Woloshin and Lisa Schwartz – “Influence of medical journal press releases on the quality of associated newspaper coverage.”***

This is an important contribution to our understanding of the food chain of the dissemination of research news to the American public:  medical journals feed journalists who feed the American public what they get out of journals – sometimes driven largely by what’s in journal news releases.  If the information at the source is complete and high quality, the flow of information from journalists to the public is more likely to be complete and high quality as well.  But this analysis also suggests that “low quality press releases might make (associated newspaper stories) worse.”

Excerpts:

Higher quality press releases issued by medical journals were associated with higher quality reporting in subsequent newspaper stories. In fact, the influence of press releases on subsequent newspaper stories was generally stronger than that of journal abstracts. Fundamental information such as absolute risks, harms, and limitations was more likely to be reported in newspaper stories when this information appeared in a medical journal press release than when it was missing from the press release or if no press release was issued. Furthermore, our data suggest that poor quality press releases were worse than no press release being issued: fundamental information was less likely to be reported in newspaper stories when it was missing from the press release than where no press release was issued at all (although the findings were generally not statistically significant).

…

Reporting on medical research is challenging:…

***HealthNewsReview.org is a group of independent professional journalists who review medical stories in newspapers, magazines, etc

Please see my earlier blog posting for a fuller description

Related articles

• Quality medical journal news releases can help newspapers do a better job informing public (medicalxpress.com)
• Quality medical journal news releases can help newspapers do a better job informing public (eurekalert.org)
• 5 Ways to Reinvent the Press Release With SEO (prnewsonline.com)

***Influence of medical journal press releases on the quality of associated newspaper coverage: retrospective cohort study

• Source:  British Medical Journal

  Objective

  To determine whether the quality of press releases issued by medical journals can influence the quality of associated newspaper stories.

  Design

  Retrospective cohort study of medical journal press releases and associated news stories.

  Setting

  We reviewed consecutive issues (going backwards from January 2009) of five major medical journals (Annals of Internal Medicine, BMJ, Journal of the National Cancer Institute, JAMA, and New England Journal of Medicine) to identify the first 100 original research articles with quantifiable outcomes and that had generated any newspaper coverage (unique stories ≥100 words long). We identified 759 associated newspaper stories using Lexis Nexis and Factiva searches, and 68 journal press releases using Eurekalert and journal website searches. Two independent research assistants assessed the quality of journal articles, press releases, and a stratified random sample of associated newspaper stories (n=343) by using a structured coding scheme for the presence of specific quality measures: basic study facts, quantification of the main result, harms, and limitations.

  Main outcome

  Proportion of newspaper stories with specific quality measures (adjusted for whether the quality measure was present in the journal article’s abstract or editor note).

  Results

  We recorded a median of three newspaper stories per journal article (range 1-72). Of 343 stories analysed, 71% reported on articles for which medical journals had issued press releases. 9% of stories quantified the main result with absolute risks when this information was not in the press release, 53% did so when it was in the press release (relative risk 6.0, 95% confidence interval 2.3 to 15.4), and 20% when no press release was issued (2.2, 0.83 to 6.1). 133 (39%) stories reported on research describing beneficial interventions. 24% mentioned harms (or specifically declared no harms) when harms were not mentioned in the press release, 68% when mentioned in the press release (2.8, 1.1 to 7.4), and 36% when no press release was issued (1.5, 0.49 to 4.4). 256 (75%) stories reported on research with important limitations. 16% reported any limitations when limitations were not mentioned in the press release, 48% when mentioned in the press release (3.0, 1.5 to 6.2), and 21% if no press release was issued (1.3, 0.50 to 3.6).

  Conclusion

  High quality press releases issued by medical journals seem to make the quality of associated newspaper stories better, whereas low quality press releases might make them worse.

New Federal Policy Initiatives To Boost Health Literacy Can Help The Nation Move Beyond The Cycle Of Costly ‘Crisis Care’ [With Related Resources]

Health literacy used to be thought of as a problem individuals had in understanding health information and making health decisions. Now health literacy is beginning to be viewed in more holistic terms. For example, health care providers (from nurses to institutions) now view themselves as having roles in providing relevant understandable information to patients and the public.
What brought about this change in focus? According to the article below, major health policy initiatives at the federal level, including the “Plain Writing Act of 2010, which requires all new publications, forms, and publicly distributed documents from the federal government to be written in a “clear, concise, well-organized” manner.”

A good summary of this change in direction and focus may be found within the article…
New Federal Policy Initiatives To Boost Health Literacy Can Help The Nation Move Beyond The Cycle Of Costly ‘Crisis Care’

Here is an abstract of the article (in the journal Health Affairs, January 12, 2012)

Health literacy is the capacity to understand basic health information and make appropriate health decisions. Tens of millions of Americans have limited health literacy—a fact that poses major challenges for the delivery of high-quality care. Despite its importance, health literacy has until recently been relegated to the sidelines of health care improvement efforts aimed at increasing access, improving quality, and better managing costs. Recent federal policy initiatives, including the Affordable Care Act of 2010, the Department of Health and Human Services’ National Action Plan to Improve Health Literacy, and the Plain Writing Act of 2010, have brought health literacy to a tipping point—that is, poised to make the transition from the margins to the mainstream. If public and private organizations make it a priority to become health literate, the nation’s health literacy can be advanced to the point at which it will play a major role in improving health care and health for all Americans…

In years past, clinicians and researchers alike largely viewed these issues and outcomes in terms of individual patient deficits—that is, a patient’s lack of knowledge and skills regarding health issues. We now recognize that health literacy is a dynamic systems issue,2 reflecting the complexity of both the health information being presented and the health care system being navigated.3 As summarized by the Institute of Medicine, addressing the challenge of health literacy requires system-level changes for both health professionals and organizations…

It is impossible to list all relevant related resources here!
A small sampling..

Health Literacy Library Guides (while aimed at professionals, librarians, etc, some have links to materials for the rest of us)

• [Especially for the rest of us] Medical Information in Plain Language: A Health Literacy Resource Guide(Univ of CT Health Center) , including
  • News about health

• Improving Health Education Literacy (Nebraska Methodist College), especially
  • Evaluating health information on the Web (with a video)

• Health Literacy (Duke University), especially
  • Tools for Patients (Patient Care)

Great places to start for health information on many topics (diseases, conditions, talking with health care professionals, etc)

(More Great Places here)

• MedlinePlus – Over 750 topics on conditions, diseases, and wellness.  Information ondrugs, herbs, and supplements. Links to directories (health care providers, health care facilities, etc) and organizations which provider health information. Surgery videos, informative slideshows, and more.
• Agency for Healthcare Research and Quality – Consumers and Patients
  the latest evidence based information for improving your health, including podcasts and videos
• Familydoctor.org includes health information for the whole family
  Short generalized information on Diseases and Conditions (with A-Z index), Health Information for Seniors, Men, and Women, Healthy Living Topics, pages geared to Parents & Kids, and videos.  Numerous health tools in the left column (as health trackers, health assessments, and a Search by Symptom page.
  
  
• KidsHealth provides information about health, behavior, and development from before birth through the teen years. Material is written by doctors in understandable language at three levels: parents, kids, and teens
  KidsHealth also provides families with perspective, advice, and comfort about a wide range of physical, emotional, and behavioral issues that affect children and teens.

Understanding Health Research

• “Summaries for Patients”  are short summaries of studies and clinical guidelines (how medicine is best practiced) are  published in Annals of Internal Medicine.
  [Go to Summaries for Patients, scroll down a little, the right column has  link to all summaries and a search box ]Summaries about studies describe how researchers did the published study and what they found.
  Summaries about clinical guidelines describe the official recommendations for patient care
• patientINFORM plain language summary Web sites are provided by participating publishers to help patients or their caregivers more fully understand the implications of research and to provide links to the full text of research articles they’ve selected from participating journals. The publishers allow readers following links from patientINFORM material on the health organizations’ sites to access the full text of these articles without a subscription, and they provide patients and caregivers with free or reduced-fee access to other articles in participating journals.
• Cochrane Collaboration provides systematic reviews (thorough summaries) of the strongest evidence available about healthcare interventions (as drugs and medical procedures).  It does not cover all interventions, but those covered were reviewed  in-depth by experts in the medical and library fields.
  • Here is how to find plain language  and audio summaries of Cochrane reviewsGo to the Cochrane Collaboration home page and scroll down to Browse Free Summaries.
    Topics include Breast Cancer, Dementia and Cognitive Improvement, and Complementary Medicine.
    Click on To the Cochrane Library in the upper right corner of the Cochrance Collaboration home page.
    This Cochrane Library search page has a Help page , and an Advanced Search option.
• HealthNewsReview.org – Independent Expert Reviews of News Stories
  The site is dedicated to
  • Improving the accuracy of news stories about medical treatments, tests, products and procedures.
  • Helping consumers evaluate the evidence for and against new ideas in health care.

Health News Review includes

• Select (about 2-3 a day out of hundreds!) reviews of news articles. These reviews are very thorough, including how the article stacks up against criteria as treatment costs, scientific evidence, conflicts of interest
• Toolkit with links to how to understand studies, evaluate claims, etc

• Understanding Medical Research (MedlinePlus) has great links, including
  • Increasing Knowledge — How to Read a Research Paper(Lewy Body Dementia Association)
  • JAMA Patient Page: Basic Science Research(American Medical Association) – PDF
    Also available in Spanish

Related articles

• Patient engagement is the holy grail of health care (kevinmd.com)
• Patient Portal Is Now Open! (johnwarrenod.wordpress.com)
• October is Health Literacy Month (mayorshealthline.wordpress.com)
• Study Reveals Health-Literate Patients Not Always Adept At Managing Heart Failure Care (jflahiff.wordpress.com)
• NCI Announces Guide to Communicating Data to Lay Audiences (jflahiff.wordpress.com)
• Health Communication, Health Literacy and e-Health | Health.gov (ODPHP) (policyabcs.wordpress.com)
• Search the Office of Minority Health’s Library Catalog Online | Health Information Literacy – for health and well being (jflahiff.wordpress.com)
• Understanding Health Literacy (aa47.wordpress.com)
• 3rd Annual Kentucky Health Literacy Summit – March 22-23, 2012 (aa47.wordpress.com)
• Why Consumers Struggle to Understand Healthcare (money.usnews.com)
• Two New Health Literacy Papers (aa47.wordpress.com)
• Low Health Literacy: Impact on Costs (thielst.typepad.com)
• Health Information Literacy in Your Hospital (aa47.wordpress.com)

HealthNewsReview.org – Independent Expert Reviews of News Stories

HealthNewsReview.org – Independent Expert Reviews of News Stories

Health News Review includes reviews of health articles in the news.Their objective criteria includes these factors…

• What’s the total cost?
• How often do benefits occur?
• How often do harms occur?
• How strong is the evidence?
• Is this condition exaggerated?

• Are there alternative options?
• Is this really a new approach?
• Is it available to me?
• Who’s promoting this?
• Do they have a conflict of interest?

The Web site also includes a toolkit – “a number of tipsheets, primers, links and other resources to help journalists and consumers do a better job of evaluating claims about health care interventions”

• A list of tips for understanding studies.

• A collection of other resources for evaluating claims & analyzing news coverage

• A list of just some of the websites and blogs we like

• For journalists, we’ve posted a list of independent experts who can help them with their stories

• A HealthNewsReview.org widget – that you can display on your website – that posts headlines of the latest things published on our site

From their About Page

HealthNewsReview.org is a website dedicated to:

• Improving the accuracy of news stories about medical treatments, tests, products and procedures.
• Helping consumers evaluate the evidence for and against new ideas in health care.

We support and encourage the ABCs of health journalism.

• Accuracy
• Balance
• Completeness

Related Resources

How to Read a Research Paper (and also Understand Health News Research Items) (jflahiff.wordpress.com)

patientInform
 “The goal of patientINFORM is to allow patients, their family members and anyone interested in learning more about a specific disease or its treatment to access the most important new research articles through the web sites ofparticipating health organizations or publishers. Participating health organizations provide interpretation of research articles, in the form of summaries or news items written to be understood by nonphysician, nonscientist readers..

Understanding medical research (MedlinePlus) – links to overviews, related issues, and information from organizations

Related articles

• Misleading information on health social sites (and tips on how to evaluate health/medical information) (jflahiff.wordpress.com)
• Are Health Apps the Cure for Anything That Ails You? (jflahiff.wordpress.com)
• Health Literacy Month Is Back – Tell a Friend (engagingthepatient.com)
• Three common mistakes in medical journalism (boingboing.net)
• Looking for free, online, trustworthy health information that requires paid subscriptions? Check your local library’s home page!(jflahiff.wordpress.com)
• It is a rare medical story that gets high marks (jflahiff.wordpress.com)
• Don’t Be Fooled By Health Fraud Scams (jflahiff.wordpress.com)
• How the News Media May Hurt – Not Help – Health Literacy Efforts (engagingthepatient.com)
• Essential tips for medical journalists (kevinmd.com)
• PubMed Health – A Growing Resource for Clinical Effectiveness Information (jflahiff.wordpress.com)
• Misleading information on health social sites (and tips on how to evaluate health/medical information) (jflahiff.wordpress.com)
• Are Health Apps the Cure for Anything That Ails You? (jflahiff.wordpress.com)
• Health Literacy Month Is Back – Tell a Friend (engagingthepatient.com)
• Three common mistakes in medical journalism (boingboing.net)