[News release] Scientists unknowingly tweak experiments

Scientists in a laboratory of the University of La Rioja. (Photo credit: Wikipedia)

From the 18 March 2015 Australian National University news release

A new study has found some scientists are unknowingly tweaking experiments and analysis methods to increase their chances of getting results that are easily published.

The study conducted by ANU scientists is the most comprehensive investigation into a type of publication bias called p-hacking.

P-hacking happens when researchers either consciously or unconsciously analyse their data multiple times or in multiple ways until they get a desired result. If p-hacking is common, the exaggerated results could lead to misleading conclusions, even when evidence comes from multiple studies.

“We found evidence that p-hacking is happening throughout the life sciences,” said lead author Dr Megan Head from the ANU Research School of Biology.

The study used text mining to extract p-values – a number that indicates how likely it is that a result occurs by chance – from more than 100,000 research papers published around the world, spanning many scientific disciplines, including medicine, biology and psychology.

“Many researchers are not aware that certain methods could make some results seem more important than they are. They are just genuinely excited about finding something new and interesting,” Dr Head said.

“I think that pressure to publish is one factor driving this bias. As scientists we are judged by how many publications we have and the quality of the scientific journals they go in.

“Journals, especially the top journals, are more likely to publish experiments with new, interesting results, creating incentive to produce results on demand.”

Dr Head said the study found a high number of p-values that were only just over the traditional threshold that most scientists call statistically significant.

“This suggests that some scientists adjust their experimental design, datasets or statistical methods until they get a result that crosses the significance threshold,” she said.

“They might look at their results before an experiment is finished, or explore their data with lots of different statistical methods, without realising that this can lead to bias.”

The concern with p-hacking is that it could get in the way of forming accurate scientific conclusions, even when scientists review the evidence by combining results from multiple studies.

For example, if some studies show a particular drug is effective in treating hypertension, but other studies find it is not effective, scientists would analyse all the data to reach an overall conclusion. But if enough results have been p-hacked, the drug would look more effective than it is.

“We looked at the likelihood of this bias occurring in our own specialty, evolutionary biology, and although p-hacking was happening it wasn’t common enough to drastically alter general conclusions that could be made from the research,” she said.

“But greater awareness of p-hacking and its dangers is important because the implications of p-hacking may be different depending on the question you are asking.”

The research is published in PLOS Biology.

[Reblog] A problem with precision medicine: It’s not quite precise – at least not yet

From the 20 February 2015 post by Joseph Burns at Covering Health (Association of Health Care Journalist)

At a conference last year, Michael Laposata, M.D., Ph.D., one of the nation’s best known pathologists, explained how clinical laboratories could deliver more value to patients, physicians, and health insurers. To do so, pathologists and laboratory scientists need to provide more detailed explanations about lab test results because even physicians who order genetic and molecular tests are often confused about the results, said Laposata, chairman of the Department of Pathology at the University of Texas Medical Branch.

When he explains test results to ordering physicians, he frequently refers to an “allele” which is one of two or more versions of a gene, he said. When he does, physicians sometimes ask, “What’s an allele?”

His anecdote is telling following President Obama’s announcement last month that he recommended spending $215 million on the precision medicine initiative. The announcement was correctly hailed as an important and needed investment in medical technology. “Precision medicine” is described by the National Institutes of Health as “an emerging approach for disease treatment and prevention that takes into account individual variability in genes, environment, and lifestyle for each person.”

Given that there is and will continue to be a lot of hype about precision and personalized medicine, we may want to check our expectations, because the hurdles are daunting, as Tabitha M. Powledge wrote for the Genetic Literacy Project. “The plan embodies a wonderfully human let’s-climb-Everest-because-it’s-there aspiration. But you also have to wonder about the practicality of such a sweeping program,” she explained.

…

There is not much good, proven, scientific, medical uses for what we’re talking about as personalized or precision medicine,” Greely said. “And yet, we … sell and we hype as if there is much more.”

…

“Here’s the problem: because personalized medicine is in the realm of OMG-that’s-too-complicated science, the usual watchdogs don’t see it. Plus the big academic medical centers love the grants that it generates.”

Even health insurers are struggling to understand the full implications of genetic and molecular testing…

 

Related article

Precision medicine to prevent diabetes? Researchers develop personalized way to steer prevention efforts

[Reblog] Precision Medicine Initiative: Some quick resources

 

From the 30 January 2015 post BY PIA CHRISTENSEN  at Covering Health – Monitoring the Pulse of Healthcare Journalism 

 White House has announced its anticipated “Precision Medicine Initiative,” which it describes as an “emerging field of medicine that takes into account individual differences in people’s genes, microbiomes, environments, and lifestyles – making possible more effective, targeted treatments for diseases like cancer and diabetes. ”

….

The practice of medicine has always been personal regarding the treatment of individual patients, but science has fostered a new era of so-called personalized medicine that takes into account each person’s specific clinical, genetic, genomic and environmental information in designing tailored treatment plans

…

The White House released this fact sheet today.

This interview with the director of the program in personalized health at the University of Utah offers a good explanation of what personalized care is and examples of what it could do.

For Science magazine, Jocelyn Kaiser writes that the Obama precision medicine plan would create huge U.S. genetic biobank. She follows up with more details about the price tag and budget.

…

The White House released this fact sheet today.

This interview with the director of the program in personalized health at the University of Utah offers a good explanation of what personalized care is and examples of what it could do.

For Science magazine, Jocelyn Kaiser writes that the Obama precision medicine plan would create huge U.S. genetic biobank. She follows up with more details about the price tag and budget.

 

Related articles

• Five Accelerants to the Adoption of Connected Health (From the 28 January 2015 item at The cHealth blog)

“I have been thinking lately about the cultural and business phenomena that are currently shaping and accelerating the adoption of connected health and, in that context, came up with five accelerants.  The best part of the story is that four of the five are already going on and we can see their early-stage effects.

So, at the risk of ‘dumbing down’ adoption, here is my list of five accelerants.  If we could make these go faster, the adoption of connected health would accelerate too.”

1. Increase value-based reimbursement for providers.
2. Create more mechanisms for provider reimbursement for non face-to-face care (like the new CMS CPT code that just took effect).
3. Accelerate consumer choice in the marketplace as well as ‘consumer-driven health care’ (i.e., high deductible plans, health savings accounts (HSAs), etc.).
4. Make the consumer-facing technology truly frictionless.
5. Create a universal privacy/security technology and make it a public good.

• Integrated approach to customer relationship management and patient relationship management (From the 28 January 2015 post at Health Care Conversation)

  A comprehensive consumer and patient engagement model should help providers attract and engage individuals in the key areas they value:

  • Help in understanding and navigating the health care system
  • Personalize information and care based on an individual’s needs
  • Easy access and communication with provider and care team
  • Support in managing an acute episode or a chronic illness
  • Secure access to personal health records

 

Read the entire article here

Related articles

• Obama to detail personalized medicine initiative
• Obama Precision Medicine Plan Would Create Huge U.S. Genetic Biobank
• Got genes: Obama proposes genetic biobank of 1mn Americans’ DNA to fight disease

Accelerating Medicines Partnership – National Institutes of Health (NIH)

Accelerating Medicines Partnership – National Institutes of Health (NIH)

Great component – sharing data! Note below the bolded underlined portion

From the 4 March 2014 press release

The Accelerating Medicines Partnership (AMP) is a bold new venture between the NIH, 10 biopharmaceutical companies and several non-profit organizations to transform the current model for developing new diagnostics and treatments by jointly identifying and validating promising biological targets of disease. The ultimate goal is to increase the number of new diagnostics and therapies for patients and reduce the time and cost of developing them.

AMP will begin with three to five year pilot projects in three disease areas:

• Alzheimer’s disease
• type 2 diabetes
• autoimmune disorders of rheumatoid arthritis and systemic lupus erythematosus (lupus)

For each pilot, scientists from NIH and industry have developed research plans aimed at characterizing effective molecular indicators of disease called biomarkers and distinguishing biological targets most likely to respond to new therapies.

Through this cross-sector partnership, which will be managed through the Foundation for the NIH (FNIH), NIH and industry partners are sharing expertise and resources — $230 million — in an integrated governance structure that enables the best informed contributions to science from all participants. A critical component of the partnership is that industry partners have agreed to make the AMP data and analyses publicly accessible to the broad biomedical community. These pilot projects will set the stage for broadening AMP to other diseases and conditions.

AMP Partners

Government	Industry	Non-Profit Organizations
FDA NIH	AbbVie Biogen Idec Bristol-Myers Squibb GlaxoSmithKline Johnson & Johnson Lilly Merck Pfizer Sanofi Takeda	Alliance for Lupus Research Alzheimer’s Association American Diabetes Association Lupus Foundation of America Lupus Research Institute Foundation for the NIH Geoffrey Beene Foundation PhRMA Rheumatology Research Foundation USAgainstAlzheimer’s

Budget: 5 years [$230 Million (Rounded) Total Project Funding]

($Millions)	Total Project	Total NIH	Total Industry
Alzheimer’s Disease	129.5	67.6	61.9
Type 2 Diabetes	58.4	30.4	28
Rheumatoid Arthritis and Lupus	41.6	20.9	20.7
Total	229.5	118.9	110.6

The Opportunity

The Challenge

The Impact

Governance

 

Less Research Is Needed

Research being carried out at the Microscopy lab of the . This photo was taken on July 28, 2006 using a Nikon D70. For more information about INL’s research projects and career opportunities, visit the lab’s facebook site. http://www.facebook.com/idahonationallaboratory (Photo credit: Wikipedia)

From a thoughtful 25 June 2012 post at The Health Care Blog

he most over-used and under-analyzed statement in the academic vocabulary is surely “more research is needed”.

These four words, occasionally justified when they appear as the last sentence in a Masters dissertation, are as often to be found as the coda for a mega-trial that consumed the lion’s share of a national research budget, or that of a Cochrane review which began with dozens or even hundreds of primary studies and progressively excluded most of them on the grounds that they were “methodologically flawed”.

Yet however large the trial or however comprehensive the review, the answer always seems to lie just around the next empirical corner.

With due respect to all those who have used “more research is needed” to sum up months or years of their own work on a topic, this ultimate academic cliché is usually an indicator that serious scholarly thinking on the topic has ceased. It is almost never the only logical conclusion that can be drawn from a set of negative, ambiguous, incomplete or contradictory data…

Related articles

• Understanding the nuances of evidence-informed healthcare (thehindu.com)

Looking for historical biomedical information? Try the redesigned IndexCat, a product of the US National Library of Medicine

In my last position as a medical librarian, IndexCat was the first place to go for finding historical biomedical articles and related information. Searches on IndexCat can find in minutes what took up to an hour or more in the print version, providing access to over 3.7 million items as information about books, journal articlesd,issertations, pamphlets, reports, newspaper clippings, case studies, obituary notices, letters, portraits, as well as rare books and manuscripts.

IndexCat has recently been redesigned for even easier access to the records of historical biomedical information.

IndexCat is “the [free!]online version of The Index-Catalogue of the Library of the Surgeon-General’s Office (Index-Catalogue) [, the]..multi-part printed bibliography or list of items in the Library of the Surgeon-General’s Office, U.S. Army. It contains material dated from the 1400s through 1950 and is an important resource for researchers in the history of medicine, history of science, and for clinical research.”[http://www.nlm.nih.gov/hmd/indexcat/abouticatalogue.html]

While IndexCat does not contain the full text of items, it provides enough information on them so they can be located at libraries.
If you need the full text of the items, the best place to start is your local public or academic library. Ask for a reference librarian.
He or she can help you find the item or assist you in getting a copy through interlibrary loan. And remember, most academic libraries will be happy to help those who are not affiliated with their institution. Just call ahead and ask how they assist the public.

Related blog post

How to obtain free/low cost medical and scientific articles (jflahiff.wordpress.com)