[Reblog] JAMA papers raise questions about FDA drug and device approval

JAMA papers raise questions about FDA drug and device approval.

JAN24 2014

Posted by Gary Schwitzer in FDA

An important series of papers was published in the Journal of the American Medical Association this week.

“Clinical Trial Evidence Supporting FDA Approval of Novel Therapeutic Agents, 2005-2012,” by Dr. Joseph Ross and colleagues, concluded that the quality of clinical trial evidence used by the FDA as the basis of approving new drugs varies widely.  A couple of interesting data points:

• in the seven-year period of analysis, 37% of drugs were approved on the basis of a single pivotal trial.
• trials using surrogate end points as their primary outcome formed the exclusive basis of approval for 45% of drugs approved. (See our primer, “Surrogate markers may not tell the whole story.”)

In an opinion piece,  “Opening the FDA Black Box,” Drs. Steven Goodman and Rita Redberg said the study:

“…raises a host of questions needing further exploration. Despite the FDA requirement for evidence from a minimum of 2 randomized clinical trials supporting an effect on health outcomes, 37% of product approvals were based on only 1 trial, 53% of cancer trials were nonrandomized, and an active comparator was used in only 27% of non–infectious disease trials. Surrogate end points were used in almost all approvals via the accelerated approval process and in 44% of nonaccelerated approvals. Trials were comparatively short, with most lasting less than 6 months, even those assessing chronic treatments for chronic diseases. Cancer drugs, perhaps predictably, were more often approved via the accelerated process and with weaker designs.”

Another paper looked at the reasons that FDA marketing approval for new drugs was delayed or denied.

And a fourth paper looked at FDA regulation of medical devices, “a process that has received relatively little attention,” according to Goodman and Redberg, who continued:

In USA Today, Liz Szabo wrote a good summary of the JAMA papers under the headline, “Not all FDA-approved drugs get same level of testing: Evidence behind FDA-approved drugs and devices often has major limitations.”

Read the entire article here

Related Resources

• ClinicalTrials.gov

ClinicalTrials.gov is a registry and results database of publicly and privately supported clinical studies of human             participants conducted around the world.
When available, study results information is included in the study record under the Study Results tab. See How to Find Results of Studies for more information on finding results entered in the results database.

Results (after 2008, only those required by US federal requirements) include
–Participant data (how many started the trial, dropped out, etc)
—  Information about participants (age, gender, blood pressure readings, etc)

[Speaking of gender…]

Good to know, most drugs can affect women and men differently

    —   Outcome (results of taking the drug plus any placebo), with statistics

    –Adverse effects , serious and other (this was not required before 2008)

• Evaluating Health Information (from my personal Google site)

  • Research Articles Summarized for Us All

  • How to Read a Research Paper

Related articles

Another take on the “relaxation of standards”

• Drug Approvals Take Customized Approach Based on Disease
• [Reblog] Last week “clinical trial system broken”; this week “luxury journals distort/damage science”
• Newly published survey shows drug shortages still have major impact on patient care
• [Reblog] Misleading BMJ news releases may be one reason journalists report on more observational studies
• Does taking multiple medicines increase risk of being admitted to hospital? Yes and no
• [Reblog] Who is making your medical app?
• [Reblog] What happens to medicine when it has no heroes?

10 (strongly suggested yet humorous) commandments for physicians when prescribing treatments

Unnecessary reliance on screening tests and the underuse of personalized medicine are two major concerns I have with the present practice of medicine. Hence the selection of this article for a blog item.

Mayo’s Dr. Victor Montori and his team argue that medical intervention success is best be measured in holistic terms as death, quality of life, and ability to function. This is in direct opposition to current industry and professional guideline standards which emphasizes narrow (and often misleading) outcomes as blood pressure reduction, lipid levels, and glucose levels.

The team’s analysis with “10 commandments” for physicians is published in the 28 December 2011 article The idolatry of the Surrogate. The article, unfortunately, is only available through paid subscription through the BMJ (British Medical Journal – Helping Doctors Make Better Decisions).***
The commandments basically encourage physicians to be careful with statistical results from clinical trials, information from industry experts, and to treat and respect the patient as individual with treatment related statistics as guides.

On a related note, I am very impressed with folks who empower themselves in treatment decisions by keeping up with biomedical breakthroughs, new treatments, and new ways of looking at diseases. I have posted related blogs as ePatients: The hackers of the healthcare world and Meet e-patient Dave – a voice of patient engagement.

Here is the abstract of the article
Easier to measure surrogate outcomes are often used instead of patient important outcomes such as death, quality of life, or functional capacity when assessing treatments. John Yudkin, Kasia Lipska, and Victor Montori argue that our obsession with surrogates is damaging patient care
Diabetes care is largely driven by surrogates. The US Institute of Medicine defines surrogates as “biomarker[s] intended to substitute for a clinical endpoint [and] expected to predict clinical benefit (or harm . . .) based on epidemiologic, therapeutic, pathophysiologic, or other scientific evidence.”1 In diabetes, concentrations of glycated haemoglobin (HbA1c) are used as a surrogate marker for outcomes that are important to patients, such as blindness or amputation. Other surrogates such as blood pressure, lipids, albumin excretion rates, and C reactive protein have been used to predict outcomes of cardiovascular disease and to guide clinical practice in people with or without diabetes. Much of the evidence for clinical interventions is based on their effect on surrogate outcomes rather than those that matter to patients such as quality of life or avoidance of vision loss or renal failure. Moreover, because these “hard” end points generally show much smaller responses to interventions than surrogate markers, many of the widely accepted strategies for diabetes may be based on artificially inflated expectations.
Recent studies have challenged the assumption that reliance on surrogates can accurately predict the effect of treatment on hard outcomes. There are the oral hypoglycaemic drugs that reduce HbA1c but increase the risk of cardiovascular events,2 antihypertensive drugs that do not reduce the risk of stroke,3 and drugs that improve cholesterol profiles but do not reduce cardiovascular events.4 Explanations for such phenomena include unwanted effects of the drug or an incomplete understanding of the pathophysiology of the disease.5 But why have …

Below are listed the the ten commandments**** with definitions and paraphrasing. I have forgotten much more than I have remembered from a college statistics class 30+ years ago! The “explanations” are a result of finding quality information on the Internet.

(For a great “translation” with less math, please go to the blog posting …Get Your Doctor to Treat You Right)

The New Therapeutics: Ten Commandments

•     Thou shalt treat according to level of risk rather than level of risk factor.

Level of risk – these levels are experienced by everyone, not just those having the disease being treated
[good summary of risk levels (minimal, less than minimal, greater than minimal) ]

Risk factor -anything that makes it more likely you will get a disease, either something you do (smoking)or something you have no control over (as being over 50 makes it more likely you will get colon cancer.  People should be given treatments based on the risks associated with the treatment on anyone,not  individual factors (age, blood pressure, other conditions)

• Thou shalt exercise caution when adding drugs to existing polypharmacy.



Polypharmacy – (poly is Greek for many) Whenever a person is taking a drug, any additional drugs may interact and cause bad reactions, including death.

• Thou shalt consider benefits of drugs as proven only by hard endpoint studies.


Endpoint study – research study involving humans where the outcomes (results) directly address the question. For example, if a drug was tested on how it reduced heart disease, the hard endpoint would be the reduction of heart disease.  However, hard endpoint studies are usually not accomplished in short periods of time, because it takes time for diseases to develop.  This paragraph sums up endpoint workarounds well.
From Deciphering Media stories on Diet (Harvard.edu)
4. Did the study look at real disease endpoints, like heart disease or osteoporosis? Chronic diseases,
like heart disease and osteoporosis, often take many decades to develop. To get around waiting that
long, researchers will sometimes look at markers for these diseases, like narrowing of the arteries or
bone density. These markers, though, don’t always develop into the disease.

•     Thou shalt not bow down to surrogate endpoints, for these are but graven images.



Surrogate endpoint – a substitute endpoint in a clinical trial. It is not the item being measured directly (as  heart disease), but an item related to what is being studied (as blood pressure). During the study these substitutes will be used to check on the health of the people in the clinical trial, the usefulness of the drug being treated, and if there are any complications. Surrogate endpoints are substitutes for (true) clinical endpoints (as survival for 5 years after the treatment).  Surrogate endpoints don’t always guarantee a clinical endpoint (just because blood pressure goes down, heart disease may not be treated). However studies with good endpoints are expensive (require testing on many people) are take long periods of time.
[Adapted from an eHow article with good references]

•      Thou shalt not worship Treatment Targets, for these are but the creations of Committees.

Correlation doesn’t always meant causality!A treatment target is a goal of a treatment intervention.  An example would be to reduce a specific protein in order to prevent a specific cancer (Potential New Treatment Target for Retinoblastoma, 13 January 2012 Medscape article) .  The “cause/effect” relationship between something measurable (as a protein) and a disease may not truly exist.  It is also possible that the  presence of the protein and the onset of a disease may be due to other factors in a web of events.

• Thou shalt apply a pinch of salt to Relative Risk Reductions, regardless of P values, for the population of their provenance may bear little relationship to thy daily clientele.

Relative Risk – The number of times more likely (RR > 1) or less likely (RR < 1) an event is to happen in one group compared with another. [From the BMJ glossary]
 P value – a number used to show how a variable (as a drug treatment) has a different result thano variable (no treatments). So, a high P value would seem to point to an effective drug treatment.

• Thou shalt honour the Numbers Needed to Treat, for therein rest the clues to patient-relevant information and to treatment costs.

The Numbers Needed to Treat (NNT) tells how many patients need a specific treatment in order to prevent an additional bad outcome (as a heart attack or stroke). So if a drug has an NNT of 10, 10 people have to be treated with the drug in order to prevent one additional bad outcome.
For example, if a drug is found to reduce the risk of a bad outcome from 50% to 40%, the absolute risk .1 (the difference). And the NNT is the inverse of the absolute  or 10.  [From Numbers Needed to Treat (Patient.co.UK)

•  Thou shalt not see detailmen, nor covet an Educational Symposium in a luxury setting.

Detailmen are pharmacy representatives who present doctors with their company’s drug information with the aim of persuading the doctor to presribe their drugs. These representatives often sponsor educational talks (often “focusing” on conditions rather than drugs) at physician meetings. These “luxury settings” may included free buffets and hospitality rooms. [From Influencing Doctors :How Pharmaceutical Companies Use Enticement to ‘Educate’ Physicians (ABC News)]

•    Thou shalt share decisions on treatment options with the patient in the light of estimates of the individual’s likely risks and benefits.

•  Honour the elderly patient, for although this is where the greatest levels of risk reside, so do the greatest hazards of many treatments.

*** Click here for suggestions on how to get this article for free or at low cost.
In past blogs I have posted items on initiatives for the wider sharing of scientific articles to the public with subsidies, open access, etc.
****Richard Lehman’s Journal Review – 3 January 2012