HealthNewsReview.org | Independent Expert Reviews of News Stories | Holding Health and Medical Journalism Accountable |
|
|
This criterion may apply most often in regard to news coverage about drugs in clinical trials. We believe that journalists writing about an investigational drug (or an investigation into new uses for an existing drug) should provide their readers with information about what phase of research the drug is in.
-
An unsatisfactory score may be given because the story may:
- Not be clear that the drug is not currently available.
- Treat FDA approval of an investigational drug as a fait accompli.
- Make predictions about how the drug "could be" or "should be" approved and on the market within a given time frame.
- Make predictions about a time frame for future availability using only a company spokesman as a source.
- But the criterion also applies to questions of availability of a new test, device or procedure.
- An Unsatisfactory story may:
- Fail to give a sense of how widespread is the adoption of the approach.
- Fail to address issues like the availability of trained personnel to deliver the approach.
Journalists should provide appropriate coverage of the likely cost to the individual or community of new treatments, tests, products or procedures. The proposed selling price is likely to be known at the time of launching of a new drug, although new procedures may be more difficult to cost. Journalists should be able to get an estimate from their sources.
-
An unsatisfactory score may be given because the story may:
- Fail to discuss the costs of an approach.
- Downplay cost as an issue without satisfactorily explaining why
- Fail to satisfy the reviewer's desire to compare the costs of a new approach with the costs of existing alternatives.
- Fail to address whether a new approach is likely to be covered by insurance.
- A Not Applicable score may be given if:
- The story was about an experimental approach that is very early in its development and does not cite costs. However, this is a judgment call.
Our rule of thumb: If it's not too early to talk about how well something might work, then it.s not too early to start discussing what it may cost. - The reviewer believes that a discussion of costs is relatively unimportant (e.g., things whose cost most people would know such as aspirin, dietary factors, etc.).
This item tries to analyze news stories that exaggerate or over-sell a condition. There are different forms of "mongering" - turning risk factors into diseases (e.g., low bone mineral density becomes osteoporosis); misrepresentation of the natural history and/or severity of a disease (e.g., early-stage low-grade prostate cancer); medicalization of minor or transient variations in function (e.g. temporary erectile dysfunction or female 'sexual dysfunction'); medicalization of normal states (baldness, wrinkles, shyness, menopause); or exaggeration of the prevalence of a disorder (e.g., using rating scales to 'diagnose' erectile dysfunction). A news story may contain more than one example of mongering.
Identifying disease mongering is a matter of judgment. Sometimes it is obvious. But how do you decide if an article on irritable bowel disorder, erectile dysfunction, restless leg syndrome or osteoporosis (all of which can be serious for some sufferers) is misrepresenting the condition to the public?
-
An unsatisfactory score may be given because the story may:
- Present spurious statistics.
- Exaggerate the human consequences.
- Create undue fear.
- Treat surrogate markers endpoints as if they were disease.
- Include interviews with "worst-case" patients - holding such patients up as examples as if their experiences were representative of all with this condition.
We are dealing with stories about new treatments, tests, products or procedures; the relevant standard is that the journalist is expected to present information about the quality of the clinical evidence on which claims are based. In the case of a major claim about the efficacy of a new treatment this should be based on the results of a randomized trial with relevant clinical endpoints. Sometimes, the first story about a new treatment is in the form of .promising. results from a case series. There is nothing intrinsically wrong with writing about this, but the story should make clear the limitations of the evidence, and contain a caution about interpretation of uncontrolled data. For example, a story about a non-randomized cohort or observational study should explain that researchers were not able to adjust for all factors that might account for an observed difference. The hierarchy of evidence is an important factor for journalists to consider, and to explain to readers and viewers, and we should look for it in these stories.
-
An unsatisfactory score may be given because the story may:
- Fail to point out the limitations of the evidence.
- Fail to include a caution about interpretation of uncontrolled data.
- Fail to make clear the limitations of a small study.
- Fail to point out if a primary outcome is a surrogate marker or fail to caution readers/viewers about extrapolating this to health outcomes.
- Fail to point out the limited peer review that may have taken place with findings presented at a scientific meeting.
- Present findings from an animal or lab experiment without cautioning readers/viewers about the limited applicability to human health.
- Present anecdotes as evidence of a treatment's harms or benefits - rather than as a single illustration of its use.
The topic of the quality of the evidence may also be addressed in the "Our Review" summary comments area, as a question of whether the story should have been assigned at all.
The expectation is that the journalist will always mention potential adverse effects of any new treatment, test, product or procedure that is being covered in a story. Ideally the story should mention both the frequency and severity of adverse effects of treatments. Some so-called "minor" side effects may have dramatic impact on individuals' lives. Balanced coverage should take even "minor" side effects into consideration. If we feel the reporting minimized the potential harms of treatment, we will give the story an Unsatisfactory rating on this criterion. Further, if there is no attempt to quantify the potential for harms, this would be unsatisfactory. Patient anecdotes alone are not sufficient.
-
An unsatisfactory score may be given because the story may:
- Fail to mention potential harms.
- Fail to quantify potential harms.
- Fail to describe the severity of potential harms.
- Fail to account for "minor" side effects that could have a significant impact on a patient's life.
- Rely too heavily on a patient anecdote about safety.
- Rely too heavily on an investigator's comment that an approach appears to be safe - without supporting data.
Many "new" treatments, tests, products or procedures are not really novel. The agent may be another member of a well-established therapeutic class of drug; even if it represents a new class, it may offer no more than the drugs that are widely available. In the press release for a new drug this essential information may be lost in the hype and the drug is portrayed as "novel" in order to increase initial sales. We believe that journalists should be able to make these distinctions by doing their research or by putting appropriate questions to independent sources; they should put an informative statement in their story about the novelty of a new product.
-
An unsatisfactory score may be given because the story may:
- In the judgment of the reviewers, make inaccurate, misleading or incomplete statements about the novelty of a treatment, test, product or procedure.
- Fail to establish what's new about the new approach compared with existing alternatives.
- Describe what is clearly an off-label use of an approved drug without making clear that this is the case.
The expectation is that the journalist will report a quantitative estimate of the benefits of a new treatment, test, product or procedure; generally this information is reported in the trials that are the basis of the therapeutic claims in the news story. However it is important that the information is communicated accurately and in a way that helps readers understand the true value of the treatment. Surveys have shown that journalists have a tendency to report benefits as the relative reduction in the frequency of an adverse outcome. For instance a reduction in the risk of hip fracture with treatment from 2% to 1% might be reported as a "halving" or a '50% reduction'. In such situations we believe that the journalist should report the change in absolute risk (this includes reporting the risk with and without treatment) and/or a derived variable such as the Number Needed to Treat (NNT). In some cases, for instance, in a preliminary uncontrolled case series there may be no comparative data to report, so a score of N/A may be applied.
-
An unsatisfactory score may be given because the story may:
- Fail to quantify benefits.
- Use only relative, not absolute, risk or benefit data.
- Rely too heavily on what may be unrepresentative patient anecdotes about benefits.
- Cite statistics appropriately, but allow even a single unchallenged exaggerated quote to throw the story out of balance.
It is simply not sound journalistic practice to lift material directly from a news release without stating that this is the case. This criterion will rarely be employed. If, on the front end, we know that a news release was sent out and if we can get a copy of that release, we will review it along with the story.
- An Unsatisfactory story may:
- For a story to be rated unsatisfactory, we must have clear evidence that the exact same wording from a news release appears in a story.
- Lacking such evidence, the worst a story can rate is N/A.
- With a story that has several independent sources, we can safely assume that it did NOT rely on a news release and thus, give it a Satisfactory score.
In order to get a satisfactory score on this criterion, two things must occur: There must be an independent expert source quoted (someone not directly connected with the research) and there must be some attempt to let the public know about potential conflicts of interest.
-
An unsatisfactory score may be given because the story may:
- Fail to identify the source of the story (news release, journal article, editorial, scientific meeting presentation, etc.) and fail to make obvious the extent to which that source is likely to be conflicted (e.g., .a PR consultant working for the company said.. or .Dr Smith, who received a company grant to perform the study said..)
- Fail to include the input of a trusted independent source with expertise on the subject to comment on the claims made. (Ideally, more than one such independent source would be used.)
The story should put the idea being discussed into the context of existing alternatives, mentioning availability of alternative treatments or tests or procedures with some assessment of their relative performance. If the treatment is genuinely novel, then the article should mention that there is no alternative treatment, or that only general supportive/symptomatic care has been provided up until now.
-
An unsatisfactory score may be given because the story may:
- Discuss a surgical approach without mentioning non-surgical alternative approaches.
- Discuss a new test without mentioning other tests that are available, including the option of not being screened in the case of a screening test.
- Fail to discuss the advantages/disadvantages of the new idea compared with existing approaches.
- Fail to discuss how the new treatment, test, product or procedure fits into the realm of existing alternatives.
The U.S. Preventive Services Task Force is considered the gold standard of preventive health recommendations - including on screening tests. It's a good source for journalists and consumers.
About 70% of the stories reviewed from 2006-9 failed to adequately discuss costs, or to explain how big (or small) are the potential benefits and harms of treatments, tests, products and procedures.
We have documented a disturbing trend of news stories taking an advocacy stance, promoting certain screening tests outside the boundaries of scientific evidence.
Stories on new technologies like Cyberknife, DaVinci robotic surgery systems, and proton beam cancer therapy often fail to scrutinize the evidence and/or to discuss the costs involved.
Rather than suggesting that everyone should be screened for everything, news stories could explain: "All screening tests cause harm; some may do good."
The first 38 network TV network morning health news stories reviewed in 2009 earned an average score of 1.2 stars. 13 of the 38 stories got ZERO stars.
Both TIME magazine and BusinessWeek have published terrific stories explaining the importance of the Number Needed to Treat - or NNT.
Knowing relative risk reduction is like knowing you have a 50% off coupon but not knowing whether it's for a Lexus or a lollipop. Absolute risk reduction tells you what the "coupon" is worth. Read more.
The website NoFreeLunch.org posts "a database of health care professionals who have pledged to accept no gifts from industry and to rely on non-promotional sources of information."
To help journalists cover stories responsibly, we post a list of independent experts who state that they do not have financial ties to drug or medical device manufacturers.
We apply the same ten standardized criteria to the review of every story.
We have about 30 story reviewers. Each story is reviewed by 3 different people.
Gary Schwitzer's seven words you shouldn't use in medical news: cure, miracle, breakthrough, promising, dramatic, hope, victim. Read why.
Our reviewers include two former CNN medical reporters and a former editor of the Washington Post health section.