Researching the evidence
For each therapy, we search published medical studies for evidence of the following:
- Does this therapy improve treatment outcomes? (Improving Treatment Outcomes)
- For example, improving survival or reducing cancer growth
- For example, improving survival or reducing cancer growth
- Does this therapy help manage side effects? (Managing Side Effects)
- For example, reducing pain or nausea, or reducing anxiety or depression
- For example, reducing pain or nausea, or reducing anxiety or depression
- Does this therapy help reduce the risks of cancer or recurrence of cancer? (Reducing Cancer Risk/Recurrence)
- For example, reducing incidence or severity of precancerous lesions or increasing time before recurrence
- Does this therapy help your body be less hospitable to cancer? (Optimizing Body Terrain)
- For example, reducing inflammation, reducing insulin resistance, or improving hormone balance
- What are the risks associated with this therapy? (Safety)
- For example, interactions with traditional cancer treatments or side effects of the therapy
Strength of evidence
We review the published literature and assess the strength of evidence based on the quality of the studies that are available. Study quality is assessed based on two criteria: study design and study size.
We focus on clinical (human) studies since these are most likely to represent real world, relevant effects. However, if no clinical studies have been done using this therapy, we may include evidence from lab-based (in vitro) or animal studies.
For clinical studies, the study designs we consider from strongest to weakest are:
- Meta-analysesa statistical analysis that combines the results of two or more research studies; the results of smaller research studies addressing the same or similar questions can be analyzed as though they are one bigger, more powerful study of RCTs or observational studies
- Randomized controlled trials (RCTs)a study design in which people are randomly assigned to either an experimental group or a control group to compare the outcomes from different treatments; an RCT is considered a strong design for determining a therapy’s effects
- Observational studiesa type of study in which individuals are observed or certain outcomes are measured, but no attempt is made to affect the outcome (for example, no treatment is given); an example is a study that records people’s diets, but doesn’t try to alter their diets, and looks for patterns of disease or other outcomes related to different foods
- Case studiesa descriptive and exploratory analysis of a person, group, or event regarding changes observed over time; because changes due to treatment are not compared to similar changes over time without treatment, a case study is considered a weak study design or other uncontrolled studiesa study in which a therapy is used, but without a comparison group against which to judge outcomes; an uncontrolled study is considered a weak study design
Larger studies offer more confidence in findings because their results are less likely to be impacted by chance or unexpected influences. Study sizes are based on the number of participants and type of study:
- Small: <100 participants
- Mid-sized: between 100-1,000 for observational studies; 100-500 for RCTs
- Large: between 1,000-5,000 for observational studies; 500-1,000 for RCTs
- Very large: >5,000 for observational studies; >1,000 for RCTs
Therapy ratings
Ratings are based on the quality (design and size) and number of studies and the degree of consistency of reported effects among the available studies. In other words, whether many or most studies point to the same conclusion. For ‘Improving Treatment Outcomes’ and ‘Managing Side Effects and Promoting Wellness’ we use the following ratings:
- 0=No evidence: No quality relevant studies available OR multiple studies available but substantially conflicting evidence of effect
- 1=Weak evidence: Few preclinical or small clinical studies showing consistent effect OR several studies with some conflicting evidence, but an overall trend in one direction
- 2=Modest evidence: Several small or at least one mid-sized RCT or observational study AND most of the available evidence pointing in one direction
- 3=Good evidence: Several mid-sized or at least one large, well-designed RCT or observational study AND little to no conflicting evidence compared to the overall evidence available
| Study Quantity and Quality | Consensus (agreement across studies) | ||
|---|---|---|---|
| 0: No evidence | No quality relevant studies available | OR | Multiple studies available but substantially conflicting evidence of effect |
| 1: Weak evidence | Few preclinical or small clinical studies showing consistent effect | OR | Several studies with some conflicting evidence, but an overall trend in one direction |
| 2: Modest evidence | Several small or at least one mid-sized RCT or observational study | AND | Most of the evidence pointing in one direction |
| 3: Good evidence | Several mid-sized or at least one large RCT or observational study | AND | Little to no conflicting evidence compared to overall evidence available |
Previous CancerChoices rating system
We previously used a 0-5 scale system, some of our reviews have not been updated to this scale. Read about the original system here >