Making Decisions With the Research Evidence
This page can support you in making decisions using the research evidence-based content on this website and from other sources. Unless otherwise stated, the evidence about treatment effect on this site has been assessed by our team to be strong and relevant. Assessment of the evidence is based on the qualities listed below and whether there is sufficient evidence to make claims about treatment effect. If there are any reasons to be cautious about the evidence, it will be explicitly stated in the article.
Quality of research evidence
The type of research study and features of study design affect the quality of the evidence produced. When considering all the evidence from all the studies on a topic, the quality and features of each study contribute to the overall quality of evidence on that topic. Some factors to look at when considering a body of evidence include:
Sample size
The sample size is how many participants are in a research study. Small sample sizes may not have enough power to show whether a treatment works. Larger sample sizes are more likely to reflect the general population and accurately show the effect of a treatment.
Blinding, control groups, and randomization
Blinding, control groups, and randomization are strategies used to minimize bias in study results. Randomized controlled trials (RCTs) are a study type that uses all three of these strategies to try and eliminate the effect of confounding factors and bias in research results. RCTs are the gold standard for studying treatments for this reason.
Amount of research studies
The amount of research studies is important because each new study can validate, verify, or contradict the results of previous studies. If there are many studies on a specific topic with consistent results, the evidence is more likely to be reliable and be applicable to a more general population.
Consistency of results
Consistency is whether all the studies on a topic have similar results. When different studies produce opposing results and there is no explanation for the inconsistency, one should be cautious about making decisions using the evidence.
Systematic reviews and meta-analyses
Systematic reviews and meta-analyses combine the findings from all the studies on a topic together. This includes doing a systematic search for all the studies that address that topic, assessing the quality of each study, and interpreting the combined findings of all the studies. Bias and error can still be found in systematic reviews and meta-analyses, but they are an excellent place to start looking at the evidence if available.
For more in-depth descriptions of the types of research studies and study design features, please see our article on Understanding Research Evidence.
Other forms of evidence
The following types of evidence originate outside of research but are important in the interpretation of research evidence and decision-making, especially if there is no or limited research available on a particular issue.
Expert opinion
Expert opinion is a view or statement on a topic from a person who is an expert in the given field, based on clinical experiences or reasoning using foundational medical principles.
Clinical consensus
Clinical consensus is a statement of agreement from a group of experts based on the best available evidence and expert clinical reasoning, that has taken contradictory opinions and biases into account.
Lived experience
Lived experience is the knowledge a person gains from direct, first-hand experience. There is value in understanding the impact and meaning of direct experiences for the development of research and treatments. Views of the same experience vary based on the person, their unique experience, and their environment.
Research in spinal cord injury
Low study participation
You may notice that many SCI studies have small sample sizes. SCI is not a common condition, so the number of people with SCI in a given location is often small. Even within that population, the level of injury and level of function will be very diverse. To make sure that this variation in injury types does not impact research results, studies often have strict participant criteria that require specific ranges for the level of injury, level of function, time since injury, secondary health issues, medication use, etc. Also, people with SCI are more likely to have trouble accessing and maintaining participation in a study due to transportation, mobility, and ongoing health issues. All these factors contribute to the small sample sizes in studies of SCI and SCI treatments.
RCT challenges
Although RCTs are considered the gold standard for treatment research, they may not always be possible or ethical. There are ethical concerns over some of the strategies used in RCTs to reduce study bias when used for certain treatments. For example, randomizing participants to a non-treatment group in an exercise study, when it is commonly known that exercise is beneficial to health could be considered unethical. Invasive procedures such as surgeries are also difficult to study in RCTs because blinding participants might require a “sham” treatment (e.g. prepping the patient and making incisions but not doing the procedure). If a sham treatment is invasive and carries some risk, recruitment of willing participants in an already small participant pool becomes even more challenging. It can also be difficult to recruit participants for a matched control group because of large variations in level of function, and level of injury.
Questions to ask before making a decision:
- Does this address your problem?
- Based on the potential risks and benefits, is this suitable for you? (make a pros and cons list!)
- Is this accessible for you? (finances, location, transportation)
- How will this impact your life? (work, school, activities)
- Do you have sufficient social, emotional, and physical supports? (family, friends, caregivers, other supports)
- What are your personal preferences/goals?
- What questions do you have?
- What are the next steps that need to be taken?
