The Meaning of Significant in MS Studies

scientist evaluating clinical trial data on computer
Jason Butcher/Cultura/Getty Images

What does "significant" mean? In a clinical trial of a medication's safety and effectiveness, the finding that the resulting data (information) is "statistically significant" is a scientific way of saying that the result was unlikely to have happened by chance. Therefore, the result is likely due to the effects of the medication that was studied. 

Understanding P-Values

Of course, coming to that conclusion is not as simple as it sounds. Researchers typically use a well-known and reliable statistical method to measure and evaluate results from study to study. It's called the "p-value" and it measures the likelihood that a study's results happened by chance. 

The p-value provides a percentage of that likelihood, based on statistical tests of the study results. So, if a p-value is 0.01, there is a 1% likelihood that the result was due to chance and a 99% chance that it wasn't—that, instead, it was due to the effect of the medication.

The most common cut-off for p-values is 0.05 – that is, if a p-value is 0.06, then it is considered not statistically significant. On the other hand, if a p-value is 0.04, then the result is statistically significant.

What Is the "Null Hypothesis"?

You may know that the word "null" is associated with "zero." In this type of statistical measurement, researchers start by assuming zero difference between, for example, a new medication and an older one. This may seem odd since they want to find out if the new one is better than the older one. But it works. Here's how:

Let's say a study is indeed designed to see if a new medication is better than an older one. The null hypothesis is stated as, "There is no difference in the effect (patient outcome) between new medication X and older medication Y." A p-value of 0.04 then translates into: Based on the study data, there is a 4% chance that there is no difference between the two medications. Of course, that means that there's a 96% chance that there is a difference between them.

What Does "Significant" Mean? A Real-Life Example 

To use a real example, let’s look at the REGARD study of people with multiple sclerosis (MS). This study compared the medication Copaxone to Rebif.

One outcome (result) studied was the amount of time that passed before patients' first MS relapse after 96 weeks of being on the medications. (The research term for this is "time to first relapse.") The p-value for this difference was p=0.64, meaning that, because the p-value was higher than 0.05, there was no statistically significant difference between the times until the first relapse in patients on either medication. Stated differently, there was a 64% chance that there was no statistically significant difference.

However, another outcome studied was the number of active lesions that were seen on MRI scans of the two groups. It turned out that study participants treated with Rebif had an average of 0.24 MS lesions per scan, while those taking Copaxone had an average of 0.41 lesions per scan. In this case, p=0.0002, meaning this was a statistically significant finding.

What Does "Significant" Mean to Individual Patients and Their Doctors?

It's important to keep in mind that "statistically significant" does not necessarily mean that something is clinically significant or meaningful to individuals. For instance, the difference in the numbers of active MS lesions in the study discussed above is small, even though it's statistically significant. So it probably wouldn't be the main reason a doctor chooses one of the medications over the other. The doctor might give other factors more weight in the treatment decision. For example, the medications' side effects, cost, and injection frequency.

Things to Keep in Mind When Looking at a Clinical Study Report

As you might suspect, there are many more factors (for example, how many participants are studied or how the outcomes are measured) that can influence a clinical study's final p-value results. Nevertheless, knowing what p-values mean is a huge advantage for understanding what the information from a clinical study means for researchers, doctors, and patients.

Read at least the abstract (short summary) of the study. It can provide more details about a medication than you can get from a one-line blurb in a marketing piece or a brochure headline. 

Was this page helpful?

Article Sources

  • Mikol DD, Barkhof F, Chang P, Coyle PK, Jeffery DR, Schwid SR, Stubinski B, Uitdehaag BM; REGARD Study Group. Comparison of subcutaneous interferon beta-1a with glatiramer acetate in patients with relapsing multiple sclerosis (the REbif vs Glatiramer Acetate in Relapsing MS Disease [REGARD] study): a multicentre, randomised, parallel, open-label trial. Lancet Neurol. 2008 Oct;7(10):903-914.