Review Question - QID 104026

In order to understand the idea of P value, it is first important to consider the two possible results from any study. Using this study as an example, the first possibility is that statin A and statin B are equivalent, that is to say on average they will have the same effect. This possibility is known as the null hypothesis. The other possibility is that there is a difference between statin A and statin B, which is to say that on average one will be better than the other. This possibility is known as the alternative hypothesis.

In this study, we see that the group taking statin A does better than the other group, but we do not know if that difference is due to the drugs or whether the difference is due to random chance. Therefore, to decide whether there is a real difference in the drugs, we ask "What is the likelihood that we would have gotten this result just by random chance if the two drugs are the same (the null hypothesis)?". The answer to this question is known as the p-value, which is defined as the likelihood of getting this result or a more extreme result if the null hypothesis is true. The smaller this number, the less likely it is that the result was due to chance alone. Therefore in this case a p value of 0.052 means that there is only a 5.2% chance that this reduction of 11 or a greater reduction would be seen even if the drugs are the same.

Biau et al. discuss the p-value and hypothesis testing. A common misunderstanding is to consider the p-value as the probability the null hypothesis is true instead of the probability of obtaining the observed value or one that is more extreme, assuming the null hypothesis is true.

Goodman reviews several common p-value misconceptions. Interpretation of a p-value should never occur in a vacuum; consideration should be given to the context of the experiment as well. The probability of a conclusion cannot be calculated from the data from a single experiment; rather, one must reference external evidence. The Bayes factor is often a superior alternative to a p-value due to its simpler interpretability; however, this statistical tool has yet to gain sufficient popularity to be put into widespread use in the medical literature.

Illustration A shows a graphical representation of the p-value. The bell curve depicts a normal probability distribution; the p-value is depicted by the shaded green area under the curve and represents the probability that a given observed data point (or a more extreme value) is due to chance.

Incorrect Answers:
Answer 1: This answer choice most nearly resembles the definition of a 95% confidence interval. This statement in the answer choice does not reflect the true definition of a p-value. A p-value itself cannot be used to directly determine whether there is a real difference between two values.
Answers 3&5: These do not reflect the correct definition of a p-value, which is the probability that results as or more extreme than those seen in the study would be observed if the null hypothesis were true.
Answer 4: Statistical significance depends on the cut-off value set by the researchers and/or statisticians. By convention, statistical significance is usually reached when the calculated p-value is less than 0.05.