Learn Before
p value
A p value is the probability, calculated under a specified statistical model, of getting a result at least as extreme as the one observed. The model combines the null hypothesis with the test's other assumptions, such as how the data were collected and how they are distributed. A p value is not the probability that the null hypothesis is true, and it is not the probability that the result occurred purely by chance. A small p value means the data would be unusual if the whole model were true; on its own it cannot say whether the null hypothesis or another assumption is at fault. By a common convention in psychology, a result is called statistically significant when p is .05 or less, that is, when p is at or below a significance level (alpha) of .05: under the model, results at least as extreme as the observed one would occur no more than 1 time in 20. A p value does not measure how large or important an effect is, and a conclusion should not rest only on whether p passes a threshold. The table below lists conventional descriptors for ranges of p; its third column shows typical but incautious interpretations that claim more than a p value can show, not correct readings.
Source: Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133; section 2 and principles 1–3, 5 and 6 (pp. 131–132). https://doi.org/10.1080/00031305.2016.1154108
0
2
Tags
User experience (UX) research
Design Science
KPU
Research Methods in Psychology - 4th American Edition @ KPU
Related
Learn After
Don't make such black and white interpretations
Factors Determining the p-value
Test Statistic
Misinterpretations of the p-value
Alpha (Level of Significance)
p-hacking
The 2015 Ban on Null Hypothesis Testing
Alpha Criterion
What does a p value represent in the context of psychological research?
If a psychology researcher calculates a p-value of 0.08, they will typically declare the experimental result to be statistically significant.
A social psychologist is analyzing data from four different studies. Based on the standard thresholds for statistical significance provided in the table, match each study's obtained p value with its correct interpretation.
A psychologist is comparing findings across four different research studies. Based on the definition of a p value and the significance thresholds provided in the table, arrange these findings in order from the outcome LEAST likely to have occurred by chance to the outcome MOST likely to have occurred by chance.
Misinterpretation of the p Value
Determinants of the Value
A researcher obtains a value of and labels the result 'highly significant' (). According to standard significance thresholds, where and , this labeling is _____.
A cognitive psychologist conducts an experiment on learning styles and calculates a value of . Applying the standard threshold of significance () described in the text, the researcher should declare this result to be statistically significant.
Evaluate the following three experimental results and arrange them in order from the outcome representing the lowest probability of occurring purely by chance (1) to the outcome representing the highest probability of occurring purely by chance (3).
Define what a value represents in the context of an experiment and state the standard threshold for declaring an experimental result to be statistically significant.
Based on the standard definition of a value and its typical threshold, explain in your own words what it means if the researchers determined the probability of their outcome occurring purely by chance was less than 1 in 20.
A psychology student runs an analysis on their study's data and discovers the probability of their results occurring purely by chance is . Applying the standard threshold of significance, what conclusion should the student draw about their experimental result, and why?
Reporting Effect Sizes in Null Hypothesis Significance Testing
Using the convention stated in this question for marking results with asterisks (a cut-off value itself counts as meeting it), match each label with the range of values it marks.
By a common convention in psychology, a result is declared statistically significant when . For a result exactly at this threshold (), results at least as extreme would occur, under the test's model (the null hypothesis plus the test's assumptions), time in how many?
By a common convention in psychology, a result is declared statistically significant when . For a result exactly at this threshold (), if the null hypothesis and the test's assumptions were true, results at least as extreme would occur 1 time in _____.