Don't make such black and white interpretations

Factors Determining the p-value

Test Statistic

Misinterpretations of the p-value

Alpha (Level of Significance)

p-hacking

The 2015 Ban on Null Hypothesis Testing

Alpha Criterion

What does a p value represent in the context of psychological research?

If a psychology researcher calculates a p-value of 0.08, they will typically declare the experimental result to be statistically significant.

A social psychologist is analyzing data from four different studies. Based on the standard thresholds for statistical significance provided in the table, match each study's obtained p value with its correct interpretation.

Misinterpretation of the p Value

Determinants of the $p$ Value

A researcher obtains a $p$ value of $0.02$ and labels the result 'highly significant' ($**$). According to standard significance thresholds, where $* = p \leq 0.05$ and $** = p \leq 0.01$, this labeling is _____.

A cognitive psychologist conducts an experiment on learning styles and calculates a $p$ value of $0.04$. Applying the standard threshold of significance ($p = 0.05$) described in the text, the researcher should declare this result to be statistically significant.

Define what a $p$ value represents in the context of an experiment and state the standard threshold for declaring an experimental result to be statistically significant.

Based on the standard definition of a $p$ value and its typical threshold, explain in your own words what it means if the researchers determined the probability of their outcome occurring purely by chance was less than 1 in 20.

A psychology student runs an analysis on their study's data and discovers the probability of their results occurring purely by chance is $0.03$. Applying the standard threshold of significance, what conclusion should the student draw about their experimental result, and why?

Reporting Effect Sizes in Null Hypothesis Significance Testing

Using the convention stated in this question for marking results with asterisks (a cut-off value itself counts as meeting it), match each label with the range of $p$ values it marks.

By a common convention in psychology, a result is declared statistically significant when $p \leq 0.05$. For a result exactly at this threshold ($p = 0.05$), results at least as extreme would occur, under the test's model (the null hypothesis plus the test's assumptions), $1$ time in how many?

By a common convention in psychology, a result is declared statistically significant when $p \leq 0.05$. For a result exactly at this threshold ($p = 0.05$), if the null hypothesis and the test's assumptions were true, results at least as extreme would occur 1 time in _____.

Arrange the following three $p$ values in ascending numerical order, from the smallest (1) to the largest (3).

A psychologist has $p$ values from four different studies. Arrange the $p$ values in ascending numerical order, from the smallest to the largest. (This orders the numbers only: $p$ values from different studies, each calculated under its own model, do not by themselves rank how strong the evidence is.)