Don't make such black and white interpretations

Factors Determining the p-value

Test Statistic

Misinterpretations of the p-value

Alpha (Level of Significance)

p-hacking

The 2015 Ban on Null Hypothesis Testing

Alpha Criterion

What does a p value represent in the context of psychological research?

If a psychology researcher calculates a p-value of 0.08, they will typically declare the experimental result to be statistically significant.

A social psychologist is analyzing data from four different studies. Based on the standard thresholds for statistical significance provided in the table, match each study's obtained p value with its correct interpretation.

Misinterpretation of the p Value

Determinants of the $p$ Value

A researcher obtains a $p$ value of $0.02$ and labels the result 'highly significant' ($**$). According to standard significance thresholds, where $* = p \leq 0.05$ and $** = p \leq 0.01$, this labeling is _____.

A cognitive psychologist conducts an experiment on learning styles and calculates a $p$ value of $0.04$. Applying the standard threshold of significance ($p = 0.05$) described in the text, the researcher should declare this result to be statistically significant.

Define what a $p$ value represents in the context of an experiment and state the standard threshold for declaring an experimental result to be statistically significant.

Based on the standard definition of a $p$ value and its typical threshold, explain in your own words what it means if the researchers determined the probability of their outcome occurring purely by chance was less than 1 in 20.

Reporting Effect Sizes in Null Hypothesis Significance Testing

Using the convention stated in this question for marking results with asterisks (a cut-off value itself counts as meeting it), match each label with the range of $p$ values it marks.

By a common convention in psychology, a result is declared statistically significant when $p \leq 0.05$. For a result exactly at this threshold ($p = 0.05$), results at least as extreme would occur, under the test's model (the null hypothesis plus the test's assumptions), $1$ time in how many?

By a common convention in psychology, a result is declared statistically significant when $p \leq 0.05$. For a result exactly at this threshold ($p = 0.05$), if the null hypothesis and the test's assumptions were true, results at least as extreme would occur 1 time in _____.

Arrange the following three $p$ values in ascending numerical order, from the smallest (1) to the largest (3).

A psychologist has $p$ values from four different studies. Arrange the $p$ values in ascending numerical order, from the smallest to the largest. (This orders the numbers only: $p$ values from different studies, each calculated under its own model, do not by themselves rank how strong the evidence is.)

A psychology student runs an analysis on their study's data and obtains a $p$ value of $0.03$. Using the common convention of a significance level of $0.05$ (a result is significant when $p \leq 0.05$), what conclusion should the student draw about their experimental result, and why?