Intuitive Judgment of Statistical Significance

When determining statistical significance in null hypothesis testing, how do sample size and relationship strength interact?

A researcher finds only a very weak correlation between sleep quality and exam performance, yet the result is statistically significant. This outcome is plausible if the study used a very large sample of participants.

In psychological research, statistical significance depends on a trade-off between sample size and relationship strength. Match each hypothetical research scenario with the statement that best explains its likely statistical outcome.

In null hypothesis testing, the statistical significance of a result is determined by the trade-off between the number of participants and the strength of the relationship. Rank the following hypothetical research studies in order from the study that requires the strongest observed relationship to reach statistical significance (1) to the study that requires the weakest observed relationship to reach statistical significance (4).

Imagine you are developing a research proposal to investigate a rare psychological condition. Because the condition is so rare, you can only construct a sample of $10$ participants ($N = 10$). To formulate a design that is still capable of reaching a statistically significant result ($p < .05$), which of the following strategies should you prioritize in your plan to address the trade-off between sample size and relationship strength?

In null hypothesis testing, a weak statistical relationship can never reach statistical significance, even if the sample size is extremely large.

A researcher evaluates a study and concludes that its significant $p$-value ($p < .05$) proves a 'powerful and highly impactful' effect, despite the study having 30,000 participants and a relationship strength of only $r = .02$. This conclusion is likely an overestimation because it fails to account for the _____ between sample size and relationship strength in null hypothesis testing.

Due to the trade-off in null hypothesis testing, a very strong statistical relationship can reach statistical significance even if the study's _____ is small.

Based on the trade-off between sample size and relationship strength in null hypothesis testing, match each research scenario with the most appropriate methodological application.

In null hypothesis testing, the likelihood of a result being statistically significant depends on how the sample size and relationship strength combine. Analyze the four hypothetical study outcomes below and arrange them in order from the study that is MOST likely to achieve statistical significance (1) to the study that is LEAST likely to achieve statistical significance (4).