What is the primary risk associated with conducting multiple unplanned exploratory analyses in a research study?

If a researcher discovers an interesting, unexpected relationship during an exploratory analysis, they should present it as a planned analysis in their research report to make the finding appear more robust.

Match each research scenario or outcome with the appropriate concept related to distinguishing between planned and exploratory data analyses.

A researcher discovers an unexpected relationship in their data that was not part of their original hypotheses. Arrange the following steps in the correct scientific sequence to ensure the finding is reported transparently and validated properly.

You are designing a manuscript template for a psychology lab to ensure that researchers properly differentiate between planned and exploratory analyses. Which organizational strategy for the 'Results' section best constructs a report that maintains scientific transparency when unexpected relationships are discovered?

Match each term related to the differentiation of research analyses with its correct definition.

To accurately evaluate a study's claims, a researcher must identify which analyses were unplanned, because conducting multiple unplanned tests on the same dataset increases the risk of a Type $1$ error—meaning those results must be treated with _____ and replicated before being accepted as reliable conclusions.

A psychology researcher pre-registered a single hypothesis, ran that planned analysis, and found no significant effect. They then explored the dataset further by running 12 additional unplanned tests; one yielded p < .05. Reporting that single result as a confirmed, hypothesis-driven finding—rather than labeling it exploratory—is acceptable scientific practice.

In a research report, clearly separating planned analyses from exploratory ones is essential because every additional _____ test performed on the same dataset increases the probability that a statistically significant pattern is a false positive rather than a true effect.

A researcher finishes data collection and notices an unexpected relationship that was never part of the original research plan. Evaluate the following actions and arrange them in the order that best reflects responsible scientific practice for incorporating this discovery into a research report.

Explain the primary statistical risk associated with conducting multiple unplanned exploratory analyses on a dataset, and state the two actions researchers should take when they discover unexpected, interesting relationships in their data.

Based on the principles of differentiating planned and exploratory analyses, explain why the team must not present this sophomore chemistry major finding as a confirmed, planned result. How should they address this finding in their research report?

A researcher's planned hypothesis test yields a non-significant result ($p > .05$). They decide to conduct an exploratory post-hoc analysis and discover a surprising, statistically significant relationship between two other variables. Write a brief statement (1-3 sentences) that the researcher could include in their discussion section to report this finding in accordance with responsible scientific practices.