You reject the null hypothesis when your sample data are so unlikely under the “no effect / no difference” assumption that this assumption is no longer plausible, typically when the p-value is less than or equal to your chosen significance level (like 0.05).

What the null hypothesis is

The null hypothesis H0H_0H0​ usually says “nothing interesting is happening”: no effect, no difference, no relationship.

Examples:

  • “The new drug has the same effect as placebo.”
  • “The mean score of Group A equals Group B.”
  • “There is no association between smoking and a certain disease.”

The alternative hypothesis H1H_1H1​ (or HaH_aHa​) is the “something is happening” claim you’d like to support, such as a real effect or difference.

Core rule: p-value vs alpha

In most standard tests you follow this decision rule:

  • Choose a significance level α\alpha α (often 0.05, sometimes 0.01).
  • Compute a test statistic and its p-value from your data.
  • Compare p-value and α\alpha α:

You reject the null hypothesis when:

  • p-value ≤ α\alpha α.
    • Your result is called statistically significant.
* Mnemonic: “If the p is low, the null must go.”

You fail to reject the null hypothesis when:

  • p-value > α\alpha α.
    • The data are not statistically significant.
* The sample does not provide strong enough evidence that an effect exists.

A simple example: with α=0.05\alpha =0.05α=0.05, if your p-value is 0.004, you reject H0H_0H0​; if your p-value is 0.12, you fail to reject H0H_0H0​.

Intuition: what “reject” really means

A p-value is the probability of observing data at least as extreme as yours, assuming the null hypothesis is true.

  • If that probability is very small (≤ α\alpha α), your data are “too weird” for the null to comfortably explain.
  • So you decide that H0H_0H0​ is unlikely and reject it in favor of the alternative.

Think of it like this: you start by giving H0H_0H0​ the benefit of the doubt; only very unusual results (relative to H0H_0H0​) justify throwing it out.

Important caveats (Type I and II errors)

When you reject H0H_0H0​, you always risk a Type I error : rejecting a true null.

  • α\alpha α is the long-run probability of making this error if the null is actually true.
  • Smaller α\alpha α (like 0.01 instead of 0.05) means you reject less often but are more conservative.

When you fail to reject H0H_0H0​, you risk a Type II error : missing a real effect because your study lacked power (small sample, noisy data, etc.).

That’s why good practice is to look not only at the p-value, but also:

  • Effect size (how large is the difference?).
  • Confidence intervals (range of plausible values).
  • Study design and sample size.
  • Whether the assumptions of the test are met.

“Reject” vs “accept” the null

Modern statistics usually avoids saying “accept the null hypothesis.”

  • If p-value > α\alpha α, you say you fail to reject H0H_0H0​.
  • Lack of evidence against H0H_0H0​ is not strong evidence that H0H_0H0​ is true; it might just mean the data are inconclusive (for example, your sample is too small).

So the key decision wording is:

  • “We reject the null hypothesis and conclude there is evidence of an effect.”
  • “We fail to reject the null hypothesis; the data do not provide strong evidence of an effect.”

Quick checklist: when to reject H0H_0H0​

You reject the null hypothesis when:

  1. You have clearly specified H0H_0H0​ and H1H_1H1​ before looking at the results.
  2. You have chosen a significance level α\alpha α (often 0.05 or 0.01).
  3. The test’s assumptions (e.g., independence, distribution, variance conditions) are reasonably met.
  4. The computed p-value is less than or equal to α\alpha α.
  5. The effect size and context make the result meaningful, not just “technically significant.”

If any of these fail—especially if p-value > α\alpha α or assumptions are badly violated—you should not reject H0H_0H0​.

TL;DR: Reject the null hypothesis when your test yields a p-value less than or equal to your chosen significance level, the test’s assumptions are met, and the result is both statistically and practically meaningful.

Information gathered from public forums or data available on the internet and portrayed here.