A simple proof that the p-value distribution is uniform when the null hypothesis is true

Someone asked this question in my linear modeling class: why is it that the p-value has a uniform distribution when the null hypothesis is true? Proof is remarkably simple.

First, notice that when a random variable Z comes from a $Uniform(0,1)$ distribution, then the probability that $Z$ is less than (or equal to) some value $z$ is exactly $z$: $P(Zleq z)=z$.

Next, we prove the following proposition:

Proposition:
If a random variable $Z=F(T)$, i.e., if $Z$ is the CDF for a random variable $T$, then $Z sim Uniform(0,1)$.

Note here that the p-value is a random variable, call it $Z$. The p-value is the CDF of the random variable $T$ that is a transformation of the random variable $bar{X}$: $T=(bar{X}-mu)/(sigma/sqrt{n})$. This random variable has a CDF $F(T)$.

So, if we can prove the above proposition, we have shown that the p-value’s distribution under the null hypothesis is $Uniform(0,1)$.

Proof:

Let $Z=F(T)$.

$P(Zleq z) = P(F(T)leq z) = P(Z leq F^{-1}(z) ) = F(F^{-1} (z))= z$.

Since $P(Zleq z)=z$, Z is uniformly distributed, that is, Uniform(0,1).

Related