# p-value

$\renewcommand{\Pr}[1]{\text{Pr}\left[ #1 \right]} \def\eps{\varepsilon} \def\E{\mathbb{E}} \def\pv{p\text{-value}}$

## Overview #

• The p-value is a r.v.
• For a given observation, it's the probability of obtaining observing results at least as "extreme" as the currently observed result, given the null hypothesis is correct.
• Imagine $f(x)$ is some criterion r.v. which returns the probability of finding more extreme results than some observation $x$. In this case, the null hypothesis is also a distribution which we're assuming $x$ is sampled from.
$\pv(x) = p \in [0, 1] \;:\; \Pr{f(x) \ge p \;|\; x \sim H_0}$
• Used in null-hypothesis significance testing.
• Different from the transposed conditional probability
$\times \;\; \Pr{x \sim H_0 \;|\; f(x) \ge \pv}$ \begin{align*} \Pr{x \sim H_0 \;|\; f(x) \ge \pv} &= \Pr{f(x) \ge \pv \;|\; x \sim H_0} \cdot \frac{\Pr{f(x) \ge \pv}}{\Pr{x \sim H_0}} \\ \end{align*}

## Traps #

1. p-value is conditional probability, not probability the null-hypothesis is true nor probability alternative-hypothesis is false.
2. p-value is only statement about the relation of the observed data to the null-hypothesis.
3. p-value significance of 0.05 is only convention.
4. p-value doesn't indicate size or importance of an effect.