p-values

A p-value associated with a hypothesis test of some null hypothesis $H_0$ vs. some alternative $H_A$ is the probability, under the null hypothesis, of observing a result at least as extreme as the statistic you observed.

Extreme here means in the direction of the rejection region.

What does extreme mean?

• One sided lesser: pnorm(z)
• One sided greater: 1 - pnorm(z)
• Two sided: 2*(1- pnorm(abs(z)))

Example: Let’s say $\overline{Y}=25.7$, $H_0:\mu =30$, $H_1:\mu <30$, $n=100$, ${\sigma }^2=25,known$:

z <- (25.7 - 30)/(sqrt(25/100))
pnorm(z)

## [1] 3.985805e-18

The p-value is not

A p-value is the probability, under the null hypothesis, of observing a result at least as extreme as the result that was actually observed.

A p-value is NOT the probability of the null hypothesis being true!

There is a 3.57% chance that the mean is truly 12.

There is a 3.57% chance of observing a z-statistic at least this far from zero when the mean truly is 12.

American Statistical Association Statement on p-values

p-values as measures of evidence against the null hypothesis

From Statistical Sleuth

p-values as measures of evidence against the null hypothesis

E.g. if $p<0.001$, for $H_0:\mu =30$, $H_A:\mu <30$.

• “There is convincing evidence against the hypothesis that the mean time spent preparing for class by freshman at OSU is equal to 30”.

• Also OK: “There is convincing evidence the mean time spent preparing for class by freshman at OSU is less than 30”.

• Not OK: “There is convincing evidence the mean time spent preparing for class by freshman at OSU is equal to 30” - wrong direction

• Not OK: “There is no evidence the mean time spent preparing for class by freshman at OSU is equal to 30” - p-values don’t give evidence for null

Your turn:

Now imagine the p-value is, $p=0.5$. Which of the following are correct conclusions?

• “There is no evidence the mean time spent preparing for class by freshman at OSU is less than 30”.

• “There is no evidence the mean time spent preparing for class by freshman at OSU is equal to 30”.

• “There is convincing evidence the mean time spent preparing for class by freshman at OSU is equal to 30”.

• “There is no evidence against the hypothesis that the mean time spent preparing for class by freshman at OSU is equal to 30”.

p-values and rejection regions

$p\le \alpha$ $⟺$ Reject $H_0:\mu ={\mu }_0$ at significance level $\alpha$

$p>\alpha$ $⟺$ Fail to reject $H_0:\mu ={\mu }_0$ at significance level $\alpha$

XKCD picture

https://xkcd.com/1478/

Inference: Confidence Intervals

A confidence interval gives a range of plausible values for the parameter.

A hypothesis test asks if a value is plausible.

Leads to:

• A $(1-\alpha )100$% confidence interval is the set of all null hypotheses that would not be rejected at level $\alpha$.

• That is, ${\mu }_0$ is in a two-sided $(1-\alpha )100$% confidence interval for $\mu$ if $H_0:\mu ={\mu }_0$ would not be rejected at level $\alpha$ vs. a two-sided alternative.

CI for Z-test

Rejection region for two-sided alternative: $|Z\left({\mu }_0\right)|>z_{1-\alpha /2}$

We want all ${\mu }_0$ that satisfy

$$\left|\frac{\overline{Y}-{\mu }_0}{\sigma /\sqrt{n}}\right|<z_{1-\alpha /2}$$ Or equivalently,

$$z_{\alpha /2}<\frac{\overline{Y}-{\mu }_0}{\sigma /\sqrt{n}}<z_{1-\alpha /2}$$

CI for Z-test

Leads to $(1-\alpha )100$% confidence intervals of the form

$$\left(\overline{Y}-z_{1-\alpha /2}\frac{\sigma }{\sqrt{n}},\overline{Y}+z_{1-\alpha /2}\frac{\sigma }{\sqrt{n}}\right)$$

Sometimes called a Z-confidence interval.

$z_{1-{\alpha }_2}=1.96\approx 2$

Interpretation of CIs

• $(1-\alpha )100$% of the time that you perform this experiment, the interval you construct will contain the true value of $\mu$.
• E.g. in $\alpha 100$% of possible random samples from the population, this intervalcontains the true $\mu$.
• It is incorrect to say probability the true mean is inside a specific interval is, e.g. 95%.
• The correct statement is “95% of the time, intervals constructed in this manner will include $\mu$”

A statistical summary

When summarizing an analysis, state:

• Point estimate
• Confidence interval estimate with confidence level
• p-value and conclusion against the null, worded in context without notation.

(Any other information neccessary to understand what analysis was undertaken)

A statistical summary: example

Let’s say our sample mean time spent preparing for class from our sample of 100 OSU freshman is $\overline{Y}=25.7$. (Still assuming a known variance of ${\sigma }^2=25$)

• There is convincing evidence OSU freshman (Fall 2017) spend less than 30 hours per week preparing for classes (one-sided p-value, p < 0.001 , from Z-test).
• We estimate that the mean time OSU freshman (Fall 2017) spent preparing for class was 25.7 hours per week.
• With 95% confidence, the mean time OSU freshman spend preparing for class is between 24.72 and 26.68 hours per week.

use p-value rounded to 2 significant figures if > 0.001

Your turn:

A random sample of $n=25$ Corvallis residents had an average IQ score of 104. Assume a population variance of ${\sigma }^2=225$. What’s the mean IQ for Corvallis residents? Is it plausible the mean for Corvallis residents is greater than 100?

Write a statistical summary.

Next time

What if we don’t know the population variance?