Hypothesis test

Hypothesis tests are a powerful tool, with a solid mathematical foundation. However, criticisms of hypothesis testing abound.

For example, one criticism of hypothesis testing is that statistical significance is often confused with scientific significance. Suppose we run this t-test: H0: mean IQ of Boston residents = mean IQ of NYC residents, Ha: they are not equal. Say we get a very small p-value and conclude that residents from Boston and NYC have different IQ’s, on average. We then proceed to actually check the two different sample means: mean Boston IQ = 125.3 and mean NYC IQ = 125.2. While our data finds that 0.1 difference *statistically* significant, it’s not *scientifically* significant. That 0.1 IQ difference is definitely there, but it’s too small for us to notice in our every day lives. For all intents and purposes, Bostonians and New Yorkers have the same mean IQ. Our t-test does not address the magnitude of the difference, just whether a difference exists.

There are many other criticisms of hypothesis testing. Please pick one (different from the example I gave; I encourage you to Google “criticisms of hypothesis testing”) and discuss an example of it. You can either make up your own example or find someone else’s example on the internet. If you do the latter, be sure to cite references. Then comment on two of your classmate’s posts. Either suggest how you would rectify the situation, or discuss why the situation can’t be rectified i.e. why a hypothesis test isn’t the correct tool for the job.