The Power of Statistics (and I don’t mean statistical power!)*

The more I know about statistics, the more I understand why they’ve acquired such a bad reputation over the years. Statistics do not lie, liars use statistics. There are individuals who will deliberately use statistics to their advantage. No doubt, it’s bad. However, what scares me even more than a skilled dishonest statistician is an unskilled wanna-be statistician.

I recently discovered that quantitative analysis is a requirement in undergrad Communication/Journalism in some universities. Students learn SPSS (Statistical Package of Social Sciences) and the basic of statistics. Of course, one course doesn’t teach you enough to really understand stats, but it gives you just enough knowledge to make you believe that you can conduct your own statistical analysis.

Many crucial details will influence the results of an analysis, most of which people with limited statistics experience are probably not even aware of. Let’s look at one example. When you hear that a finding is statistically significant, there is a lot more going on than you probably think. There are literally dozens of statistical tests. Some are liberal (LSD), some are more conservative (Bonferroni). A relationship may be significant using a liberal test, but not significant using a more conservative one. The confidence interval you choose (95%, 99%) will also affect the significance. With a large sample size, most relationships will be significant anyway, but that doesn’t mean they are meaningful.

Statistics are extremely powerful tool. There is a reason why only trained professionals such as police officers are allowed to use guns. There is also a reason why only medical professionals perform surgeries and engineers build bridges. If statistics were left to statisticians (and trained sociologists/psychologists obviously!), the field would have a much better reputation.

* I feel the need to explain the title. It’s a joke! Statistical power is the probability of getting a statistically significant result. It’s the odds of confirming your theory when in fact there is a relationship. A large sample size, for example, increases your statistical power. It’s a good thing, but it may lead to Type I Errors i.e. the probability of saying that a relationship is statistically significant when it’s not. Using a 99% confidence interval decreases statistical power and your chance of making Type I Errors.