Probabilistic World

The Variance: Measuring Dispersion

Posted on Written by Leave a Comment

Small pawn-shaped magnets of different color, representing varianceA few posts ago I introduced you to the “three M’s” of statistics — the concepts of mean, mode, and median. Today I want to talk to you about a related concept called variance.

While the three M’s measure the central tendency of a collection of numbers, the variance measures their dispersion. That is, it measures how different the numbers are from each other.

Measuring dispersion is another fundamental topic in statistics and probability theory. On the one hand, it tells you how much you can trust the central tendency measures as good representatives of the collection. High variance usually means a lot of the numbers in the collection will be far away from those measures.

[Read more…]

The Mean, the Mode, And the Median

Posted on Written by 2 Comments

Mean, median, and mode of a distribution shown on a Spaghetti Western background

The concepts of mean, median, and mode are fundamental to statistics, probability theory, and anything related to data analysis as a whole. Being this important, they deserve their own introduction.

In statistics, these 3 concepts are examples of measures of central tendency. This is a fancy way of saying that they are single values that summarize collections of values.

Let’s unpack the last sentence. What exactly do I mean by ‘values’ and ‘collections of values’?

[Read more…]

Frequentist and Bayesian Approaches in Statistics

Posted on Written by 43 Comments

Dennis Lindley vs. Ronald Fisher

What is statistics about?

Well, imagine you obtained some data from a particular collection of things. It could be the heights of individuals within a group of people, the weights of cats in a clowder, the number of petals in a bouquet of flowers, and so on.

Such collections are called samples and you can use the obtained data in two ways. The most straightforward thing you can do is give a detailed description of the sample. For example, you can calculate some of its useful properties:

  • The average of the sample
  • The spread of the sample (how much individual data points differ from each other), also known as its variance
  • The number or percentage of individuals who score above or below some constant (for example, the number of people whose height is above 180 cm)
  • Etc.

You only use these quantities to summarize the sample. And the discipline that deals with such calculations is descriptive statistics.

But what if you wanted to learn something more general than just the properties of the sample? What if you wanted to find a pattern that doesn’t just hold for this particular sample, but also for the population from which you took the sample? The branch of statistics that deals with such generalizations is inferential statistics and is the main focus of this post.

The two general “philosophies” in inferential statistics are frequentist inference and Bayesian inference. I’m going to highlight  the main differences between them — in the types of questions they formulate, as well as in the way they go about answering them.

But first, let’s start with a brief introduction to inferential statistics.

[Read more…]

An Intuitive Explanation of P-Values

Posted on Written by 9 Comments

"P-values are tricky to understand, p

If you ever took an introductory course in statistics or attempted to read a publication in a scientific journal, you know what p-values are. Оr at least you’ve seen them. Most of the time they appear in the “results” section of a paper, attached to claims that need verification. For example:

  • “Ratings of the target person’s ‘dating desirability’ showed the predicted effect of prior stimuli, […], p < 002.”

The stuff in the square brackets is usually other relevant statistics, such as the mean difference between experimental groups. If the p-value is below a certain threshold, the result is labeled “statistically significant” and otherwise it’s labeled “not significant”. But what does that mean? What is the result significant for? And for whom? What does all of that say about the credibility of the claim preceded by the p-value?

There are common misinterpretations of p-values and the related concept of “statistical significance”. In this post, I’m going to properly define both concepts and show the intuition behind their correct interpretation.

If you don’t have much experience with probabilities, I suggest you take a look at the introductory sections of my post about Bayes’ theorem, where I also introduce some basic probability theory concepts and notation.

[Read more…]