Probability Theory & Statistics Archives

Intuitive Explanation of Expected Value

Posted on November 24, 2018 Written by The Cthaeh 20 Comments

Expected value is another central concept in probability theory. It is a measure of the “long-term average” of a random variable (random process). I know this doesn’t sound too clear, but in this post I’m going to explain exactly what it means.

There are many areas in which expected value is applied and it’s difficult to give a comprehensive list. It is used in a variety of calculations by natural scientists, data scientists, statisticians, investors, economists, financial institutions, and professional gamblers, to name just a few.

[Read more…]

The Law of Large Numbers: Intuitive Introduction

Posted on November 13, 2018 Written by The Cthaeh 17 Comments

The law of large numbers is one of the most important theorems in probability theory. It states that, as a probabilistic process is repeated a large number of times, the relative frequencies of its possible outcomes will get closer and closer to their respective probabilities.

For example, flipping a regular coin many times results in approximately 50% heads and 50% tails frequency, since the probabilities of those outcomes are both 0.5.

The law of large numbers demonstrates and proves the fundamental relationship between the concepts of probability and frequency. In a way, it provides the bridge between probability theory and the real world.

[Read more…]

The Mean, the Mode, And the Median

Posted on October 1, 2018 Written by The Cthaeh 3 Comments

The concepts of mean, median, and mode are fundamental to statistics, probability theory, and anything related to data analysis as a whole. Being this important, they deserve their own introduction.

In statistics, these 3 concepts are examples of measures of central tendency. This is a fancy way of saying that they are single values that summarize collections of values.

Let’s unpack the last sentence. What exactly do I mean by ‘values’ and ‘collections of values’?

[Read more…]

Occam’s Razor: A Probabilistic View

Posted on March 13, 2018 Written by The Cthaeh 7 Comments

“Simple explanations are better than complex explanations.” — have you heard this statement before? It’s the most simplified version of the principle called Occam’s razor. More specifically, the principle says:

A simple theory is always preferable to a complex theory, if the complex theory doesn’t offer a better explanation.

Does it make sense? If it’s not immediately convincing, that’s okay. There have been debates around Occam’s razor’s validity and applicability for a very long time.

In this post, I’m going to give an intuitive introduction to the principle and its justification. I’m going to show that, despite historical debates, there is a sense in which Occam’s razor is always valid. In fact, I’m going to try to convince you this principle is so true that it doesn’t even need to be stated on its own.

[Read more…]

Not All Zero Probabilities Are Created Equal

Posted on August 20, 2017 Written by The Cthaeh 8 Comments

What does a probability of zero mean? When people use it in everyday conversations, a statement like “the probability of something is zero” usually implies that that something isn’t going to happen. Or that it is impossible to happen. Or that it will never happen.

There’s zero chance I’m passing this exam!

Is this true? Can we really say that zero probability events are impossible to occur? I’m going to show you that this is, in fact, false. You will see zero probability events are more than possible: they happen all the time.

[Read more…]

When Dependence Between Events Is Conditional

Posted on November 26, 2016 Written by The Cthaeh 9 Comments

In this post, I want to talk about conditional dependence and independence between events. This is an important concept in probability theory and a central concept for graphical models.

In my two-part post on Bayesian belief networks, I introduced an important type of graphical models. You can read Part 1 and Part 2 by following these links.

This is actually an informal continuation of the two Bayesian networks posts. Even though I initially wanted to include it at the end of Part 2, I decided it’s an important enough topic that deserves its own space.

[Read more…]