Probabilistic World

Introduction To Probability Distributions

Posted on Written by 3 Comments

If you want to take your understanding of probabilities to the next level, it’s crucial to be familiar with the concept of a probability distribution.

In short, a probability distribution is an assignment of probabilities or probability densities to all possible outcomes of a random variable.

For example, take the random process of flipping a regular coin. The outcome of each flip is a random variable with a probability distribution:

  • P(“Heads”) = 0.5
  • P(“Tails”) = 0.5

Depending on the type of random variable you’re working with, there are two general types of probability distributions: discrete and continuous. In this post, I’m going to give an overview of both kinds. And in follow-up posts I’m going to individually introduce specific frequently used probability distributions from each kind.

[Read more…]

The Variance: Measuring Dispersion

Posted on Written by Leave a Comment

Small pawn-shaped magnets of different color, representing varianceA few posts ago I introduced you to the “three M’s” of statistics — the concepts of mean, mode, and median. Today I want to talk to you about a related concept called variance.

While the three M’s measure the central tendency of a collection of numbers, the variance measures their dispersion. That is, it measures how different the numbers are from each other.

Measuring dispersion is another fundamental topic in statistics and probability theory. On the one hand, it tells you how much you can trust the central tendency measures as good representatives of the collection. High variance usually means a lot of the numbers in the collection will be far away from those measures.

[Read more…]

An Intuitive Explanation Of Expected Value

Posted on Written by 4 Comments

A pile of poker chips and a few dice

Expected value is another central concept in probability theory. It is a measure of the “long-term average” of a random variable (random process). I know this doesn’t sound too clear, but in this post I’m going to explain exactly what it means.

There are many areas in which expected value is applied and it’s difficult to give a comprehensive list. It is used in a variety of calculations by natural scientists, data scientists, statisticians, investors, economists, financial institutions, and professional gamblers, to name just a few.

[Read more…]

The Law Of Large Numbers: Intuitive Introduction

Posted on Written by 4 Comments

The silhouette of an infinity symbol
The law of large numbers is one of the most important theorems in probability theory.  It states that, as a probabilistic process is repeated a large number of times, the relative frequencies of its possible outcomes will get closer and closer to their respective probabilities.

For example, flipping a regular coin many times results in approximately 50% heads and 50% tails frequency, since the probabilities of those outcomes are both 0.5.

The law of large numbers demonstrates and proves the fundamental relationship between the concepts of probability and frequency. In a way, it provides the bridge between probability theory and the real world.

[Read more…]

The Mean, The Mode, And The Median

Posted on Written by Leave a Comment

Mean, median, and mode of a distribution shown on a Spaghetti Western background

The concepts of mean, median, and mode are fundamental to statistics, probability theory, and anything related to data analysis as a whole. Being this important, they deserve their own introduction.

In statistics, these 3 concepts are examples of measures of central tendency. This is a fancy way of saying that they are single values that summarize collections of values.

Let’s unpack the last sentence. What exactly do I mean by ‘values’ and ‘collections of values’?

[Read more…]

Occam’s Razor: A Probabilistic View

Posted on Written by 7 Comments

The word simple written by blocks of "complex" words

“Simple explanations are better than complex explanations.” — have you heard this statement before? It’s the most simplified version of the principle called Occam’s razor. More specifically, the principle says:

A simple theory is always preferable to a complex theory, if the complex theory doesn’t offer a better explanation.

Does it make sense? If it’s not immediately convincing, that’s okay. There have been debates around Occam’s razor’s validity and applicability for a very long time.

In this post, I’m going to give an intuitive introduction to the principle and its justification. I’m going to show that, despite historical debates, there is a sense in which Occam’s razor is always valid. In fact, I’m going to try to convince you this principle is so true that it doesn’t even need to be stated on its own.

[Read more…]