Probabilistic World

Discrete Probability Distributions: An Overview

Posted on Written by 2 Comments

A discrete probability distribution, two hands holding dice, and a background referencing the movie The Matrix

In my previous two posts I sketched the frame of the big picture around probability distributions. In my introductory post I gave some intuition about the general concept and talked about the two major kinds: discrete and continuous distributions. And in the follow-up post I related the concepts of mean and variance to probability distributions. I showed that this connection itself goes through two fundamental concepts from probability theory: the law of large numbers and expected value.

Now I want to build on all these posts. My plan is to start introducing commonly used discrete and continuous distributions in separate posts dedicated to each one. And I want to start with the former, since they are significantly easier to understand.

The goal of the current post is to be a final warm-up before delving into the details of specific distributions.

[Read more…]

Mean And Variance Of Probability Distributions

Posted on Written by 4 Comments

In my previous post I introduced you to probability distributions.

In short, a probability distribution is simply taking the whole probability mass of a random variable and distributing it across its possible outcomes. Since every random variable has a total probability mass equal to 1, this just means splitting the number 1 into parts and assigning each part to some element of the variable’s sample space (informally speaking).

In this post I want to dig a little deeper into probability distributions and explore some of their properties. Namely, I want to talk about the measures of central tendency (the mean) and dispersion (the variance) of a probability distribution.

[Read more…]

Introduction To Probability Distributions

Posted on Written by 3 Comments

If you want to take your understanding of probabilities to the next level, it’s crucial to be familiar with the concept of a probability distribution.

In short, a probability distribution is an assignment of probabilities or probability densities to all possible outcomes of a random variable.

For example, take the random process of flipping a regular coin. The outcome of each flip is a random variable with a probability distribution:

  • P(“Heads”) = 0.5
  • P(“Tails”) = 0.5

Depending on the type of random variable you’re working with, there are two general types of probability distributions: discrete and continuous. In this post, I’m going to give an overview of both kinds. And in follow-up posts I’m going to individually introduce specific frequently used probability distributions from each kind.

[Read more…]

An Intuitive Explanation Of Expected Value

Posted on Written by 7 Comments

A pile of poker chips and a few dice

Expected value is another central concept in probability theory. It is a measure of the “long-term average” of a random variable (random process). I know this doesn’t sound too clear, but in this post I’m going to explain exactly what it means.

There are many areas in which expected value is applied and it’s difficult to give a comprehensive list. It is used in a variety of calculations by natural scientists, data scientists, statisticians, investors, economists, financial institutions, and professional gamblers, to name just a few.

[Read more…]

The Law Of Large Numbers: Intuitive Introduction

Posted on Written by 11 Comments

The silhouette of an infinity symbol
The law of large numbers is one of the most important theorems in probability theory.  It states that, as a probabilistic process is repeated a large number of times, the relative frequencies of its possible outcomes will get closer and closer to their respective probabilities.

For example, flipping a regular coin many times results in approximately 50% heads and 50% tails frequency, since the probabilities of those outcomes are both 0.5.

The law of large numbers demonstrates and proves the fundamental relationship between the concepts of probability and frequency. In a way, it provides the bridge between probability theory and the real world.

[Read more…]

Occam’s Razor: A Probabilistic View

Posted on Written by 7 Comments

The word simple written by blocks of "complex" words

“Simple explanations are better than complex explanations.” — have you heard this statement before? It’s the most simplified version of the principle called Occam’s razor. More specifically, the principle says:

A simple theory is always preferable to a complex theory, if the complex theory doesn’t offer a better explanation.

Does it make sense? If it’s not immediately convincing, that’s okay. There have been debates around Occam’s razor’s validity and applicability for a very long time.

In this post, I’m going to give an intuitive introduction to the principle and its justification. I’m going to show that, despite historical debates, there is a sense in which Occam’s razor is always valid. In fact, I’m going to try to convince you this principle is so true that it doesn’t even need to be stated on its own.

[Read more…]