Probability Theory Archives - Probabilistic World

Occam’s Razor: A Probabilistic View

Posted on March 13, 2018 Written by The Cthaeh 7 Comments

“Simple explanations are better than complex explanations.” — have you heard this statement before? It’s the most simplified version of the principle Occam’s razor. More specifically, the principle says:

A simple theory is always preferable to a complex theory, if the complex theory doesn’t offer a better explanation.

Does it make sense? If it’s not immediately convincing, that’s okay. There have been debates around Occam’s razor’s validity and applicability for a very long time.

In this post, I’m going to give an intuitive introduction to the principle and its justification. I’m going to show that, despite historical debates, there is a sense in which Occam’s razor is always valid. In fact, I’m going to try to convince you this principle is so true that it doesn’t even need to be stated on its own.

[Read more…]

Not All Zero Probabilities Are Created Equal

Posted on August 20, 2017 Written by The Cthaeh Leave a Comment

What does a probability of zero mean? When people use it in everyday conversations, a statement like “the probability of something is zero” usually implies that that something isn’t going to happen. Or that it is impossible to happen. Or that it will never happen.

There’s zero chance I’m passing this exam!

Is this true? Can we really say that zero probability events are impossible to occur? I’m going to show you that this is, in fact, false. You will see zero probability events are more than possible: they happen all the time.

[Read more…]

When Dependence Between Events Is Conditional

Posted on November 26, 2016 Written by The Cthaeh 2 Comments

In this post, I want to talk about conditional dependence and independence between events. This is an important concept in probability theory and a central concept for graphical models.

In my two-part post on Bayesian belief networks, I introduced one of the main categories of graphical models. You can read Part 1 and Part 2 by following these links.

This is actually an informal continuation of the two Bayesian networks posts. Even though I initially wanted to include it at the end of Part 2, I decided it’s an important enough topic that deserves its own space.

[Read more…]

What Are Bayesian Belief Networks? (Part 2)

Posted on November 20, 2016 Written by The Cthaeh Leave a Comment

In the first part of this post, I gave the basic intuition behind Bayesian belief networks (or just Bayesian networks) — what they are, what they’re used for, and how information is exchanged between their nodes.

In this post, I’m going to show the math underlying everything I talked about in the previous one. It’s going to be a bit more technical, but I’m going to try to give the intuition behind the relevant equations.

If you stick to the end, I promise you’ll get a much deeper understanding of Bayesian networks. To the point of actually being able to use them for real-world calculations.

[Read more…]

2016 US Presidential Election Predictions (November 7 Update)

Posted on November 7, 2016 Written by The Cthaeh 2 Comments

The moment of truth! The year-long battle for the White House is coming to its conclusion!

Tomorrow, voters in all states will determine the next US president in one of the most unpredictable elections in US history. Many people will be celebrating and many will be disappointed when the final votes are counted a little over 24 hours from now.

Donald Trump managed to achieve a miraculous recovery of his campaign just 2 weeks before election day. Currently, Hillary Clinton has about 76% chance of winning the election. Donald Trump’s chances for a direct win are about 11%. There is also a significant probability of 13-14% for a tie between the candidates, which would send the contest to the US Congress!

[Read more…]

What Are Bayesian Belief Networks? (Part 1)

Posted on November 3, 2016 Written by The Cthaeh 11 Comments

In my introductory Bayes’ theorem post, I used a “rainy day” example to show how information about one event can change the probability of another. In particular, how seeing rainy weather patterns (like dark clouds) increases the probability that it will rain later the same day. Bayesian belief networks, or just Bayesian networks, are a natural generalization of these kinds of inferences to multiple events or random processes that depend on each other.

This is going to be the first of 2 posts specifically dedicated to this topic. Here I’m going to give the general intuition for what Bayesian networks are and how they are used as causal models of the real world. I’m also going to give the general intuition of how information propagates within a Bayesian network.

[Read more…]