Probability Theory & Statistics Archives

What Are Bayesian Belief Networks? (Part 2)

Posted on November 20, 2016 Written by The Cthaeh 11 Comments

In the first part of this post, I gave the basic intuition behind Bayesian belief networks (or just Bayesian networks) — what they are, what they’re used for, and how information is exchanged between their nodes.

In this post, I’m going to show the math underlying everything I talked about in the previous one. It’s going to be a bit more technical, but I’m going to try to give the intuition behind the relevant equations.

If you stick to the end, I promise you’ll get a much deeper understanding of Bayesian networks. To the point of actually being able to use them for real-world calculations.

[Read more…]

What Are Bayesian Belief Networks? (Part 1)

Posted on November 3, 2016 Written by The Cthaeh 22 Comments

In my introductory Bayes’ theorem post, I used a “rainy day” example to show how information about one event can change the probability of another. In particular, how seeing rainy weather patterns (like dark clouds) increases the probability that it will rain later the same day.

Bayesian belief networks, or just Bayesian networks, are a natural generalization of these kinds of inferences to multiple events or random processes that depend on each other.

This is going to be the first of 2 posts specifically dedicated to this topic. Here I’m going to give the general intuition for what Bayesian networks are and how they are used as causal models of the real world. I’m also going to give the general intuition of how information propagates within a Bayesian network.

[Read more…]

Predicting the 2016 US Presidential Election

Posted on September 20, 2016 Written by The Cthaeh 6 Comments

November 7 UPDATE: click here to view our final update post where we give our latest analyses just a day before the election!
Click here to go to our daily predictions page.

In the first 10 posts I mostly concentrated on theoretical topics. But the general focus of this blog is much broader. For the first time I’m going to show an actual application of probability theory for estimating real life events.

An ongoing event many people are closely following right now is the US presidential election. The primary season officially concluded at the end of July and now the general election battle is in full swing. The main clash is between former Secretary of State Hillary Clinton (D) and businessman Donald Trump (R). Clinton and Trump are challenged by 3^rd party candidates Gary Johnson from the Libertarian Party (a two-time former governor of New Mexico) and Jill Stein from the Green Party (a physician and a political activist).

[Read more…]

Frequentist and Bayesian Approaches in Statistics

Posted on June 16, 2016 Written by The Cthaeh 53 Comments

What is statistics about?

Well, imagine you obtained some data from a particular collection of things. It could be the heights of individuals within a group of people, the weights of cats in a clowder, the number of petals in a bouquet of flowers, and so on.

Such collections are called samples and you can use the obtained data in two ways. The most straightforward thing you can do is give a detailed description of the sample. For example, you can calculate some of its useful properties:

The average of the sample
The spread of the sample (how much individual data points differ from each other), also known as its variance
The number or percentage of individuals who score above or below some constant (for example, the number of people whose height is above 180 cm)
Etc.

You only use these quantities to summarize the sample. And the discipline that deals with such calculations is descriptive statistics.

But what if you wanted to learn something more general than just the properties of the sample? What if you wanted to find a pattern that doesn’t just hold for this particular sample, but also for the population from which you took the sample? The branch of statistics that deals with such generalizations is inferential statistics and is the main focus of this post.

The two general “philosophies” in inferential statistics are frequentist inference and Bayesian inference. I’m going to highlight the main differences between them — in the types of questions they formulate, as well as in the way they go about answering them.

But first, let’s start with a brief introduction to inferential statistics.

[Read more…]

Calculating Compound Event Probabilities

Posted on April 29, 2016 Written by The Cthaeh 5 Comments

You can think of probabilities as measures of uncertainty in the occurrence of an event, the truth of a hypothesis, and so on.

These measures are numbers between 0 and 1. Zero means the event is impossible to occur and 1 means the event is certain to occur.

If you have many events of interest, you can measure their probabilities separately, but you can also measure probabilities of different combinations of these events.

Say you’re following the national soccer championships of England, Spain, and Italy. You want to calculate the probabilities of Arsenal, Barcelona, and Juventus becoming national champions next season. These probabilities are:

P(Arsenal)
P(Barcelona)
P(Juventus)

But what if you want to calculate the probability of both Arsenal and Barcelona becoming champions? Or the probability that at least one of the three teams does?

In this post, I’m going to show how probabilities of such combinations of events are calculated. I’m going to give the general formulas, as well as the intuition behind them. To do that, I’m first going to introduce a few relevant concepts from probability theory.

[Read more…]

Probability: What Is It, Really?

Posted on April 8, 2016 Written by The Cthaeh 12 Comments

Throughout history, we have come up with better and more accurate ways to measure physical quantities like time, length, mass, and temperature. This has been crucial for our scientific and technological development.

Each of these quantities has a precise definition and is informative about some aspect of the current state of the physical world. For example, the mass of an object can tell you how much work is necessary to lift it at a certain height. The outside air temperature determines the kind of clothes you would wear when you go out. And so on.

Probabilities are also quantities that measure something — they have a very precise and unambiguous mathematical definition. But still, they don’t relate to things in the physical world as straightforwardly and as intuitively as measures like mass and length.

[Read more…]