Probability is a measure which quantifies the dregree of (un)certainty associated with an event. We will discuss three common approaches to probability.
Laplace’s classical definition of probability is based on equally likely outcomes. He postulates the following properties of events:
- the sample space is composed of a finite number of basic outcomes
- the random process generates exactly basic outcome and hence one elementary event
- the elementary events are equally likely, i.e. occur with the same probability
Accepting these assumptions, the probability of any event (subset of the sample space) can be computed as
Example: Rolling a six-sided dieSample space: Define event ‘even number’Elementary events in : ,,
Richard von Mises originated the relative frequency approach to probability: The probability for an event is defined as the limit of the relative frequency of , i.e. the value the relative frequency will converge to if the experiment is repeated an infinite number of times. It is assumed that replications are independent of each other.
Let denote the absolute frequency of occurring in repetitions. The relative frequency of is then defined as According to the statistical concept of probability we have
Since it follows that .
Example: Flipping a coin
Denote by the event ‘a head appears’. Absolute and relative frequencies of after trials are listed in the table below. This particular sample displays a non monotonic convergence to , the theoretical probability of a head occuring in repeated flips of a ’fair’ coin..
Visualizing the sequence of relative frequencies as a function of sample size provides some intuition into the character of the convergence.
A central objective of statistics is to estimate or approximate probabilities of events using observed data. These estimates can then be used to make probabilistic statements about the process generating the data, (e.g., confidence intervals which we will study later), tto test propositions about the process and to predict the likelihood of future events
Axiomatic Foundation of Probability
is a probability measure. It is a function which assigns a number to each event of the sample space .
Axiom 1 is real-valued with .
Axiom 3If two events and are mutually exclusive (), then
Some basic properties of probabilityLet be events and a probability measure. Then the following properties follow from the above three axioms
- If for , then
Addition Rule of Probability
Let and be any two events. Then
Extension to three events , , :
Assume you have shuffled a standard deck of 52 playing cards. You are interested in the probability of a randomly drawn card being a queen or a ’heart’.We are thus interested in probability of the event .
Following Laplace’s notion of probability, we proceed as follows: There are 4 queens and 13 hearts in the deck. Hence,
But there is also one card which is both a queen and a heart. As this card is included in both counts, we would overstate the probability of either queen or heart appearing if we simply added both probabilities. In fact, the addition rule of probability requires one to deduct the probability of this joint event: Here,
The probability of drawing queen’s face and/or heart suit is .
The event can be rewritten as a union of two disjoint sets and as follows
as illustrated on the Venn diagram below:
The probability is, according to axiom 3, which implies
We rewrite the event as a union of two disjoint sets and so that
The probability follows from axiom 3 Now we obtain the desired result by calculating using the formula given in part one:
Proof of Property 5:Let us show that for it follows that .
The event can be rewritten as , where and are disjoint sets.According to axiom 3 we have the following: . Nonnegativity of the probability implies that .
This rule can be illustrated using a Venn diagram:
Proof of Property 7:Let us prove that .
We have and , where and are clearly disjoint.Using axiom 3 the probability of can be calculated as
This result is displayed on the following Venn diagram: