Probability Concepts

From MM*Stat International

Jump to: navigation, search

Probability is a measure which quantifies the dregree of (un)certainty associated with an event. We will discuss three common approaches to probability.

Classical Probability

Laplace’s classical definition of probability is based on equally likely outcomes. He postulates the following properties of events:

  • the sample space is composed of a finite number of basic outcomes
  • the random process generates exactly basic outcome and hence one elementary event
  • the elementary events are equally likely, i.e. occur with the same probability

Accepting these assumptions, the probability of any event (subset of the sample space) can be computed as Properties:

Example: Rolling a six-sided dieSample space: Define event ‘even number’Elementary events in : ,,

Statistical Probability

Richard von Mises originated the relative frequency approach to probability: The probability for an event is defined as the limit of the relative frequency of , i.e. the value the relative frequency will converge to if the experiment is repeated an infinite number of times.  It is assumed that replications are independent of each other. Let denote the absolute frequency of occurring in repetitions. The relative frequency of is then defined as According to the statistical concept of probability we have Since it follows that . Example: Flipping a coin Denote by  the event ‘a head appears’.  Absolute and relative frequencies of after trials are listed in the table below. This particular sample displays a non monotonic convergence to , the theoretical probability of a head occuring in repeated flips of a ’fair’ coin..

10 7 0.700
20 11 0.550
40 17 0.425
60 24 0.400
80 34 0.425
100 47 0.470
200 92 0.460
400 204 0.510
600 348 0.580
800 404 0.505
1000 492 0.492
2000 1010 0.505
3000 1530 0.510
4000 2032 0.508
5000 2515 0.503

Visualizing the sequence of relative frequencies as a function of sample size provides some intuition into the character of the convergence.

En folimg535.gif

A central objective of statistics is to estimate or approximate probabilities of events using observed data.  These estimates can then be used to make probabilistic statements about the process generating the data,  (e.g., confidence intervals which we will study later), tto test propositions about the process and to predict the likelihood of future events

Axiomatic Foundation of Probability

is a probability measure.  It is a function which assigns a number to each event of the sample space . Axiom 1 is real-valued with . Axiom 2. Axiom 3If two events and are mutually exclusive (), then Some basic properties of probabilityLet be events and a probability measure. Then the following properties follow from the above three axioms


  1. If for , then

Addition Rule of Probability

Let and be any two events. Then

En folnode7 c 02.gif

Extension to three events , , :

En folnode7 c k 1 1.gif

Assume you have shuffled a standard deck of 52 playing cards. You are interested in the probability of a randomly drawn card being a queen or a ’heart’.We are thus interested in probability of the event . Following Laplace’s notion of probability, we proceed as follows: There are 4 queens and 13 hearts in the deck. Hence,

But there is also one card which is both a queen and a heart. As this card is included in both counts, we would overstate the probability of either queen or heart appearing if we simply added both probabilities. In fact, the addition rule of probability requires one to deduct the probability of this joint event: Here,

Thus, The probability of drawing queen’s face and/or heart suit is .

  1. The event can be rewritten as a union of two disjoint sets and as follows

    as illustrated on the Venn diagram below:

    En folnode7 c mi 2 1k.gif

    The probability is, according to axiom 3, which implies

  2. We rewrite the event as a union of two disjoint sets and so that

    En folnode7 c mi 2 1l.gif

    The probability follows from axiom 3 Now we obtain the desired result by calculating using the formula given in part one:

    En folnode7 c mi 2 1m.gif

Proof of Property 5:Let us show that for it follows that . The event can be rewritten as , where and are disjoint sets.According to axiom 3 we have the following: . Nonnegativity of the probability implies that . This rule can be illustrated using a Venn diagram:

En folnode7 c mi 01.gif

Proof of Property 7:Let us prove that . We have and , where and are clearly disjoint.Using axiom 3 the probability of can be calculated as This result is displayed on the following Venn diagram:

En folnode7 c mi 02.gif