Probability and Statistics
In probability we have a sample space \(S\), which abstractly is just some set, and events which are subsets of \(S\). You should be familiar with the best set theoretic operations of union, intersection and taking the complement:
- Union: \( A \cup B = \{ x \in S \text{ such that } x \in A \text{ or }x\in B\} \)
- Intersection: \( A \cap B = \{ x \in S \text{ such that } x \in A \text{ and }x\in B\}\)
- Complement: \( A^c = \{ x \in S \text{ such that } x \not\in A\} \) (this last part reads "\(x \) is not in \(A\)")
These can be thought of as new events where "event \(A\) happens or \(B\) happens", "event \(A\) happens and event \(B\) happens" and "event \(A\)" does not happen respectively.
Demorgan's laws tell us how these interact:
- \( (A \cup B )^c = A^c \cap B^c \)
- \( (A \cap B)^c = A^c \cup B^c \)
A probability function is an assignment of a "likelihood" to each event in the sample space. More precisely it is a function \(P\) from the space of events to the interval \([0,1]\) with
- \( P(S) =1 \)
- If \(A_i\) is a sequence of mutually disjoint (i.e having empty intersection) events then \( P(A_1 \cup A_2 \cup \cdots A_n) = \sum_{i=1}^nP(A_i). \)
Note that, for independent events, we have \(P(A\cap B) = P(A)P(B).\)
Conditional probability deals with the likelihood of an event happening given that another event has already occurred. We write \(P(A|B)\) to mean the probability of \(A\) given \(B.\) We have the following basic formula which can be taken as a definition:
| \[ P(A|B) = \frac{P(A\cap B)}{P(B)}. \] |
The following, which can be derived from the above equation, is known as Bayes Theorem:
| \[ P(A|B) = \frac{P(B|A)P(A)}{P(B)}. \] |
This interpretation gave birth to an entire area of statistics, unsurprisingly called Bayesian Statistics, where philosophy is taken that a degree of belief can be formed based on evidence known.
© 2021 Imperial College London. All rights reserved. Terms of use (opens in new tab)