1.1. Bernoulli and Binomial#
Bernoulli processes and the Binomial distribution: do I need to study for my probability exam?
Extreme value analysis and the study of extremes has a foundational basis in the theory of Bernoulli processes and the Bernoulli and Binomial distributions.
Bernoulli trials and Bernoulli process#
One student wants to know if she will pass her favourite course (of course, probability) without studying. Thus, she has the simplest possible experiment: one trial with two possible outcomes, pass (success!) and not pass (usually called failure). Note that they are mutually exclusive (if you pass, you haven’t failed) and collectively exhaustive (there’s no third possible outcome from the experiment). The results from such a trial are a two-sided Bernoulli random variable and, thus, the distribution of probabilities of the two outcomes is a Bernoulli distribution. The Bernoulli distribution is a discrete distribution function whose probability mass function (pmf) is given by
\( p_X(x) = P[X=x|p] = p^x(1-p)^{1-x} \hspace{1cm} for \ x = 0 \ or \ 1 \ and \ p \in [0,1]\)
\( p_X(x) = P[X=x|p] = 0 \hspace{2.7cm} otherwise \)
where \(p\) is the probability of success (\(x = 1\)), and \(x = 0 \ and \ 1\) are the possible outcomes.
Imagine that the student is not studying and wants to leave it to chance. She would have to take the course several times (several trials) and since she’s not studying between trials, each trial would be independent (she’s not learning from previous experiences). This will constitute what it’s called a Bernoulli process.
Bernoulli process criteria
The governing criteria for a Bernoulli process are:
There are only two possible outcomes which are mutually exclusive and collectively exhaustive: success vs. failure.
There is a constant probability of success, \(p\).
Each trial is independent. That is, the result of the next trial does not depend on the previous one.
Binomial distribution#
The student loved the probability course and she doesn’t mind retaking it, but since she doesn’t want to delay her graduation, she needs to pass the course in the current year. In the course, there are three exams and needs to pass at least two of them. Therefore, she is interested in determining the probability of passing two exams without studying in 3 trials.
The student knows that needs the probability of success, \(p\), since she remembers from the probability lectures that it is a Bernoulli process. Thus, she performs a survey to other students from previous courses. 50 students admitted not studying for a probability exam and 2 of them passed it. Then, she determines the success probability as \(p = 2/50 = 0.04\).
And now?
Let’s go step by step. Considering that the outcomes (passing each exam) are independent, the probability of having two successes and a failure is \(p^2(1-p)\). However, there are three different sequences which may lead to that situation and have the same joint probability: passing the first two exams, passing the first and the third exam or passing the last two exams. Hence, the probability of two successes and one failure in three trials can be obtained from the addition rule[1] as follows:
\( p_X(2)=P[X=2|3, p]=\binom{3}{2}p^2(1-p)^{3-2}=\frac{3!}{2!(3-2)!}p^2(1-p)=3p^2(1-p) \)
which is the pmf of the Binomial distribution. Thus, we went from the Bernoulli distribution to the Binomial distribution. The pmf of the Binomial distribution is then defined as
\( p_X(x) = P[X=x|n, p] = \binom{n}{x}p^x(1-p)^{n-x} \hspace{1cm} for \ x = 0, 1, ..., n; \ p \in [0,1]\)
\( p_X(x) = P[X=x|n, p] = 0 \hspace{3.3cm} otherwise \)
where
\( \binom{n}{x} = \frac{n!}{x!(n-x)!} \)
which is the total number of possible combinations when selecting x successes from n trials. Thus, in our example, the random variable X represents the number of successes in n trials follows a Binomial distribution with parameters \(p\) (probability of success) and \(n\) (number of trials). The cumulative distribution function (cdf) of a Binomial distribution is given by
\( F_X(x) = \sum_{k=0}^x\binom{n}{x}p^k(1-p)^{n-k} \)
Note that the maximum value of the cdf is reached for \(X = n\).
Binomial-distributed variable
The conditions for a random variable to follow a Binomial distribution are:
A series of Bernoulli trials is made with two possible outcomes: success or failure.
There is a constant probability of success, \(p\).
There is a fixed number of trials, \(n\).
The outcomes of the trials are independent.
The random variable X is the total number of successes in \(n\) trials and the order of the successes is irrelevant.
Finally, let’s answer the student questions. Applying the Binomial distribution: \( p_X(x) = P[X=2|3, 0.04] = \binom{3}{2}p^2(1-p)^{3-2} \approx 0.005 \)
Sadly, there is a very low chance that the student can make it without studying! However, now she knows Bernoulli processes and Binomial distribution and that for sure increases the chances of passing!