Title: Default for 35.2 Elementary Probability
Course: HELM Workbooks 2023-2024

Introductory Probability

Probability as an informal idea is something you will have been familiar with for a long time. In conversation with friends, you must have used sentences such as

- ‘It might start raining soon’
- ‘I might be lucky and pass all my examinations’
- ‘It is very unlikely that my team will not win the Premiership this year’
- ‘Getting a good degree will improve my chances of getting a good job’

Essentially, when you are talking about whether some event is likely to happen, you are using the concept of probability. In reality, we need to agree on some terminology so that misunderstanding may be avoided. To start with there are four terms \(-\) experiment, outcome, event and sample space \(-\) that need formal definition. There will, of course, be others as you progress through this quiz.

1. Experiment: - an activity with an observable result, or set of results, for example
  1. tossing a coin, the result being a Head or a Tail
  2. testing a component, the result being a defective or non-defective component
  3. maximum speed testing of standard production cars;
  4. testing to destruction armour plating intended for use on tanks.
Some of the experiments outlined above have a very limited set of results (tossing a coin) while others (destruction testing) may give a widely variable set of results. Also it is worth noting that destruction testing is not appropriate for all products. Companies manufacturing trucks or explosives, for example, could not possibly test to destruction on a large scale - they would have little or nothing left to market!
2. Outcome - an outcome is simply an observable result of an experiment, for example
  1. tossing a coin, the possible outcomes are Heads or Tails
  2. testing a component, the outcome being a defective or non-defective component
  3. maximum speed testing of standard production cars, the outcomes being a set of numbers representing the maximum speeds of a set of vehicles
  4. testing to destruction armour plating intended for use on tanks, the outcomes might be (for example) the numbers of direct hits sustained before destruction.
3. Event - this is just an outcome or set of outcomes to an experiment of interest to the experimenter.
4. Sample Space - a sample space is the set of all possible outcomes of an experiment.

For example, if we throw a 6-sided die then the sample space is \(\{1,2,3,4, 5,6\}\) and two possible events are

1. a score of 3 or more, represented by the set: \(\{3,4,5,6\}\)
2. a score which is even, represented by the set: \(\{2,4,6\}\)

Everyday examples include games of chance.

If the sample space can be written in the form of a list (possibly infinite) then it is called a discrete sample space (e.g. number of tosses of a fair coin before Heads occurs). If this is not possible then it is called a continuous sample space (e.g. positions where shells land in a tank battle). The following example, therefore, concerns a discrete sample space.

Example

Obtain the sample space of the experiment throwing a single coin.

Solution

Consider the experiment of throwing a coin which can land Heads up \((H)\) or Tails up \((T)\). We list the outcomes as a set \(\{H, T\}\) − the order being unimportant. This set, \(\{H, T\}\), is the sample space. On any particular throw of a coin, Heads or Tails are equally likely to occur. We say that, for a fair coin, \(H\) and \(T\) are equally likely outcomes.

Exercise

Give the set of equally likely outcomes of the experiements:

1. Throwing a fair die with \([value]\) faces numbered \(1\) to \([value]\). An example of one outcome is 3.
2. Throwing three fair coins. An example of one outcome is \(THT\) where \(T\) and \(H\) represent tails and heads respectively.

Give your answers as discrete sets in the form \(\{\)outcome 1, outcome 2,\(\dots\}\).

2. Enter each outcome here inside speech marks, i.e. write the outcome \(THT\) as "THT":
Feedback: Solution 1. On any fair die with distinct faces, any single outcome is exactly as probable as any other. Therefore, on a fair die with \([value]\) faces the set of equally likely outcomes is \([value]\). 2. In a similar fashion, when tossing three fair coins all single outcomes are equally likely. Therefore, the set of equally likely outcomes is the set of all possible outcomes, \([value]\).

Example

For the following list of experiments, list (if possible) a suitable sample space. If you cannot write out a suitable sample space, describe one in words.

1. Test a light switch
2. Count the daily traffic accidents in Loughborough involving cyclists
3. Measure the tensile strength of small gauge steel wire
4. Test the maximum current carrying capacity of household mains cabling
5. Test the number of on-off switchings that a new type of fluorescent tube will cope with before failure
6. Pressure test an underwater TV camera.

Solution

Sample spaces might be

1. \(\{\)works, fails\(\}\)
2. \(\{0,1, 2, 3, \dots \}\), hopefully a small upper limit!
3. \(\{\) Positive continuous range \((0, \infty)\)\(\}\) or \(\{\)Suitable bounded continuous interval depending on the wire or measuring device\(\}\)
4. \(\{\) Positive continuous range \((0, \infty)\)\(\}\) or \(\{\)Suitable bounded continuous range depending on the type of cable\(\}\)
5. \(\{0,1, 2, 3, \dots\}\), hopefully a high upper limit!
6. \(\{\) Positive continuous range \((0, \infty)\)\(\}\) or \(\{\)Suitable bounded continuous interval\(\}\)

Example

A car manufacturer offers certain options on its family cars. Customers may order:

- either automatic gearboxes or manual gearboxes
- either sunroof or air-conditioning
- either steel wheels or alloy wheels
- either solid colour paint or metallic paint

Find the number of outcomes in the sample space of options that it is possible to order and represent them using a suitable diagram.

Solution

A suitable diagram is shown in the figure below. The diagram makes it easy to find the number of outcomes simply by counting. It also points the way to a formula for calculating the number of outcomes.

[Image: Tree diagram bifurcating where choices in car equipment differ]

Figure: Tree diagram.

In this case there is a total of 16 outcomes in the sample space of options. Note that in each case the customer makes two choices. This implies that there are \[2 \times 2 \times 2 \times 2 = 16\]options in total.

Diagrams such as the one above are called tree diagrams. They are only suitable in simple situations.

Events

As we have already noted a collection of some or all of the outcomes of an experiment is called an event. So an event is a subset of the sample space. For example, if we throw a 6-sided die then the sample space is \(\{1, 2, 3, 4, 5, 6\}\) and two possible events are

- a score of 3 or more, represented by the set: \(\{3, 4, 5, 6\}\)
- a score which is even, represented by the set: \(\{2, 4, 6\}\).

Example

Two coins are thrown. List the ordered outcomes for the event when just one Tail is obtained.

Solution

\(\{HT, TH\}\). Note that here the order does matter, i.e. \(TH\) is distinct from \(HT\).

Exercise

Three coins are thrown. List the ordered outcomes which belong to each of the following events.

1. two Tails are obtained
2. at least two Tails are obtained
3. at most two Tails are obtained

Recall that you should type an outcome such as \(TTH\) as "TTH":
Feedback: Solution 1. Two Tails are obtained: \([value]\) 2. At least two Tails are obtained: \([value]\) 3. At most two Tails are obtained: \([value]\) Note that (a) is a subset of (b) and (a) is also a subset of (c).

Exercise

A new type of paint to be used in the manufacture of garden equipment is tested for impact shock resistance to damage and scratch resistance to damage. The results of the \([value]\) tests are as follows.

Of the tests
- \([value]\) showed both good shock and scratch resistance.
- \([value]\) showed good shock resistance but poor scratch resistance.
- \([value]\) showed poor shock resistance but good scratch resistance.
- \([value]\) showed both poor shock and scratch resistance.

Now, if \(A\) is the event \(\{\)Good shock resistance\(\}\) and \(B\) is the event \(\{\)Good scratch resistance\(\}\), the consider the following:

1. \(A\cup B\)
2. \(A\cap B\)
3. \(A'\)
4. \(B'\)

and determine the number of samples in each:

1. \(|A\cup B|=\quad\)
2. \(|A\cap B|=\quad\)
3. \(|A'|=\quad\)
4. \(|B'|=\quad\)

Note: here the notation \(|A|\) represents the number of elements in \(A\).
Feedback: Solution 1. The event \(A\cup B\) consists of those samples which have either good shock and good scratch resistance (or both) so that \(|A\cup B|=[value]\). 2. The event \(A\cap B\) consists of those samples which have both good shock and good scratch resistance so that \(|A\cap B|=[value]\). 3. The event \(A'\) consists of those samples which do not have good shock resistance so that \(|A'|=[value]\). 4. The event \(B'\) consists of those samples which do not have good scratch resistance so that \(|B'|=[value]\).

Complement

We have met the complement before in relation to sets. We consider it again here in relation to sample spaces and events. The complement of an event is the set of outcomes which are not members of the event.

For example, the experiment of throwing a 6-faced die has sample space \(S = \{1,2,3,4,5,6\}\).

- The event “score of 3 or more is obtained” is the set \(\{3,4,5,6\}\). The complement of this event is \(\{1,2\}\) which can be described in words as “score of 3 or more is not obtained” or “score of 1 or 2 is obtained”.
- The event “even score is obtained” is the set \(\{2,4,6\}\). The complement of this event is \(\{1,3,5\}\) or, in words “even score is not obtained” or “odd score is obtained”.

In a previous exercise concerning tossing three coins:

- the complement of event (a) is \(\{TTT,THH,HTH,HHT,HHH\}\).
- the complement of event (b) is \(\{THH,HTH,HHT,HHH\}\).
- the complement of event (c) is \(\{TTT\}\).

Notation: It is customary to use a capital letter to denote an event. For example, \(A = \{\)two Heads are thrown\(\}\). The complementary event is denoted \(A'\).

Example

State, in words, what are the complements of each of the following events in relation to the experiment of throwing three coins (avoid using the word not):

1. two Heads.
2. at least two Heads.
3. at most two Heads.

Solution

1. no Heads, one Head or three Heads.
2. no Heads or one Head (or equivalently, at most one Head).
3. three Heads.

Definitions of Probability

Relative frequency applied to probability

Consider the experiment of throwing a single coin many times.

Suppose we throw a coin 10 times and obtain six Heads and four Tails; does this suggest that the coin is biased? Clearly not! What about the case when we obtain 9 Heads and 1 Tail? We conducted an experiment in which a coin was thrown 100 times and the result recorded each time as 1 if a Head appeared face up and 0 if a Tail appeared. In the figure below we have plotted the average score \(r/n\), where \(r\) is the number of Heads and \(n\) is the total number of throws, against \(n\) for \(n=10,20,\dots,100\). The quantity \(r/n\) is called the relative frequency of Heads.

[Image]

Figure: Plot of the relative frequency, \(r/n\).

As \(n\) increases the relative frequency settles down near the value \(1/2\). This is an experimental estimate of the probability of throwing a Head with this particular coin. Note that when \(n = 50\) this estimate was \(0.49\) and when \(n = 100\) this estimate was \(0.51\). When we repeated the whole experiment again, the value of \(r/n\) when \(n = 100\) was 0.46. Hence the use of the word estimate. Normally, as the number of trials is increased the estimate tends to settle down but this is not certain to occur. Theoretically, the probability of obtaining a Head when a fair coin is thrown is \(1/2\). Experimentally, we expect the relative frequency to approach \(1/2\) as \(n\) increases.

Equi-probable spaces and the principle of equally likely outcomes

An equi-probable space is a sample space in which the chance that any one sample point occurs is equal to the chance that any other sample point occurs. Whether a sample space is an equi-probable space is usually determined by inspection or logic.

1. Tossing a coin. The sample space \(S\) is, using an obvious notation\[S = \{H,T\}\]\(H\) and \(T\) are the two simple outcomes and are equally likely to occur.
2. Rolling a 6-faced die. A sample space is:\[S = \{1,2,3,4,5,6\}\]Each number from \(1\) to \(6\) is a simple outcome and is equally likely to occur.
3. Tossing three coins. A sample space comprised of simple outcomes is:\[S = \{HHH,HHT,HTH,HTT,THH,THT,TTH,TTT\}\]Each of the eight outcomes stated is equally likely to occur.
4. Counting the number of heads when three coins are tossed. Here\[S = \{3\text{ Heads},2\text{ Heads},1\text{ Head},0\text{ Heads}\}.\]This is not an equi-probable space and the outcomes are not equally likely. (For example: the event \(\{2\text{ Heads}\}\) is the union of three simple events \(\{HHT,HTH,THH\}\) so must occur more often than the event \(\{HHH\}\).

Of the sample spaces above all are equi-probable spaces except for (d).

Key Point

The Principle of Equally Likely Outcomes

This states that each simple outcome in an equi-probable space is equally likely to occur. This principle enables us to deduce the probabilities that simple events (and hence more complicated events which are combinations of simple events) occur.

Notation

If \(A\) is an event associated with a sample space \(S\) then the probability of \(A\) occurring is denoted by \(\mathbb{P}(A)\). Referring to the examples above we may immediately deduce that

1. \(\displaystyle \text{P}(H) = \text{P}(T) =\frac12\)
2. \(\displaystyle \text{P}(1) = \text{P}(2) = \text{P}(3) = \text{P}(4) = \text{P}(5) = \text{P}(6) = \frac16\)
3. \(\displaystyle \text{P}(HHH) = \text{P}(HHT) = \text{P}(HTH) = \text{P}(THH)= \text{P}(HTT)\)\[ =\text{P}(THT) = \text{P}(TTH) = \text{P}(TTT) = \frac18\]

Definition

We can now define probability using the Principle of Equally Likely Outcomes as follows: If a sample space \(S\) consists of \(n\) simple outcomes which are equally likely and an event \(A\) consists of \(m\) of those simple outcomes, then\[\text{P}(A) = \frac{m}{n} = \frac{\text{the number of simple outcomes in }A}{\text{the number of simple outcomes in }S}.\]It follows from definition that \(0\leq\text{P}(A)\leq1\).

- If \(\text{P}(A) = 1\) we say that the event \(A\) is certain because \(A\) is identical to \(S\).
- If \(\text{P}(A) = 0\) we say that the event \(A\) is impossible because \(A\) is empty.

The set with no outcomes in it is called the empty set and is written \(\varnothing\); therefore \(\text{P}(\varnothing)=0\).

Exercise

For each of the following events, \(A\), \(B\) and \(C\), determine the probabilities of them occuring, i.e. determine the probabilities \(\text{P}(A)\), \(\text{P}(B)\) and \(\text{P}(C)\). Therefore, calculate the probabilities of their complements, \(\text{P}(A')\), \(\text{P}(B')\) and \(\text{P}(C')\).

1. \(A = \{\text{Throwing }[value]\text{ or higher with a fair }[value]\text{-sided die}\}\)
2. \(B = \{\text{Throwing exactly two Heads with three coins}\}\)
3. \(C = \{\text{Throwing a total score of }[value]\text{ with a pair of fair }[value]\text{-sided dice}\}\)

Give your answers as integers or fractions, not decimals:

1. \(\text{P}(A)=\quad\)\(\quad\)and therefore \(\text{P}(A')=\quad\)
2. \(\text{P}(B)=\quad\)\(\quad\)and therefore \(\text{P}(B')=\quad\)
3. \(\text{P}(C)=\quad\)\(\quad\)and therefore \(\text{P}(C')=\quad\)
Feedback: Solution 1. There are \([value]\) equally like outcomes in the sample space, of these \([value]\) are greater than or equal to \([value].\) Therefore, \(\text{P}(A) = \displaystyle [value]\) and \(\text{P}(A') = \displaystyle [value].\) 2. There are eight equally likely outcomes of which three, \(\{HHT,HTH,THH\}\) are elements of \(B.\) Therefore, \(\text{P}(B) = \displaystyle[value]\) and \(\text{P}(B') = \displaystyle[value].\) 3. It is not possible to score \([value]\) on a pair of \([value]\)-sided die. Therefore, \(\text{P}(C) = [value]\) and \(\text{P}(C') = [value].\)

Not surprisingly, the probabilities of an event \(A\) and its complement are related. The probability of the event \(A'\) is easily found from the identity\[\frac{\text{number of outcomes in }A}{\text{total number of outcomes}} + \frac{\text{number of outcomes not in }A}{\text{total number of outcomes}} \equiv 1\]so that \(\text{P}(A) + \text{P}(A') \equiv 1\).

Key Point

The Complement Rule

\[\text{P}(A') = 1 - \text{P}(A)\]In words: The probability of the complement of \(A\) occurring is equal to 1 minus the probability of \(A\) occurring.

The use of the event \(A'\) can sometimes simplify the calculation of the probability \(\text{P}(A)\). For example, suppose that two 6-faced dice are thrown and we require the probability of the event\[A\text{: that we obtain a total score of at least four.}\]There are many combinations that produce a total score of at least four; however there are only \(3\) combinations that produce a total score of two or three which is the complementary event to the one of interest. The event \(A'=\{(1,1),(1,2),(2,1)\}\) (where we use an obvious notation of stating the total score on the first die followed by the score on the second die) is the complement of \(A\).

Now \(\text{P}(A')=\frac{3}{36}\) since there are \(6\times 6\) possible combinations in throwing two dice. Thus \[\qquad \text{P}(A)=1-\text{P}(A')=1-\frac{3}{36}=\frac{33}{36}=\frac{11}{12}.\]

Exercise

Find the probability of obtaining a total score of at least five when three fair \([value]\)-sided dice are thrown. Hint: identify \(A\) and \(A'\), then calculate \(\text{P}(A')\), and then \(\text{P}(A)\):

\(\text{P}(A)=\quad\)
Feedback: Solution Three fair \([value]\)-sided dice are thrown.There are \([value]\times[value]\times[value] = [value]\) possible outcomes. If, for example, \((1,3,4)\) denotes the scores of \(1\) on die one, \(3\) on die two and \(4\) on die three and if \(A\) is the event ‘a total score of five or more’ then \(A'\) is the event ‘a total score of less than 5’ i.e.\[A' = \{(1,1,1),(2,1,1),(1,2,1),(1,1,2)\}\] There are four outcomes in \(A'\) and hence \(\text{P}(A') = \dfrac{4}[value]\) so that \(\text{P}(A) = \dfrac[value][value] = \displaystyle [value] .\)

Exercise

Consider the following experiments. In each case state the sample space.

1. The number, \(n\), of defective items in a batch of \([value]\) is noted.
2. The weight, \(w\), in kg, of lubricating oil drained from a machine is determined using a spring balance. Assume that there is at least some oil drained from the machine.
3. The natural logarithm of the weight, \(\ln{w}\), in kg, according to a spring balance, of lubricating oil drained from a machine, is noted. Assume that there is at least some oil drained from the machine.

In STACK connected intervals on the real line are typed the following way:

- \(\texttt{oo(a,b)}\) for the "open-open" interval \((a,b)\).
- \(\texttt{oc(a,b)}\) for the "open-closed" interval \((a,b]\).
- \(\texttt{co(a,b)}\) for the "closed-open" interval \([a,b)\).
- \(\texttt{cc(a,b)}\) for the "closed-closed" interval \([a,b]\).

The discrete set should be typed as \(\texttt{{a,b,c,d,...}}\). If you need to use \(\infty\), then type \(\texttt{inf}\) (type \(\texttt{-inf}\) for \(-\infty\)).

1. The random variable \(n\) is discrete and is an element of the set
2. The random variable \(w\) is continuous and is an element of the set
3. The random variable \(\ln{w}\) is continuous and is an element of the set
Feedback: 1. \(n\in[value]\). 2. \(w\in[value]\). 3. \(\ln{w}\in[value]\).

Exercise

An experiment consists of throwing two \([value]\)-faced dice.

1. Write down the sample space of one of the die in this experiment.
2. If \(A\) is the event that the total score is less than \(5\), find \(\text{P}(A)\).
3. If \(B\) is the event that a double, i.e. \((1,1),(2,2),(3,3),...\), is thrown, find \(\text{P}(B')\).
4. What is the probability that a double is not thrown and the score is less than \(5\)? The event here is notated \(A\cap B'\), so that the probability we are seeking is \(\text{P}(A\cap B')\).

Give all probabilities as fractions, not decimals:

1. The sample space of one of the die in this experiment is .
2. \(\text{P}(A)=\quad\)
3. \(\text{P}(B')=\quad\)
4. \(\text{P}(A\cap B')=\quad\)
Feedback: 1. The sample space is a set consisting of all pairs \((a,b)\) such that \(a,b\) are elements of \([value].\) 2. The pairs that make a total score less than \(5\) are \(\{(1,1),(1,2),(1,3),(2,1),(2,2),(3,1)\}\) so that \(\text{P}(A)= \dfrac{6}[value] = \displaystyle [value].\) 3. There are \([value]\) possible doubles so that \(\text{P}(B) = \dfrac[value][value] = \dfrac{1}[value]\) and therefore \(\text{P}(B')=1 - \left( \dfrac{1}[value] \right) = \displaystyle [value].\) 4. The set \(A\cap B'\) consists only of the pairs \((1,2),(1,3),(2,1),(3,1)\) so that \(\text{P}(A\cap B')= \displaystyle [value].\)

Exercise

A lot consists of \([value]\) good articles, \([value]\) articles with only minor defects and \([value]\) with major defects. One article is chosen at random from the lot.

1. If \(G\) is the event that the article has no defects, find \(\mathbb{P}(G)\).
2. If \(M\) is the event that the article has major defects, find \(\mathbb{P}(M')\).
3. Find the probability that the article is either good or has major defects. This event is denoted \(G\cup M\), so that we are seeking the probability \(\mathbb{P}(G\cup M)\).

Give all probabilities as fractions, not decimals:

1. \(\mathbb{P}(G)=\quad\)
2. \(\mathbb{P}(M')=\quad\)
3. \(\mathbb{P}(G\cup M)=\quad\)
Feedback: Solution The total number of articles is \([value] + [value] + [value] = [value]\). 1. Only the \([value]\) good articles have no defects, therefore \( \mathbb{P}(G) = [value] / [value] = [value] \). 2. The event \(M'\) corresponds to the article having no major defects, this occurs if we pick one of the \( [value] \) good articles or one of the \( [value] \) articles with only minor defects, therefore \(\mathbb{P}(M') = ([value] + [value]) / [value] = [value]\). 3. There are \( [value] \) good articles and \( [value] \) articles with major defects, therefore \( \mathbb{P}(G\cup M) = ([value] + [value]) / [value] = [value]\).

Exercise

Propeller shafts for marine applications are inspected to ensure that they satisfy both diameter requirements and surface finish requirements. The results of [value] inspections are as follows.

From the inspections it was found that:

- \([value]\) shafts satisfy both diamater and surface finish requirements;
- \([value]\) shafts satisfy diameter requirements, but not surface finish requirements;
- \([value]\) shafts do not satisfy diameter requirements, but do satisfy surface finish requirements;
- \([value]\) shafts satisfy neither diameter or surface finish requirements.

A shaft is selected at random. Let \(D\) be the event that it satsifies diameter requirements, and \(F\) the event that it satisfies surface finish requirements. Find the probability that:

1. the shaft satisfies surface finish requirements, i.e. \(\text{P}(F)\);
2. the shaft satisfies both diameter and surface finish requirements, i.e. \(\text{P}(D\cap F)\);
3. the shaft satisfies either diameter or surface finish requirements, i.e. \(\text{P}(D\cup F)\);
4. the shaft satisfies neither diameter or surface finish requirements, i.e. \(\text{P}(D'\cap F')\).

Give all probabilities as fractions, not decimals:

1. \(\text{P}(F)=\quad\)
2. \(\text{P}(D\cap F)=\quad\)
3. \(\text{P}(D\cup F)=\quad\)
4. \(\text{P}(D'\cap F')=\quad\)
Feedback: 1. There are \([value] + [value] = [value]\) shafts that satisfy the surface finish conditions so that \(\text{P}(F) = \dfrac{[value] + [value]}[value] = \displaystyle [value].\) 2. There are \([value]\) shafts that satisfy both requirements, therefore \(\text{P}(D\cap F) = \dfrac[value][value] =\displaystyle [value].\) 3. There are \([value] + [value] + [value] = [value]\) shafts that satisfy either requirement, therefore \(\text{P}(D\cup F) = \dfrac{[value] + [value] + [value]}[value] = \displaystyle [value]. \) 4. There are \([value]\) shafts that do not satisfy either requirement so that \(\text{P}(D'\cap F') = \dfrac[value][value] = \displaystyle [value]. \)