Expectation and Variance

Expectation and Variance

The expectation is the expected value of X, written as E(X) or sometimes as μ.

The expectation is what you would expect to get if you were to carry out the experiment a large number of times and calculate the 'mean'.

To calculate the expectation we can use the following formula:

E(X) = ∑ xP(X = x)

It may look complicated, but in fact is quite easy to use.

You multiply each value of x with its corresponding probability. If we then add all these up we obtain the expectation of X. This is best seen in an example.

Example:

Suppose you roll a die and let X be the number on the uppermost face. Find the expectation of X, E(X).

The probability distribution is given by:

So we expect 3.5. This is what we would expect if we were to roll the dice a large number of times and find the mean.

This is a 'special' discrete random variable as all the probabilities are the same.

P(X = x) = 1/6. This distribution is said to be a uniform distribution.

With uniform distributions it is possible to calculate the expectation by using the symmetry of the table. The expectation, E(X) is calculated by finding the halfway point.

Note: For our example using the symmetry of the distribution we find halfway of 1 2 3 4 5 6, which is 3.5 (just like the median).

If X is a discrete random variable and f(x) is any function of x, then the expected value of f(x) is given by:

E[f(x)] = ∑ f(x)P(X = x)

Again, another tricky-looking formula best explained using an example.

Example:

The number of heads obtained when flipping 3 coins is the discrete random variable, X which has the following probability distribution.

x 0 1 2 3
P(X = x) 1/8 3/8 3/8 1/8

We can work out the expectation as follows...

E(X) = (0 × 1/8) + (1 × 3/8) + (2 × 3/8) + (3 × 1/8) = 1.5

What though if we want to work out the value of E(X2)?

Let's first add an extra row to our table of x2...

x 0 1 2 3
x2 0 1 4 9
P(X = x) 1/8 3/8 3/8 1/8

This time instead of multiplying each probability by x, we multiply by x2.

So:

E(X2) = ( 0 × 1/8) + (1 × 3/8) + (4 × 3/8) + (9 × 1/8) = 3

Similarly, if we wanted to calculate E(X3) we would have to add up all the probabilities multiplied by x3

Question:

What would I need to multiply the probabilities by in order to calculate the following?

There are a few general results we should remember to help with our calculations...

E(a) = a

E(aX + b) = aE(X) + b

Where a and b are both constants.

This means by knowing just the expectation, E(X), we can calculate other expectations quickly.

Example:

If E(X) = 3 then:

  1. E(2X) = 2E(X) = 6
  2. E(5X) = 5E(X) = 15
  3. E(4X + 2) = 4E(X) + 2 = 14

Try for yourself

The variance is a measure of how spread out the values of X would be if the experiment leading to X were repeated a number of times.

The variance of X, written as Var(X) is given by:

Var(X) = E(X2) − (E(X))2

If we write E(X) = μ then:

Var(X) = E(X2) − μ2

Or:

Var(X) = E(X − μ)2

This tells us that Var(X) ≥ 0

Example:

Calculate the expectation and variance of X of the following distribution:

x 0 1 2 3
P(X = x) 0.1 0.2 0.5 0.2

We already know how to calculate E(X) and E(X2).

E(X) = (0 × 0.1) + (1 × 0.2) + (2 × 0.5) + (3 × 0.2) = 1.8

E(X2) = (0 × 0.1) + (1 × 0.2) + (4 × 0.5) + (9 × 0.2) = 4

So:

Var(X) = E(X2) − (E(X))2 = 4 − 1.82 = 0.76

There are a few general results we should remember to help with our calculations...

Var(aX) = a2Var(X)

Var(aX + b) = a2Var(X)

Where a and b are both constants.

This means by knowing just the variance, Var(X), we can calculate other variances quickly.

Example:

If Var(X) = 2.5 then,

Var(2X) = 22 × Var(X) = 4 × 2.5 = 10

Var(4X − 3) = 42 × Var(X) = 16 × 2.5 = 40

Try for yourself:

The square root of the Variance is called the Standard Deviation of X. Standard deviation is given the symbol σ.

√ Var(X) = σ, or Var(X) = σ2

S-cool exclusive!!