In this article probability theory and random variables are introduced. If more information is required, the book "Probability and Random Processes" by Grimmett & Stirzaker (1982) is a good place to start.

Suppose a needle is dropped onto a floor made up of planks of wood. The needle may or may not intersect one of the joints between the planks. A single throw of the needle is called an experiment or trial. There are two possible outcomes. Either the needle intersects a joint(s) or it lands between the joints. By repeating the experiment a large number of times, the probability P of a particular outcome or event can be calculated. Let A be the event "needle intersects a joint". Let N(A) be the number of occurrences of A over n trials. As n tends to infinity, N(A)/n converges to the probability that A occurs, P(A), on any particular trial. On occasions, a probability can be assigned to an outcome without experiment.  Buffon (1777) found this to be the case for the needle throwing experiment. The set of all possible outcomes of an experiment is called the sample space and is denoted by W. A random variable is a function X : W ® Â.

Uppercase letters will be used to represent generic random variables, whilst lowercase letters will be used to represent possible numerical values of these variables. To describe the probability of possible values of X, consider the following definition. The distribution function of a random variable X is the function F_X : Â ® [0, 1] given by F_X(x) = P(X £ x).

Discrete random variables

The random variable X is discrete if it takes values in some countable subset {x₁, x₂, …}, only, of Â. The distribution function of such a random variable has jump discontinuities at the values x₁, x₂, … and is constant in between. The function f_X : Â ® [0, 1] given by f_X(x) = P(X = x) is called the (probability) mass function of X. The mean value, or expectation, or expected value of X with mass function f_X, is defined to be

(1)

The expected value of X is often written as m.

It is often of great interest to measure the extent to which a random variable X is dispersed. The variance of X or Var(X) is defined as follows:

(2)

The variance of X is often written as s², while its positive square root is called the standard deviation. Since X is discrete, (2) can be re-expressed accordingly:

(3)

In the special case where the mass function f_X(x) is constant and X takes n real values, (3) reduces to a well known equation determining the variance of a set of n numbers:

(4)

Events A and B are said to be independent if and only if the incidence of A does not change the probability of B occurring. An equivalent statement is P(A Ç B) = P(A)P(B). Similarly, the discrete random variables X and Y are called independent if the numerical value of X does not affect the distribution of Y. In other words, the events {X = x} and {Y = y} are independent for all x and y. The joint distribution function F_{X, Y} : ² ® [0, 1] of X and Y is given by F_{X, Y}(x, y) = P(X £ x and Y £ y). Their joint mass function f_{X, Y} : ² ® [0, 1] is given by f_{X, Y}(x, y) = P(X = x and Y = y). X and Y are independent if and only if f_{X, Y}(x, y) = f_X(x)f_Y(y) for all x, y Î Â.

Consider an archer, shooting arrows at the target shown in Figure 1. Suppose the archer is a very poor shot and hits the target randomly - in other words, target regions of equal area will have the same probability of being hit. For simplicity, it is assumed the archer always hits the target. If the archer is allowed to fire two arrows, the sample space

W = { AA, AB, AC, AD, AE, BA, BB, …, DD, DE, EA, EB, EC, ED, EE }.

Figure 1: An archery target. A hit in region A scores 4 points, B scores 3 points, C scores 2 points, D scores 1 point and E scores nothing.

Let the variable X(w) represent the score of a particular outcome. The scoring guidelines outlined in Figure 1 imply

X(AA) = 8, X(AB) = X(BA) = 7, X(AC) = X(BB) = X(CA) = 6, …,
X(CE) = X(DD) = X(EC) = 2, X(DE) = X(ED) = 1, X(EE) = 0.

Clearly X is a discrete random variable, mapping the sample space W to scores (real numbers).

The probability that an arrow hits a target region is directly proportional to the area of the region. The regions A to E are annuli with inner and outer radii as shown in Figure 1. The probabilities of hitting A to E are 1/25, 3/25, 5/25, 7/25 and 9/25 respectively. The mass function of X, f_X(x), is then

f_X(0) = P(X = 0) = P(Hit E)P(Hit E) = 81/625,
f_X(1) = P(X = 1) = 2 . P(Hit D)P(Hit E) = 126/625,
f_X(2) = P(X = 2) = 2 . P(Hit C)P(Hit E) + P(Hit D)P(Hit D) = 139/625 and so on.

From (1), the expected value of X is E(X) = 0.81/625 + 1.126/625 + 2.139/625 + 3.124/625 + … = 2.4. From (3), the variance of X is Var(X) = (2.4²)×81/625 + (1.4²)×126/625 + (0.4²)×139/625 + (0.6²)×124/625 + … = 2.72. The distribution function of X, F_X(x), is then

F_X(0) = P(X £ 0) = f_X(0),
F_X(1) = P(X £ 1) = f_X(1) + f_X(0),
F_X(2) = P(X £ 2) = f_X(2) + f_X(1) + f_X(0) and so on.

The distribution function F_X(x) is shown in Figure 2.

Figure 2: The distribution function F_X of X for the archery target.

Continuous random variables

The random variable X is continuous if its distribution function can be expressed as

(5)

for some integrable function f_X : Â ® [0, ¥). In this case, f_X is called the (probability) density function of X. The fundamental theorem of calculus and (5) imply

f(x)dx can be thought of as the element of probability P(x £ X £ x + dx) where

(6)

If B₁ is a measurable subset of  (such as a line segment or union of line segments) then

(7)

where P(X Î B₁) is the probability that the outcome of this random choice lies in B₁. The expected value (or expectation) of X with density function f_X is

(8)

whenever this integral exists. The variance of X or Var(X) is defined by the already familiar (2). Since X is continuous, (2) can be re-expressed accordingly:

(9)

The joint distribution function of the continuous random variables X and Y is the function F_{X, Y} : ² ® [0, 1] given by F_{X, Y}(x, y) = P(X £ x, Y £ y). X and Y are (jointly) continuous with joint (probability) density function f_{X, Y} : ² ® [0, ¥) if

for each x, y Î Â.  The fundamental theorem of calculus suggests the following result

f_{X, Y}(x, y)dxdy can be thought of as the element of probability P(x £ X £ x + dx, y £ Y £ y + dy) where

(10)

If B₂ is a measurable subset of ² (such as a rectangle or union of rectangles and so on) then

(11)

where P((X, Y) Î B₂) is the probability that the outcome of this random choice lies in B₂. X and Y are independent if and only if {X £ x} and {Y £ y} are independent events for all x, y Î Â. If X and Y are independent, F_{X, Y}(x, y) = F_X(x)F_Y(y) for all x, y Î Â. An equivalent condition is f_{X, Y}(x, y) = f_X(x)f_Y(y) whenever F_{X, Y} is differentiable at (x, y).

An example of a continuous random variable can be found in the needle throwing described earlier. A needle is thrown onto the floor and lands with random angle w relative to some fixed axis. The sample space W = [0, 2p). The angle W is equally likely in the real interval [0, 2p). Therefore, the probability that the angle lies in some interval is directly proportional to the length of the interval. Consider the continuous random variable X(w) = w. The distribution function of X, shown graphically in Figure 3, is

F_X(0) = P(X £ 0) = 0,
F_X(x) = P(X £ x) = x/2p, (0 £ x < 2p)
F_X(2p) = P(X £ 2p) = 1.

The density function, f_X, of F_X is as follows:

Figure 3: The distribution function F_X of X for the needle.

References

BUFFON, G. L. L. Comte de. Essai d'Arithmétique Morale. In: Supplément à l'Histoire Naturelle, v. 4. Paris: Imprimerie Royale (1777).

GRIMMETT, G. and STIRZAKER, D. Probability and Random Processes, Clarendon Press, Oxford (1982).