Data Representation

Definition. The mean of $n$ values $x_1, x_2, \ldots, x_n$ is

$\bar{x} = \frac{1}{n}\sum_{i=1}^{n}x_i$

Theorem. The function $S(a) = \displaystyle\sum_{i=1}^{n}(x_i - a)^2$ is minimised when $a = \bar{x}$.

Proof. Expand $S(a)$:

$S(a) = \sum(x_i^2 - 2ax_i + a^2) = \sum x_i^2 - 2a\sum x_i + na^2$

$\frac{dS}{da} = -2\sum x_i + 2na$

Setting $\dfrac{dS}{da} = 0$: $2na = 2\sum x_i \implies a = \dfrac{\sum x_i}{n} = \bar{x}$.

Check: $\dfrac{d^2S}{da^2} = 2n \gt 0$So this is a minimum. $\blacksquare$

Intuition. The mean is the “centre of mass” of the data. It is the single value that best Represents all the data points in the sense of least squares — no other value produces a smaller Total squared error. This is why the mean is the foundation of regression and estimation theory.

The median is the middle value when data are arranged in order. For $n$ values:

• If $n$ is odd: median = $\dfrac{n+1}{2}$-th value.
• If $n$ is even: median = average of $\dfrac{n}{2}$-th and $\left(\dfrac{n}{2}+1\right)$-th values.

The mode is the most frequently occurring value. A dataset can be unimodal, bimodal, or have no Mode.

• The mean uses all data values but is affected by outliers.
• The median is robust to outliers but ignores the magnitude of extreme values.
• The mode is useful for categorical data.

Caution: Warning Mean. A few extreme values can pull the mean far from the centre of the data.

The variance of $x_1, \ldots, x_n$ is

$\sigma^2 = \frac{1}{n}\sum_{i=1}^{n}(x_i - \bar{x})^2$

The standard deviation is $\sigma = \sqrt{\sigma^2}$.

Theorem. $\sigma^2 = \dfrac{\sum x_i^2}{n} - \bar{x}^2$

Proof.

For sample data, the unbiased estimator of the population variance is

$s^2 = \frac{1}{n-1}\sum_{i=1}^{n}(x_i - \bar{x})^2 = \frac{\sum x_i^2 - n\bar{x}^2}{n-1}$

The division by $n-1$ (Bessel”s correction) accounts for the fact that $\bar{x}$ is estimated from The same data, losing one degree of freedom.

• $Q_1$ (lower quartile): the median of the lower half of the data.
• $Q_2$ (median): the median of all the data.
• $Q_3$ (upper quartile): the median of the upper half.

The interquartile range (IQR) is $Q_3 - Q_1$.

A box plot displays:

• Minimum and maximum values (or whisker endpoints)
• $Q_1$, $Q_2$ (median), $Q_3$
• The box spans from $Q_1$ to $Q_3$
• The median is marked inside the box

An outlier is a value that lies more than $1.5 \times \mathrm{IQR}$ below $Q_1$ or above $Q_3$:

$\mathrm{Lower fence} = Q_1 - 1.5 \times \mathrm{IQR}$ $\mathrm{Upper fence} = Q_3 + 1.5 \times \mathrm{IQR}$

Values outside these fences are potential outliers.

Caution: Warning Some use $1.5 \times$ IQR, others use different multipliers.

Definition. Coding transforms data using $y = \dfrac{x - a}{c}$ where $a$ and $c$ are constants ($c \neq 0$).

If $y_i = \dfrac{x_i - a}{c}$Then:

$\bar{y} = \frac{\bar{x} - a}{c}, \qquad \sigma_y = \frac{\sigma_x}{|c|}$

Proof.

$\bar{y} = \frac{1}{n}\sum y_i = \frac{1}{n}\sum\frac{x_i - a}{c} = \frac{1}{c}\left(\frac{\sum x_i}{n} - a\right) = \frac{\bar{x} - a}{c}$

$\sigma_y^2 = \frac{1}{n}\sum(y_i - \bar{y})^2 = \frac{1}{n}\sum\left(\frac{x_i - a}{c} - \frac{\bar{x}-a}{c}\right)^2 = \frac{1}{c^2}\cdot\frac{1}{n}\sum(x_i-\bar{x})^2 = \frac{\sigma_x^2}{c^2}$

Hence $\sigma_y = \sigma_x/|c|$. $\blacksquare$

For data with frequencies $f_1, f_2, \ldots, f_k$:

$\bar{x} = \frac{\sum f_i x_i}{\sum f_i}, \qquad \sigma^2 = \frac{\sum f_i x_i^2}{\sum f_i} - \bar{x}^2$

Use the midpoint of each class as the representative value. This introduces an approximation Since we lose information about the distribution within each class.

Skewness measures the asymmetry of a distribution about its centre. A distribution is:

• Positively skewed (right-skewed): the right tail is longer; mean $\gt$ median.
• Negatively skewed (left-skewed): the left tail is longer; mean $\lt$ median.
• Symmetric: mean = median (and mode, for unimodal distributions).

Pearson’s first coefficient uses the mean, median, and standard deviation:

$S_1 = \frac{3\left(\bar{x} - Q_2\right)}{\sigma}$

Pearson’s second coefficient uses only the quartiles:

$S_2 = \frac{Q_3 + Q_1 - 2Q_2}{Q_3 - Q_1}$

Interpretation:

• $S \gt 0$: positive skew (right tail longer).
• $S \lt 0$: negative skew (left tail longer).
• $S = 0$: symmetric distribution.

For a unimodal distribution:

• Symmetric: mean = median = mode.
• Positively skewed: mode $\lt$ median $\lt$ mean.
• Negatively skewed: mean $\lt$ median $\lt$ mode.

The mean is pulled in the direction of the longer tail, while the mode remains at the peak and the Median lies between them.

As introduced in Section 3.3, the $1.5 \times \mathrm{IQR}$ rule defines fences. Some boards further Distinguish between mild and extreme outliers:

• Mild outlier: a value between $1.5 \times \mathrm{IQR}$ and $3 \times \mathrm{IQR}$ from the nearest quartile.
• Extreme outlier: a value more than $3 \times \mathrm{IQR}$ from the nearest quartile.

$\mathrm{Extreme lower fence} = Q_1 - 3 \times \mathrm{IQR}$ $\mathrm{Extreme upper fence} = Q_3 + 3 \times \mathrm{IQR}$

The modified z-score uses the median absolute deviation (MAD). For a dataset $x_1, x_2, \ldots, x_n$ with median $\tilde{x}$:

$\mathrm{MAD} = \mathrm{median}\left(|x_i - \tilde{x}|\right)$

The modified z-score for each observation is:

$M_i = \frac{0.6745\left(x_i - \tilde{x}\right)}{\mathrm{MAD}}$

An observation is flagged as an outlier if $|M_i| \gt 3.5$.

To construct a box plot:

1. Draw a horizontal (or vertical) number line covering the range of the data.
2. Draw a rectangular box from $Q_1$ to $Q_3$.
3. Mark the median $Q_2$ as a line inside the box.
4. Extend a whisker from $Q_1$ to the smallest data value within the lower fence, and from $Q_3$ to the largest data value within the upper fence.
5. Plot any values outside the fences as individual points (these are the outliers).

Compare the distances from $Q_2$ to each quartile:

• $Q_3 - Q_2 \gt Q_2 - Q_1$: the upper half is more spread out, indicating positive skew.
• $Q_3 - Q_2 \lt Q_2 - Q_1$: the lower half is more spread out, indicating negative skew.
• $Q_3 - Q_2 \approx Q_2 - Q_1$: the distribution is approximately symmetric.

Outliers on one side also indicate skewness in that direction.

When two or more box plots are drawn on the same scale, compare:

• Location: which distribution has the higher median?
• Spread: which has the larger IQR or total range?
• Skewness: do the distributions differ in shape?
• Outliers: does one distribution have more extreme values?

A back-to-back stem-and-leaf diagram places two distributions on either side of a shared stem, Enabling direct visual comparison of shape, spread, and outliers.

Example. Comparing test scores of two classes (Class A | Stem | Class B):

 8 7 5 | 5 | 3 4 6
 4 3 1 | 6 | 0 2 7 9
 6 2 0 | 7 | 1 3 5 8
 9 5   | 8 | 2 4
       | 9 | 1 7

Reading from the diagram: Class A has scores 55, 57, 58, 61, 63, 64, … While Class B has 53, 54, 56, 60, 62, 67, … Both classes share the stem (tens digit), with Class A on the left and Class B On the right.

A cumulative frequency curve (ogive) plots cumulative frequency against the upper class boundary Of each group. To compare two distributions:

1. Plot both ogives on the same axes.
2. Read off medians, quartiles, and percentiles from each curve.
3. Compare location (medians), spread (IQR), and shape (skewness).

When asked to compare two distributions in an exam, structure your response around four points:

1. Average: compare means or medians, stated in the context of the data.
2. Spread: compare standard deviations or IQRs, stated in context.
3. Shape: compare skewness where apparent.
4. Outliers: mention any unusual values and their effect.

Always relate numerical comparisons to the original context of the data.

When data are grouped into classes, quantiles are estimated using linear interpolation. For the $p$-th percentile:

$x_p = L + \left(\frac{p \cdot n}{100} - c_f\right) \cdot \frac{w}{f}$

Where:

• $L$ = lower class boundary of the class containing the $p$-th percentile
• $n$ = total frequency
• $c_f$ = cumulative frequency of all classes below $L$
• $w$ = class width
• $f$ = frequency of the class containing the $p$-th percentile

For the median ($p = 50$):

$Q_2 = L + \left(\frac{n}{2} - c_f\right) \cdot \frac{w}{f}$

For quartiles ($p = 25$ and $p = 75$):

$Q_1 = L + \left(\frac{n}{4} - c_f\right) \cdot \frac{w}{f}, \qquad Q_3 = L + \left(\frac{3n}{4} - c_f\right) \cdot \frac{w}{f}$

Find the median from the following grouped frequency distribution:

$n = 47$. The median position is $n/2 = 23.5$.

Cumulative frequencies: 5, 17, 35, 43, 47. The 23.5th value falls in the class $20 \lt x \le 30$.

$Q_2 = 20 + \left(23.5 - 17\right) \cdot \frac{10}{18} = 20 + 6.5 \cdot \frac{10}{18} = 20 + \frac{65}{18} \approx 23.6$

Class A | Stem | Class B
  9 7 3 |   4  | 1 2 5
  8 5 4 |   5  | 0 3 6 8
  6 2 0 |   6  | 1 4 7
  4     |   7  | 2 5 9
        |   8  | 3 6

1. Forgetting to check that solutions satisfy the original equation (especially with squaring both sides or dividing by variables).

2. Misreading the question, particularly with ‘hence’ vs ‘hence or otherwise’. The former requires using previous work.

3. Forgetting the $+c$ constant of integration in indefinite integrals, or misusing boundary conditions in definite integrals.

4. Losing marks by not showing sufficient working. Always write out each step, especially in proof questions.

• Correlation and Regression — The mean and standard deviation are foundational for calculating PMCC and regression coefficients.
• Probability — Probability distributions provide the theoretical basis for understanding data variation.
• Hypothesis Testing — Summary statistics from this topic feed directly into significance tests and confidence intervals.