A confidence interval gives an estimated range of values which is likely to include an unknown population parameter, the estimated range being calculated from a given set of sample data

This is the definition of confidence interval given in the Statistics Glossary v1.1 by Valerie J. Easton & John H. McColl. The “unknown population parameter” is usually the population mean, so in the following I will just assume that the “unknown population parameter” is indeed the mean.

Thus we are dealing with a sample of a population and we want to measure how close we get to the population mean using only data about a sample.

If independent samples are taken from the same population and confidence interval evaluated for each sample then a certain percentage (called confidence level) of the intervals will include the population mean. The confidence level is usually 95%, but we can get to 99%, 90% or any other percentage we fancy.

I’m always a bit let down when I read a paper and authors do not report the confidence interval of their experimental results. It means that whatever measure they are reporting you have to guess whether it is significant or not.

I think it’s good to make a habit of including the confidence interval for any measurement you are reporting.

In most practical settings, we don’t actually know what is the population distribution and we just assume that it is normally distributed. For samples from other population distributions what I am going to describe is approximately correct by the Central Limit Theorem.

For a population with unknown mean \(\mu\), unknown standard deviation \(\sigma\), a confidence interval for the population mean, based on a random sample of size, is \(\overline{x}\pm t^*\frac{s}{\sqrt{n}}\) where:

  • \(\overline{x}\) is the sample mean;
  • \(n\) is the sample size;
  • \(s\) is the estimated standard deviation (also known as standard error);
  • \(t^*\) is the upper \(\frac{1-C}{2}\) critical value for the Student’s t-distribution with \(n-1\) degrees of freedom.

The most difficult element is to evaluate \(t^*\).

Assume that we are given the height in cm of 30 one year old toddlers: 63.5, 81.3, 88.9, 63.5, 76.2, 67.3, 66.0, 64.8, 74.9, 81.3, 76.2, 72.4, 76.2, 81.3, 71.1, 80.0, 73.7, 74.9, 76.2, 86.4, 73.7, 81.3, 68.6, 71.1, 83.8, 71.1, 68.6, 81.3, 73.7, 74.9.

The average height is 74.8 cm. What is the 95% confidence interval of this mean?

Mean Confidence Interval in Java

The Apache Commons Math 3 can give critical values for the Student’s t-distribution. So download it or use your dependency manager to use it. Here is the code that calculates the 95% confidence interval:

import org.apache.commons.math3.distribution.TDistribution;
import org.apache.commons.math3.exception.MathIllegalArgumentException;
import org.apache.commons.math3.stat.descriptive.SummaryStatistics;

public class ConfidenceIntervalApp {

    public static void main(String args[]) {
        // data we want to evaluate: average height of 30 one year old male and female toddlers
        // interestingly, at this age height is not bimodal yet
        double data[] = new double[] { 63.5, 81.3, 88.9, 63.5, 76.2, 67.3, 66.0, 64.8, 74.9, 81.3, 76.2, 72.4, 76.2, 81.3, 71.1, 80.0, 73.7, 74.9, 76.2, 86.4, 73.7, 81.3, 68.6, 71.1, 83.8, 71.1, 68.6, 81.3, 73.7, 74.9 };
        // Build summary statistics of the dataset "data"
        SummaryStatistics stats = new SummaryStatistics();
        for (double val : data) {
            stats.addValue(val);
        }

        // Calculate 95% confidence interval
        double ci = calcMeanCI(stats, 0.95);
        System.out.println(String.format("Mean: %f", stats.getMean()));
        double lower = stats.getMean() - ci;
        double upper = stats.getMean() + ci;
        System.out.println(String.format("Confidence Interval 95%%: %f, %f", lower, upper));
    }

    private static double calcMeanCI(SummaryStatistics stats, double level) {
        try {
            // Create T Distribution with N-1 degrees of freedom
            TDistribution tDist = new TDistribution(stats.getN() - 1);
            // Calculate critical value
            double critVal = tDist.inverseCumulativeProbability(1.0 - (1 - level) / 2);
            // Calculate confidence interval
            return critVal * stats.getStandardDeviation() / Math.sqrt(stats.getN());
        } catch (MathIllegalArgumentException e) {
            return Double.NaN;
        }
    }

}

The output of this program is:

Mean: 74.806667
Confidence Interval 95%: 72.328860, 77.284474

Mean Confidence Interval in Python

The code to do the same calculation in Python is very similar. We will use numpy and scipy:

#!/usr/bin/env python

from scipy.stats import t
from numpy import average, std
from math import sqrt

if __name__ == '__main__':
    # data we want to evaluate: average height of 30 one year old male and
    # female toddlers. Interestingly, at this age height is not bimodal yet
    data = [63.5, 81.3, 88.9, 63.5, 76.2, 67.3, 66.0, 64.8, 74.9, 81.3, 76.2,
            72.4, 76.2, 81.3, 71.1, 80.0, 73.7, 74.9, 76.2, 86.4, 73.7, 81.3,
            68.6, 71.1, 83.8, 71.1, 68.6, 81.3, 73.7, 74.9]
    mean = average(data)
    # evaluate sample variance by setting delta degrees of freedom (ddof) to
    # 1. The degree used in calculations is N - ddof
    stddev = std(data, ddof=1)
    # Get the endpoints of the range that contains 95% of the distribution
    t_bounds = t.interval(0.95, len(data) - 1)
    # sum mean to the confidence interval
    ci = [mean + critval * stddev / sqrt(len(data)) for critval in t_bounds]
    print "Mean: %f" % mean
    print "Confidence Interval 95%%: %f, %f" % (ci[0], ci[1])

The output is exactly the same of the Java version.