Time Series Analysis

A time series consists of observations at discrete equispaced intervals of time.  For example, sales in month t could be denoted as zt and in the previous month by zt-1.  Typically, the objective is to forecast future values of z (such as zt+1) based on present and past values of z and perhaps explanatory variables such as price.

A model in which future values are forecast purely on the basis of past values of the time series is called an autoregressive (AR) process.

A model in which future values are forecast purely on the basis of past shocks (or noise or random disturbances) is called a moving average (MA) process.

A model that uses both past values of the time series and past shocks is called an autoregressive-moving average (ARMA) process.

These models assume that the time series is stationary - that is the series fluctuates around a time invariant mean, and the variance and autocovariance (ie covariance between zt and zt-s for all values of s) do not vary with time.  In practice, most time series need to be transformed to achieve stationarity.  To stabilise variance a logarithm transform is often used - appropriate where the variance of the series increases in proportion to the mean.  To stabilise the mean, differencing is usually employed.  For example, first order differencing is xt = zt - zt-1.  First order differencing eliminates "drift" but often it needs to be applied twice to eliminate trend and seasonal differencing is often necessary too.  An ARMA model of a differenced series is called an ARIMA model, where the I stands for Integrated because the output needs to be anti-differenced, or integrated, to forecast the original series.

Most forecasting models also need to account for explanatory variables such as price, advertising, or income.  Often, there are lagged impacts as well as (or instead of) immediate impacts - that is past values of explanatory variables can be important.  Such a model is called a Transfer Function model.  It is similar to a distributed lag regression model except that the residuals (or error series) are modelled as an ARMA model.  The ARMA component makes the model "adaptive" to shocks and changes in trend and level.

Collectively, these models along with the process of identification, fitting, and diagnostic checking are called Box Jenkins models after the authors of the "Bible" on the subject (Time Series Analysis, George E. P. Box and Gwilym M. Jenkins, Holden-Day, 1976).