Smoothing Time Series

Read:

Chapter fpp3 8: Exponential Smoothing

• Local Smoothing examples will be given
  • Moving Average:

\(\begin{equation} \hat{T}_{t} = \frac{1}{m} \sum_{j=-k}^k y_{t+j} \end{equation}\)

• Local Regression (locally estimated scatterplot smoothing - LOESS, locally weighted scatterplot smoothing - LOWESS)
  

• Spline

• Global Smoother Examples: Linear models, generalizes linear modes, generalized additive models are some examples of global smoother

• In general, any model that tries to noise removal may be seen as a smoother

Antidiabetic Drug Sales Data

Linear Model (aka, linear regression)

Quadratic Model (Orthogonal)

Cubic Model (Orthogonal)

Re-Weighted Least Square

GAM

Moving Average (Covered in Previous Lecture)

LOESS (Local Polynomial Regression, With Various Span Values)

timeSeries package contians some smoothing functions that are wrapper of base functions lowess, supmsu and smooth.spline :
- smoothLowess(): locally weighted regression (Cleveland, 1981)
- smoothSpline(): Chambers & Hastie (1992)
- smoothSupsmu(): Friedman’s (1984) SuperSmoother

Exponential Smoothing

Read:

Chapter 8 of FPP

Exponential Smoothing

Forecasting time series using R, R. Hyndman

Simple Exponential Smoothing

• Simple Exponential Smoothing is not suitable for time series with seasonal and trend/cycle components

\(\hat{x}_{T+1|T} = \alpha{x_t} + \alpha(1-\alpha)x_{t-2} + \cdots+ \alpha(1-\alpha)^{t-1}x_{1} + \dots\)

where \(0<\alpha<1\) is a smoothing parameter. This can be written as;

\(\hat{x}_{t+1} = l_{t}\)

\(l_{t} = \alpha{x_{t}} + (1 - \alpha)l_{t-1}\)

• The one-step-ahead forecast for time \(T+1\) is a weighted average of all of the observations in the series \(x_1,\dots,x_T\)

Weighted average form

• Let the first fitted value at time 1 be denoted by \(\ell_0\)

\(\begin{align*} \hat{x}_{2|1} &= \alpha x_1 + (1-\alpha) \ell_0\\ \hat{x}_{3|2} &= \alpha x_2 + (1-\alpha) \hat{x}_{2|1}\\ \hat{x}_{4|3} &= \alpha x_3 + (1-\alpha) \hat{x}_{3|2}\\ \vdots\\ \hat{x}_{T|T-1} &= \alpha x_{T-1} + (1-\alpha) \hat{x}_{T-1|T-2}\\ \hat{x}_{T+1|T} &= \alpha x_T + (1-\alpha) \hat{x}_{T|T-1}. \end{align*}\)

• Substituting each

\(\begin{align*} \hat{x}_{3|2} & = \alpha x_2 + (1-\alpha) \left[\alpha x_1 + (1-\alpha) \ell_0\right] \\ & = \alpha x_2 + \alpha(1-\alpha) x_1 + (1-\alpha)^2 \ell_0 \\ \hat{x}_{4|3} & = \alpha x_3 + (1-\alpha) [\alpha x_2 + \alpha(1-\alpha) y_1 + (1-\alpha)^2 \ell_0]\\ & = \alpha x_3 + \alpha(1-\alpha) x_2 + \alpha(1-\alpha)^2 y_1 + (1-\alpha)^3 \ell_0 \\ & ~~\vdots \\ \hat{x}_{T+1|T} & = \sum_{j=0}^{T-1} \alpha(1-\alpha)^j x_{T-j} + (1-\alpha)^T \ell_{0}. \end{align*}\)

Component form

\(\begin{align*} \text{Forecast equation} && \hat{x}_{t+h|t} & = \ell_{t}\\ \text{Smoothing equation} && \ell_{t} & = \alpha x_{t} + (1 - \alpha)\ell_{t-1}, \end{align*}\)

Estimation

• Initial value of \(\alpha,\ell_0\) must be selected (sometimes these parameters are selected subjectively)

\(\begin{equation} \text{SSE}=\sum_{t=1}^T(y_t - \hat{y}_{t|t-1})^2=\sum_{t=1}^Te_t^2 \end{equation}\)

• Non-linear minimisation using an optimisation tool to solve this equation

Series: Exports 
Model: ETS(A,N,N) 
  Smoothing parameters:
    alpha = 0.8399875 

  Initial states:
   l[0]
 39.539

  sigma^2:  35.6301

     AIC     AICc      BIC 
446.7154 447.1599 452.8968 

Forecast

A Trend + Seasonal Time Series Example

Holt’s Linear Trend Method

• Additive

\(\hat{x}_{t+1} = \ell_t + kT_t\)

\(\ell_t\) is estimated level
\(T_t \: or\: b_t\) is estimated trend

\(\ell_t = \alpha{x_t} + (1-\alpha) (\ell_{t-1} + T_{t-1})\)

\(0\le\alpha\le1\)

\(T_t = \beta^*(\ell_t - \ell_{t-1}) + (1-\beta^*)T_{t-1}\)
\(0<\beta^*<1\)

or like in FPP (section 8.2)

\(\begin{align*} \text{Forecast equation}&& \hat{x}_{t+h|t} &= \ell_{t} + hb_{t} \\ \text{Level equation} && \ell_{t} &= \alpha x_{t} + (1 - \alpha)(\ell_{t-1} + b_{t-1})\\ \text{Trend equation} && b_{t} &= \beta^*(\ell_{t} - \ell_{t-1}) + (1 -\beta^*)b_{t-1}, \end{align*}\)

• Multiplicative

\(\hat{x}_{t+1} = l_t \times kT_t\)

\(L_t = \alpha{x_t} + (1-\alpha) (l_{t-1} \times T_{t-1})\)

\(T_t = \beta(\frac{l_t}{l_{t-1}}) + (1-\beta)T_{t-1}\)

Forecast method: Holt's method

Model Information:
Holt's method 

Call:
holt(y = qcement, h = 10)

  Smoothing parameters:
    alpha = 0.3028 
    beta  = 1e-04 

  Initial states:
    l = 0.5049 
    b = 0.0081 

  sigma:  0.1414

     AIC     AICc      BIC 
364.4692 364.7335 381.7244 

Error measures:
                       ME      RMSE       MAE        MPE     MAPE     MASE
Training set 0.0002894225 0.1401761 0.1074737 -0.5524654 7.181068 1.052087
                   ACF1
Training set 0.09672077

Forecasts:
        Point Forecast    Lo 80    Hi 80    Lo 95    Hi 95
2014 Q2       2.427549 2.246344 2.608754 2.150420 2.704678
2014 Q3       2.435677 2.246340 2.625014 2.146111 2.725243
2014 Q4       2.443805 2.246665 2.640944 2.142306 2.745303
2015 Q1       2.451932 2.247283 2.656581 2.138949 2.764916
2015 Q2       2.460060 2.248163 2.671958 2.135991 2.784129
2015 Q3       2.468188 2.249277 2.687098 2.133393 2.802983
2015 Q4       2.476316 2.250605 2.702026 2.131122 2.821510
2016 Q1       2.484443 2.252128 2.716759 2.129148 2.839739
2016 Q2       2.492571 2.253829 2.731313 2.127447 2.857695
2016 Q3       2.500699 2.255695 2.745703 2.125998 2.875400

Forecast Plot

Residuals

• Observe the Seasonality on the ACF graph

Holt-Winter’s Method

• Good for time series with trend and seasonality.

• Apllicable to Additive and Multiplicative time series

Additive:

\(\begin{align*} \hat{x}_{t+h|t} &= \ell_{t} + hb_{t} + s_{t+h-m(k+1)} \\ \ell_{t} &= \alpha(x_{t} - s_{t-m}) + (1 - \alpha)(\ell_{t-1} + b_{t-1})\\ b_{t} &= \beta^*(\ell_{t} - \ell_{t-1}) + (1 - \beta^*)b_{t-1}\\ s_{t} &= \gamma (x_{t}-\ell_{t-1}-b_{t-1}) + (1-\gamma)s_{t-m}, \end{align*}\)

\(\ell_t\) is the level, and \(S_t\) is the seasonal component. \(k\) is integer part of \((h-1)/m\), \(m\) denotes the period of the seasonality, for example, \(m=12\) for yearly data.

\(0<\alpha, \beta^* \; and \; \gamma<1\) are smoothing parameters.

Multiplicative:

\(\begin{align*} \hat{x}_{t+h|t} &= (\ell_{t} + hb_{t})s_{t+h-m(k+1)} \\ \ell_{t} &= \alpha \frac{x_{t}}{s_{t-m}} + (1 - \alpha)(\ell_{t-1} + b_{t-1})\\ b_{t} &= \beta^*(\ell_{t}-\ell_{t-1}) + (1 - \beta^*)b_{t-1} \\ s_{t} &= \gamma \frac{x_{t}}{(\ell_{t-1} + b_{t-1})} + (1 - \gamma)s_{t-m}. \end{align*}\)

Example: Quarterly Australian Portland Cement production

• Let’s fit both additive and multiplicative H-W model.

• Here are the estimated parameters

Additive Model: Cement Example

Series: value 
Model: ETS(A,A,A) 
  Smoothing parameters:
    alpha = 0.6281542 
    beta  = 0.008504617 
    gamma = 0.2007587 

  Initial states:
      l[0]       b[0]        s[0]      s[-1]     s[-2]       s[-3]
 0.5159829 0.01504697 -0.01162037 0.02647345 0.0295095 -0.04436257

  sigma^2:  0.0074

     AIC     AICc      BIC 
135.3290 136.1361 166.3883 

Multiplicative

Series: value 
Model: ETS(M,A,M) 
  Smoothing parameters:
    alpha = 0.7504791 
    beta  = 0.002970008 
    gamma = 0.0001000013 

  Initial states:
      l[0]        b[0]     s[0]    s[-1]   s[-2]    s[-3]
 0.5042987 0.008074177 1.029664 1.048807 1.01635 0.905178

  sigma^2:  0.0022

      AIC      AICc       BIC 
 6.783846  7.591021 37.843192 

Together

Components

• Here is the first ten estimated components

• Plot of the components

Model Performance

And the forecast