Theory


Vector autoregressions (VARs) are one of the most popular econometric models, and they find wide usage both for forecasting and structural analysis in academia and in central banks. However, they are not without problems. Many applications of these models are heavily parameterized and overfit the historical data, leading to poor generalisability out-of-sample. There are a number of responses to this problem:

  • The frequentist response to this problem is that the model is ‘over-parameterized’ and that the number of parameters should be chosen according to an information criterion or cross-validation.

  • The modern frequentist response is that the problem is not the number of parameters, per say, but more the fact that we are not applying sufficient regularization to the parameters, especially longer lags.

  • The Bayesian response puts the regularization argument in a coherent theoretical framework through the introduction of informative priors.

Below we show how PyFlux can be used to estimate Bayesian VARs.

PyFlux

We will load quarterly UK macroeconomic data for the past 25 years.

import pandas as pd
import matplotlib.pyplot as plt
import pyflux as pf
%matplotlib inline

data = pd.read_csv("uk_economy.csv")
data.index = pd.to_datetime(data['Date'])
data = data.iloc[:,1:]
data.head()

plt.figure(figsize=(15,5))
plt.plot(data)
plt.legend(data.columns.values,loc=3);
plt.ylabel("Rate (%)")
plt.title("UK Macroeconomic Data")
plt.show()
png

Next we specify the model. We will use a VAR with 4 lags:

var_model = pf.VAR(data=data,lags=4)

There are many approaches in the literature to construct priors. For example, Litterman’s ‘Minnesota Prior’ specifies the location of the first lag as 1 and the rest 0 as a random walk is often appropriate for economic time series. The Minnesota prior also contains a way to downweight the importance of past lags through the covariance hyperparameters. Here we follow the Litterman prior for the first moments, and set the location of the first own lags to 1 and the rest to 0. For the downweighting, we do something quick and dirty and penalize the longer lags with a discount rate \delta.

delta = 0.10 # harsh discount rate (we don't have much data)

var_model.adjust_prior([0,13,26],pf.Normal(0,100)) # constant prior - fairly uninformative

for i in range(4): # 4 lags
    if i == 0:
        var_model.adjust_prior([el+3*i for el in [1,14,27]],pf.Normal(1,1.0*(delta**(i)))) # AR(1) lags
    else:
        var_model.adjust_prior([el+3*i for el in [1,14,27]],pf.Normal(0,0.5*(delta**(i)))) # AR(1) lags
    var_model.adjust_prior([el+3*i for el in [2,3,15,16,28,29]],pf.Normal(0,0.5*(delta**(i)))) # AR(2+) lags
print(var_model.latent_variables)
Index    Latent Variable           Prior           Prior Latent Vars         V.I. Dist  Transform 
======== ========================= =============== ========================= ========== ==========
0        Inflation Rate Constant   Normal          mu0: 0, sigma0: 100       Normal     None      
1        Inflation Rate AR(1)      Normal          mu0: 1, sigma0: 1.0       Normal     None      
2        Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.5       Normal     None      
3        Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.5       Normal     None      
4        Inflation Rate AR(2)      Normal          mu0: 0, sigma0: 0.05      Normal     None      
5        Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.05      Normal     None      
6        Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.05      Normal     None      
7        Inflation Rate AR(3)      Normal          mu0: 0, sigma0: 0.005     Normal     None      
8        Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.005     Normal     None      
9        Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.005     Normal     None      
10       Inflation Rate AR(4)      Normal          mu0: 0, sigma0: 0.0005    Normal     None      
11       Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.0005    Normal     None      
12       Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.0005    Normal     None      
13       Interest Rate Constant    Normal          mu0: 0, sigma0: 100       Normal     None      
14       Interest Rate AR(1)       Normal          mu0: 1, sigma0: 1.0       Normal     None      
15       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.5       Normal     None      
16       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.5       Normal     None      
17       Interest Rate AR(2)       Normal          mu0: 0, sigma0: 0.05      Normal     None      
18       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.05      Normal     None      
19       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.05      Normal     None      
20       Interest Rate AR(3)       Normal          mu0: 0, sigma0: 0.005     Normal     None      
21       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.005     Normal     None      
22       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.005     Normal     None      
23       Interest Rate AR(4)       Normal          mu0: 0, sigma0: 0.0005    Normal     None      
24       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.0005    Normal     None      
25       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.0005    Normal     None      
26       Unemployment Rate Constan Normal          mu0: 0, sigma0: 100       Normal     None      
27       Unemployment Rate AR(1)   Normal          mu0: 1, sigma0: 1.0       Normal     None      
28       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.5       Normal     None      
29       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.5       Normal     None      
30       Unemployment Rate AR(2)   Normal          mu0: 0, sigma0: 0.05      Normal     None      
31       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.05      Normal     None      
32       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.05      Normal     None      
33       Unemployment Rate AR(3)   Normal          mu0: 0, sigma0: 0.005     Normal     None      
34       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.005     Normal     None      
35       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.005     Normal     None      
36       Unemployment Rate AR(4)   Normal          mu0: 0, sigma0: 0.0005    Normal     None      
37       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.0005    Normal     None      
38       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.0005    Normal     None      
39       Cholesky Diagonal 0       Uniform         n/a (non-informative)     Normal     exp       
40       Cholesky Off-Diagonal (1, Uniform         n/a (non-informative)     Normal     None      
41       Cholesky Diagonal 1       Uniform         n/a (non-informative)     Normal     exp       
42       Cholesky Off-Diagonal (2, Uniform         n/a (non-informative)     Normal     None      
43       Cholesky Off-Diagonal (2, Uniform         n/a (non-informative)     Normal     None      
44       Cholesky Diagonal 2       Uniform         n/a (non-informative)     Normal     exp       

We use black box variational inference to estimate latent variables with fit:

x = var_model.fit('BBVI',iterations=15000)
10% done : ELBO is -677.720094719
20% done : ELBO is -661.87767258
30% done : ELBO is -535.684632563
40% done : ELBO is -548.88707709
50% done : ELBO is -480.473314911
60% done : ELBO is -377.536675354
70% done : ELBO is -295.52917053
80% done : ELBO is -305.158759779
90% done : ELBO is -237.031168261
100% done : ELBO is -253.893858625

Final model ELBO is -168.909877996


Shrinkage for Inflation Latent Variables (for own lags)

Plotting the parameters for inflation’s own lags gives some indication as to the strength of our priors on the posterior for the longer lags.

var_model.plot_parameters([1],figsize=(15,1.5))
var_model.plot_parameters([1,4],figsize=(15,1.5))
var_model.plot_parameters([1,4,7],figsize=(15,1.5))
var_model.plot_parameters([1,4,7,10],figsize=(15,1.5))
png png png png

Shrinkage for Unemployment Latent Variables (for own lags)

var_model.plot_parameters([27],figsize=(15,1.5))
var_model.plot_parameters([27,30],figsize=(15,1.5))
var_model.plot_parameters([27,30,33],figsize=(15,1.5))
var_model.plot_parameters([27,30,33,36],figsize=(15,1.5))
png png png png

We can view predictions of our macroeconomic variables with plot_predict:

var_model.plot_predict(h=4,figsize=(15,5))
png png png

This was before the ‘Brexit thing’ happened…

Specifying a prior on the covariance matrix

Above we assumed non-informativeness about the covariance matrix. If we want to include prior information on the covariance matrix we can specify a Inverse Wishart prior. First, we can display the covariance matrix for our results above:

var_model.custom_covariance(var_model.latent_variables.get_z_values(transformed=False))
array([[ 0.48318869,  0.07586724 ,  -0.01367291],
       [ 0.07586724 ,  0.37076457, -0.03349389],
       [ -0.01367291, -0.03349389,  0.12133547]])

As an arbitrary example – merely to show the effects of the prior – we can specify the Inverse Wishart as I\left(3\right) and \nu=10 using construct_wishart:

import numpy as np
var_model.construct_wishart(10,np.eye(3))
var_model.fit('BBVI',iterations=15000)
10% done : ELBO is -591.29489848
20% done : ELBO is -407.427509007
30% done : ELBO is -366.373801007
40% done : ELBO is -353.011970078
50% done : ELBO is -295.351808742
60% done : ELBO is -266.511009254
70% done : ELBO is -202.983249715
80% done : ELBO is -224.763716642
90% done : ELBO is -317.18302103
100% done : ELBO is -201.125203562

Final model ELBO is -145.846007915

<pyflux.results.BBVIResults at 0x7f5d60d1df60>
var_model.custom_covariance(var_model.latent_variables.get_z_values(transformed=False))
array([[ 0.4316595,  0.05773277,  0.00947594],
       [ 0.05773277,  0.38248824, -0.02096291],
       [ 0.00947594, -0.02096291,  0.10627838]])