## Theory

Vector autoregressions (VARs) are one of the most popular econometric models, and they find wide usage both for forecasting and structural analysis in academia and in central banks. However, they are not without problems. Many applications of these models are heavily parameterized and overfit the historical data, leading to poor generalisability out-of-sample. There are a number of responses to this problem:

• The frequentist response to this problem is that the model is ‘over-parameterized’ and that the number of parameters should be chosen according to an information criterion or cross-validation.

• The modern frequentist response is that the problem is not the number of parameters, per say, but more the fact that we are not applying sufficient regularization to the parameters, especially longer lags.

• The Bayesian response puts the regularization argument in a coherent theoretical framework through the introduction of informative priors.

Below we show how PyFlux can be used to estimate Bayesian VARs.

## PyFlux

We will load quarterly UK macroeconomic data for the past 25 years.

import pandas as pd
import matplotlib.pyplot as plt
import pyflux as pf
%matplotlib inline

data.index = pd.to_datetime(data['Date'])
data = data.iloc[:,1:]

plt.figure(figsize=(15,5))
plt.plot(data)
plt.legend(data.columns.values,loc=3);
plt.ylabel("Rate (%)")
plt.title("UK Macroeconomic Data")
plt.show()


Next we specify the model. We will use a VAR with 4 lags:

var_model = pf.VAR(data=data,lags=4)


There are many approaches in the literature to construct priors. For example, Litterman’s ‘Minnesota Prior’ specifies the location of the first lag as $1$ and the rest $0$ as a random walk is often appropriate for economic time series. The Minnesota prior also contains a way to downweight the importance of past lags through the covariance hyperparameters. Here we follow the Litterman prior for the first moments, and set the location of the first own lags to $1$ and the rest to $0$. For the downweighting, we do something quick and dirty and penalize the longer lags with a discount rate $\delta$.

delta = 0.10 # harsh discount rate (we don't have much data)

var_model.adjust_prior([0,13,26],pf.Normal(0,100)) # constant prior - fairly uninformative

for i in range(4): # 4 lags
if i == 0:
var_model.adjust_prior([el+3*i for el in [1,14,27]],pf.Normal(1,1.0*(delta**(i)))) # AR(1) lags
else:
var_model.adjust_prior([el+3*i for el in [1,14,27]],pf.Normal(0,0.5*(delta**(i)))) # AR(1) lags
var_model.adjust_prior([el+3*i for el in [2,3,15,16,28,29]],pf.Normal(0,0.5*(delta**(i)))) # AR(2+) lags

print(var_model.latent_variables)

Index    Latent Variable           Prior           Prior Latent Vars         V.I. Dist  Transform
======== ========================= =============== ========================= ========== ==========
0        Inflation Rate Constant   Normal          mu0: 0, sigma0: 100       Normal     None
1        Inflation Rate AR(1)      Normal          mu0: 1, sigma0: 1.0       Normal     None
2        Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.5       Normal     None
3        Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.5       Normal     None
4        Inflation Rate AR(2)      Normal          mu0: 0, sigma0: 0.05      Normal     None
5        Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.05      Normal     None
6        Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.05      Normal     None
7        Inflation Rate AR(3)      Normal          mu0: 0, sigma0: 0.005     Normal     None
8        Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.005     Normal     None
9        Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.005     Normal     None
10       Inflation Rate AR(4)      Normal          mu0: 0, sigma0: 0.0005    Normal     None
11       Interest Rate to Inflatio Normal          mu0: 0, sigma0: 0.0005    Normal     None
12       Unemployment Rate to Infl Normal          mu0: 0, sigma0: 0.0005    Normal     None
13       Interest Rate Constant    Normal          mu0: 0, sigma0: 100       Normal     None
14       Interest Rate AR(1)       Normal          mu0: 1, sigma0: 1.0       Normal     None
15       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.5       Normal     None
16       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.5       Normal     None
17       Interest Rate AR(2)       Normal          mu0: 0, sigma0: 0.05      Normal     None
18       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.05      Normal     None
19       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.05      Normal     None
20       Interest Rate AR(3)       Normal          mu0: 0, sigma0: 0.005     Normal     None
21       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.005     Normal     None
22       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.005     Normal     None
23       Interest Rate AR(4)       Normal          mu0: 0, sigma0: 0.0005    Normal     None
24       Inflation Rate to Interes Normal          mu0: 0, sigma0: 0.0005    Normal     None
25       Unemployment Rate to Inte Normal          mu0: 0, sigma0: 0.0005    Normal     None
26       Unemployment Rate Constan Normal          mu0: 0, sigma0: 100       Normal     None
27       Unemployment Rate AR(1)   Normal          mu0: 1, sigma0: 1.0       Normal     None
28       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.5       Normal     None
29       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.5       Normal     None
30       Unemployment Rate AR(2)   Normal          mu0: 0, sigma0: 0.05      Normal     None
31       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.05      Normal     None
32       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.05      Normal     None
33       Unemployment Rate AR(3)   Normal          mu0: 0, sigma0: 0.005     Normal     None
34       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.005     Normal     None
35       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.005     Normal     None
36       Unemployment Rate AR(4)   Normal          mu0: 0, sigma0: 0.0005    Normal     None
37       Inflation Rate to Unemplo Normal          mu0: 0, sigma0: 0.0005    Normal     None
38       Interest Rate to Unemploy Normal          mu0: 0, sigma0: 0.0005    Normal     None
39       Cholesky Diagonal 0       Uniform         n/a (non-informative)     Normal     exp
40       Cholesky Off-Diagonal (1, Uniform         n/a (non-informative)     Normal     None
41       Cholesky Diagonal 1       Uniform         n/a (non-informative)     Normal     exp
42       Cholesky Off-Diagonal (2, Uniform         n/a (non-informative)     Normal     None
43       Cholesky Off-Diagonal (2, Uniform         n/a (non-informative)     Normal     None
44       Cholesky Diagonal 2       Uniform         n/a (non-informative)     Normal     exp


We use black box variational inference to estimate latent variables with fit:

x = var_model.fit('BBVI',iterations=15000)

10% done : ELBO is -677.720094719
20% done : ELBO is -661.87767258
30% done : ELBO is -535.684632563
40% done : ELBO is -548.88707709
50% done : ELBO is -480.473314911
60% done : ELBO is -377.536675354
70% done : ELBO is -295.52917053
80% done : ELBO is -305.158759779
90% done : ELBO is -237.031168261
100% done : ELBO is -253.893858625

Final model ELBO is -168.909877996


### Shrinkage for Inflation Latent Variables (for own lags)

Plotting the parameters for inflation’s own lags gives some indication as to the strength of our priors on the posterior for the longer lags.

var_model.plot_parameters([1],figsize=(15,1.5))
var_model.plot_parameters([1,4],figsize=(15,1.5))
var_model.plot_parameters([1,4,7],figsize=(15,1.5))
var_model.plot_parameters([1,4,7,10],figsize=(15,1.5))


### Shrinkage for Unemployment Latent Variables (for own lags)

var_model.plot_parameters([27],figsize=(15,1.5))
var_model.plot_parameters([27,30],figsize=(15,1.5))
var_model.plot_parameters([27,30,33],figsize=(15,1.5))
var_model.plot_parameters([27,30,33,36],figsize=(15,1.5))


We can view predictions of our macroeconomic variables with plot_predict:

var_model.plot_predict(h=4,figsize=(15,5))


This was before the ‘Brexit thing’ happened…

### Specifying a prior on the covariance matrix

Above we assumed non-informativeness about the covariance matrix. If we want to include prior information on the covariance matrix we can specify a Inverse Wishart prior. First, we can display the covariance matrix for our results above:

var_model.custom_covariance(var_model.latent_variables.get_z_values(transformed=False))

array([[ 0.48318869,  0.07586724 ,  -0.01367291],
[ 0.07586724 ,  0.37076457, -0.03349389],
[ -0.01367291, -0.03349389,  0.12133547]])


As an arbitrary example – merely to show the effects of the prior – we can specify the Inverse Wishart as $I\left(3\right)$ and $\nu=10$ using construct_wishart:

import numpy as np
var_model.construct_wishart(10,np.eye(3))
var_model.fit('BBVI',iterations=15000)

10% done : ELBO is -591.29489848
20% done : ELBO is -407.427509007
30% done : ELBO is -366.373801007
40% done : ELBO is -353.011970078
50% done : ELBO is -295.351808742
60% done : ELBO is -266.511009254
70% done : ELBO is -202.983249715
80% done : ELBO is -224.763716642
90% done : ELBO is -317.18302103
100% done : ELBO is -201.125203562

Final model ELBO is -145.846007915

<pyflux.results.BBVIResults at 0x7f5d60d1df60>

var_model.custom_covariance(var_model.latent_variables.get_z_values(transformed=False))

array([[ 0.4316595,  0.05773277,  0.00947594],
[ 0.05773277,  0.38248824, -0.02096291],
[ 0.00947594, -0.02096291,  0.10627838]])