15. Bayesian Methods

c A. Colin Cameron & Pravin K. Trivedi 2006

These transparencies were prepared in 2003. They can be used as an adjunct to Chapter 13 of our subsequent book Microeconometrics: Methods and Applications Cambridge University Press, 2005.

Original version of slides: May 2003 Outline

1. Introduction

2. Bayesian Approach

3. Bayesian Analysis of Linear Regression

4. Monte Carlo Integration

5. Markov Chain Monte Carlo Simulation

6. MCMC Example: Gibbs Sampler for SUR

7. Data Augmentation

8. Bayesian Model Selection

9. Practical Considerations 1 Introduction

Bayesian regression has grown greatly since books by Arnold Zellner (1971) and Leamer (1978).

Controversial. Requires specifying a probabilistic model of prior beliefs about the unknown parameters. [Though role of prior is negligible in large samples and relatively uninformative priors can be speci…ed.]

Growth due to computational advances.

In particular, despite analytically intractable posterior can use simulation (Monte Carlo) methods to

– estimate posterior moments

– make draws from the posterior. 2 Bayesian Approach

1. Prior () Uncertainty about parameters explicitly modelled by density ().

e.g. is an income elasticity and on basis of eco- nomic model or previous studies it is felt that Pr[0:8 1:2] = 0:95. Possible prior is [1; 0:12]. N

2. Sample joint density or likelihood f(y ) j Similar to ML framework. In single equation case y is N 1 vector and depen- dence on regressors X is suppressed.

3. Posterior p( y) j Obtained by combining prior and sample. 2.1 Bayes Theorem

Bayes inverse law of probability gives posterior f(y )() p( y) = j ; (1) j f(y) where f(y) is marginal (wrt ) prob. distn of y

f(y) = f(y )()d: (2) Z j

Pr[A B] Pr[B A] Pr[A] Proof: Use Pr[A B] = \ = j : j Pr[B] Pr[B]

f(y) in (1) is free of , so can write p( y) as j proportional to the product of the pdf and the prior p( y) f(y )(): (3) j _ j

Big di¤erence

– Frequentist: 0 is constant and is random.

– Bayesian: is random. b 2.2 Normal-Normal iid Example

1. Sample Density f(y ) j Assume y [; 2] with unknown and 2 ij N given. N N=2 f(y ) = 22 exp (y )2 =22 j 8 i 9 < iX=1 = N exp (y :)2 ; ; _ 2 2

2. Prior (). Suppose [; 2] where and 2 are given. N 1=2 () = 2 2 exp ( )2 =2 2 1 n o exp ( )2 ; _ 2 2

3. Posterior density p( y) j N 1 p( y) exp (y )2 exp ( )2 _ 2 2 j 2 2 After some algebra (completing the square) 1 ( )2 (y )2 p( y) exp + _ 2 12 2 j (2 " N + #) 2 1 ( 1) _ exp 2 (2 " 1 #)

2 2 2 1 = 1 Ny= + = 2 2 2 1 1 = N= + 1= : Properties of posterior: – Posterior density is y [ ; 2]: j N 1 1

– Posterior mean 1 is weighted average of prior mean and sample average y.

2 – Posterior precision 1 is sum of sample precision of y, N=2, and prior precision 1= 2. [Precision is the reciprocal of the variance.]

– As N , y [y; 2=N] ! 1 j N Normal-Normal Example with 2 = 100; = 5, 2 = 3, N = 50 and y = 10. 2.3 Speci…cation of the Prior

Tricky. Not the focus of this talk.

Prior can be improper yet yield proper posterior.

A noninformative prior has little impact on the re- sulting posterior distribution. Use Je¤reys prior, not uniform prior, as invariant to reparametrization.

For informative prior prefer natural conjugate prior as it yields analytical posterior. Exponential family prior, density and posterior. ) e.g. normal-normal, Poisson-gamma

Hierarchical priors popular for multilevel models. 2.4 Measures Related to Posterior

Marginal Posterior: p(k y) = p(1; :::; d y)d1::dk 1dk+1::dd: j j R

Posterior Moments: mean/median; standard devn.

Point Estimation: no unknown to estimate. 0 Instead …nd value of that minimizes a loss function.

Posterior Intervals (95%): Pr ) y = 0:95. k;:025 k k;:975 j h i

Hypothesis Testing: Not relevant. Bayes factors.

Conditional Posterior Density: p(k j; j k; y) = p( y)=p(j k y): j 2 j 2 j 2.5 Large Sample Behavior of Posterior

Asymptotically role of prior disappears.

If there is a true then the posterior mode (max- 0 imum of the posterior) is consistent for this. b

Posterior is asymptotically normal a y ; () 1 ; (4) j N I h i centered around the posteriorb mode,b where 1 @2 ln p( y) = j : I " @@ # 0 =

b b Called a Bayesian central limit theorem. 3 Bayesian Linear Regression

Linear regression model y X; ;2 [X ;2I ]: j N N

Di¤erent results with noninformative and informative priors. Even within these get di¤erent results according to setup. 3.1 Noninformative Priors

Je¤reys’priors: ( ) c and (2) 1=2. j _ _ All values of j equally likely. Smaller values of 2 are viewed as more likely. 2 2 ( ; ) _ 1= :

Posterior density after some algebra p( ;2 y; X) j 1 K=2 1 1 exp ( ) (X X)( ) _ 2 0 2 0 2 1 (N K)=2+1 b (N K) s2 b exp 2 2 2 !

2 2 1 Cond. posterior p( ; y; X) is [ ; X X ] j N OLS 0 b Marginal posterior p( y; X) (integrate out 2) is j multivariate t-distribution centered at with N K 2 1 dof and variance s (N K) X X = (N K 2). 0 b Marginal posterior p(2 y; X) is inverse gamma. j

Qualitatively similar to frequentist analysis in …nite samples.

Interpretation is quite di¤erent.

E.g. Bayesian 95 percent posterior interval for j is j t:025;N K se j h i b b

– means that j lies in this interval with posterior probability 0.95

– not that if we had many samples and constructed many such intervals 95 percent of them will con- tain the true j0. 3.2 Informative Priors

Use conjugate priors: – Prior for 2 is [ ; 2 ]: j N 0 0 – Prior for 2 is inverse-gamma.

Posterior after much algebra is p( ;1= y; X) j (0+N)=2 1 s1 K=2 2 exp 2 _ 2 2 1 exp 0 22 1 where 1 = 0 + X0X ( 0 0 + X0X ) 1 = 0 + X0X b 1 1 s = s + u u + 0 + X X 1 0 0 0 0 b b Conditional posterior p( 2; y; X) is [ ; ]: j N 1

Marginal posterior p( y; X) (integrate out 2) is j multivariate t-distribution centered at :

Here is average of and prior mean : OLS 0 And precision is sum of prior and sample precisions. b 4 Monte Carlo Integration

Compute key posterior moments, without …rst ob- taining the posterior distribution.

Want E[m( y)], where expectation is wrt to pos- j terior density p( y). j For notational convenience suppress y:

So wish to compute E [m()] = m()p()d: (5) Z

Need a numerical estimate of an integral: - Numerical quadrature too hard. - Direct Monte Carlo with draws from p() not possible. - Instead use importance sampling. 4.1 Importance Sampling

Rewrite E [m()] = m()p()d Z m()p() = g()d; Z g() ! where g() > 0 is a known density with same support as p().

The corresponding Monte Carlo integral estimate is 1 S m(s)p(s) E [m()] = ; (6) S g(s) sX=1 where s,bs = 1; :::; S, are S draws from of from g() not p().

To apply to posterior need also to account for constant of integration in the denominator of (1). Let pker() = f (y ) () be posterior kernel. j

Then posterior density is pker() p() = ; pker()d with posterior moment R pker() E [m()] = m() d ker Z p ()d! m() pker()d = R ker R p ()d ker mR () p ()=g() g()d = : R pker()=g() g()d R The importance sampling-based estimate is then 1 S m(s)pker(s)=g(s) E [m()] = S s=1 ; (7) 1 S ker s s PS s=1 p ( )=g( ) b where s, s = 1; :::;P S, are S draws of from the importance sampling density g(): Method was proposed by Kloek and van Dijk (1978).

Geweke (1989) established consistency and asymp- totic normality as S if ! 1 – E[m()] < so the posterior moment exists 1 – p()d = 1 so the posterior density is proper. May require ()d < . R 1 R – g() > 0 over the support of p()

– g() should have thicker tails than the p() to ensure that the importance weight w() = p()=g() remains bounded. e.g. use multivariate- t.

The importance sampling method can be used to estimates many quantities, including mean, standard deviation and percentiles of the posterior. 5 Markov Chain Monte Carlo Sim- ulation

If can make S draws from the posterior, 1 s E[m()] can be estimated by S s m( ). P But hard to make draws if no tractable closed form expression for the posterior density.

Instead make sequential draws that, if the sequence is run long enough, converge to a stationary distribution that coincides with the posterior density p().

Called Markov chain Monte Carlo, as it involves simulation (Monte Carlo) and the sequence is that of a Markov chain.

Note that draws are correlated. 5.1 Markov Chains

A Markov chain is a sequence of random variables xn (n = 0; 1; 2; :::) with

Pr [xn+1 = x xn; xn 1; :::; x0] = Pr [xn+1 = x xn] ; j j so that the distribution of xn+1 given past is com- pletely determined only by the preceding value xn.

Transition probabilities are txy = Pr [x = y xn = x] : n+1 j

For …nite state Markov chain with m states form an m m transition matrix T:

Then for transition from x to y in n steps (stages) the transition probability is given by Tn, the n-times matrix product of T. (n) The rows t of the matrix Tn give the marginal j distribution across the m states at the nth stage.

The chain is said to yield a stationary distribution or invariant distribution t (x) if

t (x) Tx;y = t (y) y A: x A 8 2 X2

(n) For Bayesian application the chain is not xn:

We want the chain (n): (1) to converge to a stationary distribution and (2) this stationary distribution to be the desired posterior. 5.2 Gibbs Sampler

Easy to describe and implement.

Let = [ ] have posterior density p() =p( ; ). 10 20 0 1 2

Suppose know p( ) and p( ). 1j 2 2j 1

Then alternating sequential draws from p( ) and 1j 2 p( ) in the limit converge to draws from p( ; ). 2j 1 1 2 5.2.1 Gibbs Sampler Example

Let y = (y ; y ) [; ] 1 2 N where = (1; 2)0 and has diagonal entries 1 and o¤-diagonals .

Then given a uniform prior for the posterior y [y;N 1]: j N

So the conditional posterior distributions are ; y (y + ( y )) ; (1 2)=N 1j 2 N 1 2 2 ; y h(y + ( y )) ; (1 2)=Ni ; 2j 1 N 2 2 2 h i

Can iteratively sample from each conditional normal distribution using updated values of 1 and 2.

If the chain is run long enough then it will converge to the bivariate normal. 5.2.2 Gibbs Sampler

More generally, suppose is partitioned into d blocks. 2 e.g. = [ ]0 in a linear regression example.

Let be the kth block k and k denote all components of aside from k.

Assume the full conditional distributions p k k , j k = 1; :::; d are known.

Then sequential sampling from the full conditionals can be set up as follows. (0) (0) (0) 1. Let the initial values of be = (1 ; :::; d ):

2. The next iteration involves sequentially revising all (1) (1) (1) components of to yield = (1 ; :::; d ) generated using d draws from the d conditional distributions as follows: (1) (0) (0) p(1 2 ; :::; d ) (1) j(1) (0) (0) p(2 1 ; 3 :::; d ) j . (1) (1) (1) (1) p(d 1 ; 2 ; :::; d 1) j

3. Return to step one, reinitialize the vector at (1) and cycle through step 2 again to obtain the new draw (2): Repeat the steps until convergence is achieved. Geman and Geman (1984) showed that the stochastic sequence (n) is a Markov chain with the cor- rect stationaryn distribution.o See also Tanner and Wong (1987) and Gelfand and Smith (1990).

These results mentioned do not tell us how many cycles are needed for convergence, which is model dependent.

It is very important to ensure that su¢ cient number of cycles are executed for the chain to converge. Dis- card the earliest results from the chain, the so-called ‘burn-in’phase. Diagnostic tests are available. 5.3 Metropolis Algorithm

The Gibbs sampler is the best known MCMC algorithm.

Limited applicability as it requires direct sampling from the full conditional distributions which may not be known.

Two extensions that allow MCMC to be applied more generally are the Metropolis algorithm and the Metropolis- Hastings algorithm.

In applying MCMC we use a sequence of approxi- mating posterior distributions; are transition distributions or transition kernels or proposal densities:

(n) (n 1) Use the notation Jn( ) which emphasizes j that the transition distribution varies with n. 1. Draw a starting point (0) from an initial approxi- mation to the posterior for which p((0)) > 0. e.g. draw from a multivariate t-distribution centered on the posterior mode.

2. Set n = 1: Draw from a symmetric jumping (1) (0) distribution J1( ), j a b a b i.e. for any arbitrary pair ( ; );Jn( ) = b a j Jn( ) j e.g. (1) (0) [(0); V] for some …xed V. j N

(0) 3. Calculate the ratio of densities r = p()=p( ):

4. Set

(1) with probability min(r; 1) = (0) ( with probability (1 min(r; 1)) :

5. Return to step 2, increase the counter and repeat the following steps. Can view as an iterative method to maximize p(). (n) If increases p() then = always. (n) If decreases p() then = with prob r.

Similar in spirit to accept-reject sampling but with no requirement that a …xed multiple of the jumping distribution always covers the posterior.

Metropolis generates a Markov chain with properties of reversibility, irreducibility and Harris recurrence that ensure convergence to a stationary distribution.

To see that the Metropolis stationary distribution is the desired posterior p() do as follows ... Let a and be points such that p( ) p(a). b b

(n 1) (n) If = a and = then = with b b certainty and (n) (n 1) Pr[ = ; = a] = Jn( a)p(a): b bj

(n 1) If order is reversed and = b and = a (n) then = a with probability r = p(a)=p(b) and

(n) (n 1) p(a) Pr[ = a; = b] = Jn(a b)p(b) j p(b) = Jn(a )p(a) j b = Jn( a)p(a) bj as symmetric jumping distribution.

Symmetric joint distribution marginal distributions of (n) and (n 1) are ) same p() is the stationary distribution ) 5.4 Metropolis-Hastings (M-H) Algorithm

The Metropolis-Hastings (M-H) algorithm is the same as the Metropolis algorithm, except that in step 2 the jumping distribution need not be symmetric.

Then in step 3 the acceptance probability (n 1) p( )=Jn( ) r = n (n 1) j (n 1) p( )=Jn( ) (n 1) j p( )Jn( ) = : (n 1) (jn 1) p( )Jn( ) j

Any normalizing constants present in either p( ) or Jn( ) cancel in rn. So both posterior and jump probabilities need only be computed up to this constant. 5.5 M-H Examples

Di¤erent jumping distributions lead to di¤erent M-H algorithms.

Gibbs sampler is a special case of M-H. If is partitioned into d blocks, then there are d Metropolis steps at the nth step of the algorithm. The jumping distribution is the conditional distribution given in subsection 5.2 and the acceptance probability is always 1. Gibbs sampling is also called alternating conditional sampling.

Mixed strategies can be used. e.g. an M-H step combined with a Gibbs sampler.

The independence chain makes all draws from a …xed density g () : A random walk chain sets the draw (n 1) = + "; where " is a draw from g("):

Gelman et al. (1995, p.334) consider [; ]. N For Metropolis with

(n 1) [(n 1); c2]; j N c 2:4=pq leads to greatest e¢ ciency relative to ' direct draws from q variate normal. The e¢ ciency is about 0:3, compared to 1=q for the 2 Gibbs sampler for = Iq. 6 Gibbs Sampler for SUR

Two-equation example with ith observation y1i = 11 + 12x1i + "1i y2i = 21 + 22x2i + "2i;

" 0 1i ; = 11 12 : " "2i # N "" 0 # " 21 22 ##

Assume independent informative priors, with 1 [ ; B ]; N 0 0 1 Wishart[ ; D ]; 0 0

Some algebra yields the conditional posteriors N ; y; X [C (B + x 1y ); C ]; j N 0 0 0 i0 i 0 iX=1 N 1 1 1 ; y; X Wish[ + N; (D + " " ) ] j 0 0 i0 i iX=1 N 1 1 where C0 = B0 + i=1 xi0 xi . P Gibbs sampler can be used since conditionals known.

Simulation: N = 1000 or N = 10000. x [0; 1] and x [0; 1]: 1iN 2iN 11 = 12 = 21 = 22 = 1 11 = 22 = 1; 12 = :5: 1 Priors 0 = 0, B0 = I (with = 10; 1; 0:1), D0 = I and 0 = 5.

Gibbs sampler samples recursively from the conditional posteriors. Reject the …rst 5000 replications - “burn-in”. Use subsequent 50000 and 100000 replications.

Table reports mean and st.dev. of the marginal posterior distribution of the 7 parameters. First three columns: not sensitive to di¤erent values of :

Fourth column vs. …rst shows doubling number of reps has very little e¤ect.

Fifth column vs. …rst shows that increasing sample size ten-fold to 100000 has relatively small impact on point estimates though precision is much higher.

When number of reps is small 1000 the autocor- ' relation coe¢ cients of parameters are found to be as high as 0.06. When number of reps 50000 serial ' correlation is much lower < 0:01. 7 Data Augmentation

Gibbs sampler can sometimes be applied to a wider range of models by introduction of auxiliary variables.

In particular, this is the case for models involving latent variables, such as discrete choice, truncated and censored models.

Observe only y = g(y ) for given g( ) and latent dependent variable y. e.g. Probit / logit have y = 1(y > 0).

Data augmentation replaces y by imputed values and treats this as observed data.

Essential insight, due to Tanner and Wong (1987), is that posterior based only on observed data is intractable, but that obtained after data augmentation is often tractable using the Gibbs sampler. 8 Bayesian Model Selection

Method uses Bayes factors.

Two hypotheses under consideration - H1 and H2 possibly non-nested. - Prior probabilities Pr[H1] and Pr[H2]: - Sample dgp’sPr[y H ] and Pr[y H ]: j 1 j 2

Posterior probabilities by Bayes Theorem Pr[y H ]Pr[H ] Pr[H y] = j k k : kj Pr[y H ]Pr[H ] + Pr[y H ]Pr[H ] j 1 1 j 2 2

The posterior odds ratio Pr[H1 y] Pr[y H1]Pr[H1] Pr[H1] j = j B12 ; Pr[H y] Pr[y H ]Pr[H ] Pr[H ] 2j j 2 2 2 where B =Pr[y H ] =Pr[y H ] is called Bayes fac- 12 j 1 j 2 tor. Hypothesis 1 preferred if posterior odds ratio > 1.

Bayes factor = posterior odds in favor of H 1 if Pr[H1] =Pr[H2].

Bayes factor has form of a likelihood ratio. But depends on unknown parameters k eliminated by integrating over parameter space wrt prior, so

P r [y Hk] = P r [y k;Hk] (k Hk) d: j Z j j

This expression depends upon all the constants that appear in the likelihood. These constants can be neglected when evaluating the posterior, but are required for the computation of the Bayes factor. 9 Practical Considerations

WinBUGS package (Bayesian inference Using Gibbs Sampling) package especially useful for hierarchical models and missing data problems.

For more complicated models use Matlab or Gauss.

Practical issue of how long to run the chain. Diagnostic checks for convergence are available, but often do not have universal applicability. Graphs of output for scalar parameters from the Markov chain is a visually attractive way of con…rming convergence, but more formal approaches are available (Geweke, 1992). Gelman and Rubin (1992) use multiple (parallel) Gibbs samplers each beginning with di¤erent starting values to see if di¤erent chains converge to the same posterior distribution. Zellner and Min (1995) propose several convergence criteria that can be used if the posterior can be writ- ten explicitly. 10 Bibliography

Useful books include Gamerman (1997), Gelman, Carlin, Stern and Rubin (1995), Gill (2002) and Koop (1993) plus older texts by Zellner (1971) and Leamer (1978).

Numerous papers by Chib and his collaborators, and Geweke and his collaborators, cover many topics of interest in microeconometrics. See Chib and Green- berg (1996), Chib (200) and Geweke and Keane (2000).

Albert, J.H. (1988), “Computational Methods for Using a Bayesian Hierarchical Generalized Linear Model”. Journal of American Statis- tical Association, 83, 1037-1045:

Casella, G. and E. George (1992), “Explaining the Gibbs Sampler”, The American Statistician, 46, 167-174. Chib, S. (2000), “Markov Chain Monte Carlo Methods: Computa- tion and Inference”, chapter 57 in J.J. Heckman and E.E. Leamer, Editors, Handbook of Econometrics Volume 5, 3570-3649.

Chib, S., and E. Greenberg (1995), “Understanding the Metropolis- Hastings Algorithm”, The American Statistician, 49, 4, 327-335.

Chib, S., and E. Greenberg (1996), “Markov Chain Monte Carlo Simulation Method in Econometrics”, Econometric Theory, 12, 409- 431.

Gamerman, D. (1997), Markov Chain Monte Carlo: Stochastic Sim- ulation for Bayesian Inference, London: Chapman and Hall.

Gelfand, A.E. and A.F.M. Smith (1990) “Sampling Based Approaches to Calculating Marginal Densities”, Journal of American Statistical Association, 85, 398-409.

Gelman, A., J.B. Carlin, H.S. Stern and D.B. Rubin (1995), Bayesian Data Analysis, London: Chapman and Hall.

Gelman, A., and D.B. Rubin (1992), “Inference from Iterative Sim- ulations Using Multiple Sequences”, Statistical Science, 7, 457-511. Geman, S. and D. Geman (1984), “Stochastic Relaxation, Gibbs Dis- tributions and Bayesian Restoration of Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741.

Geweke, J. (1989), “Bayesian Inference in Econometric Models Using Monte Carlo Integration”, Econometrica, 57, 1317-1339.

Geweke, J. (1992), “Evaluating the Accuracy of Sampling-based Ap- proaches to the Calculation of Posterior Moments (with discussion)”, in J. Bernardo, J. Berger, A.P. Dawid, and A.F.M. Smith, Editors, Bayesian Statistics, 4, 169-193. Oxford: Oxford University Press.

Geweke, J. and M. Keane (2000), “Computationally Intensive Meth- ods for Integration in Econometrics”, chapter 56 in Heckman, J.J. and E.E. Leamer, Editors, Handbook of Econometrics Volume 5, 3463-3567.

Gill, J. (2002), Bayesian Methods: A Social and Behavioral Sciences Approach, Boca Raton (FL): Chapman and Hall.

Hastings, W.K. (1970), “Monte Carlo Sampling Methods Using Markov Chain and Their Applications”, Biometrika, 57, 97-109.

Kass, R.E. and A.E. Raftery (1995), “Bayes Factors”, Journal of American Statistical Association, 90, 773-795. Kloek, T. and H.K. van Dijk (1978), “Bayesian Estimates of Equa- tion System Parameters: An Application of Integration by Monte Carlo”, Econometrica, 46, 1-19.

Koop, G. 2003), Bayesian Econometrics, Wiley.

Leamer, E.E. (1978), Speci…cation Searches: Ad Hoc Inference with Nonexperimental Data, New York: John Wiley.

Robert, C.P., and G. Casella (1999), Monte Carlo Methods, New York: Springer-Verlag.

Tanner, M.A., and W.H. Wong (1987), “The Calculation of Pos- terior Distributions by Data Augmentation”, Journal of American Statistical Association, 82, 528-549.

Zellner, A. (1971), An Introduction to Bayesian Inference in Econo- metrics, New York: John Wiley.

Zellner, A. (1978), “Je¤reys-Bayes Posterior Odds Ratio and the Akaike Information Criterion for Discriminating Between Models”, Economics Letters, 1, 337-342.

Zellner, A., and C-k. Min (1995), “Gibbs Sampler Convergence Criteria”, Journal of American Statistical Association, 90, 921-927. Table 1: Mean and Standard deviation of the Posterior Distribution of a two-equation SUR Model calculated by Gibbs Sampling.

= 10 = 1 = 1=10 = 10 = 10 N 1000 1000 1000 1000 10000 reps 50000 50000 50000 100000 100000 11 0.971 1.013 0.983 1.020 1.010 (0.0310) (0.0312) (0.0316) (0:0324) (0:0100) 12 1.026 0.9835 1.006 1.006 1.015 (0.0265) (0.0271) (.0265) (:0268) (0:0086) 21 1.016 0.972 0.993 1.017 0.991 (0.0309) (0.0325) (0.0322) (0:0326) (0:0100) 22 0.983 0.992 0.979 1.005 1.007 (0.0256) (0.0285) (0.0272) (0:0277) (0:0085) 11 0.960 0.969 1.012 1:043 1.010 (0.0429) (0.0434) (0.0453) (0:0466) (0:0143) 12 -0.499 -0.507 -0.519 -0.576 -0.515 (0.0340) (0.0358) (0.0368) (0:0379) (0:0113) 22 0.950 1.066 1.049 1.062 1.002 (0.425) (0.0476) (0.0467) (0:0472) (0:0141)