Deciphering Federal Reserve Communication Via Text Analysis of Alternative FOMC Statements ∗

Home , Federal funds, The Forstall System

Deciphering Federal Reserve Communication via Text Analysis of Alternative FOMC Statements ∗

Taeyoung Doh† Dongho Song‡ Shu-Kuei Yang§

October 20, 2020

Abstract

We apply a natural language processing algorithm to FOMC statements to construct a new measure of monetary policy stance, including the tone and novelty of a policy statement. We exploit cross-sectional variations across alternative FOMC statements to identify the tone (for example, dovish or hawkish) and contrast the current and previous FOMC statements released after Committee meetings to identify the novelty of the announcement. We then use high-frequency bond prices to compute the surprise component of the monetary policy stance. Our text-based estimates of monetary policy surprises are not sensitive to the choice of bond maturities used in estimation, are highly correlated with forward guidance shocks in the literature, and are associated with lower stock returns after unexpected policy tightening. The key advantage of our approach is that we are able to conduct a counterfactual policy evaluation by replacing the released statement with an alternative statement, allowing us to perform a more detailed investigation at the sentence and paragraph level.

JEL Classiﬁcation: E30, E40, E50, G12. Keywords: Alternative FOMC statements, counterfactual policy evaluation, monetary policy stance, text analysis, natural language processing. ∗We thank John Rogers for sharing his dataset on monetary policy shock estimates with us. We thank Kyungmin Kim and Suyong Song as well as seminar participants at the Federal Reserve Bank of Kansas City, the Interfed Brown Bag, the KAEA webinar, and National University of Singapore who provided helpful comments. All the remaining errors are ours. The opinions expressed herein are those of the authors and do not reﬂect the views of the Federal Reserve Bank of Kansas City or Federal Reserve System. †Federal Reserve Bank of Kansas City; [email protected] ‡Carey Business School, Johns Hopkins University; [email protected] §Federal Reserve Bank of Kansas City; [email protected] 1

1 Introduction

Central banks have increasingly relied on public communications to provide guidance regarding future policy actions, e.g., Woodford(2005) and Blinder et al.(2008). The practice became more prevalent when monetary policy is constrained by the eﬀective lower bound, see Bernanke(2010). In this regard, both quantitative decisions (e.g., interest rates or asset purchases) made by central banks and their qualitative descriptions of the economic factors that lead to the decisions serve as important information variables for understanding monetary policy.

While the profession has moved toward treating policy statements and speeches by central bank officials as data to be analyzed, important limitations exist in the parsing of textual content. First, quantifying the tone (between a dovish stance and a hawkish one) automat- ically from only the publicly released text is difficult because the tone is often relative to what could have been released. Second, even when using asset prices as instruments, identifying which part of a communication is perceived as most crucial by the markets is difficult to do because texts like statements and speeches are multi-dimensional objects. In addition, assumptions are required on the maturity structure of bond prices to translate bond price movements into particular parts of the statement. Third, it is hard to evaluate the (counterfactual) impact of alternative language in the statement on the markets within the commonly used text analysis methods largely based on word counting when the alternative language mainly changes the contextual meaning of words rather than the frequency pattern of words.

The purpose of this paper is to contribute in all three dimensions to enhance our understanding of the transmission of monetary policy to the financial markets, which is important for both policy makers and market participants. We work with the Federal Open Market Committee’s (FOMC) post-meeting statements in this paper. We differ from the existing literature on two fronts in achieving our objectives. First, we refine the information in the FOMC statements using a novel natural language processing algorithm known as the Universal Sentence Encoding (USE). In contrast to the word-counting methods that ignore the local context (e.g., Term Frequency-Inverse Document Frequency (TF-IDF) and Latent Semantic Analysis (LSA)), the USE provides context-aware representation of words in the document. Second, and more importantly, we consider alternative policy statements created 2 by the staff of the Federal Reserve Board as well as the statement released right after the meeting in parsing of policy statements. Alternative statements are available for each FOMC meeting since March 2004 and contain a more dovish (Alt A) or a more hawkish (Alt C or Alt D) statement than the benchmark one (Alt B). The different views of economic outlook and policy prescription contained in the alternative statements provide important anchoring points for interpreting the tone of the policy statement released after the meeting.1

We provide a novel measure of the surprise component of monetary policy announcements based on our text analysis. We do this in two steps. First, we characterize the “monetary policy stance” communicated by each FOMC statement. For this, we identify the “tone” of monetary policy announcements by computing the similarities in terms of the USE representation between the released statement and the alternative statements.2 We define the “novelty” of monetary policy announcements by computing the distance in terms of the USE representation between the current statement and the previous statement. By taking the product of the tone and novelty of monetary policy announcements, we obtain the monetary policy stance. The first step only relies on the text analysis of the FOMC statements. In the second step, we introduce (high-frequency) bond prices to compute the surprise component of the monetary policy stance. The expected component of the monetary policy stance is defined as a weighted average of the hawkish and dovish policy stances of the alternative statements. We back out the weight that maximizes the rank correlation of the (high-frequency) bond prices and the surprise component of monetary policy announcements (aka, monetary policy shocks), namely, the difference between the monetary policy stance and its expected component.

We then use (high-frequency) stock prices to show that a tightening policy surprise ac- cording to our measure generates a negative stock price reaction. Speciﬁcally, a positive one-standard-deviation surprise leads to a 20 basis points (bps) drop in stock prices on average. Also, we verify that our measure of monetary policy shocks is highly correlated (about 50%) with forward guidance shocks identiﬁed in the existing literature (that relies

1Like the FOMC transcripts, alternative statements are made public ﬁve years after they were created. Although published with a signiﬁcant lag, dovish and hawkish alternative statements generally incorporate information on market expectations regarding the upcoming policy statement because they are written to surprise the market in respective directions. See FRB(2004) for the detailed description of statement language. 2The released statement is typically very similar to the benchmark statement created before each FOMC meeting but may not be exactly same because some phrases can change during the FOMC deliberation. 3 on high-frequency bond prices). Both serve as external validation of our measure.

The key advantage of our approach is that we are able to conduct a counterfactual policy evaluation by replacing the released statement with either one of the alternative statements. Specifically, we work with FOMC statements released in September 2007 and December 2010 as two illustrative examples.3 The post-meeting statement in September 2007 lowered the federal funds rate target by 50 bps but it noted that “the Committee judges that some inflation risks remain, and it will continue to monitor inflation developments carefully.” We consider the counterfactual statement that would lower the federal funds rate target by 25 bps but mention “the downside risks to economic growth outweigh the upside risks to inflation.” In spite of a smaller rate cut, we find that this counterfactual statement might be a more dovish statement, leading to an 1.94% increase of stock prices instead of the 0.1% increase after the release of the post-meeting statement.4 Another example is the December 2010 FOMC meeting in which the counterfactual statement suggests a more explicit time- dependent forward guidance on the future path of the federal funds rate by replacing “for an extended period” by “at least through the mid-2012’ to indicate how long the interest rate would stay low. It also announces additional asset purchases. The actual released statement omits such an explicit forward guidance and the expansion of asset purchases. We find that adopting the counterfactual in December 2010 would lead to about 0.77% increase in the stock market return instead of 0.05% increase after the release of the post-meeting statement. These two episodes highlight the importance of narrative information in the Fed communication, which is hard to measure without using text-analysis tools.

Related Literature. Our paper is related to multiple lines of research. First, our work draws on papers that identify monetary policy shocks using high-frequency bond data, (e.g., G¨urkaynak et al.(2005), Swanson(2017), Nakamura and Steinsson(2018), Bu et al.(2020), Bauer and Swanson(2020), Hoesch et al.(2020)). Our empirical ﬁnding on the negative stock market response to an unexpected monetary policy tightening is consistent with Bu et

3See the appendix for the detailed description of alternative statements. 4We regress stock returns on our measure of monetary policy surprises and obtain the respective OLS coefficient estimates. We replace the monetary policy stance with the counterfactual one and subtract the bond price-implied expected monetary policy stance to compute the counterfactual monetary policy surprise component. We multiply the counterfactual monetary policy surprise component to the OLS slope coefficient to assess the counterfactual impact of alternative policy prescription. It is important to note that we are only replacing one data point (that corresponds to the September 2007 FOMC statement release date) while keeping all else equal in this exercise. 4 al.(2020), Bauer and Swanson(2020), Hoesch et al.(2020) which imply that the FOMC’s communication was largely effective in inducing the intended asset market responses during our sample period (from March 2004 to December 2014). Hoesch et al.(2020) and Lunsford (2020) find that the stock market response coefficient was unstable over time but became largely negative to policy tightening since 2004. Since this timing coincides with the cre- ation of alternative statements (March 2004), we suspect that the intense monetary policy deliberation focused on effective communication might account for the timing of the break.5 However, our paper is distinguished from all these papers in our ability to evaluate the counterfactual implications of alternative policy prescriptions.

Second, our work is also related to the increasingly popular application of text analysis in economics and finance, (e.g., Gentzkow et al.(2019), Ke et al.(2019), Hansen et al.(2017), Schonhardt-Bailey(2013), Shapiro and Wilson(2019), Jegadeesh and Wu(2017), Meade and Acosta(2015), Giavazzi et al.(2020), Lucca and Trebbi(2009)). In particular, Ke et al. (2019) propose a method to combine new information and sentiment scores from news articles to predict stock returns. We adapt their method to our measure of monetary policy stance. Nonetheless our methodology is differentiated from most of these papers in the literature because we do not apply methods based on word-counting.6 We combine a context-aware representation of the text with alternative FOMC statements to better identify the tone of the released FOMC statement. In this paper, we highlight the importance of capturing the contextual meaning using the alternative FOMC statements prepared for the October 2013 meeting as an example. We also demonstrate that our text-based policy surprises are not sensitive to the choice of target maturities used in the statistical factor analysis popular in the existing literature and highly correlated with different estimates (e.g., Nakamura and Steinsson(2018) and Bu et al.(2020)).

Outline of the structure of the paper is as follows. Section2 describes our natural language processing technique and identiﬁcation scheme of monetary policy surprises using alternative statements. Section3 discusses empirical results and policy implications. Section4

5Lunsford(2020) argues that the FOMC started to issue a more explicit policy inclination from the August 2003 meeting statement and that it was a break point in the stock market response to FOMC announcements. While we cannot disentangle our story from his interpretation because no alternative statement exists before the March 2004, our story is not inconsistent with his ﬁnding. 6The exception is Giavazzi et al.(2020). who use the Doc2vec algorithm. The USE is distinct from the Doc2vec in the heavy use of the “self-attention” channel that is known to capture richer patterns of the contextual meaning of the text. 5 concludes.

2 Text-based Identiﬁcation of Monetary Policy Stance

The recent development in the natural language processing provides tools to better capture the contextual meaning of a word in a text. We rely on the USE to quantify information in texts. The encoding algorithm was pre-trained based on the Stanford Natural Language Inference (SNLI) dataset. Under the USE, the entire statement is encoded as a large-dimensional vector capturing various features of the whole text including the context between words or sentences. We highlight its features in this section relative to the commonly used word counting method but leave a more complete description of technical details to the appendix. We then provide the details about the FOMC statements and explain how we identify the monetary policy stance from the alternative statements using the USE. Finally, we explain how we leverage high-frequency asset prices to measure the surprise component of monetary policy stance.

2.1 Universal sentence encoding versus word-counting methods

Cer et al.(2018) describes two versions of the USE; 1) deep averaging of word embeddings, 2) transformer-based approach using the self-attention channel. We apply the transformer- based version of the USE to calculate the similarity between texts because the self-attention channel is powerful in capturing the context-dependent meaning of sentences as we see in the following example. The USE is able to capture the dependencies between even distant words by training deep neutral networks that can recognize complex dependencies of diﬀerent words based on large corpora. Hence, it can score the similarity between texts in a more sensible way. For example, imagine that there are two sentences consisting of n1 and n2 words respectively:

S1 = (w1,1, ··· , w1,n1 ) ,S2 = (w2,1, ··· , w2,n2 ) (1) ↓ U1 = (U1,1, ··· ,U1,512) ,U2 = (U2,1, ··· ,U2,512) . 6

USE will ﬁnd out numerical representations of S1 and S2 by two 512 dimensional vectors (U1 and U2) using a deep neural network architecture. The embedding representation is trained to perform a variety of tasks such as text classiﬁcation similar to humans and predicting some part of the text based on the rest of it. USE is available through Google TensorFlow. We calculate the similarity between the two texts based on the cosine similarity between two embedding vectors:

U 0 U Sim (Text , Text ) = cosine(U ,U ) = 1 2 . (2) USE 1 2 1 2 p 0 p 0 U1U1 U2U2

Notice that we are not restricting on pre-fixed features (e.g., the frequency of overlapping words) of the text to calculate sentence embeddings and similarity scores. The training process of embedding representations capture rich features of the text not necessarily confined to the frequency of words. This is the main difference of the USE from methods relying on word-counting that we describe below.

Term Frequency-Inverse Document Frequency. We provide a comparison with the Term Frequency-Inverse Document Frequency (TF-IDF) method. Here, similarities between multiple (N) documents are determined by the frequency of words that show up in all of these documents. Speciﬁcally,7

ni,j N + 1 Wi,j = q ln( + 1), P 2 dfj + 1 k nk,j (3)

SimTF-IDF(Text1, Text2) = cosine(W·,1,W·,2). where ni,j is the count of the j-th word in the i-th document and dfj is the number of documents that contain the j-th word. The main problem of this method is that word counting does not consider the semantic similarity between diﬀerent words and the algorithm cannot be trained to incorporate the contextual meaning.

Latent Semantic Analysis. A more sophisticated word counting method is available known as the Latent Semantic Analysis (LSA). LSA considers the high co-frequency of words in calculating the similarity score between texts. Instead of W·,1 and W·,2, LSA uses low-dimensional objects obtained by the singular value decomposition of W = [W·,1,W·,2] to

7We follow the default method used in Python. 7

Table 1: Sentence similarity

TF-IDF LSA USE

Sim(S1,S2) 0 0 0.91 Sim(S1,S3) 0.78 0.78 0.28

calculate the similarity between texts.8 Speciﬁcally,

W = UΣV 0, (4) SimLSA(Text1, Text2) = cosine(X·,1,X·,2) ,X = U·,1:2Σ1:2,1:2.

By rotating term frequency vectors to maximize the co-frequency of words across multiple documents, the LSA extracts representations that highlight the co-frequency of words used in diﬀerent documents. For this reason, it is widely used in identifying a few key topics from a large number of texts. However, like the TF-IDF, it does not take into account complex dependencies between diﬀerent words beyond the co-frequency, which is important for understanding semantic similarity.

A simple illustration. We illustrate the advantage of the USE in capturing the contextual meaning by comparing the similarity between the following sentences. We repeat the same exercise with TF-IDF and LSA for comparison.

(S1) How old are you?

(S2) What is your age?

(S3) How are you?

It is obvious that S1 and S2 are asking the same question, whereas S3 is not. Hence, the ideal classiﬁer should recognize that S1 is more similar to S2 than S3. However, the similarity score under the TF-IDF or LSA provides an opposite ranking whereas the USE provides a more sensible similarity score. We highlight the results in Table1.

8The dimension reduction is especially powerful when we try to capture the common theme from large text corpora although it does not matter in the simple example consisting of two sentences. 8

Table 2: Alternative FOMC statement similarity: October 2013 FOMC Meeting

TF-IDF USE

Sim(FOMCA,t,FOMCt) 0.975 0.895 Sim(FOMCC,t,FOMCt) 0.972 0.990

The mechanism that makes the USE capture the contextual similarity of diﬀerent texts

is the self-attention channel behind the deep neural network architecture. “How” (w1,1) is contextually connected with “old” (w1,2) in S1 while “How” (w3,1) is related to “are” in S3. The contextual representation of “How” should be able to reﬂect this diﬀerence, which is not possible in TF-IDF and LSA because they represent words as an item in the dictionary without encoding contextual linkages in embeddings. For example, if the dictionary contains

|V | words, wi,j is represented by a |V |-dimensional vector in which the jth element is one and all the other elements are zeros under these approaches. This representation is known as a one-hot vector encoding. The USE does not restrict word embedding to one-hot vector encoding, allowing multiple non-zero elements. Furthermore, it transforms the given word embedding by applying the self-attention channel that can capture the contextual dependence between a group of words in the text. The self attention channel in the USE generates the contextual representation of any word in the text by taking a weighted average of all the word embeddings in the text.

Arrows in Figure1 illustrates how attention weights link a particular word in S1 with all the other words.9 Unlike TF-IDF and LSA, the USE does not use one hot vector encodings and any element in wi,j can be non-zero. The attention-weighted average transforms word embeddings to perform tasks in the training stage better such as text classiﬁcation and word prediction by making contextually linked words have close embeddings. Notice that unlike one-hot vector encoding, elements in word embeddings are parameters set to minimize the loss function in the training stage.10

The superiority of USE over word-counting methods in capturing the local context can be further illustrated by considering alternative FOMC statements prepared for the October

9This way of illustrating the self-attention channel follows Vaswani et al.(2017). 10One of the training dataset contains web-based question and answer texts, which facilitate the USE to detect the contextual diﬀerences in the meaning of “How” in S1 and S3 better. 9

Figure 1: Illustration of self-attention: S1

How old are you?

Notes: The red arrow highlights the contextual link between “How” and “old”.

2013 FOMC meeting. As shown in the appendix, the ﬁrst paragraph of Alt A starts by acknowledging the challenge in interpreting economic data released during the intermeeting period due to the temporary shutdown of the federal government whereas Alt C does not mention this challenge. By pointing the near-term uncertainty, the reference to the government shutdown reveals that policymakers are not so sure about the recent improvement in the data. Otherwise, the description of the current outlook is fairly similar between Alt A and Alt C. The actual released statement dropped the reference to the temporary shutdown of the federal government like Alt C although it was rather close to Alt A otherwise. The textual similarity results in Table2 suggest that the USE captures the large impact of dropping the reference to the government shutdown while TF-IDF does not.11

2.2 Implementation of text analysis

We characterize the “monetary policy stance” communicated by each FOMC statement. For this, we identify the “tone” of monetary policy announcements by computing the similarities between the released statement and the alternative statements. We deﬁne the “novelty” of monetary policy announcements by computing the diﬀerence between the current statement and the previous statement released after respective FOMC meetings. By taking the product of the tone and novelty of monetary policy announcements, following Ke et al.(2019), we obtain the monetary policy stance.

11When we calculate the paragraph-by-paragraph similarity, only the first paragraph makes a large difference between USE and TF-IDF. Other paragraphs do not create significant differences between the two methods. 10

Speciﬁcally, we deﬁne the (benchmark) monetary policy stance as

Sim(FOMCt,FOMCC,t) − Sim(FOMCt,FOMCA,t) MP stance (t) = (1 − Sim(FOMCt,FOMCt−1)) (5) | {z } 1 − Sim(FOMCA,t,FOMCC,t) Novelty | {z } Tone and the (alternative) dovish and hawkish monetary policy stance as12

Dovish MP stance (t) = −|1 − Sim(FOMCt,FOMCt−1)|, (6)

Hawkish MP stance (t) = |1 − Sim(FOMCt,FOMCt−1)|, respectively. Novelty in the current benchmark FOMC statement relative to the previous one quantiﬁes the change in the FOMC’s intended policy stance. Note the monotonicity

tone(Dovish MP stance) ≤ tone(MP stance) ≤ tone(Hawkish MP stance). (7)

As conventional, we sign a positive tone as a hawkish stance and a negative tone as a dovish stance. We further normalize the tone measure between -1 and 1.

The above analysis is based on the USE representation at the statement level. But statements typically consist of multiple paragraphs and we may be interested in isolating the relevance of a particular paragraph. To do this, we compute the USE representation of the j-th paragraph in the i-type FOMC statement where i denotes the diﬀerent versions of the i i l statement. Let this be Pj,t. By comparing Pj,t with Pk,t0 , we can decompose which paragraph contributes mostly to the change in the similarity score between statements.13

To identify monetary policy surprises around FOMC announcements, we have to proxy the market expectations for the MP stance right before the FOMC meeting. We do this by using a weighted average of the dovish MP stance and the hawkish MP stance based on the assumption that alternative statements mimic expectations of market participants with

12Here, we assume alternative stances differ from the benchmark stance only in terms of the tone but not novelty. 13Note that the statement level USE representation is not an equal average of paragraph-level USE representations. When we approximate the statement level USE representation by a weighted average of paragraph-level USE representations, the first and second paragraphs take most weights. While this is not the exact replication of the statement-level USE representation, the finding suggest that the ordering language may matter because it sets the context in which the subsequent words are interpreted. 11 more extreme views.14

Et−∆MP stance(t) = (1 − pt) × Hawkish MP stance (t) + pt × Dovish MP stance (t). (8)

Note that the weight, pt, can vary over time.

2.3 Measuring the market expectations from bond prices

At the time of the FOMC announcement, the reaction of the high-frequency asset i’s prices can be captured by

P ri ≡ ln i,t+∆h = α + β MPS(p ; t − ∆) + ε , ε ∼ (0, σ2). (9) t−∆l,t+∆h i i t i,t i,t i Pi,t−∆l

The surprise component of the FOMC announcement measured at t − ∆ is

MPS(pt; t − ∆) = MP stance (t) − Et−∆MP stance (t). (10)

It is important to understand that MPS(pt; t − ∆) > 0 corresponds to tightening monetary policy.15

14Lucca and Trebbi(2009) constructs a measure of monetary policy stance based on the systematic co- occurrence of words with different sentiments in FOMC statements. However, they equate market expectations to the previous meeting’s monetary policy stance, ignoring the market reaction to developments during the inter-meeting period. Their alternative measure using newspaper discussions of FOMC announcements before and after the meeting do not face this issue but it is hard to identify the impact of changing the language in the statement under this approach because newspaper discussions instead of statements themselves are used to identify the policy stance. 15The underlying assumption of (8) is that the market is aware of the two alternative stances and they provide bounds when evaluating the benchmark statements, which may sound too strong given the fact that alternative statements are available only with a long time lag. One practical justification is that if we look at the bluebook in 2004, they rationalize alternative statements by intentionally beating market expectations in the hawkish or the dovish direction. Hence, alternative statements describe the Board staff’s best guess for two extreme market expectations. Roughly speaking, the hawkish stance represents the most hawkish person in the financial market while the dovish stance represents the most dovish person. We are capturing the marginal investor’s expectation as a weighted average of these two extreme expectations. As long as the marginal investor’s expectation is within the bound set by the survey of market participants and we do not know exactly who would be the marginal investor, this assumption seems to be plausible. 12

2.4 Constructing the surprise component of monetary policy stance.

T We calibrate the weight {pt}t=1 that maximizes the rank correlation of the high-frequency bond returns (left-side of (9)) and the surprise component of monetary policy announcements

MPS(pt; t − ∆), aka, monetary policy shocks. When pt is not time-varying, (pt = p) our estimate is identical to the maximum rank correlation estimator, see Han(1987) and Sherman (1993). Here, we assume that a dovish surprise should lead to a positive bond return because bond prices move inversely with bond yields.16 Speciﬁcally, we maximize the following rank correlation function with respect to pt:

T X b b (pτ ) = argmax 1(r > r )1(MPS(pτ ) < MP S(pτ )). (11) i i=1 τt−∆l,τt−∆h τt0 −∆l,τt0 −∆h t t0 t6=t0

This (negative) rank correlation is maximized by calibrating pt based on the sorted bond return. Speciﬁcally, the time series of bond returns {rb }T are sorted from most t−∆l,t+∆h t=1 negative to most positive. Let the ordering of the sorted-returns be indicated with new time subscripts {τ1, ..., τT }:

rb = min{rb }T (12) τ1−∆l,τ1+∆h t−∆l,t+∆h t=1 rb = max{rb }T . τT −∆l,τT +∆h t−∆l,t+∆h t=1

For a strictly negative value of βb in (9), we have that

MPS(pτT ) ≤ ... ≤ MPS(pτt ) ≤ ... ≤ MPS(pτ1 ) (13) rb ≤ ... ≤ rb ≤ ... ≤ rb τ1−∆l,τ1+∆h τt−∆l,τt+∆h τT −∆l,τT +∆h

where τt ∈ {τ1, ...τT }. Because it is possible that there are (potentially) multiple realizations of {pτ1 , ..., pτT } that satisfy (13), we pick the one that achieves the largest negative correlation between {MPS(p ), ..., MP S(p )} and {rb , ..., rb }. This can be done τ1 τT τ1−∆l,τ1+∆h τT −∆l,τT +∆h

via grid search (with respect to pτt ). Once we select {pτ1 , ..., pτT }, we can sort them back to match the original time subscript {p1, ..., pT } and construct the corresponding MPS(pt) for

each pt, t ∈ {1, ..., T }.

16 The sign of the correlation is negative because pt corresponds to a dovish probability, which contributes to a negative surprise in monetary policy stance. 13

Figure 2: Monetary policy stance

-5 MP stance Dovish MP stance Hawkish MP stance 2006 2008 2010 2012 2014

Notes: We normalize these measures to have a unit variance. 3 Empirical Results

3.1 Data for alternative FOMC statements

The Federal Reserve Board staﬀ started to prepare alternative FOMC statements from the March 2004 FOMC meeting. The latest available statement is the one prepared for the December 2014 FOMC meeting. We have 87 FOMC statements (March 2004 to December 2014) excluding two inter-meeting announcements (Aug 2007, Jan 2008). When multiple versions of hawkish or dovish alternative statements are available (e.g., Alt A1 or Alt D), we use the most extreme one to identify the tone of the released statement.17

3.2 Monetary policy stance and surprises

Figure2 provides the time series of (5) and (6) constructed based on the USE. Our measure captures the change in the policy stance including both the current action (e.g., change in the federal funds rate target) and the expected future action (e.g., forward guidance about

17For four meetings (September 2008, June 2009, June 2011, August 2013), we drop the first paragraph of each statement to calculate the textual similarity because the original version including the first paragraph has not shown enough dissimilarity between the dovish alternative statement and the hawkish statement which is crucial for our identification of the tone. 14

the future interest rates). As in (7), we observe the monotonicity across the three measures. While the sign of the monetary policy stance is determined by the tone, its magnitude is largely governed by the dissimilarity from the previous statement.

To extract monetary policy surprises, we construct the market expectation of monetary policy stance which is the weighted average of the dovish and hawkish monetary policy stance. The market-based probability that the dovish and hawkish alternative statement would be

released are parameterized by pt and 1−pt, respectively. To calibrate the market-based prob-

ability pt, we use high-frequency bond market return data around FOMC announcements. One virtue of our maximum rank correlation approach is that it sidesteps the burden of 2 estimating αi, βi, σi when identifying p1:T in (9). To provide robustness to our claim, we rely

on bond futures returns of various combinations of ∆l, ∆h ∈ {10, ..., 120}min to obtain p1:T and the corresponding MPS(p1:T ). We provide the results in Figure3. The median values ofp ˆ1:T are highly correlated with each other, e.g., 0.96 or higher. This finding implies that the dovish probabilities extracted from bond returns are fairly robust to different window intervals or instruments. The robustness of our result is different from Bu et al.(2020) who find large differences in monetary policy shock estimates depending on the maturity of the bond data. However, when both use the five-year bond return, our measures are highly correlated with each other. We interpret this as demonstrating the value of our text-based analysis in isolating the common component from the response of an individual bond return to a policy announcement without using the statistical factor analysis of bond returns of multiple maturities.

3.3 Stock market responses to monetary policy surprises.

Since the Federal Reserve intervenes in Treasury markets to influence the interest rate, it is not surprising that bond returns react to monetary policy surprises. But the ultimate goal of the monetary policy is to achieve the dual mandate of maximum employment and price stability by affecting the real economy. For this, understanding the links between monetary policy and asset prices above and beyond bond returns is important as highlighted by Bernanke and Kuttner(2005) who find that an unanticipated 25 bps cut in the federal funds rate leads to about 1 percent increase in the stock market return. We turn to the 15

Figure 3: Dovish probability comparison

0.5

0 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

0.5

0 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

0.5

0 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

Notes: We rely on the 5-year and 10-year Treasury bond futures returns and the 1-year Eurodollar futures returns. Returns are deﬁned with the following interval ∆l, ∆h ∈ {10, ..., 120}min. The median values are indicated with solid lines. stock market reaction to our measure of monetary policy surprise to check if our text-based measure captures similar stock market responses. For the benchmark case, we select the 5-year Treasury bond futures returns with window intervals ∆l = ∆h = 10min to back out the probability weights and construct MPS(ˆpt). Con- ditional on this output, we conduct the regression analysis using stock returns as an external validation check. Speciﬁcally, we regress stock returns rs on the bond market-implied t−∆l,t+∆h

MPS(ˆpt). In essence, we are estimating (9) using an OLS with stock returns. The estimation results summarized in Table3 imply that the bond market-implied MPS(ˆpt) signiﬁcantly predict stock returns measured at various window intervals. Because we normalized MPS(ˆpt) to have a unit variance, we can directly interpret the magnitude of β coeﬃcient in assessing the 16

Table 3: Stock returns: Regression results

2 [∆l ∆h] α β t-stat (α) t-stat (β) R [−10 10] 0.05 -0.23 1.08 [1.07] -4.75 [-4.43] 0.19 [0.19] [−20 20] 0.04 -0.20 0.75 [0.74] -4.78 [-4.24] 0.12 [0.12] [−30 30] 0.10 -0.18 1.49 [1.48] -4.45 [-3.81] 0.08 [0.08] [−40 40] 0.16 -0.19 2.25 [2.24] -3.33 [-2.92] 0.07 [0.07] [−50 50] 0.16 -0.18 2.21 [2.13] -3.20 [-2.77] 0.07 [0.07] [−60 60] 0.20 -0.22 2.56 [2.59] -3.35 [-2.96] 0.08 [0.08] [−90 90] 0.19 -0.21 2.25 [2.21] -2.43 [-2.18] 0.06 [0.06] [−120 120] 0.17 -0.21 1.72 [1.69] -1.85 [-1.64] 0.05 [0.05]

Notes: Based on the median value ofp ˆ1:T , we regress stock returns (deﬁned at various window intervals) 2 on MPS(ˆp1:T ). Numbers in square brackets are alternative t statistics and R based on bootstrap standard errors to account for the fact that our policy surprise measure is a generated regressor.

economic significance of MPS(ˆpt). On average, we find that a positive one-standard-deviation surprise leads to a 20 bps drop in stock prices. The R2 values are higher for returns defined with shorter window intervals.

Nakamura and Steinsson(2018) argue that unexpected policy easing or tightening by the FOMC identified by five different high-frequency interest rate futures data (current month and next month federal funds futures, 2, 3, 4 quarter ahead Eurodollar futures) around policy announcements often did not move the private sector’s expectation of economic growth in the intended direction. For instance, they show that the Bluechip forecast of real GDP growth declined after unexpected policy tightening in many cases because the dovish announcement might have revealed the Federal Reserve’s private information on the gloomy economic outlook. However, Bauer and Swanson(2020) provide evidence that the Federal Reserve does not have an information advantage over the private sector and suggest that the finding in Nakamura and Steinsson(2018) can be explained by the relatively low-frequency nature of the private sector forecast data. In addition, they use the daily-frequency stock market return data. Bauer and Swanson(2020) show that the unexpected policy tightening leads to a decline in the high-frequency stock market return even though they use the same monetary policy shock measure as Nakamura and Steinsson(2018). Our finding is consistent with Bauer and Swanson(2020) in terms of the sign of the stock market response to a monetary policy surprise. As in Bauer and Swanson(2020), our finding is not driven by 17

the diﬀerence between our measure and monetary policy shock estimates of Nakamura and Steinsson(2018) because both measures are highly correlated as shown in Table4.

Our MPS(ˆpt) is also highly correlated with other measures of monetary policy shocks based on the high-frequency asset market data around FOMC announcements. Bu et al. (2020) construct a monetary policy shock using the idea that the variance of the daily bond return is higher on FOMC days relative to non-FOMC days due to the monetary policy announcement. In addition to near-term maturities, they use information from the entire yield curve (up to the maturity of thirty years). They argue that the information channel effect highlighted by Nakamura and Steinsson(2018) is present mainly because Nakamura and Steinsson(2018) consider only the near-term interest rate data in constructing a measure of monetary policy shock. However, we do not find any significant difference in our sample in terms of the correlation of our measure with the two measures provided by Nakamura and Steinsson(2018) and Bu et al.(2020). Also, given the robustness of our measure to the maturity of the bond return used to back out policy surprises, we do not think that the maturity of the bond return is critical in accounting for the lack of the information channel effect in our study. More plausibly, we suspect that more efforts made by the Federal Reserve staff to fine tune statement language since 2004 might have increased the effectiveness of monetary policy communications on the asset markets. This hypothesis is consistent with the finding in Lunsford(2020) who shows that the information channel effect was present before August 2003 but disappeared for the later sample as the FOMC provides a more explicit policy inclination in statements.18

Swanson(2017) identifies multiple dimensions of monetary policy shocks using eight different asset prices consisting of three Treasury bond yields (maturities of 2, 5, 10 years) on top of the five interest rate futures used in Nakamura and Steinsson(2018). He computes the three principal components that account for common variations in these eight different asset prices around FOMC announcements. He rotates three components to get 1) federal funds rate (FFR) factor that affects the current month federal funds rate futures, 2) forward guidance (FG) factor that is orthogonal to the change in the current month federal funds rate futures, and 3) large-scale asset purchase (LSAP) factor that is also orthogonal to the change in the current month federal funds rate futures and plays a minimum role in explaining the data before the federal funds rate reached the effective lower bound in December 2008. In 18A similar observation was made by Hoesch et al.(2020). 18 practice, these factors are largely distinguished by their different loadings on the maturity spectrum of the underlying interest rate data. The FFR factor has a large non-zero loading on the current month federal funds futures while the other factors have zero loadings. In addition, the loadings of FG factor are concentrated in the one-to-five year maturity spectrum while the LSAP factor has the largest loading on the ten-year Treasury yield. Our measure is particularly highly correlated with the FG factor, indicating that the communication strategy by different wording can be mostly effective in moving the medium-term interest rates.19

Bauer and Swanson(2020) emphasize that it is important to control information from economic indicators released during the inter-meeting period because the FOMC responds to it. Our results in Table4 showing a big discrepancy in the correlation with the forward guidance factor in Swanson(2017) between policy surprises and policy stance is consistent with their ﬁnding because controlling the market expectation before the FOMC meeting is most important for the forward guidance factor that is overweighting the short-to-medium term maturity bond returns. On the other hand, the LSAP factor which is loaded onto the slope of the yield curve and overweighting the long-term bond return response is less sensitive to controlling changes in market expectations during the inter-meeting period as the correlation with policy surprise is largely same as that with policy stance.

Table 4: Comparison with other measures

MP stance: surprise MP stance: level Bu et al.(2020) 0.50 0.16 Nakamura and Steinsson(2018) 0.50 -0.10 Swanson(2017) (FFR+FG+LSAP) 0.50 0.01 FFR 0.20 0.03 FG 0.52 -0.16 LSAP -0.12 -0.17

Notes: Based on the median value ofp ˆ1:T , we construct MPS(ˆp1:T ) and compute correlation with other existing measures of monetary policy factors. The last three factors are from Swanson(2017): 1) federal funds rate (FFR) factor; 2) forward guidance (FG) factor; and 3) large-scale asset purchase (LSAP) factor.

19The negative correlation with the LSAP factor is due to the fact that Swanson(2017) normalized a positive innovation to the LSAP as larger asset purchases than expected, resulting in policy easing. 19

3.4 Counterfactual policy evaluation.

The key advantage of our approach is that we are able to conduct a counterfactual policy evaluation by replacing the released statement with either one of the alternative statements. We also look at the paragraph-level USE representation instead of the statement-level one in order to further investigate the impact of changing a particular part of the statement on the monetary policy shock.

We consider two episodes in our sample period. First, we look at the FOMC statement released in September 2007.20 The policy decision suggested in Alt B was to lower its target for the federal funds rate 50 bps whereas that in Alt C was to lower it by 25 bps. Thus, the market could view Alt C more hawkish than Alt B. On the other hand, in the assessment of risk, Alt C stated that the downside risks to economic growth outweigh the upside risks to inflation (we refer to it as “balance of risk statement”), which was omitted in Alt B. Our similarity measures suggest that the statement about the balance of risk can be perceived to be more accommodative than the 50 bps cut in the target rate.21 If Alt C were the only available hawkish alternative statement, this might have posed a risk to our identification of the tone. But the Board staff made a more extreme hawkish version of the alternative statement (Alt D) for this meeting perhaps because they recognized the possibility that Alt C may sound dovish rather than hawkish due to the “balance of risk” language. The released statement was similar to Alt B by lowering the federal funds rate 50 bps but without stressing downside risk. In this section, we consider the counterfactual adoption of Alt C and quantify its impact on stock prices.

Conditional on βˆ = −0.23 in (the ﬁrst row of) Table3, we multiply the counterfactual monetary policy surprise component to asset the impact on the stock returns (deﬁned in the 10-minute interval). For this, we replace the monetary policy stance with the counterfactual one and subtract the bond price-implied expected monetary policy stance to compute the counterfactual monetary policy surprise component. It is important to note that we are only replacing one data point (that corresponds to the September 2007 FOMC statement release

20The detailed description of alternative statements is provided in the appendix. 21We confirm this by looking paragraph level USE representations of statements. See the appendix for further discussion. To some extent, the Board staff recognized this issue and provided a more extreme hawkish version of alternative statement (Alt D) at this FOMC cycle. Alt D does have the balance of risk statement like Alt C. Our identification of the tone of the monetary policy stance compares the released statement with Alt A and Alt D, sidestepping this ambiguity in the tone of Alt C. 20 date) while keeping all else equal in this exercise. We find that the counterfactual monetary policy stance turns out to be much more dovish leading to an 1.94% increase of stock prices relative to the 0.1% increase without that statement. Based on the estimate of Bernanke and Kuttner(2005), this might be equivalent to an additional 50 bps cut in the federal funds rate.22 While suggestive, caution is needed in interpreting this result because the counterfactual policy surprise is more than three standard deviation away from the original estimate of policy surprise, making our assumption on the stability of the private sector’s expectations somewhat fragile.23 If the counterfactual shock is much bigger than the realized one, the linearity of our stock market return regression may not be a good assumption for this.

We perform another counterfactual exercise for the December 2010 FOMC meeting that can be a modest intervention.24 In this case, the Alt A replaces a qualitative forward guidance on the future path of the federal funds rate like “for an extended period” by a more explicit time-dependent guidance such as “at least through the mid-2012”. It also expands asset purchases by $200 billion. The actual statement released after the December 2010 meeting was close to Alt B and did not include languages on forward guidance and asset purchases used in Alt A. We ﬁnd that the stock market return might have increased by about 0.7% based on our regression if the Alt A were released.25 Since the counterfactual policy surprise is less than one standard deviation away from the estimate of the original policy surprise, we regard this result as a modest policy change that is less likely to materially aﬀect the expectations of the private sector.26

22Since Alt C actually contained a 25 bps cut in the federal funds rate target rather than a 50 bps cut in the released statement, our analysis suggest that the change in risk assessment actually might have had an impact equivalent to a 75 bps cut in the federal funds rate target. 23This is an example of well known Lucas(1976) critique. While the critique is valid for any reduced- form model that does not model the private sector’s expectations formation process explicitly, the practical relevance may be small if the counterfactual surprise is relatively modest. 24The detailed description of alternative statements is available in the appendix. 25Our paragraph level USE representations suggest that changing the language on asset purchased might have made a slightly bigger difference than changing the language on forward guidance. 26Eventually, the FOMC adopted a more explicit time-dependent forward guidance on the future path of the federal funds rate in the August 2011 meeting and Lunsford(2020) highlights the effectiveness of including such a language in influencing financial market conditions. 21

4 Conclusion

The central bank’s public communications about current and future policy actions have increasingly received attention as a policy tool. Since March 2004, the FOMC has deliberated on alternative policy statements prepared by the Federal Reserve staff before each FOMC meeting. Two alternative statements capture the hawkish or dovish deviation from the central tendency of the market expectation right before the meeting, providing cross-sectional variations around the released statement. We apply a novel natural language processing algorithm based on a deep learning architecture to alternative FOMC statements in order to identify the tone of the released statement. This USE algorithm detects the contextual meaning of words in the statement and quantifies the information provided by language in alternative statements. We construct a new measure of monetary policy surprises by combining the high-frequency bond returns around FOMC announcements with the text analysis of alternative statements by the USE. Our text-based measure is able to capture the common factor in an individual bond return response to a policy announcement without using the statistical factor analysis common in the existing literature, which can be sensitive to the choice of target maturities. We find that an unexpected policy tightening leads to a decline in the stock market return on average. The finding vindicates that the FOMC’s communication achieved mostly its intended effect on the stock market return at least since 2004, which is consistent with the recent empirical findings. Two suggestive counterfactual exercises show 1)involving alternative statements implies that changing the language describing the risk assessment might have had a much bigger impact on the stock market return than changing the federal funds rate target by 25 bps and 2)a more explicit forward guidance on the future path of the federal funds rate could have been effective in easing financial market conditions. 22

References

Bauer, Michael and Eric T Swanson, “The Fed’s Response to Economic News Explains the,” 2020.

Bernanke, Ben S., “Central Bank Independence, Transparency, and Accountability,” in “Speech of Chairman Ben S. Bernanke at the Institute for Monetary and Economic Studies International Conference” 2010.

Bernanke, Ben S and Kenneth N Kuttner, “What explains the stock market’s reaction to Federal Reserve policy?,” The Journal of ﬁnance, 2005, 60 (3), 1221–1257.

Blinder, Alan S., Michael Ehrmann, Marcel Fratzscher, Jakob De Haan, and David-Jan Jansen, “Central Bank Communication and Monetary Policy: A Survey of Theory and Evidence,” Journal of Economic Literature, 2008, 46 (4), 910–945.

Bu, Chunya, John Rogers, and Wenbin Wu, “Forward-Looking Monetary Policy and the Transmission of Conventional Monetary Policy Shocks,” 2020.

Cer, Daniel, Yinfei Yang, Shengyi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun- Hsuan Sung, Brian Strope, and Ray Kurzweil, “Universal Sentence Encoder,” 2018. Manuscript.

FRB, “Some General Issues Concerning Statement Language,” in “Bluebook, 2004 March FOMC Meeting” 2004.

Gentzkow, Matthew, Bryan Kelly, and Matt Taddy, “Text as Data,” Journal of Economic Literature, 2019, 57 (3), 535–574.

Giavazzi, Francesco, Felix Iglhaut, Giacomo Lemoli, and Gaia Rubera, “Terror- ist Attacks, Cultural Incidents and the Vote for Radical Parties: Analyzing Text from Twitter,” Technical Report, National Bureau of Economic Research 2020.

G¨urkaynak, Refet S, Brian Sack, and Eric Swanson, “Do Actions Speak Louder Than Words? The Response of Asset Prices to Monetary Policy Actions and Statements,” International Journal of Central Banking, 2005, 1, 55–93. 23

Han, Aaron K., “Non-parametric Analysis of a Generalized Regression Model,” Journal of Econometrics, 1987, 35, 303–316.

Hansen, Stephen, Michael McMahon, and Andrea Prat, “Transparency and Delib- eration Within the FOMC: A Computational Linguistics Approach,” Quarterly Journal of Economics, 2017, 133 (2), 801–870.

Hoesch, Lukas, Barbara Rossi, and Tatevik Sekhposyan, “Has the information channel of monetary policy disappeared? Revisiting the empirical evidence,” 2020.

Jegadeesh, Narasimhan and Di Andrew Wu, “Deciphering Fedspeak: The information content of FOMC meetings,” Available at SSRN 2939937, 2017.

Ke, Zheng Tracy, Bryan T. Kelly, and Dacheng Xiu, “Predicting Returns with Text Data,” 2019. Manuscript.

Lucas, Robert E J., “Econometric policy evaluation: A critique,” in “Carnegie-Rochester Conference Series on Public Policy,” Vol. 1 Elsevier 1976, pp. 19–46.

Lucca, David O and Francesco Trebbi, “Measuring central bank communication: an automated approach with application to FOMC statements,” Technical Report, National Bureau of Economic Research 2009.

Lunsford, Kurt G, “Policy Language and Information Eﬀects in the Early Days of Federal Reserve Forward Guidance,” American Economic Review, 2020.

Meade, Ellen E. and Miguel Acosta, “Hanging on Every Word: Semantic Analysis of the FOMC’s Postmeeting Statement,” FEDS Notes, Board of Governors of the Federal Reserve System (U.S.) 2015.

Nakamura, Emi and JónSteinsson, “High-frequency identification of monetary non- neutrality: the information effect,” The Quarterly Journal of Economics, 2018, 133 (3), 1283–1330.

Schonhardt-Bailey, Cheryl, Deliberating American Monetary Policy: A Textual Analysis, MIT Press, 2013. 24

Shapiro, Adam and Daniel J Wilson, “The Evolution of the FOMCs Explicit Inﬂation Target,” Federal Reserve Bank of San Francisco Working Paper 2019-02, 2019, 2019, 12.

Sherman, Robert P., “The Limiting Distribution of the Maximum Rank Correlation Es- timator,” Econometrica, 1993, 61 (1), 1123–137.

Swanson, Eric T, “Measuring the eﬀects of Federal Reserve forward guidance and asset purchases on ﬁnancial markets,” Technical Report, National Bureau of Economic Research 2017.

Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,Lukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” in “Advances in neural information processing systems” 2017, pp. 5998–6008.

Woodford, Michael, “Central Bank Communication and Policy Eﬀectiveness,” Proceedings of the Federal Reserve Bank of Kansas City Economic Symposium at Jackson Hole, 2005, pp. 399–474. A-1

Appendix

A Illustration

Figure A-1: FOMC statement in November 2005

Class I FOMC - Restricted Controlled (FR) Page 13 of 36 Table 1: Alternative Language fo r the November FOMC Announcement September FOMC Alternative A Alternative B Alternative C 1. The Federal Open Market Committee decided The Federal Open Market The Federal Open Market The Federal Open Market Policy today to raise its target for the federal funds Committee decided today to leave Committee decided today to raise its Committee decided today to raise its Decision rate by 25 basis points to 3¾ percent. its target for the federal funds rate target for the federal funds rate by target for the federal funds rate by unchanged. 25 basis points to 4 percent. 25 basis points to 4 percent. 2. Output appeared poised to continue growing at a Elevated energy prices and Elevated energy prices and The disruptive effects of recent good pace before the tragic toll of Hurricane hurricane-related disruptions in hurricane-related disruptions in Katrina. The widespread devastation in the Gulf hurricanes seem likely to be economic activity seem to have economic activity seem to have temporary, especially in light of region, the associated dislocation of economic slowed the growth of spending, set temporarily slowed the growth of activity, and the boost to energy prices imply that increased spending associated with spending, production, and employment will be set back employment, and weakened spending and set back employment. rebuilding efforts. Economic back in the near term. In addition to elevating consumer and business confidence. However, monetary policy growth continues to be supported premiums for some energy products, the disruption The persistence of such effects is accommodation, coupled with by robust underlying growth in to the production and refining infrastructure may uncertain, but robust underlying robust underlying growth in productivity. add to energy price volatility. growth of productivity and productivity, is providing ongoing While these unfortunate developments have monetary policy accommodation are support to economic activity. Rationale increased uncertainty about near-term economic providing support to economic Spending will also be boosted by performance, it is the Committee's view that they activity. rebuilding efforts in hurricane- do not pose a more pers istent threat. Rather, affected areas. monetary policy accommodation, coupled with robust underlying growth in productivity, is

providing ongoing support to economic activity. 3. Higher energy and other costs have the High energy and other costs have The cumulative rise in energy and Core inflation and longer-term potential to add to inflation pressures. added to inflation pressures. other costs has added to inflation inflation expectations remain However, core inflation has been relatively However, core inflation has been pressures. However, core inflation contained. However, high energy and low in recent months , and longer-term relatively low in recent months, and has been relatively low in recent other costs have boosted near-term inflation expectations remain contained. longer-term inflation expectations months, and longer-term inflation inflation expectations and price remain contained. expectations remain contained. pressures, likely ma king further policy firming necessary. 4. The Committee perceives that, with appropriate monetary po licy action, the upside [no change] [no change] [none] and downside risks to the attainment of both sustainable growth and price stability should be kept roughly equal. 5. With underlying inflation expected to be With underlying inflation expected Assessment contained, the Committee believes that policy to be contained, the Committee of Risk accommodation can be removed at a pace that believes that remaining policy [no change] [none] is likely to be measured. Nonetheless, the accommodation can be removed at Committee will respond to changes in a pace that is likely to be measured. economic prospects as needed to fulfill its Nonetheless, the Committee will obligation to maintain price stability. respond to changes in economic prospects as needed to fulfill its obligation to maintain price stability.

Figure A-1 provides the oﬃcial FOMC statement released in November 2005. The largest discrepancy between Alt A and Alt C under USE

sim(FOMCt,FOMCA,t) = 0.984, sim(FOMCt,FOMCC,t) = 0.858. (A-1)

Two key changes in Alt C: (i) drops the “measured pace” language for the ﬁrst time; (ii) eliminates the balance of risk in the previous statement.

Figure A-2 provides the oﬃcial FOMC statement released in September 2007. Alt C was supposed to be more hawkish than Alt B but the balance of risk statement overwhelmed a A-2

Figure A-2: FOMC statement in September 2007

Class I FOMC - Restricted Controlled (FR) Page 22 of 49

Table 1: Alternative Language for the September 2007 FOMC Announcement Alternative A Alternative B Alternative C Alternative D 1. The Federal Open Market The Federal Open Market Committee The Federal Open Market Committee The Federal Open Market Committee Committee decided today to lower decided today to lower its target for the decided today to lower its target for the decided today to keep its target for the Policy its target for the federal funds rate federal funds rate 50 basis points to federal funds rate 25 basis points to federal funds rate at 5¼ percent. Decision 50 basis points to 4¾ percent. 4¾ percent. 5 percent.

2. Tighter credit conditions and the Economic growth was moderate during Economic growth was moderate during Economic growth was moderate during intensification of the housing the first half of the year, but the the first half of the year, but the the first half of the year. Financial correction appear likely to exert tightening of credit conditions has the tightening of credit conditions has the market conditions have deteriorated in appreciable restraint on economic potential to intensify the housing potential to intensify the housing recent weeks, leading to tighter credit growth. Moreover, the potential for correction and to restrain economic correction and to restrain economic and an intensification of the housing significant spillovers from credit growth more generally. Today's action growth more generally. Today's action correction. These developments have market disruptions to business and is intended to help forestall some of the is intended to help forestall some of the the potential to restrain growth in household spending poses a risk to adverse effects on the broader economy adverse effects on the broader economy economic activity. Nonetheless, the the outlook. Today’s action is that might otherwise arise from the that might otherwise arise from the economy seems likely to continue to intended to help mitigate the disruptions in financial markets and to disruptions in financial markets and to expand at a moderate pace over coming adverse effects on the broader promote moderate growth over time. promote moderate growth over time. quarters, supported by solid growth Rationale economy arising from the outside the housing sector and a robust disruptions in financial markets and global economy. to promote moderate growth over time. 3. Readings on core inflation have Readings on core inflation have Readings on core inflation have Readings on core inflation have improved modestly this year. improved modestly this year. However, improved modestly this year. However, improved modestly this year. However, However, the Committee judges the Committee judges that some the Committee judges that some a sustained moderation in inflation that some inflation risks remain, and inflation risks remain, and it will inflation risks remain, and it will pressures has yet to be convincingly it will continue to monitor inflation continue to monitor inflation continue to monitor inflation demonstrated. Moreover, the high level developments carefully developments carefully. developments carefully. of resource utilization has the potential to sustain those pressures. 4. Even after today’s action, the The Committee will continue to closely Even after today’s action, the In the current circumstances, the Committee judges that the downside follow timely indicators of economic Committee judges that the downside Committee judges that the downside risks to economic growth outweigh prospects and will act as needed to risks to economic growth outweigh the risks to economic growth are now Assessment the upside risks to inflation. Future foster price stability and sustainable upside risks to inflation. Future policy roughly balanced by the upside risks to of Risk policy adjustments will depend on economic growth. adjustments will depend on the outlook inflation. Future policy adjustments will the outlook for both inflation and for both inflation and economic growth, depend on the outlook for both economic growth, as implied by as implied by incoming information. inflation and economic growth, as incoming information. implied by incoming information.

smaller rate cut. The released statement does not include a change in the risk assessment while the cutting the rate by 50 basis points as in Alt B.

sim(FOMCA,t,FOMCC,t) = 0.99 > 0.968 = sim(FOMCA,t,FOMCt) (A-2)

sim(FOMCC,t,FOMCD,t) = 0.897 < 0.968 = sim(FOMCt,FOMCD,t).

The similarity calculation indicates that not just the size of the rate cut but also the statement of the balance of risk matters. Our measures suggest that the balance of risk statement could have provided more accommodation than a 25 bps rate cut.

Figure A-4 compares alternative statements for the December 2010 FOMC meeting. Alt A A-3

Figure A-3: FOMC tone identiﬁcation: TF-IDF versus USE (normalized)

FOMC Tone FOMC Tone 5 5 TF-IDF 4 4 USE 3 3

2 2

1 1

0 0

-1 -1

-2 -2

-3 -3 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Difference in Tone Difference in Tone 4 4

3 3

2 2

1 1

0 0

-1 -1

-2 -2

-3 -3 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

provides a more explicit time-dependent forward guidance on the interest rate by mentioning that rates would be low “at least through mid-2012” instead of “for an extended period.” The change in forward guidance together with the expanded asset purchases made the tone of Alt A clearly much more dovish than the released statement which was close to Alt B.

Figure A-3 compares the approach of TF-IDF with USE. Note that TF-IDF finds a small difference between Alt A and Alt C while USE detects a large difference.

USE: sim(FOMCt,FOMCA,t) = 0.895, sim(FOMCt,FOMCc,t) = 0.99 (A-3)

TF-IDF: sim(FOMCt,FOMCA,t) = 0.975, sim(FOMCt,FOMCc,t) = 0.972 (A-4) A-4

Figure A-4: Alternative A FOMC statement in December 2010

Class I FOMC - Restricted Controlled (FR) December 9, 2010

Table 1: Overview of Alternatives for the December 14 FOMC Statement

Key November December Alternatives Components Statement A B C D Economic Activity

Recent pace of recovery recovery is economic recovery pace of recovery continues to be slow Developments continues to be slow continuing is proceeding pace of recovery recovery is pace of recovery continues to be slow continues to be slow; high continuing unemployment; Labor high unemployment n.a. employers remain Market n.a. reluctant to add to employers remain reluctant to add to payrolls; payrolls; unemployment unemployment rate is elevated rate is elevated gradual return to higher resource utilization with gradual return to higher resource utilization with price stability price stability Outlook progress has been progress remains progress has been progress has been disappointingly slow disappointingly disappointingly slow n.a. slow slow Inflation

expectations stable, expectations stable, Alternatives expectations stable, but underlying but underlying although underlying expectations stable, but but underlying inflation has inflation has inflation has trended Recent underlying inflation has inflation has continued to trend continued to trend lower, expectations Developments trended lower; measures trended lower; downward; downward; have remained are somewhat low measures are measures are low measures are stable somewhat low somewhat low same as “Economic Outlook same as “Economic Activity” outlook above Activity” outlook above Target Federal Funds Rate Intermeeting 0 to ¼ percent 0 to ¼ percent Period exceptionally low Forward exceptionally low levels exceptionally low levels low levels for some levels at least Guidance for an extended period for an extended period time through mid-2012 SOMA Portfolio Policy $800 billion of $400 billion of Treasuries ($200b $600 billion of Treasuries ($200b $600 billion of Treasuries discontinue program more than Nov.), Treasuries by end of less than Nov.), $50 by end of 2011:Q2, $75 announced in $75 billion per 2011:Q2, $75 billion per month, billion per month November Approach month, through billion per month through 2011:Q2 2011:Q3 for the time being, maintain reinvestment maintain reinvestment policy maintain existing policy reinvestment policy Future Policy Action will adjust program as needed; will employ will employ policy policy tools as necessary will adjust program as needed; will employ policy tools as tools as necessary to to support the recovery Approach necessary to support the recovery and to help ensure that promote maximum and to help ensure that inflation, over time, is at levels consistent with its mandate employment and inflation, over time, is at price stability levels consistent with its mandate

Page 13 of 46

A change in the description of the interpretation of the incoming data seems to play a bigger role in USE than TF-IDF. We can see that in Figure A-5 - Figure A-7. A-5

Figure A-5: Alternative A FOMC statement in October 2013

Authorized for Public Release Class I FOMC – Restricted Controlled (FR) October 24, 2013

FOMC STATEMENT—OCTOBER 2013 ALTERNATIVE A

1. The effects of the temporary shutdown of the federal government [ , including delays in releases of some key data, ] have made the evolution of economic conditions during the intermeeting period somewhat more difficult to assess. However, information received since the Federal Open Market Committee met in July September generally suggests that economic activity has been expanding at a moderate modest pace. Some Indicators of labor market conditions have shown some further improvement in recent months, but the unemployment rate remains elevated. Available data suggest that household spending and business fixed investment advanced, and but that the recovery in the housing sector has been strengthening, but mortgage rates have risen further has slowed in response to higher mortgage rates. and Fiscal policy is restraining economic growth. Apart from fluctuations due to changes in energy prices, inflation has been running below the Committee’s longer-run objective, but even though longer-term inflation expectations have remained stable. 2. Consistent with its statutory mandate, the Committee seeks to foster maximum employment and price stability. The Committee expects that, with appropriate policy accommodation, economic growth will pick up from its recent pace and the unemployment rate will gradually decline toward levels the Committee judges Alternatives consistent with its dual mandate. The Committee sees the downside risks to the outlook for the economy and the labor market as having diminished, on net, since last fall, but the tightening of financial conditions observed in recent months since the spring, if sustained, could slow the pace of improvement in the economy and labor market. The Committee recognizes that inflation persistently below its 2 percent objective could pose risks to economic performance, but it anticipates that inflation will move back toward its objective over the medium term. 3. Taking into account the extent of federal fiscal retrenchment over the past year, the Committee sees the improvement in economic activity and labor market conditions since it began its asset purchase program a year ago as consistent with growing underlying strength in the broader economy. However, the Committee decided to await more evidence that progress will be sustained before adjusting judges that progress toward its objectives for the labor market and inflation is not yet sufficient to warrant reducing the pace of its purchases. Accordingly, the Committee decided to continue purchasing additional agency mortgage-backed securities at a pace of $40 billion per month and longer-term Treasury securities at a pace of $45 billion per month. The Committee is maintaining its existing policy of reinvesting principal payments from its holdings of agency debt and agency mortgage-backed securities in agency mortgage-backed securities and of rolling over maturing Treasury securities at auction. Taken together, these actions should maintain downward pressure on longer-term interest rates, support mortgage markets, and help to make broader financial conditions more accommodative, which in turn should promote a stronger economic recovery and help to ensure that inflation, over time, is at the rate most consistent with the Committee’s dual mandate. 4. The Committee will closely monitor incoming information on economic and financial developments in coming months and will continue its purchases of Treasury and

Page 23 of 64 A-6

Figure A-6: Released FOMC statement in October 2013

Press Release

Release Date: October 30, 2013

For immediate release

Information received since the Federal Open Market Committee met in September generally suggests that economic activity has continued to expand at a moderate pace. Indicators of labor market conditions have shown some further improvement, but the unemployment rate remains elevated. Available data suggest that household spending and business fixed investment advanced, while the recovery in the housing sector slowed somewhat in recent months. Fiscal policy is restraining economic growth. Apart from fluctuations due to changes in energy prices, inflation has been running below the Committee's longer-run objective, but longer-term inflation expectations have remained stable.

Consistent with its statutory mandate, the Committee seeks to foster maximum employment and price stability. The Committee expects that, with appropriate policy accommodation, economic growth will pick up from its recent pace and the unemployment rate will gradually decline toward levels the Committee judges consistent with its dual mandate. The Committee sees the downside risks to the outlook for the economy and the labor market as having diminished, on net, since last fall. The Committee recognizes that inflation persistently below its 2 percent objective could pose risks to economic performance, but it anticipates that inflation will move back toward its objective over the medium term.

Taking into account the extent of federal fiscal retrenchment over the past year, the Committee sees the improvement in economic activity and labor market conditions since it began its asset purchase program as consistent with growing underlying strength in the broader economy. However, the Committee decided to await more evidence that progress will be sustained before adjusting the pace of its purchases. Accordingly, the Committee decided to continue purchasing additional agency mortgage-backed securities at a pace of $40 billion per month and longer-term Treasury securities at a pace of $45 billion per month. The Committee is maintaining its existing policy of reinvesting principal payments from its holdings of agency debt and agency mortgage- backed securities in agency mortgage-backed securities and of rolling over maturing Treasury securities at auction. Taken together, these actions should maintain downward pressure on longer- term interest rates, support mortgage markets, and help to make broader financial conditions more accommodative, which in turn should promote a stronger economic recovery and help to ensure that inflation, over time, is at the rate most consistent with the Committee's dual mandate.

The Committee will closely monitor incoming information on economic and financial developments in coming months and will continue its purchases of Treasury and agency mortgage- backed securities, and employ its other policy tools as appropriate, until the outlook for the labor market has improved substantially in a context of price stability. In judging when to moderate the pace of asset purchases, the Committee will, at its coming meetings, assess whether incoming information continues to support the Committee's expectation of ongoing improvement in labor market conditions and inflation moving back toward its longer-run objective. Asset purchases are not on a preset course, and the Committee's decisions about their pace will remain contingent on the Committee's economic outlook as well as its assessment of the likely efficacy and costs of such

B Technical Details of Universal Sentence Encoding Architecture

The USE architecture in this paper is based on six neutral network layers, each of which has two sublayers with a self-attention channel. We ﬁrst describe tho original architecture and A-7

Figure A-7: Alternative C FOMC statement in October 2013

Authorized for Public Release Class I FOMC – Restricted Controlled (FR) October 24, 2013

FOMC STATEMENT—OCTOBER 2013 ALTERNATIVE C

1. Information received since the Federal Open Market Committee met in July September suggests that economic activity has been expanding continues to expand at a moderate pace. Some Indicators of labor market conditions have shown some further improvement in recent months; in particular, but the unemployment rate, remains though still elevated, has continued to decline. Household spending and business fixed investment advanced, and the housing sector has been strengthening, but continued to strengthen, even though mortgage rates have risen further on balance in recent months and fiscal policy is restraining economic growth. Apart from fluctuations due to changes in energy prices, inflation has been running somewhat below the Committee’s longer-run objective, but longer-term inflation expectations have remained stable. 2. Consistent with its statutory mandate, the Committee seeks to foster maximum employment and price stability. The Committee expects that, with appropriate policy accommodation, economic growth will pick up from its recent pace and the unemployment rate will gradually decline toward levels the Committee judges consistent with its dual mandate. The Committee sees the downside risks to the outlook for the economy and the labor market as having diminished, on net, since last fall. but the tightening of financial conditions observed in recent months, if sustained, Alternatives could slow the pace of improvement in the economy and labor market. The Committee recognizes that inflation persistently below its 2 percent objective could pose risks to economic performance, but it anticipates The Committee has become more confident that labor market conditions will continue to improve and that inflation will move back toward its 2 percent objective over the medium term. 3. Taking into account the extent of federal fiscal retrenchment over the past year, the Committee sees the improvement in economic activity and labor market conditions since it began its asset purchase program a year ago as consistent with growing underlying strength in the broader economy. However, the Committee decided to await more evidence that progress will be sustained before adjusting the pace of its purchases. Accordingly, the Committee decided to continue purchasing additional agency mortgage-backed securities at a pace of $40 billion per month and longer-term Treasury securities at a pace of $45 billion per month. In light of the cumulative progress toward maximum employment and the improvement in the outlook for labor market conditions, the Committee decided to make modest downward adjustments in the pace of its of asset purchases. Beginning in November, the Committee will add to its holdings of agency mortgage-backed securities at a pace of [ $30 ] billion per month rather than $40 billion per month, and will add to its holdings of longer-term Treasury securities at a pace of [ $35 ] billion per month rather than $45 billion per month. The Committee is maintaining its existing policy of reinvesting principal payments from its holdings of agency debt and agency mortgage-backed securities in agency mortgage-backed securities and of rolling over maturing Treasury securities at auction. Taken together, these actions The Committee’s sizable and still-increasing holdings of longer-term securities should maintain downward pressure on longer-term interest rates, support mortgage markets, and help to make broader financial conditions more accommodative, which

Page 27 of 64

then explain how to ﬁne tune it to obtain the paragraph level decomposition of similarity scoring across statements. A-8

B.1 Deep Neural Network Layers in the USE

The ﬁrst neural network in the USE is built by linking two sublayers as shown in Figure A-8 after taking a group of word embeddings that represents the source sentence as input. The 1 1 ﬁrst layer generates the sentence embedding vector (h1, ··· , hM ) as output and feeds this as input for the second layer.27

1 1 1 Pn 1 Second Sublayer: Feed Forward Neural Network (h1 =, ··· , hM ) , hj = l=1 max(0,Wjlwbl + bj,1) + bj,2 , j = 1, ··· ,M

Pn First Sublayer: Self-Attention (wb1, ··· , wbn) , wbi = k=1 Att(wi, wk)wk , i = 1, ··· , n

Input: A Set of Word Embeddings: (w1, ··· , wn)

Figure A-8: First Neural Network Layer

Here, attention weights are determined by the distance between diﬀerent word embeddings as follows:

ni w0 w X e i,j i,k wi,j = Att(wi,j, wi,k)wi,k , Att(wi,j, wi,k) = 0 . (A-5) b Pni wi,j wi,l k=1 l=1 e

The entire USE algorithm works by vertically stacking six neural network layers which take the sentence embedding output in the previous layer as input and generate another sentence embedding as output. Figure A-9 describes the entire process.

To train parameters in the neural network architecture, we need to deﬁne the loss function that compares outcomes based on sentence embeddings from the USE with those based on human judgement. For example, if we deﬁne the relation between two texts as one of 3

27The actual USE architecture is slightly more complicated than presented below. It involves 1)sub-word (character) level embedding, 2)positional embedding in which the order of any given word is also mapped into the embedding of that word, 3)residual connection in which input bypasses attention and feed-forward neural network channels with a certain probability known as the dropout rate, 4)output from the layer is normalized to have mean zero and standard deviation of one, and 5)8 multihead attention channels are applied in the attention sublayer. A-9

5 5 Final Layer: input(h1, ··· , hn) ,output(U1, ··· ,U512)

4 4 5 5 Fifth Layer: input(h1, ··· , hn) ,output(h1, ··· , hn)

3 3 4 4 Fourth Layer: input(h1, ··· , hn) ,output(h1, ··· , hn)

2 2 3 3 Third Layer: input(h1, ··· , hn) ,output(h1, ··· , hn)

1 1 2 2 Second Layer: input(h1, ··· , hn) ,output(h1, ··· , hn)

1 1 First Layer: input(w1, ··· , wn) ,output(h1, ··· , hn)

Figure A-9: Neural Network Architecture

classes (entail,contradict,neutral), we can apply the softmax classifier (f) to the difference between two embeddings. In this case, we can choose parameters in the neural network architecture to minimize the loss function that measures the distance between the machine- i j human i classified outcome (f(U ,U )) and the one judged by humans (f (Texti, Textj)) where U

is the 512-dimensional USE representation of Texti. In addition, two other natural language processing tasks are run to train the model.

• Skip-thought task: conditional on the center sentence, predict neighboring sentences (previous and next). The training dataset is from wikipedia articles.

• Question-answer prediction: predict the correct response for a given question among a list of correct answers and other randomly sampled answers. The training dataset is from web question-answer pages and discussion forums. A-10

• Natural language inference: given a premise sentence and a hypothesis sentence, ex-

tract the relation between them. Let Up and Uh be the sentence embeddings of the premise and the hypothesis, respectively. A fully-connect layer and and a 3-way soft-

max classiﬁer are applied for the concatenated input of (Up, Uh, |Up −Uh|, and Up −Uh). The three-way classiﬁer predicts if the premise entails, contradicts, or is neutral to the hypothesis. The training dataset is the SNLI corpus.

B.2 Paragraph level USE Representations

In some cases, paragraph-by-paragraph comparison may provide more interpretable results. For instance, we may be interested in which paragraph drives the similarity score between diﬀerent statements. For this, we obtain paragraph level USE representations and approximate the statement level USE representation by a weighted average of paragraph level USE representations.

R Denote the USE representation of the released FOMC statement at time t by St . Similarly, i St , (i = A, B, C, D) denotes the USE representation of alternative statements. The USE i representation of the j-th paragraph of the FOMC statement at time t is Pj,t. To calculate i P i Pj,t, we run the USE algorithm for each paragraph j. The idea is to construct k wkPk,t that i can mimic St best in terms of minimizing the squared diﬀerence between two representations of the FOMC statement at time t.

• Step 1: Paragraph Padding Some statements are longer than others, meaning that the corpus of FOMC statements has an unequal length depending on the statement. An easy way to ﬁx this is pad a shorter statement with empty paragraph encodings.

Suppose that nmax is the maximum number of paragraph of any given FOMC statement from the entire corpus of our dataset including both released statements and alternative statements. Then, we can extract the following array of the paragraph USE representation of the FOMC statement.

R R R Pt = [P1,t, ··· ,Pnmax,t]. (A-6)

If the number of paragraphs in the statement at time t (nR,t) is smaller than nmax, we

add (nmax − nR,t) zero vectors of 512 dimensions. The purpose of this operation is to A-11

make the USE representation of any FOMC statement have the same number of the USE representations at the paragraph level.

• Step 2: Approximate the Statement Level USE Representation by a Weighted Average of Paragraph Level USE Representations

The goal is to select weights (wj , j = (1, ··· , nmax) that can mimic this statement- level USE representation using paragraph-level USE representations. We consider the following squared loss:

X X i X i T i X i (St − wjPj,t) (St − wjPj,t). (A-7) i∈R,A,B,C,D t j j

We can put the non-negativity and unit-sum constraints on wj such that wj >= P i P i 0 , j wj = 1. Once we ﬁnd the solution for weights, we can mimic Pt by j wjPj,t. But the numerical optimization routine might be non-convex when you put the con-

straints directly. So we may consider the following transformation of wj to make the problem an unconstrained minimization problem:

eαj wj = n , (A-8) P max αk k=1 e

where αj is an unconstrained parameter. Notice that wj still satisﬁes the constraints

but we are minimizing the loss function with respect to (α1, ··· , αnmax).

• Step 3: Decomposing the Similarity Scoring For the unit-vector, the cosine similarity is simply the inner product. So we can renormalize the USE representation to have a unit length. In that case, we have the following nice decomposition of the similarity scoring between texts.

nmax nmax i j X i X j X X i j Sim(Pt ,Pt ) ∝ Sim( wkPk,t, wkPk,t) = wkwk0 Sim(Pk,t,Pk0,t). (A-9) k=1 k=1 k k0