AI-powered analysis of Fed communication
Dunn et al. explore the effectiveness of generative AI models in interpreting and categorizing topics from Federal Open Market Committee (FOMC) meeting minutes. The authors extracted sentences from FOMC meeting minutes published from 2010 onwards and manually tagged 20% of approximately 5,100 sentences with one or more of seven topic categories in order to create a dataset to serve as the “ground truth” for evaluating AI model performance. They then tested 11 different generative AI models, including versions from Anthropic, Cohere, Meta, Mistral, Google, and OpenAI. The best best-performing model (GPT-4) was used to categorize all 25,000 sentences from 2010-2024 FOMC minutes and an analysis of trends in topic distributions over time was performed to reveal shifts in FOMC discussion focus.
Key findings:
- Models performed best in identifying “Balance Sheet” and “Fed Funds Rate” topics while “Financial Developments” was the most challenging category.
- More recent models generally performed better, with notable improvements between Llama2 and Llama3 versions. Small changes in interaction methods could lead to substantial performance differences.
- Topic Trends Observed
– There is an increased focus on Financial Developments and Financial Stability over time.
– There is a decline in discussions of Real Activity and Labor Markets.
– There is a dramatic evolution in inflation focus, with a marked increase in the latter half of the 2010s.
– There are brief periods of increased focus on Fed Funds Rate and Balance Sheet topics around specific policy events.
Using Generative AI Models to Understand FOMC Monetary Policy Discussions
Authors: Wendy Dunn, Ellen E. Meade, Nitish Ranjan Sinha, Raakin Kabir
From: Board of Governors of the Federal Reserve System
AI-powered analysis of ECB communication
Nicolas Fanta and Roman Horvath in this paper examine the use of artificial intelligence (AI) to analyze European Central Bank (ECB) communication. The authors used 1,769 inter-meeting verbal communication events from ECB Governing Council members between 2008-2016. AI models (ChatGPT 3.5 and 4) were then used to classify communications as leaning towards monetary policy easing, tightening, or maintaining the status quo. Finally, Fanta and Horvath conducted probit regressions to test if the AI indicator could predict future ECB monetary policy decisions, controlling for past decisions and market expectations.
Key findings:
- The AI-based indicator closely replicated a human-coded indicator, suggesting AI can decipher central bank communication as accurately as human experts.
- In regression analyses, the AI-based communication indicator was statistically significant in predicting future ECB monetary policy changes, even after controlling for past decisions and market expectations.
- Results were robust across different AI models (GPT 3.5 and 4) and approaches (zero-shot and single-shot prompting).
- The predictive power of the AI indicator held even when excluding the zero lower bound period (2014-2016).
- The authors conclude that AI can effectively analyze central bank communication, providing results comparable to human expert judgment but more quickly and cost-effectively.
Artificial Intelligence and Central Bank Communication: The Case of the ECB
Authors: Nicolas Fanta, Roman Horvath
From: Charles University
