Why Are Economists Still Uncertain About the Effects of Monetary Policy?

Despite decades of research, there remains substantial uncertainty about the quantitative effects of monetary policy. Different models produce conflicting predictions, and these predictions lack precision. This article discusses some reasons for these issues. In addition to the relative lack of data, the structure of the economy has continued to evolve, posing challenges for empirical macroeconomic analysis more generally. Economists have been confronting these challenges by developing tools to jointly consider a range of models and continuing to seek new sources of data.

Milton Friedman and Anna Schwartz first published their seminal opus on monetary policy — "A Monetary History of the United States, 1867-1960" — 60 years ago. Yet, sizeable uncertainty about the quantitative effects of monetary policy remains despite enormous progress in the field. This is one important element driving the continued debate about the appropriate conduct of monetary policy, as well as related discussions, such as when inflation will return to closer to the Federal Reserve's 2 percent target or whether the rise in interest rates will lead to a recession.

Economists rely on economic and statistical models to estimate the quantitative effects of monetary policy. These models postulate relationships across economic variables of interest, and economists use data to infer what these imply quantitatively for the path of model variables — such as inflation or output — in response to a change in the monetary policy instrument, which is typically the nominal interest rate. The estimates and the uncertainty around those estimates depend on both the model assumptions and the available data.

This article discusses the range of estimates and uncertainty around them in several steps. First, we present a variety of estimates for the effects of monetary policy from workhorse models to provide a sense of the degree of uncertainty economists face. Next, we outline reasons for the continued imprecision. Finally, we summarize several approaches to deal with the challenges highlighted. While the discussion focuses on the effects of monetary policy, many of the issues also apply more broadly to quantitative macroeconomic research and policymaking.

Types of Uncertainty

To illustrate the degree of uncertainty in present-day estimates of the effect of monetary policy, we first plot various estimates of inflation's response to a 1 percentage point increase in interest rates. The estimates rely on standard statistical models and use monthly data on production, unemployment, prices, commodity prices and interest rates from 1969 through 1996 (following the chapter "Macroeconomic Shocks and Their Propagation (PDF)" from the Handbook of Macroeconomics).

To show the uncertainty within a given model, Figure 1 below plots confidence regions (between 10 and 90 percent) from one of the estimated models.

While the model predicts a negative average response in inflation after one year, the confidence regions indicate substantial uncertainty. These regions are wide, with the 90 percent confidence regions reaching a width of about 1 percentage point. Moreover, they comfortably include zero.

There is also uncertainty across models, as shown by the seven different point estimates plotted in Figure 2 below. Each of these use similar data and arise from either a vector autoregression or local projection, both of which are standard in the macroeconomics profession.

Despite the relatively minor differences across the estimators ex ante, they produce a striking range of quantitative results. Not only does the size of the effect vary across models, but the time taken for the effects to reach their troughs varies as well. Nevertheless, the models are not completely at odds with each other — as they all suggest an eventual decline in inflation — and thus do provide useful information for policymakers.

Why Uncertainty Persists

As highlighted by the above example, quantitative macroeconomic analysis presents several practical challenges relative to the natural sciences (such as physics, chemistry and biology) and even microeconomic analysis.¹

Crucially, macroeconomists must work with a relatively small amount of data. In particular, reliable macroeconomic data typically only dates back several decades, limiting the amount of information one can obtain. For example, quarterly data from 1960 through 2022 would yield roughly 250 observations, far less than the tens of thousands or even hundreds of thousands of observations often available to microeconomists or scientists.

The problem is compounded by the correlation in the data over time. For example, there have been only nine recessions since 1960 (as dated by the National Bureau of Economic Research). Economists therefore have an extremely limited number of episodes over which to study how monetary policy impacted the economy. In the face of limited data, it is hard to draw precise estimates of model parameters and, thus, the effects of monetary policy.

Inference is further complicated by limits to how much we can draw parallels across historical episodes. The structure of the U.S. economy has evolved over the past century, and each business cycle has its own idiosyncrasies. These changes have been especially conspicuous with the COVID-19 pandemic and its ensuing effects on the economy. As a result, even if we can fully disentangle the role of monetary policy in a previous episode, we still need to be careful in projecting those insights to the current situation. Indeed, depending on which sample period one uses for estimation, we can obtain substantially different answers.

Underlying the data is an extremely complicated web of interactions across households, firms and financial institutions in the economy, even extending beyond a country's borders through the global network of trade relationships. The decisions determining these interactions rely on these economic agents forming expectations about the economy, which in turn are critical for the resulting path of the economy. Even if an econometrician observes detailed data about the eventual trajectory of economic variables, it remains a daunting task to extract the variation arising purely from monetary policy. Failure to do so can lead to inaccurate estimates. Unlike scientists, macroeconomists are unable to run experiments to isolate these effects and instead must rely on specific theories or assumptions.

While econometricians have developed an increasingly rich set of theoretical results about pitfalls of different models, these models are much harder to implement in practice. Often, it is impossible to verify if the conditions needed for a model to produce accurate estimates are satisfied for a given data set. Moreover, even if an estimator is appropriate for, say, studying the response of inflation in the short run, it may not be suited for other variables or horizons.

Dealing With the Uncertainty

In response to the lack of data, there has been a concerted effort in the profession to increase the range of data available for macroeconomists. For example, there has been a painstaking effort to collect a longer time series of data (such as, the Jordà-Schularick-Taylor Macrohistory Database), although there remain limits on the frequency of the data and questions about how much the economy and, thus, macroeconomic responses might have changed over time.

In addition, the introduction of big data can provide richer insights into more recent times. While these approaches do not extend the number of episodes one can look at, they do seem particularly promising in our effort to isolate monetary policy movements from other macroeconomic fluctuations.

Finally, there is a growing literature using microeconomic data to inform our understanding of macroeconomic responses, even if it is often less than straightforward to extrapolate from microeconomic to macroeconomic contexts.

To confront the disagreement across models, my recent paper "Averaging Impulse Responses Using Prediction Pools" (co-authored with Thomas A. Lubik and Christian Matthes) proposes averaging across the different estimates. In particular, the paper develops a methodology to optimally weight a set of estimates for the response of the economy to an economic shock. Our methodology makes progress by having the flexibility to cater to the wide range of estimators available and accounting for the uncertainty in each estimate.

In the context of monetary policy, we find that the relative weight on different models varies drastically depending on the variable and horizon of interest. The methodology thus provides policymakers a fruitful way to aggregate multiple estimators in the face of conflicting results.

Conclusion

Even though economists have made huge leaps in understanding how monetary policy impacts the economy, there remains substantial disagreement and imprecision in estimates. While the present article focuses on the narrow question of the response of the macroeconomy (specifically inflation) to a change in interest rates, the issues extend more broadly to forecasting the path of the macroeconomy and deciding on an appropriate policy response. Supply chain issues, the tight labor market and the collapse of Silicon Valley Bank are just a few examples of the barrage of economic events whose effects need to be analyzed and forecasted in real time.

Nevertheless, models do provide useful insights that can guide policy. Economic theory gives us a framework to think about incoming data. Econometric and statistical models do provide answers to quantitative questions, even if these may be imprecise at times.

Policymakers need to balance the challenges laid out in this article with the knowledge gained from careful theoretical and empirical economic analysis, both in setting policy and in communicating with the public. These trade-offs are particularly salient and delicate in our current situation, with inflation recently rising to a 40-year high on the back of a historic pandemic.

Paul Ho is an economist in the Research Department at the Federal Reserve Bank of Richmond.

To cite this Economic Brief, please use the following format: Ho, Paul. (May 2023) "Why Are Economists Still Uncertain About the Effects of Monetary Policy?" Federal Reserve Bank of Richmond Economic Brief, No. 23-15.

This article may be photocopied or reprinted in its entirety. Please credit the authors, source, and the Federal Reserve Bank of Richmond and include the italicized statement below.

Views expressed in this article are those of the authors and not necessarily those of the Federal Reserve Bank of Richmond or the Federal Reserve System.