"This book is original and powerful. It develops a Bayesian paradigm that embraces the reality of applied modeling, in which 'discoveries' of things previously unimagined are made regularly. It will be of immediate interest to all economists and statisticians who want to push Bayesian principles toward innovative practice (and who doesn't?)."--Francis X. Diebold, University of Pennsylvania

"How do we know whether a statistical model is good enough for a particular economic research problem? To answer this question, John Geweke introduces the concept of incomplete models, showing how they can be effective tools for model building. This book is a significant contribution to econometrics--and a pleasure to read."--Richard Paap, Erasmus University Rotterdam

"This excellent book seamlessly links many important econometric methods, models, and concepts."--Gary Koop, University of Strathclyde

"This book is original and powerful. It develops a Bayesian paradigm that embraces the reality of applied modeling, in which 'discoveries' of things previously unimagined are made regularly. It will be of immediate interest to all economists and statisticians who want to push Bayesian principles toward innovative practice (and who doesn't?)."--Francis X. Diebold, University of Pennsylvania

"How do we know whether a statistical model is good enough for a particular economic research problem? To answer this question, John Geweke introduces the concept of incomplete models, showing how they can be effective tools for model building. This book is a significant contribution to econometrics--and a pleasure to read."--Richard Paap, Erasmus University Rotterdam

"This excellent book seamlessly links many important econometric methods, models, and concepts."--Gary Koop, University of Strathclyde

John Geweke is Distinguished Research Professor at the University of Technology Sydney, and research professor at the University of Colorado. He is the coeditor of the Journal of Econometrics and his most recent previous book is Contemporary Bayesian Econometrics and Statistics (Wiley).

Complete and Incomplete Econometric Models

By John Geweke

Princeton University Press

Copyright © 2010 Princeton University Press
All right reserved.
ISBN: 978-0-691-14002-5

Contents

Series Editors' Introduction.........................................viiPreface..............................................................ix1 Introduction.......................................................12 The Bayesian Paradigm..............................................73 Prior Predictive Analysis and Model Evaluation.....................344 Incomplete Structural Models.......................................865 An Incomplete Model Space..........................................122References...........................................................161

Chapter One

Introduction

Models are the venue for expressing, comparing, and evaluating alternative ways of addressing important questions in economics. Applied econometricians are called upon to engage in these exercises using data and, often, formal methods whose properties are understood in decision-making contexts. This is true of work in other sciences as well.

There is a large literature on alternative formal approaches to these tasks, including both Bayesian and non-Bayesian methods. Formal approaches tend to take models as given, and the more formal the approach the more likely this is to be true. Whether the topic is inference, estimation, hypothesis testing, or specification testing, the formal treatment begins with a specific model. The same is also true of formal approaches to the tasks of prediction and policy evaluation.

Yet the ultimate success of these endeavors depends strongly on creativity, insight, and skill in the process of model creation. As the model builder entertains new ideas, casting off those that are not deemed promising and developing further those that are, he or she is engaged in a sophisticated process of learning. This process does not, typically, involve the specification of a great many models developed to the point of departure assumed in formal treatments in graduate courses, texts in econometrics and statistics, and journal articles. Discarding models that would ultimately be unsuccessful earlier rather than later in this process of learning improves the allocation of research time and talent.

Model development is inherently a task of learning under conditions of unstructured uncertainty. To assume that one's model fully accounts for the phenomenon under question is naive. A more defensible position is that of Box (1980): all models are wrong, but some are useful. To this it might be added that, with inspiration and perspiration, models can be improved. The process of information acquisition, learning, and behavior when objectives are well-specified in a utility or loss function is familiar ground in economics. In modeling the optimal behavior of economic agents in this situation the dominant paradigm is Bayesian learning, to the point that many in the profession are comfortable terming such behavior rational.

Can this paradigm be applied to model development? A number of obstacles suggest that the task may be demanding. First, the behavior of practicing econometricians regularly appears inconsistent with the Bayesian learning paradigm. In particular, the dominant statistical paradigm in econometrics has been frequentist and the inconsistencies of this approach with Bayesian inference and learning are well-known. Second, Bayesian model specification is more demanding than most non-Bayesian model specification, requiring prior distributions for inherently unobservable constructs like parameters, as well as for models themselves when multiple models are under consideration. Finally, whereas in academic treatments of Bayesian learning reality is fully specified, in applying economics to policy questions it is not even clear that the existence of a data-generating process has any epistemological standing at all.

The thesis of this monograph is that these objections can be met, and its essays are explorations of the prospects for more effective use of the Bayesian paradigm at the point where the investigator has much less information than is presumed in formal econometric approaches, be they Bayesian or non-Bayesian. At this point models are inherently incomplete: that is, they are lacking some aspect of a joint distribution over all parameters, latent variables, and models under consideration. Chapter 2 details more fully the concept of a complete model. It also establishes notation and serves as a primer on Bayesian econometrics.

Model incompleteness can take many forms, and the essays in this monograph take up three examples. Chapter 3 addresses the early steps of model construction—before the investigator has engaged the technical demands of formal inference or estimation and perhaps even before data have been collected. The emphasis in this chapter is on using formal Bayesian methods to compare and evaluate models. Model comparison at this stage amounts to prior predictive analysis, which was introduced to statistics by Box (1980) and emphasized in econometrics by Lancaster (2004) and Geweke (2005). These ideas are not new, but their potential for greatly accelerating effective research is not as yet well appreciated in the econometrics community. Model evaluation is the assessment of a specified model by absolute standards—a process in which economists regularly engage. The assertion, or conclusion, that a model is bad for a particular purpose is repeatedly heard in the economist's workday. But, as economists regularly point out, statements like this raise the question, bad compared with what? The final section of chapter 3 sets up an incomplete model as the basis of comparison implicit in such statements, and then extends the conventional apparatus of Bayesian model comparison to the complete model being evaluated and the incomplete model that is implicitly held as the standard. This treatment provides a fully Bayesian counterpoint to frequentist tests against an unspecified alternative, also known as pure significance testing.

No model is meant to specify all aspects of reality, even a sharply confined reality chosen for its relevance to a particular academic or policy question. This restriction can be straightforward for formal econometrics: for example, the stipulation that a regression model applies only over a specified range of values of the covariates, or that a structural model with several endogenous variables is intended only to provide the marginal distribution of a subset of these variables. But often the restriction is stronger. Chapter 4 takes up the case of structural models that are intended only to provide certain population moments as functions of the structural parameters of the model, a restriction that is especially common in dynamic stochastic general equilibrium models. The chapter shows that widely applied procedures, including conventional calibration, violate this restriction by taking the higher-order moments of the model literally in reaching conclusions about its structural parameters. It provides a constructive approach to this learning problem by treating explicitly the incompleteness of the structural model and then completing the model in a way that relies only on those aspects of the structural model intended to be taken literally. In the example used throughout the chapter this approach reverses widely held conclusions about the incompatibility of the U.S. equity premium with simple growth models.

Formal Bayesian methods provide a logically consistent and well-understood solution to the problem of using competing models with conflicting implications in a decision-making context. The critical element of this solution is the specification of prior model probabilities that sum to one. In so doing, the solution conditions on the process that actually generated the data being one of the models under consideration. Non-Bayesian methods that lead to rules for model choice also make the latter assumption. A widely observed characteristic of the formal Bayesian approach is that it often assigns posterior probability very close to unity for one of the models. The Bayesian solution then effectively amounts to model choice. This is not a problem for econometric theory, because in general the data-generating process is the one selected asymptotically. On the other hand, there is an evident conflict with reality: in reaching important decisions policymakers routinely wrestle with alternative models, leading to an apparent inconsistency of clear evidence with presumed rationality of the decision makers. The final chapter in this monograph steps back from the key assumption that reality lies somewhere in the space of models being considered. Replacing the assumption that the model space is completely specified with the alternative of a linear combination of predictive densities of future events that renders past events most probable, it shows that if the data-generating process is not among the group of models considered, then one will use several models. The weights given to these models will converge to positive limits asymptotically. The weights assigned in Bayesian model averaging are incorrect, under this alternative specification in which the model space is incomplete.

Chapter Two

The Bayesian Paradigm

The Bayesian paradigm provides a powerful and practical structure for managing the risk inherent in decision making. This chapter discusses the elements of this structure, which is standard in the subjective Bayesian approach to inference and decision making. A number of recent texts provide more detailed consideration of this approach in econometrics, including Poirier (1995), Koop (2003), Lancaster (2004), Geweke (2005), Rossi et al. (2005), and Greenberg (2007). This chapter also reviews the Bayesian literature on model evaluation: the effort to assess whether the structure under consideration corresponds to reality. Model evaluation is an inherently difficult question from a Bayesian perspective, and the rest of this monograph explores some ways this can be done using incomplete models.

The essential element of the Bayesian paradigm is a complete model, detailed in section 2.1. A complete model provides a coherent joint probability distribution for the evidence relevant to the decision, unknown parameters or latent variables in the model, and additional factors that will determine the consequences of the decision but are unknown at the time the decision is made. With this distribution in hand, and a utility function for preferences over all possible consequences of the decision, the decision maker can, in principle, determine the decision that maximizes expected utility conditional on the observed evidence. A complete model, therefore, amounts to the explicit econometric implementation of the classical von Neumann–Morgenstern normative theory of decision making under uncertainty. Advances in computation, and in particular in simulation methods, since 1990 have made this implementation practical using models much more realistic than those that can be used in purely analytical approaches. Section 2.3 briefly reviews the highlights of these advances.

Especially for an important problem in a complex environment, decision makers have before them alternative models. Since the models are all relevant to the decision, each specifies the distribution of the factors determining the consequences of the decision, conditional on alternative decisions that might be made. If all of the models are Bayesian with proper prior distributions, then they are all complete. By specifying prior (that is, unconditional) probabilities on the alternative models, decision makers can extend the coherent probability distributions of the individual models to all of the models under consideration, as described in section 2.2. Since the distribution over observables and factors determining the consequences of alternative decisions is coherent and complete, the familiar expected utility calculus applies. The mechanics of this process impose greater technical demands than do those of a single model. Section 2.2 reviews the substantial progress on this problem that has been made in the past decade.

The specification of the set of Bayesian models under consideration is typically recursive. It begins with a prior probability for each model. Then, within each model, the prior probability distribution of unobservables (parameters and latent variables) is followed by the distribution of observables conditional on unobservables. Finally, each model provides the distribution of factors relevant to the decision conditional on unobservables and observables. There are minor variations on this theme: in particular, the distribution of latent variables and observables conditional on parameters may be more convenient. Regardless, model specification is almost always a forward recursion from models to unobservables to observables to decision-relevant factors. Simulation from this distribution is straightforward, as outlined in section 2.3, and the convenience of this simulation emerges as an important factor in chapters 3 and 4. For a decision maker, however, the relevant distribution is conditional on the data—the observables that are known when the decision is made. This posterior distribution is neither forward nor recursive, and the corresponding simulation is not straightforward. Since the late 1980s extraordinary progress has been made in solving these simulation problems, and these solutions in turn are essential to the ongoing rapid growth in the practical application of complete models to important decisions. Section 2.3 reviews these methods very briefly. They are used only in chapter 5, which requires no detailed understanding of posterior distribution simulators. The texts mentioned at the start of this chapter all provide treatments of that topic in greater depth.

The strength of the Bayesian paradigm is its specification of a complete and coherent structure answering the question: conditional on the models specified, the available data, and a given utility function, what is the appropriate decision? Alternative utility functions provide no essential complication: the paradigm provides the answer for any utility function for which expected utility is known to exist. It can also easily address the sensitivity of the decision to alternative prior distributions of parameters. These are all critical advantages relative to non-Bayesian methods: some additional attractions of Bayesian methods include accounting for parameter uncertainty, avoiding the need to choose a single model from many, and the ability to discover the extent to which disagreements among decision makers are due to different beliefs or different utility functions. Yet, the answer that emerges can be no better than the models under consideration.

The same caveat applies to non-Bayesian methods. Model evaluation, in the form of hypothesis testing, has been central to non-Bayesian statistics since its inception in the late nineteenth century. Within the Bayesian paradigm it is possible to lose sight of the fact that the precise and relevant answers it provides are contingent on the adequacy of its complete models. There is a lively Bayesian literature on the use of predictive distributions to evaluate these models. Section 2.4 reviews these approaches, as do Lancaster (2004) and Geweke (2005), and they provide foundations for the rest of this monograph. Whether or not purely Bayesian methods can be used to evaluate models has long been an open question in this literature. Chapter 3, building on section 2.4, shows that pure Bayesian model evaluation is both possible and practical using incomplete models.

2.1 Complete Models

Denote a complete model by A, "assumptions." A complete model has four elements:

(1) A j_T x 1 observable random vector y_T , where T denotes the sample size. The notation j_T allows flexibility in the specification of y. For example, if an m x 1 vector is observable at each of T sample points, then one can write y'_T = ( [??]'₁, ..., [??]'_T) with j_T = T_m. Some data may be missing, in which case j_T ≤ T_m and j_T may be random.

(2) A k_A,T x1 unobservable random vector θ_A,T [member of] Θ_A,T. The vector may consist of parameters, latent variables, or both. The notation Θ_A,T reflects the fact that this vector is specific to the model. The dimension k_A,T may also depend on T, as is typically the case with latent variables. The number of parameters may also depend on sample size, as is the case with flexible highly parameterized models and with explicitly nonparametric Bayesian models. Missing data can be included in Θ_A,T.

(Continues...)

Excerpted from Complete and Incomplete Econometric Modelsby John Geweke Copyright © 2010 by Princeton University Press. Excerpted by permission of Princeton University Press. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.