Homepage
Narrative Description of Research on Stochastic Game Theory
Academic Vita (with links to *.pdf files for all publications)
Data and Instructions (appendices for laboratory experiments)
Y2K Bibliography (2000 listings in experimental economics)
Survey Papers on Experimental Economics
(by
topic)
Relaxing the Assumptions of Perfect Rationality
I am particularly interested in decision making
in interactive "games" when people are not assumed to be perfectly
rational. This research is based on mathematical models of strategic
behavior in which the best decision depends on what one thinks other(s)
will do, e.g. what they will bid in an auction. These beliefs can
be determined by learning, or in the absence of prior experience, by introspection.
Beliefs determine the payoffs anticipated for each decision, and payoffs
determine the choice probabilities. Even small amounts of "noise"
in decision making can have a large "snowball" effect in an interactive
game. In some contexts, these models of bounded rationality predict
that behavior will be essentially the reverse of what it would be in the
case with perfect rationality. Some of these papers are listed on
my Vita
and pertain to coordination,
social
dilemmas,
rent
seeking,
public
goods,
games
played only once, etc.
Laboratory Experiments in Economics
The new theories are tested in laboratory
experiments by letting students make decisions in markets and games in
which their earnings depend on the decisions made by themselves and others.
For example, the profitability of a particular price may depend on price
itself and on the prices charged by rival sellers. The observed choice
data usually reject the hypothesis of perfect rationality that is typically
used in economic theory, and the models of bounded rationality are used
to explain and predict anomalies. For a summary of this "stochastic
game theory" that is written for a broad scientific audience, see: "Stochastic
Game Theory: for Playing Games, Not Just for Doing Theory" (with Jacob
Goeree), in the September 1999 Proceedings of the National Academy of
Sciences.
Experimental Economics
The Y2K Bibliography
of Experimental Economics and Social Science lists about 2000 publications
and over 500 working papers, with selected abstracts and keywords that
have been used to construct special reference lists. Related work
on experimental economics can be found in the new Kluwer Academic Press
Journal, Experimental
Economics that I co-edit. For a general introduction
to experimental economics, see the 1993 Princeton University Press monograph
Experimental
Economics [Contents]
(with Doug Davis). The cover for this book is shown in the box at
the upper right part of this page. This research is also used
to devise interactive classroom games for use in teaching economics at
all levels (click on Classroom
Games).
Narrative Description of Recent Research:
Stochastic Game Theory and Human Behavior
National Science
Foundation Grants
SES 0094800
(PI, with Jacob Goeree and 8 co-PIs at 8 universities)
SBR 9818683
(PIs: Jacob Goeree and Charles Holt)
SBR 9617784
(PIs: Simon Anderson, Jacob Goeree, and Charles Holt)
Contents:
I. Game Theory and Laboratory Experiments
II. Three Complementary Approaches
Introspection and Games
Played Once
Equilibrium with Bounded
Rationality
Learning and Evolution
III. Applications of Stochastic Game Theory
Rent Seeking
Coordination
Binary Choice Games
IV. Incorporating a Broader Range of Human Motivation
Public Goods and Altruism
Bargaining and Fairness
Risk Aversion in Lottery
Choice, Games, and Auctions
V. Summary of Major Scientific Insights
Noisy Introspection:
A Model of Disequilibrium Behavior for One-Shot Games
Quantal Response Equilibrium
Analysis for Games with Continuous Strategies
Stochastic Equilibrium
Models with Altruism, Inequity Aversion, and Risk Aversion
Stochastic Learning
Equilibrium: Long Run Consequences of Learning
Stochastic Potential
VI. Abstracts of Research Papers and Publications
I. Game Theory and Laboratory Experiments: Historical Perspective
Economics in the early twentieth century pertained almost exclusively to "price theory," which assumes that traders respond passively to price signals without thinking strategically about rivals' actions and reactions. This theory can be effectively applied to `thick' markets with lots of traders and good information, but John von Neumann and Oscar Morgenstern realized that many economic interactions like bilateral bargaining or bidding in auctions are not adequately modeled by price theory. Although their Theory of Games and Economic Behavior was reviewed on the front page of the New York Times and was widely heralded in the profession, John Nash, a young mathematics graduate student at Princeton, noticed a flaw. The only games that were "solved" were zero-sum games, i.e. those with the special property that one person's gain exactly equals another's loss. Nash approached von Neumann, who was then Chair of the Princeton Mathematics Department, with a more general equilibrium analysis, which von Neumann dismissed as merely a mathematical "fixed point" argument.
The notion of equilibrium is analogous to a state of rest in a physical system. An equilibrium in an interactive social system is, roughly speaking, a state in which people choose strategies of action, and there is no tendency for the system to change if each person does not want to change their own strategy, given what they know about the strategies chosen by the others. Nash provided a formal definition of equilibrium and proved that such an equilibrium would in fact exist in a wide class of strategic games. Nash's thesis advisor, Arnold Tucker, sent the proof to the Proceedings of the National Academy of Sciences about fifty years ago, resulting in a half-page Nobel-Prize winning paper.
On the other coast, a group of mathematicians and economists at the RAND Corporation heard about Nash's proof and were immediately interested, since the von Neumann/Morgenstern zero-sum paradigm did not fit well with the possibility of mutual destruction from a nuclear exchange that loomed so large in the 1950's. The same day that word of Nash's proof arrived in Santa Monica, two mathematicians ran a laboratory experiment designed to determine whether behavior of financially motivated people would conform to the predicted behavior (it didn't). Tucker happened to pass by and see the payoff numbers written on the blackboard, and he used them to invent the story of the "prisoner's dilemma" for a talk on recent developments in game theory at the Stanford Psychology Department.
The prisoner's dilemma is a game in which the best decision one can make (in private) does not depend on what the other person does, i.e. the prosecutor makes promises and threats to two prisoners in such a manner that each is better off confessing to a crime regardless of whether the other confesses, even though both would be better off if neither confessed. The Nash equilibrium is for both to confess, which produces an (admittedly unhappy) state of rest.
The prisoner's dilemma has become a paradigm for many aspects of strategic interaction, although there are others that will be discussed below. In the meantime, game theory itself has finally assumed the central role in economics first envisioned by von Neumann and Morgenstern, although ironically, the Nash equilibrium has replaced their earlier solutions as the central element in the theory. Game theory is being increasingly applied in law, management, psychology, political science, and to some extent in systems engineering, artificial intelligence, biology, sociology and anthropology. In my opinion, game theory is the closest thing there is to a unified theory of social science.
Game theory is especially
useful in situations where well developed markets and prices are not available,
e.g. for the analysis of coordination within a firm, family, tribe, or
government agency. It is also essential in contexts where people are trying
to anticipate others' decisions, as with firms in concentrated industries,
or governments considering strategic international trade issues. To a large
extent, the focus of economic analysis is on the equilibria of complex
interactions, and the requirements for formal models of strategic behavior
limit the behavior that can be accommodated. Although the Nash equilibrium
can apply to a wide range of social situations, for example, it still requires
very restrictive rationality assumptions. An alternative approach that
emerges from computer science and artificial intelligence is to write complex
computer simulations that program adaptive responses into simulated human
players. A second alternative is to rely more on non-mathematical insights
about human behavior gleaned from the social psychology literature. Our
work is influenced by these approaches; we seek to broaden the behavioral
possibilities that can be considered, while preserving the possibility
of using formal models to produce precise predictions that increase the
power and usefulness of game theory. The next section describes three closely
related perspectives of this new stochastic game theory: introspection
for games played once, learning for repeated games, and equilibrium for
long-run steady states.
II. Three Complementary Approaches
Introspection and One-Shot Games
The increasing adoption
of game-theoretic models in law, business, and social sciences, is problematical
in light of its failures to explain human decisions in many laboratory
settings. "Ten Little Treasures of Game Theory and Ten Intuitive Contradictions"
(Goeree and Holt, American Economic Review, December 2001) is intended
to serve as a wake-up call for those who are not familiar with earlier
experiments by economists and psychologists. Each of the games considered
has one set of parameters (the "treasure treatment") for which the Nash
equilibrium predicts very well, but a change in a payoff parameter that
does not alter the Nash equilibrium causes a large shift in observed behavior
in a parallel "contradiction treatment." Moreover, these contradictions
of theory are intuitive. For a popular-press account of this paper,
see Hal Varian's article: "What, Exactly, Was on John Nash's Beautiful
Mind?" New York Times, April 11, 2002).
For example, consider a "matching pennies" game in which a soccer player can either kick to the East side or the West side, and the goalie can either dive East or West. Suppose that a "match" (the goalie dives to meet the kick) gives payoffs of 1 to the goalie and -1 to the kicker, whereas a mismatch gives -1 to the goalie and +1 to the kicker. Each player wants to be unpredictable, and the only Nash equilibrium for this symmetric game is for each player to choose each direction with probability 1/2 (otherwise at least one player would want to change their behavior). This prediction is confirmed in laboratory matching pennies games. Now consider what happens if the fans of the goalie's team, who are seated on the East side of the field, announce publicly that they will donate ten thousand dollars to the team scholarship fund if the goalie blocks a kick on the East side. This only increases the goalie's payoff for the (dive-East, kick-East) outcome, but it cannot increase the Nash equilibrium probability that the goalie dives East, since the kicker must remain indifferent over the two kick directions in a Nash equilibrium with randomized decisions. This unintuitive prediction, that a change in a player's own payoff for one of the outcomes will not change that player's probability of choosing that outcome, is dramatically rejected in the experiments based on asymmetric matching pennies games. Thus the Nash equilibrium works well in the symmetric game and it predicts poorly in the asymmetric matching pennies game.
We show that intuitive contradictions like this are explained by a simple model of "noisy introspection" (about what the other person might do, of what the other person thinks I might do, of what they think I think, etc.). It is well known that a succession of noise-free iterated best responses may not converge, or if it does, it will converge to a Nash equilibrium, which cannot explain behavior in the "contradiction" treatments. The innovation in our model is the injection of increasing amounts of noise into the iterated thought process: one is assumed to be less uncertain about the other person's decisions than about the other's beliefs about my decisions, etc. In "A Model of Noisy Introspection," (Goeree and Holt, forthcoming in Games and Economic Behavior) we show that the introspective equilibrium always exists and that it provides a good explanation of deviations from standard equilibrium predictions in a series of 37 one-shot game experiments. This model relaxes two extreme rationality assumptions of a Nash equilibrium: rational choice (no mistakes in choosing the action with the highest expected payoff) and rational expectations (belief distributions about others' actions correspond to actual distributions of decisions). This noisy introspection model is useful for the analysis of games played only once, which is a good description of many interactions in politics, war, legal disputes, and auctions.
Logit Equilibrium
Expectations will only be
accurate by coincidence for a game played once, but with repetition, players
may come to learn what to expect, and beliefs will more closely approximate
the distributions of actual decisions. We use learning models to explain
patterns of adjustment in some of the experimental papers, but when behavior
stabilizes and expectations become more accurate, the use of an equilibrium
model is appropriate. The logit equilibrium imposes the rational
expectations assumption, but relaxes the rational choice assumption by
letting the probability of making the best decision be is an increasing
function of the difference in expected payoffs, just as the probability
of judging correctly which light is brightest is positively related to
the true difference in brightness. This is essentially the notion of "probabilistic
choice" (due to the psychologist, Duncan Luce, and others) injected into
the equilibrium analysis of a game. (The "logit" equilibrium is obtained
when the popular logit probabilistic choice rule is used, which causes
choice probabilities to be proportional to exponential functions of expected
payoffs.) In order to relate these concepts, think of a Nash equilibrium
as a kind of economists' heaven (depending on your perspective) where everyone
is perfectly rational and knows that the others are too. Then the logit
equilibrium is like a situation where people make mistakes but have learned
to anticipate other's mistakes, and the noisy introspection model applies
to a "one-shot" encounter before this learning has taken place.
The effects of bounded rationality are illustrated dramatically in "Anomalous Behavior in a Traveler's Dilemma?" (Capra, Goeree, Gomez, and Holt, American Economic Review, 1999). The motivating story is that two travelers lose bags with identical value, and the airline representative asks them to fill out claims independently, with the constraint that claims must be in some pre-specified range, e.g. between $80 and $200. The travelers will be reimbursed if the claims are equal, but if claims are unequal the assumption is that the high claimant overstated the value, so both are reimbursed at the minimum of the two claims. Moreover, the low claimant receives a reward of $R and the high claimant's reimbursement is reduced by a corresponding amount. For example, if the claims are 99 and 100, then the minimum claim is 99, so the first person receives 99 + R and the second earns 99 - R. Each person would have an incentive to "undercut" the other's claim if it were known, and the unique Nash equilibrium (with rational choice and rational expectations) is for both to claim the lowest possible amount ($80 in the example). Notice that this argument does not depend on the size of the penalty/reward rate R. i.e. whether it is a matter or pennies or $100.
The traveler's dilemma is no more about travel claims than the prisoner's dilemma is about confessions. In each case, the goal is to devise a stylized situation or paradigm that captures the key elements of the strategic landscape for a class of applications. This means preserving the essential economic structure of the game and stripping away or holding constant the rich framing and contextual elements of any particular choice problem, so that the effects of changes in economic incentives can be evaluated. As noted above, the remarkable feature of the traveler's dilemma game is that the equilibrium prediction is independent of R, the penalty reward parameter. But if R is low (e.g. pennies), there is little risk in raising the travel claim, and if R is large (e.g. $50), there is considerable risk, so intuition suggests that claims will be higher with a low penalty parameter. This intuition is confirmed in the laboratory experiments: the Nash equilibrium predicts well for high penalty parameters, but the data cluster at the opposite end of the set of feasible claims when R is low. The logit equilibrium model explains data in both cases, at least after the data settle down in a steady state.
Learning and Evolution
If you look at the mathematical
expressions in the traveler's dilemma paper, you will probably be skeptical
about whether such abstract formulas can actually predict the behavior
of human subjects who typically rely on intuition instead of pencil-and-paper
calculations. People in the experiments adapt and learn, if one sees high
claims in the first rounds of the experiment, it is reasonable to expect
high claims in subsequent rounds. We use models of "belief" learning with
noisy decisions to explain the pattern of adjustments to equilibrium. In
"Stochastic Game Theory: For Playing Games, Not Just for Doing Theory,"
(Holt an Goeree, Proceedings of the National Academy of Sciences,
1999), we use computer simulations of randomly matched players to predict
both the directions and the steady state levels of the adjustment paths
observed in experiments. If the simulated learning process has enough memory
to learn the exact probability distribution of claims (e.g. "fictitious
play"), then simulations show that behavior converges to the logit equilibrium.
A more refined analysis of learning is presented in "An Experimental Study
of Imperfect Price Competition," (Capra, Goeree, Gomez, and Holt, International
Economic Review, 2002) where we show that the steady state of the learning
process is the logit equilibrium under fictitious play, but that small
differences persist when beliefs are more responsive to recent experience.
This paper introduces the idea of a stochastic learning equilibrium,
which is close to the logit equilibrium for the applications we have considered,
but which can differ from the logit equilibrium when learning is subject
to "recency effects".
One reaction that we sometimes encounter is that noise is transitory, just as random wave action may not have much effect on the predictable patterns of ocean tides. This reaction is understandable, since one might expect a small amount of decision error would only add "noise" around the predictions of a theory with no errors, e.g. the predicted Nash equilibrium claim of 80 in the traveler's dilemma example. In an interactive game, however, there can be synergies that make the effects of noise much more analogous to the momentum that accumulates in atmospheric weather patterns. For example, if noise causes an upward drift in one person's decisions, then this changes the landscape of others' expected payoffs, causing them to drift upward, and the resulting "herd effect" may move the data to a noisy equilibrium that is nowhere near the Nash equilibrium prediction. In the process, the boundedly rational players are not solving any equations, they are merely reacting to the positive and negative results of noisy local changes. We use an interactive version of the Fokker-Planck equation from theoretical physics to model a naive "hill-climbing" adjustment process (with Brownian motion) that is shown to have the logit equilibrium as a steady state. This work is reported in "Stochastic Game Theory: Adjustment to Equilibrium Under Noisy Directional Learning" (Anderson, Goeree, and Holt).
The recent history of economic
methodology supports the Lakatosian notion that a theory will not be abandoned
unless a better alternative is available. In fact, laboratory violations
of Nash predictions have had little effect on applied game theory models
in the absence of a widely accepted alternative. If anything, game theory
has become more enshrined in its mathematical armor, with new specialty
journals, recent Nobel prizes, and an international Game Theory Society
holding its first World Congress in July 2000. The isolation of game theory
is reflected in a remark by Nobel Laurate Reinhard Selten: "Game theory
is for proving theorems, not for playing games." One problem has been that
strategic interactions occur in uncontrolled situations that are subject
to various alternative interpretations. Controlled experiments have been
relatively new in mainstream economics, and until recently experimental
economics has not been taught at the top-10 U.S.graduate programs. Consequently,
game theory has undergone considerable refinement with only indirect exposure
to behavioral evidence, and indeed the adjectives that precede some theoretical
concepts sound more theological than scientific, e.g. "subgame perfectness,"
"the divinity refinement," or "purification of equilibria." Sometimes theoretical
intuition even seems to substitute for data; as Richard Thaler once quipped
at a psychology conference, "When economists say that the evidence is mixed,
they often mean that the theory says one thing and the data say something
else." The goal of the traveller's dilemma paper and the others in the
bounded rationality series is to redirect the focus to explaining intuitive
behavioral anomalies with formal models that relax the perfect rationality
assumptions. The next section describes a number of applications where
intuitive deviations from Nash predictions are explained by the incorporation
of noise into interactive models.
III. Applications of Stochastic Game Theory
Rent Seeking
Economists are typically
thought of as being apologists for the economic system, by providing a
theology that explains how economic interactions produce efficient outcomes
via an "invisible hand." Economics experiments largely support Adam Smith's
remarkable insights about market efficiency, but economists have been quick
to spot inefficiencies arising from imperfections and non-market interactions.
Perhaps the most perverse example occurs when an attempt to give away an
indivisible "good" results in a net loss to the group of potential recipients
as a whole. An example was provided to me in a recent letter from a colleague:
"...say that a franchise was going to be awarded to a company by a State government. Once the franchise is given out, the company getting it would be a monopolist. Obviously, many companies would like to be awarded such a contract and would spend resources on lawyers, lobbyists, and politicians trying to get it. When the `competition' is over, however, there is only one winner who gets the franchise and earns the `rents' available....For the losers, all of the resources they expended are lost....The conventional theory for such auctions exhibits several features. For example, in equilibrium, all rent is dissipated by the expenditures of firms trying to get the franchise.... not exactly what we observe in reality. In many cases it may be possible for firms to over-dissipate the rents by collectively spending more to receive a franchise than the franchise is worth to any one of them (a clear social inefficiency)."
This rent seeking process is like an "all-pay auction" in which the prize goes to the highest bidder, but all have to pay their bids, since bidding in done by spending money on lobbying and franchise application procedures. A Nash equilibrium for these rent-seeking games can produce under-dissipation or exact dissipation of rents, but never over-dissipation, since this would yield losses that could be avoided by not competing for the franchise. In contrast, over-dissipation is possible in models that relax the assumption of perfect rationality, as described in "Rent Seeking with Bounded Rationality: An Analysis of the All Pay Auction" (Anderson, Goeree, and Holt, 1998, Journal of Political Economy). Even though rent is always exactly dissipated in a Nash equilibrium for the all-pay auction, the logit equilibrium predicts that rent dissipation is greater when there are more competitors for the rent. This paper provides an explanation for the qualitative patterns of over-dissipation observed in some laboratory experiments.
Coordination
Despite the well-known efficiency
properties of competitive markets, economists have long been concerned
that there may be multiple equilibria in complex social systems. The possibility
of becoming mired in a bad equilibrium dates back at least to Thomas Malthus'
notion of a "general glut." The problem is one of coordination: e.g. it
may not make sense for me to go to the market to trade if you are going
to stay home on the farm. Our papers are based on a model of a coordination
game in which workers choose effort levels, and the group product is the
minimum of the efforts. Thus the output is determined by the weakest link
in the production chain. This is analogous to Rousseau's "stag hunt" where
the chances of a successful encirclement of the prey depend on the weakest
link around the circle of hunters. In the economic application, each worker's
payoff is the minimum of the efforts minus the cost of their own effort.
The game is specified so that any common effort level is a Nash
equilibrium, regardless of the number of people or the cost of effort,
since a unilateral increase in effort is costly and does not affect the
minimum, and a unilateral decrease reduces the minimum by more than the
personal cost saving. Thus the Nash equilibrium makes no prediction about
average effort, despite the intuitive argument that successful coordination
should be harder with a higher effort cost or with more people (with a
large group, it is more likely that somebody will ruin the group output
with a low effort).
This intuition, that efforts will be higher with low effort cost, is confirmed by our laboratory experiments, and again, the logit equilibrium predictions track the data nicely. Efforts are intermediate and tend to rise in the low-cost treatment, and they start at about the same initial level and tend to fall in the high-cost treatment. In each case, the effort level trajectories tend to flatten after several periods of random rematching of players, and then average levels are quite close to the predictions of the logit equilibrium ("An Experimental Study of Costly Coordination," Goeree and Holt, 1999).
The stag hunt game is another of those game-theory paradigms, which in this case has been used as a proving ground for theories designed to choose among multiple Nash equilibria. Our approach predicts a unique probability distribution over decisions, and the derivation of this distribution is analogous to the minimization of potential energy in a physical system. In game theory, a potential function is a mathematical formula that is positively related to individual players' payoffs: when a change in a player's own decision raises that player's payoff, then this change necessarily raises the value of the potential function by the same amount, and vice versa for decreases. If such a potential function exists for the game, then each person trying to increase their own payoff may produce a group result that maximizes the potential function for the game as a whole. Think of two people holding adjacent sides of a treasure box, with one pulling uphill along the East-West direction and the other pulling uphill along the North-South axis. Even though each person is only pulling in one direction, the net effect will be to take the box to the top of the hill, where there is no tendency to change (a Nash equilibrium that maximizes potential). When there is some randomness in the individual directional movements, the dynamic system maximizes a "stochastic potential," which is the expected value of ordinary potential function of the game plus a measure of dispersion (entropy). We show that the maximum of stochastic potential is a logit equilibrium, which produces a unique distribution of effort levels in the coordination game. As effort costs rise, the distribution of efforts decreases, and vice versa, a result that is intuitive but not predicted by a Nash equilibrium. The intuition is that the randomness makes behavior sensitive not just to whether the incline is increasing or decreasing, but to how steep the incline actually is. This research is reported in "Minimum Effort Coordination Games: Stochastic Potential and Logit Equilibrium" (Anderson, Goeree, and Holt, Games and Economic Behavior, 2001).
Binary Choice Games
Many games of interest to
economists and political scientists involve many players who each make
a binary choice, e.g. whether to vote, enter a market, go to a potentially
crowded restaurant, volunteer to make a dangerous rescue, or contribute
to a public good. Previous experiments have reported seemingly anomalous
results, e.g. over-entry relative to Nash predictions in one study and
under-entry in another. Another example is the "volunteer's dilemma," in
which only one volunteer is needed to provide a public good, but each person
would prefer that somebody else incur the cost of volunteering. The Nash
equilibrium exhibits an unintuitively strong level of free riding: the
probability of getting at least one volunteer is predicted to go to zero
as the number of potential volunteers becomes large. This prediction is
contradicted by experiments, which show that an increase in the number
of potential volunteers increases the probability of getting at least one.
These and other anomalies are consistent with the qualitative predictions
of a quantal response equilibrium. For example, the same structural conditions
that tend to cause over-entry in some market games are also associated
with over-participation in some voting game experiments, under both majority
and proportional rule procedures. A simple graphical device for predicting
behavior in binary choice games is provided in "An Explanation of Anomalous
Behavior in Binary-Choice Games: Entry, Voting, Public Goods, and the Volunteer's
Dilemma" (Goeree and Holt, 2000).
IV. Incorporating a Broader Range of Human Motivation
Public Goods and Altruism
Economists' assumptions
about selfish preferences generally echo Adam Smith's famous remark that
"...it is not for the benevolence of the butcher or the baker...." A number
of researchers, however, have explored the implications of other-regarding
behavior (altruism, inequity aversion, etc.) in the context of public goods
and bargaining models. The idea behind a public goods game can be illustrated
by Adam Smith's street light example. The benefits of putting a street
light in front of a home may not justify the cost to one person. But if
a large number of people who pass by will benefit from the light, then
the social benefits exceed the cost, and the light should be provided even
though the private benefits to the homeowner do not exceed the cost. The
temptation in this situation is for each person to "free ride" and rely
on others' lights, which is a rationale for public provision of this service
on streets with sufficiently high traffic. However, we do observe many
voluntary contributions to public goods and charities, and this raises
the issue of how much people are motivated by concerns for others' well
being. In "A Theoretical Model of Altruism and Decision Error in Public
Goods Games" (Anderson, Goeree, and Holt, 1998, Journal of Public Economics),
we show that altruism can be incorporated into a logit equilibrium analysis.
The resulting hybrid model produces intuitive predictions, e.g. that contributions
will go up if the marginal value of the public good increases, even though
it is a Nash equilibrium for (selfish) players to free ride. With altruism,
an increase in the number of people who enjoy the public good will increase
contributions, which is consistent with some observed "group size" effects.
Following the initial experiments of sociologists Marwell and Ames, there is a large literature on the causes of voluntary giving in controlled environments. Our experiment uses a design in which the benefits of one's own contribution to others (the "external return") is varied independently from the benefit to oneself (the "internal return"). This distinction allows a more precise estimation of the importance of altruistic concerns with others' payoffs. Some altruism is indicated by the fact that observed contributions to the public good increase as we raise the external benefit to others of one's own contribution, as reported in "Private Costs and Public Benefits: Unraveling the Effects of Altruism and Noisy Behavior" (Goeree, Holt, and Laury, Journal of Public Economics, 2002). Contributing to a public good is analogous to investing in pollution abatement that will benefit others downwind or downstream. Asymmetries in own (internal) and others' (external) benefits are evaluated in "Incentives in Public Goods Experiments: Implications for the Environment" (Goeree, Holt, and Laury, forthcoming 2003).
Bargaining and Fairness
Another context where non-selfish
preferences are critical is in bilateral bargaining. In fact, Nash conducted
his own bargaining experiments, and they largely failed due to fairness
considerations. Our bargaining experiment is designed with asymmetric fixed
payments that work in precisely the opposite direction from strategic considerations
if participants are concerned with equity. The data from the experiments
allowed us to estimate a logit equilibrium model in which players have
a disutility from earning less than the other and a somewhat weaker disutility
from earning more than the other. The parameters representing decision
error, unfavorable inequality aversion ("envy"), and favorable inequality
aversion ("guilt") were all statistically significant, and the resulting
model tracked the observed pattern of proposals and responses. One implication
is that models of decision error are complementary with the efforts of
others to model preferences that are not perfectly selfish. This research
is reported in "Asymmetric Inequality Aversion and Noisy Behavior in Alternating-Offer
Bargaining Games," (Goeree and Holt, European Economic Review, 2000).
Risk Aversion in Lottery Choice, Games, and Lottery Choice
The recent working paper,
"Quantal Response Equilibrium and Overbidding in Private-Values Auctions,"
(Goeree, Holt, and Palfrey, Journal of Economic Theory, 2002) applies
theoretical advances to the study of a richer market-like setting, where
bidders for a prize have randomly differing prize values. In an antique
auction, for example, each bidder may know what they are willing to pay
for a particular item, but the others' values depend on their own tastes
and needs, which may not be observed. In the experiment, the throw of a
die was used to determine whether the value of winning a prize was $0,
$2, $4, ... etc, so each person saw their own value but not the others'
values. The high bidder would earn the difference between their own prize
value and their bid. The treatments vary asymmetries in the risks of raising
or lowering bids in a manner that leaves the Nash equilibrium unchanged
but alters the logit predictions in an intuitive manner: bids should increase
when there is less "upside risk" from losing the auction by raising one's
bid. In other words, asymmetries in payoff function "flatness" should have
effects not predicted by standard theories of perfectly rational bidders
who always choose the decision with the highest expected payoff, regardless
of asymmetries in the risks of slight deviations. Think of it this way:
the height of a hill represents expected profits, and a rational person
is assumed to go to the highest point, even if it is on the edge of a sharp
drop.
The most strident experimental controversy in the literature to date involved an exchange several years ago about risk aversion and flat payoff functions, and this paper revisits the issue by putting flatness on only one side of the payoff maximum. Consistent with economic intuition, bids are higher in the treatment with low upside risk, and lower in the treatment with high upside risk, despite the fact that the Nash equilibrium bids are the same for the two treatments. One striking feature of the logit analysis of auctions is that the predictions are so close to the aggregate bid distributions that we had to use thin lines in graphs to distinguish them. An interesting aspect of the data analysis is that there was overbidding by an amount that is rather precisely predicted by an assumption that bidders are risk averse.
The effects of risk aversion show up clearly in lottery-choice settings, where the subject chooses between pairs of lotteries with different degrees of riskiness. This risk aversion is present in low-payoff settings (prizes of several dollars), and it increases sharply when all prizes are increased by a factor of 20, and it increases again when payoffs are scaled up by a factor of 50, yielding prizes in the hundred dollar range. Moreover, risk aversion in high payoff situations is much stronger when the payoffs are "real" as opposed to being hypothetical. These results are reported in "Risk Aversion and Incentive Effects in Lottery Choices" (C. Holt and S. Laury, American Economic Review, December 2002).
Many games involve situations
where the potential payoffs from playing one strategy are much less variable
than the potential payoffs from another strategy. In "Risk Averse
Behavior in Generalized Matching Pennies Games" (Goeree, Holt, and Palfrey,
forthcoming in Games and Economic Behavior), we consider a game
where all quantal response equilibria (for risk neutrality and any error
rate) are on one side of the set of choice probabilities for the two players,
but the data are clearly on the other side. We prove that risk aversion
shifts the stochastic best response functions toward higher probabilities
of using the "safe" strategy. The resulting quantal response equilibrium
with risk aversion is close to the observed data average. Maximum
likelihood estimates of the coefficient of relative risk aversion are relatively
stable across these different games.
V. A Summary of Major Scientific Insights
Game theory is the closest thing to a unifying theory in social science, and it evokes some of the strongest antagonism as well. Critics argue that people are not perfectly rational, and that the experimental support for game theory is mixed. The response of some theorists is that there must be something wrong with the experiments because the theory is logically correct. The problem with this normative defense is that what is optimal in a game such as the traveler's dilemma depends on what the other players actually do, not on what some theory says they should do.
Our research introduces randomness into decision making, which causes decisions to be only imperfectly related to measured economic incentives. Although the extent of this randomness is a matter of empirical estimation, the incorporation of such noise is a common element of the techniques that we use to describe the behavior of our laboratory subjects. This narrative has described three complementary modifications of classical game theory. The models of introspection, learning/evolution, and equilibrium contain the common stochastic elements that represent errors or unobserved preference shocks. These three approaches are like the "three friends" of classical Chinese gardening (pine, cherry, and bamboo), they fit together nicely, each with a different purpose. Models of iterated noisy introspection are used to explain beliefs and choices in games played only once, where surprises are to be expected, and beliefs are not likely to be consistent with choices. With repetition, beliefs and decisions can be revised via learning or evolution. Choice distributions will tend to stabilize when there are no more surprises in the aggregate, and the resulting steady state constitutes a noisy equilibrium. This general approach can usefully incorporate a broad range of human motivations, including altruism, fairness, and risk aversion, factors which improve the predictive power of equilibrium models in some of the applications we have considered. To summarize, the major scientific insights are:
1) We have developed of a new model of noisy introspection to explain anomalous behavior in games played once. This introspective solution exists and reduces to the logit quantal response equilibrium in the special case where uncertainty does not increase with successive iterations of introspective thinking.
2) We have used the logit equilibrium to explain anomalous data patterns in a range of games, especially those with a continuum of strategies, where we derive existence, uniqueness, and comparative statics proofs. The advantage of this approach is that the same formal model tracks behavior that closely conforms to Nash equilibrium predictions in one treatment and deviates sharply in another.
3) We have developed ways to incorporate broader motivations into the stochastic equilibrium models, e.g. altruism, inequity aversion, and risk aversion, with applications to public goods, bargaining, and auctions.
4) We have used learning models to explain the time patterns of directional adjustment in specific laboratory experiments. We have characterized a steady-state learning equilibrium and its relationship to the (static) logit quantal response equilibrium.
5) For potential games, we use an entropy measure of dispersion to construct a "stochastic potential function" that is maximized by a noisy evolutionary process. All maxima of this stochastic potential are logit equilibria, which generalize the popular notion of "risk dominance" for selecting among multiple equilibria in 2x2 games.
These theoretical perspectives allow us to predict initial play, adjustment patterns, and final tendencies in a series of laboratory experiments. Data patterns that our colleagues would previously characterize as "behavioral" (i.e. consistent with intuition about human cognition but not with economic theory) are being picked up by these new stochastic game-theoretic models. There are discrepancies, but the overall pattern of results is surprisingly coherent, especially considering that we are using human subjects in interactive situations. The results are often precise in comparison with data from difficult physical science experiments, where impurities and extraneous forces may be hard to control. In fact, the principal investigator with a second degree in physics (Goeree) sometimes remarks that he is getting "that old physics feeling" when something unexpected happens in an economics experiment.
Laboratory experiments have
been intimately connected with the development of game theory, starting
with the reaction to Nash's seminal theorem. Two of the three recipients
of the first Nobel Prize in Economics given to game theorists (Nash and
Reinhard Selten) conducted experiments, and the third (John Harsanyi) was
instrumental in the incorporation of random elements into equilibrium models.
Patterns of human data provide the landmarks that are needed to avoid becoming
lost in the jungle of possibilities once theorists move away from assumptions
of perfect rationality. The resulting models have the empirical content
that makes them relevant for playing games, not just for doing theory.
VI. Abstracts of Recent Research Papers on Stochastic Game Theory
General Overview
Goeree, Jacob K., and Charles A. Holt (1999) "Stochastic Game Theory: For Playing Games, Not Just for Doing Theory"Proceedings of the National Academy of Sciences, 96(September), 10564-10567, http://www.pnas.org/cgi/reprint/96/19/10564. Abstract: This paper argues that noisy models of introspection, learning, and equilibrium can explain the salient behavior patterns in game experiments, patterns that are not predicted by the Nash equilibrium or its refinements. Models of iterated noisy introspection are used to explain initial choices, models of noisy learning and evolution are used to predict dynamic adjustment paths, and logit equilibrium models explain Nash-invariant treatment effects in steady-state distributions of decisions.
Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "Logit Equilibrium Models of Anomalous Behavior: What to Do when the Nash Equilibrium Says One Thing and the Data Say Something Else," in Handbook of Experimental Economics Results, edited by C.R. Plott and V.L. Smith, New York: Elsevier Press, forthcoming. Abstract: This paper shows how the logit equilibrium can be used to explain anomalous data patterns in a wide variety of games such as social dilemma games, coordination games, contests, pricing games, etc.
Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "Bounded
Rationality in Markets and Interactive Systems," Experimental Economics,
forthcoming. Abstract: This is a selective survey of the literature on
bounded rationality. Results from experimental economics are used to motivate
models that relax the rational choice assumptions.
Noisy Introspection
Goeree, Jacob K., and Charles A. Holt (1999) "A Model of Noisy Introspection," forthcoming, Games and Economic Behavior. Abstract: The paper presents a theoretical model of noisy iterated introspection designed to explain behavior in games played only once. The equilibrium determines layers of beliefs about others' beliefs about ..., etc., but relaxes the Nash-like requirement that belief distributions coincide with distributions of decisions, i.e. it allows for systematic surprises. We prove that this thought process converges. Data from 37 one-shot matrix games are used to estimate error and introspection parameters, which allows rejection of parameter restrictions implied by Nash and logit equilibrium limit cases. Adding an introspection parameter cuts the mean squared prediction error by about half, and adding a risk aversion parameter reduces the remaining mean squared error by half.
Goeree, Jacob K., and Charles A. Holt (2001) "Ten
Little Treasures of Game Theory, and Ten Intuitive Contradictions,"American
Economic Review, 91(5), 1402-1422. Abstract: The
"treasures" are ten static and dynamic games where behavior matches the
Nash equilibrium or relevant refinement, and the contradictions show anomalous
behavior patterns. In some games, Nash seems to only work by coincidence,
e.g. if deviation losses are symmetric or very high, and in other games
the data are repelled from the Nash prediction and pile up on the opposite
side of the set of feasible decisions.
Quantal Response Equilibrium
Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (2002) "The Logit Equilibrium: A Perspective on Intuitive Behavioral Anomalies," Southern Economic Journal, 69(1) 21-47.Abstract: The paper characterizes logit equilibria for a class of N-player games with payoffs that depend on the ranking of player's decisions, including Bertrand price competition, Hotelling's location game, the traveler's dilemma, and many variants of coordination games. General existence, symmetry, uniqueness, and comparative statics proofs are presented and applied.
Anderson, Simon P., Jacob Goeree, and Charles A. Holt (1998) "Control Costs and Equilibria in Games with Bounded Rationality," Discussion Paper, presented at the Summer 1998 ESA Meetings. Abstract: Van Damme's notion of control costs is that it is more costly to implement decisions more precisely. This paper derives the relationship between control costs and noisy approaches to equilibrium in games. In two-by-two games, quantal response equilibria are equivalent to Nash equilibria with control costs. Extensions to N-player matrix games are discussed.
Goeree, Jacob K. and Charles A. Holt (2000) "An Explanation of Anomalous Behavior in Binary-Choice Games: Entry, Voting, Public Goods, and the Volunteer's Dilemma," Discussion Paper. Abstract: This paper characterizes behavior with "noisy" decision making for a general class of N-person, binary-choice games. Applications include: participation games, voting, market entry, binary step-level public goods games, the volunteer's dilemma, etc. Many anomalous data patterns in laboratory experiments based on these games can be explained using the quantal response equilibrium.
Goeree, Jacob K., Charles A. Holt, and Thomas Palfrey (2000)"Risk
Averse Behavior in Generalized Matching Pennies Games," forthcoming
in Games and Economic Behavior. Abstract: We consider a 2
× 2 game in which each player chooses between a relatively safe decision
and a risky decision with a large difference between the possible payoffs.
If players' utilities are correctly measured by financial rewards, the
unique Nash equilibrium and all quantal response generalizations reside
on one half of the set of choice probabilities. The observed choice
frequencies are well away on the other side. Risk aversion shifts
the stochastic best response functions toward each player's safe strategy,
and the resulting intersection of these functions is close to the observed
data.
Learning and Evolution
Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1997) "Stochastic Game Theory: Adjustment to Equilibrium Under Noisy Directional Learning," Discussion Paper. Abstract: This paper presents a dynamic model in which agents adjust their decisions in the direction of higher payoffs, subject to random error. This process produces a probability distribution of players' decisions whose evolution over time is determined by the Fokker-Planck equation. The dynamic process is stable for all potential games, a class of payoff structures that includes several widely studied games. In equilibrium, the distributions that determine expected payoffs correspond to the distributions that arise from the logit function applied to those expected payoffs. This "logit equilibrium" forms a stochastic generalization of the Nash equilibrium and provides a possible explanation of anomalous laboratory data.
Capra, C. Monica, Jacob K. Goeree, Rosario Gomez, and Charles A. Holt (2002) "Learning and Noisy Equilibrium Behavior in an Experimental Study of Imperfect Price Competition," International Economic Review, 43(3), August, 613-636. Abstract: The experiments implement a model of imperfect price competition in which the high price seller matches the lower price but has a lower sales quantity. The Nash-Bertrand prediction is unaffected by the market share of the high-price seller, but the data respond sharply to changes in this parameter, a response that is consistent with dynamic and equilibrium models of noisy behavior. Simulation techniques are used to explore conditions under which the steady-state "stochastic learning equilibrium" matches the logit equilibrium, and conditions under which there are small but systematic differences due to "recency effects."
Capra, C. Monica, Jacob K. Goeree, Rosario Gomez, and Charles A. Holt (1999) "Anomalous Behavior in a Traveler's Dilemma," American Economic Review, 89:3, June, 678-690. Abstract: The observed choices in a traveler's dilemma experiment are moved across the entire set of feasible decisions by changes in a treatment variable that has no effect on the unique Nash prediction. Dynamic patterns are explained by a logit learning model, and the steady state distributions are centered around the predictions of a logit equilibrium that generalizes the Nash equilibrium to allow for noisy behavior.
Goeree, Jacob K. and Charles A. Holt (2000) "Stochastic Learning
Equilibrium," Discussion Paper Presented at the June 2000 ESA Meeting .
Learning and adaptive behavior in an interactive context may be noisy,
due to errors in forecasting and/or decision-making. This paper introduces
the notion of a stochastic learning equilibrium, which is a steady
state distribution of players' histories of past decisions. We prove
existence and show that the steady states do not coincide with standard
equilibrium predictions, e.g. Nash or quantal response, except under specific
conditions. The model is general enough to include endogenous learning
rules and more mechanical reinforcement learning.
Rent Seeking
Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "Rent Seeking with Bounded Rationality: An Analysis of the All Pay Auction,"Journal of Political Economy, 106:4, August, 828-853. Abstract: When people expend real resources to compete for a prize, a significant part of the associated value (rent) may be dissipated in the process. A model of bounded rationality is used to explain why the extent of rent dissipation in an all-pay auction may be sensitive to factors such as the number of competitors, the cost of effort, etc., which have no affect in a Nash analysis. A logit equilibrium model of boundedly rational behavior is proposed and analyzed.
Goeree, Jacob K., Simon P. Anderson, and Charles A. Holt (1998) "The
War of Attrition with Noisy Players," in Advances in Applied Microeconomics,
Volume 7, edited by M.R. Baye, Greenwich, Conn.: JAI Press, 15-29.
Abstract: We show that the Nash equilibria for the two-player normal-form
war of attrition with asymmetric values involve one player choosing zero
effort (conceding immediately). Non-degenerate mixed-strategy equilibria
under different prize values are possible only when there is no maximum
effort. These equilibria have perverse comparative static properties: an
increase in one player's value leaves that player's bid distribution unaffected
and raises the other player's effort. We describe the logit equilibrium
for the game, which is symmetric when values are equal and predicts that
the player with the higher prize value exerts more effort in the asymmetric
case.
Coordination
Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (2001) "Minimum-Effort Coordination Games: Stochastic Potential and the Logit Equilibrium,"Games and Economic Behavior, 34(2), 177-199. Abstract: This paper considers coordination games with a continuum of Pareto-ranked Nash equilibria. The introduction of noise error yields a unique distribution of decisions that maximizes a stochastic potential function (expected value of the potential of the game plus entropy). As the noise vanishes, the limiting distribution converges to an outcome that is analogous to the risk-dominant outcome for 2×2 games. In accordance with experimental evidence and economic intuition, our results show that efforts decrease with increases in effort costs and the number of players, even though these parameters do not affect the Nash equilibria.
Goeree, Jacob K., and Charles A. Holt (1999) "An
Experimental Study of Costly Coordination," Discussion Paper. Abstract:
We present and test a unified view of behavior in coordination experiments
with effort-cost and numbers effects not predicted by the Nash equilibrium.
The theory is a generalization of risk dominance and maximum potential;
the maximization of stochastic potential explains (final period) effort
levels of human subjects in a series of minimum- and median-effort coordination
experiments.
Public Goods
Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "A Theoretical Analysis of Altruism and Decision Error in Public Goods Games,"Journal of Public Economics, 70:2 (November), 297-323. Abstract: This paper formalizes an equilibrium model in which altruism and decision-error parameters determine the distribution of contributions in public goods games. We prove existence of a unique, symmetric equilibrium density of contributions and show that (i) contributions increase with the marginal value of the public good, (ii) total contributions increase with the number of participants if there is altruism, and (iii) mean contributions lie between the Nash prediction and half the endowment. These predictions, which are not implied by a standard Nash analysis, are roughly consistent with laboratory data.
Goeree, Jacob K., Charles A. Holt, and Susan K. Laury (2002) "Private Costs and Public Benefits: Unraveling the Effects of Altruism and Noisy Behavior," Journal of Public Economics, 83(2) 257-278. Abstract: The effects of a contribution to a public good are decomposed into an internal return to the contributor and an external return to each of the others. Contributions in one-shot games are generally increasing in internal returns, external returns, and group size, and a logit model of individual behavior tracks treatment averages well, both for linear and non-linear altruism specifications.
Goeree, Jacob K., Charles A. Holt, and Susan K. Laury (2000)"Incentives
in Public Goods Experiments: Implications for the Environment," forthcoming
in Frontiers of Environmental Economics, J. List and A. de Zeeuw,
eds.. Abstract: This paper reports results of an experiment
designed to evaluate the effects of externalities by altering the costs
and benefits of an investment that corresponds to pollution abatement.
In this experiment, a person can make an investment with a private (internal)
return that does not cover the investment cost, but with a public (external)
return that makes the investment socially optimal. Investments are
increasing in internal and external returns, and are virtually identical
for two treatments with the same "price," defined to be the ratio of the
external benefit to the internal loss from making an investment. Individual
forecast data make it clear that many who invest nothing are free riding
on anticipated investments by others, and most who make significant investments
in the final period do not expect others to be as generous.
Bargaining and Auctions
Goeree, Jacob K., and Charles A. Holt (2000) "Asymmetric Inequality Aversion and Noisy Behavior in Alternating-Offer Bargaining Games," European Economic Review, 44, 2000, 1079-1089. Abstract: We report a two-stage alternating-offer bargaining experiment in which players receive asymmetric fixed money payments in addition to their earnings from the bargaining process. These endowments do not affect the perfect positive correlation between initial Nash offers and the remaining pie, but are selected to induce a perfectly negative relationship between the remaining pie size and the first-stage offer that would equalize final earnings of the two players. This negative relationship is apparent in the data, which suggests the importance of fairness considerations. A model of asymmetric inequality aversion and stochastic choice is used to provide estimates of utility and logit error parameters.
Goeree, Jacob K., Charles A. Holt, and Thomas R. Palfrey (2002) "Quantal
Response Equilibrium and Overbidding in Private-Value Auctions." Journal
of Economic Theory, 104(1), 247-272. Abstract: This paper reports
the results of a private-values auction experiment in which expected costs
of deviating from the Nash equilibrium bidding function are asymmetric,
with the implication that upward deviations will be more likely in one
treatment than in the other. Overbidding is observed in both treatments,
but is higher in the treatment where the costs of overbidding are lower.
We specify and estimate a noisy (logit) model of equilibrium behavior.
Estimated noise and risk aversion parameters are consistent across treatments
and are highly significant and of reasonable magnitudes. Alternative explanations
of overbidding are also considered. The estimates of parameters from a
cumulative probability weighting function yield a formulation that is essentially
equivalent to risk aversion.
Brief Survey Papers on Topics
in Experimental Economics
(citations and coauthors are listed on my vita)
Methodology
"Varying
the Scale of Financial Incentives Under Real and Hypothetical Conditions"
Industrial Organization:
"Predatory
Pricing: Rare Like a Unicorn?"
"The Exercise of Market Power in Laboratory Experiments"
"The Effects of Collusion in Laboratory Experiments"
"Industrial Organization: A Survey of Laboratory Research"
"Markets with Posted Prices: Recent Results from the Laboratory"
Public Goods:
"Theoretical
Explanations of Treatment Effects
in
Voluntary Contributions Experiments"
"Voluntary
Provision of Public Goods: Experimental Results
with
Interior Nash Equilibria".
Games and Information:
"Stochastic
Game Theory: For Playing Games, Not Just for Doing Theory,"
"Information Cascade Experiments"
"Game Theory: Ten Little Treasures and Ten Intuitive Contradictions"
"Logit
Equilibrium Models of Anomalous Behavior: What to do when the
Nash
Equilibrium Says One Thing and the Data Say Something Else"