Decision theory offers tools to handle uncertainty, such as probabilistic models and the assessment of objectives given specified decision criteria. A well-known sequential decision problem is the one facing Ulysses onhis journey home to Ithaca in Homer’s great tale from antiquity.Ulysses must make a choice about the manner in which he will sail pastan island inhabited by sweet-singing sirens. In the second choice situation, however, the minimumone stands to gain is $0 no matter which choice one makes.
- Decision theory is a study of an agent’s rational choices that supports progress in technology such as work on machine learning and artificial intelligence.
- The distinct advantage ofJeffrey’s theory is that real-world decision problems can bemodelled just as the agent perceives them; the plausibility of therationality constraints on preference do not depend on decisionproblems being modelled in a particular way.
- It then followsthat for any other proposition \(s\) that satisfies the aforementionedconditions that \(r\) satisfies, you should also be indifferentbetween \(p\cup s\) and \(q\cup s\), since, again, the two unions areequally likely to result in \(s\).
- The decision theories of Savage and Jeffrey, as well as those of theircritics, apparently concern a single or “one shot only”decision; at issue is an agent’s preference ordering, andultimately her choice of act, at a particular point in time.
- The static model has familiar tabular or normalform, with each row representing an available act/option, and columnsrepresenting the different possible states of the world that yield agiven outcome for each act.
Some of therequired conditions on preference should be familiar by now and willnot be discussed further. In particular, \(\preceq\) has to betransitive, complete and continuous (recall our discussion in Section 2.3 of vNM’s Continuity preference axiom). Decision theory is not only a theory of choice but also a theory of beliefs, desires, and other relevant attitudes. The theory has practical implications for actions, inferences, and valuing, and it addresses challenges to traditional expected utility (EU) theory. Skyrms’ (1993) “diachronic Dutch book” argument forconditionalisation can be read in this way. The agent is assumed tohave EU preferences and to take a sophisticated (backwards reasoning)approach to sequential decision problems.
Hence, we say that an ordinal utility function isunique only up to ordinal transformations. Decision theory is concerned with the reasoning underlying anagent’s choices, whether this is a mundane choice between takingthe bus or getting a taxi, or a more far-reaching choice about whetherto pursue a demanding political career. (Note that “agent”here stands for an entity, usually an individual person, that iscapable of deliberation and action.) Standard thinking is that what anagent chooses to do on any given occasion is completely determined byher beliefs and desires or values, but this is not uncontroversial, aswill be noted decision theory is concerned with below.
Grant and Quiggin (2013a, 2013b), for instance,suggest that these judgments are made based on induction from pastsituations where one experienced awareness growth. There has been recent interest in yet a further challenge to expectedutility theory, namely, the challenge from unawareness. To keep things simple, we shall however focus onSavage’s expected utility theory to illustrate the challengeposed by unawareness. It was noted from the outset that EU theory is as much a theory ofrational choice, or overall preferences amongst acts, as it is atheory of rational belief and desire. This section expands, in turn,on the epistemological and evaluative commitments of EU theory. Another important thing to notice about Jeffrey’s way ofcalculating desirability, is that it does not assume probabilisticindependence between the alternative that is being evaluated, \(p\),and the possible ways, the \(p_i\)s, that the alternative may berealised.
What is the best way to make decisions in uncertain situations?
But that is just taking a gamble that has avery small probability of being killed by a car but a much higherprobability of gaining $10! More generally, although people rarelythink of it this way, they constantly take gambles that have minusculechances of leading to imminent death, and correspondingly very highchances of some modest reward. Now, Savage’s theory is neutral about how to interpret thestates in \(\bS\) and the outcomes in \(\bO\).
Utility Theory and Decision Theory
Richard Jeffrey’s theory, whichwill be discuss next, avoids all of the problems that have beendiscussed so far. But as we will see, Jeffrey’s theory haswell-known problems of its own, albeit problems that are notinsurmountable. A highly controversial issue is whether one can replace the use of probability in decision theory with something else.
The Representation of Decisions
In any case, decision theory is as much a theoryof beliefs, desires and other relevant attitudes as it is a theory ofchoice; what matters is how these various attitudes (call them“preference attitudes”) cohere together. Decision theory is the study of how choices are and should be made.in a variety of different contexts. Here we look at the topic from a formal-philosophical point of view with a focus on normative and conceptual issues.
In that case, however,EU theory is effectively vacuous or impotent as a standard ofrationality to which agents can aspire. Moreover, it stretches thenotion of what are genuine properties of outcomes that can reasonablyconfer value or be desirable for an agent. The revival of subjective probability theory, from the work of Frank Ramsey, Bruno de Finetti, Leonard Savage and others, extended the scope of expected utility theory to situations where subjective probabilities can be used. At the time, von Neumann and Morgenstern’s theory of expected utility10 proved that expected utility maximization followed from basic postulates about rational behavior.
1 Savage’s theory
- In the 18th century, Daniel Bernoulli introduced the concept of “expected utility” in the context of gambling, which was later formalized by John von Neumann and Oskar Morgenstern in the 1940s.
- A well-known sequential decision problem is the one facing Ulysses onhis journey home to Ithaca in Homer’s great tale from antiquity.Ulysses must make a choice about the manner in which he will sail pastan island inhabited by sweet-singing sirens.
- Furthermore, it permits explicit restrictions on what countsas a legitimate reason for preference, or in other words, whatproperties legitimately feature in an outcome description; suchrestrictions may help to clarify the normative commitments of EUtheory.
- Indeed, thismay be one of the main reasons why economists have largely ignoredJeffrey’s theory.
- Nevertheless, Savage’s theory has been much moreinfluential than Ramsey’s, perhaps because Ramsey neither gave afull proof of his result nor provided much detail of how it would go(Bradley 2004).
- More generally, although people rarelythink of it this way, they constantly take gambles that have minusculechances of leading to imminent death, and correspondingly very highchances of some modest reward.
A similar “dynamic consistency” argument can be used todefend EU preferences in addition to learning in accordance withconditionalisation (see Hammond 1976, 1977, 1988b,c). It is assumed,as before, that the agent takes a sophisticated approach to sequentialdecision problems. Hammond shows that only a fully Bayesian agent canplan to pursue any path in a sequential decision tree that is deemedoptimal at the initial choice node. This makes the Bayesian agentunique in that she will never make “self-defeatingchoices” on account of her preferences and norms for preferencechange. She will never choose a strategy that is worse by her ownlights than another strategy that she might otherwise have chosen, ifonly her preferences were such that she would choose differently atone or more future decision nodes.
Nevertheless, the weather statistics differfrom the lottery set-up in that they do not determine theprobabilities of the possible outcomes of attempting versus notattempting the summit on a particular day. Not least, the mountaineermust consider how confident she is in the data-collection procedure,whether the statistics are applicable to the day in question, and soon, when assessing her options in light of the weather. Start with the Completeness axiom, which says that an agent cancompare, in terms of the weak preference relation, all pairs ofoptions in \(S\). Whether or not Completeness is a plausiblerationality constraint depends both on what sort of options are underconsideration, and how we interpret preferences over these options.
One may well wonder whether EU theory, indeed decision theory moregenerally, is neutral with respect to normative ethics, or whether itis compatible only with ethical consequentialism, given thatthe ranking of an act is fully determined by the utility of itspossible outcomes. Such a model seems at odds withnonconsequentialist ethical theories for which thechoice-worthiness of acts purportedly depends on more than the moralvalue of their consequences. The model does not seem able toaccommodate basic deontological notions like agent relativity,absolute prohibitions or permissible and yet suboptimal acts. This disanalogy is due to the fact that there is nosense in which the \(p_i\)s that \(p\) is evaluated in terms of needto be ultimate outcomes; they can themselves be thought of asuncertain prospects that are evaluated in terms of their differentpossible realisations. In most ordinary choice situations, the objects of choice, over whichwe must have or form preferences, are not like this.
Intertemporal choice
But here adifferent interpretation of preference is brought to bear on thecomparison of options. The idea is that preferences, or judgments ofdesirability, may be responsive to a salience condition. For example,suppose that the most salient feature when comparing cars \(A\) and\(B\) is how fast they can be driven, and \(B\) is no worse than \(A\)in this regard, yet the most salient feature when comparing cars \(B\)and \(C\) is how safe they are, and that \(C\) is no worse than \(B\)in this regard. In such a case, some argue (e.g.,Temkin 2012) that there is no reason why Transitivity should besatisfied with respect to the preferences concerning \(A\), \(B\) and\(C\). Others (e.g., Broome 1991a) argue that Transitivity is part ofthe very meaning of the betterness relation (or objective comparativedesirability); if rational preference is a judgment of betterness ordesirability, then Transitivity is non-negotiable. With respect to thecar example, Broome would argue that the desirability of a fullyspecified option should not vary, simply in virtue of what otheroptions it is compared with.
In effect, Non-Atomicityimplies that \(\bS\) contains events of arbitrarily small probability.It is not too difficult to imagine how that could be satisfied. Forinstance, any event \(F\) can be partitioned into two equiprobablesub-events according to whether some coin would come up heads or tailsif it were tossed. Each sub-event could be similarly partitionedaccording to the outcome of the second toss of the same coin, and soon. Is there anyprobability \(p\) such that you would be willing to accept a gamblethat has that probability of you losing your life and probability\((1-p)\) of you gaining $10? However,the very same people would presumably cross the street to pick up a$10 bill they had dropped.
Skyrms shows that any suchagent who plans to learn in a manner at odds with conditionalisationwill make self-defeating choices in some specially contrivedsequential decision situations. A conditionalising agent, by contrast,will never make choices that are self-defeating in this way. That is, the agent chooses a strategy that issurely worse, by her own lights, than another strategy that she mightotherwise have chosen, if only her learning rule was such that shewould choose differently at one or more future decision nodes. A number of people have suggested models to represent agents who areaware of their unawareness (e.g., Walker & Dietz 2013, Piermont2017, Karni & Vierø 2017). Steele and Stefánsson(forthcoming-b) argue that there may not be anything especiallydistinctive about how a decision-maker reasons about states/outcomesof which she is aware she is unaware, in terms of the confidence shehas in her judgments and how she manages risk. That said, the way shearrives at such judgments of probability and desirability is worthexploring further.
We first describe theprospects or decision set-up and the resultant expected utility rule,before turning to the pertinent rationality constraints on preferencesand the corresponding theorem. Decision theory is an interdisciplinary field that deals with the logic and methodology of making choices, particularly under conditions of uncertainty. It is a branch of applied probability theory and analytic philosophy that involves assigning probabilities to various factors and numerical consequences to outcomes. The theory is concerned with identifying optimal decisions, where optimality is defined in terms of the goals and preferences of the decision-maker. Defenders of resolute choice may have in mind a differentinterpretation of sequential decision models, whereby future“choice points” are not really points at which an agent isfree to choose according to her preferences at the time. If so, thiswould amount to a subtle shift in the question or problem of interest.In what follows, the standard interpretation of sequential decisionmodels will be assumed, and accordingly, it will be assumed thatrational agents pursue the sophisticated approach to choice (as perLevi 1991, Maher 1992, Seidenfeld 1994, amongst others).
The literature onsequential choice is primarily concerned, however, with more ambitiousquestions. The sequential-decision setting effectively offers new waysto “test” theories of rational preference and norms forpreference (or belief and desire) change. The question is whether anagent’s decision theory in this broad sense is shown to bedynamically inconsistent or self-defeating. As noted in Section 4, criticisms of the EU requirement of a complete preference orderingare motivated by both epistemic and desire/value considerations. Onthe value side, many contend that a rational agent may simply find twooptions incomparable due to their incommensurablequalities.
Further interpretive questions regarding preferences andprospects will be addressed later, as they arise. By the late 20th century, scholars like Daniel Kahneman and Amos Tversky challenged the assumptions of rational decision-making. Their work in behavioral economics highlighted cognitive biases and heuristics that influence real-world decisions, leading to the development of prospect theory, which modified expected utility theory by accounting for psychological factors. Notwithstanding these finer disputes, Bayesians agree that pragmaticconsiderations play a significant role in managing beliefs. Oneimportant way, at least, in which an agent can interrogate her degreesof belief is to reflect on their pragmatic implications. Furthermore,whether or not to seek more evidence is a pragmatic issue; it dependson the “value of information” one expects to gain withrespect to the decision problem at hand.
