An Introduction to decision theory by Martin Peterson

Key ideas & Notes

• Chapter 1: Introduction

  • Descriptive Decision Theory — How people actually make decisions. How people actually behave is likely to change over time and across cultures

  • Normative Decision Theory —How people ought to make (rational) decisions. Should be agnostic of culture and time

  • A decision can be right without being rational and can be rational without being right

  • Decision is RIGHT if and only if its actual outcome is at least as good as that of every other possible outcomes.

  • Decision is RATIONAL if and only if the decision maker chooses to do what she has the most reason to do at that point in time

  • INSTRUMENTAL rationality

    • Assumes a decision maker has some aim

    • Aims in themselves cannot be irrational and are external to the decision theory

    • To be instrumentally rational means to do what one has the most reason to expect will fulfil ones aim.

  • Decisions under RISK

    • The probability of the possible outcomes IS known by the decision maker

    • Most widely used approach is the principle of maximising the expected value

    • The total value of an act equals the sum of the values of its possible outcomes weighted by the probability for each outcome

  • Decision under UNCERTAINTY / IGNORANCE

    • The probability of the possible outcomes IS NOT known by the decision maker

  • SOCIAL CHOICE theory — Aims to establish principles for how decisions involving more than one decision maker out to be made.

• Chapter 2: The Decision Matrix

  • DECISION PROBLEM

    • States — Parts of the world that are not and outcome or an act. Devices needed for applying acts

    • Outcomes — What ultimately matters

    • Acts — Instruments to reach the outcomes.

  • SCALES

    • ORDINAL scales — merely express preference ordering. No information about by how much one preference is better/worse than another preference. Qualitative comparison of objects

    • CARDINAL scales

      • INTERVAL scales — Accurately reflect the difference between the objects being measured (but not the ratio). Example: Comparing Celsius and Fahrenheit scales. Quantitative comparison of objects.

      • RATIO scales — Reflect ratios. A pair of ratio scales are equivalent if and only if each can be transformed into the other by multiplying all values by some positive constant. Quantitative comparison of objects. Example: Height, Weight

• Chapter 3: Decisions under ignorance

  • The decision maker knows what the alternatives and what outcomes they may result it, but is unable to assign any probabilities to the states corresponding to the outcomes

  • Dominance — A1 dominates A2 if A1 always leads to better outcomes regardless no matter what states of the world happen to be true

  • Maximin

    • Focus on the worst possible outcome

    • One should maximise the minimal value obtainable with each act

    • Ai ≥ Aj if and only if min(Ai) ≥ min(Aj) for all i and j

    • Allows value to be measure on Cardinal scale

  • Leximin — Lexical Maximin

    • Addresses a scenario where worst outcomes are identical

    • If worst outcomes are equal, choose the alternative with the best second worst outcome (if equal, choose third, fourth etc)

    • Allows value to be measure on Cardinal scale

  • Maximax — Maximise the maximum value obtainable with an act. Allows value to be measure on Cardinal scale

  • Optimism-Pesimism rule

    • Generalisation of the maximin and maximax rules

    • Requires value to be measure on an Interval scale

    • Consider best and worst possible outcomes of each alternative and then choose an altertive according to the decision makers level of optimism/pesimism

  • Minimax Regret — The best alternative is the one that minimises the maximum amount of regret

  • Principle of insufficient reason

    • If one has no reason to think that one state of the world is more likely than another, then all states should be assigned equal probability

    • Enables transforming decisions under ignorance into decisions under risk

• Chapter 4: Decisions under risk

  • The decision maker knows the probabilities of the outcomes

  • Principle of maximising the expected value favoured by most experts

  • Expected Value

    • EV = p1 · v1 + p2 · v2 + ... + pn · vn

    • Value is evaluated from the decisions maker's point of view

  • Expected Utility

    • Utility of an outcome depends on how valuable the outcomes is from the decision maker's perspective

    • EU = p1 · u1 + p2 · u2 + ... + pn · un

    • Based on the law of large numbers, in the long run it is best to maximise expected utility

    • Assumes that probability of each outcome remains stable over time.

• Chapter 6: The Mathematics of probability

  • Kolmogorov (1933) — Every probability is a real number between 0 and 1: 1 ≥ p(A) ≥ 0

  • The probability of the entire sample space is 1: p(S) = 1

  • If two events are mutually exclusive, then the probability that one of them will occur equals the probability of he first + the probability of the second

    • if A ∩ B = 0, then p(A U B) = p(A) + p(B)

    • if A and B are mutually exclusive then p(A ∨ B) = p(A) + p(B)

    • p(A) + p(negA) = 1

    • If A and B are logically equivalent, then p(A) = p(B)

  • Conditional probability

    • p(A|B) = p(A ∧ B) / p(B)

    • A is independent of B if and only if p(A) = p(A|B)

    • If A is independent of B then p(A ∧ B) = p(A) · p(B)

  • Bayes Theorem

    • p(B|A) = p(B) · p(A|B) / p(A)

    • p(B) is prior probability

    • Unknown priors is a problem, but can be reduced by applying Bayes theorem over and over again.

    • The more times the Bayes theorem is applied, the closer to the truth (posterior probability value) we will get (regardless of the initial prior value!)

• Chapter 7: The Philosophy of probability

  • Objectivists — Believe that statements about probability refer to the facts in the real world

  • Subjectivists — Deny objectivists' claims. Statements about probability refer to the degree to which the speaker believes in something. Probabilities are entities that humans somehow create in their own minds. When two decision makers hold different subjective probabilities they just happen to believe something to different degrees.

  • Classical interpretation

    • Probability of an event seen as a fraction of the total number of possible ways in which the event can occur

    • Presupposes that all possible outcomes are equally likely

  • Frequency interpretation

    • The probability of an event is the ratio between the number of time the event occurred and the total number of observed cases.

    • Always defined relative to some reference class

    • Key challenge - how to determine which reference class is suitable and why?

  • Propensity Interpretation

    • Developed by Karl Popper in 1950s

    • Probability is related to certain features of the real world based on their tendency to give rise to a certain effect

    • Propensity can't be directly observed --> We need to rely on indirect evidence

    • Propensities have a temporal direction: if A has a propensity to give rise to B, then A cannot occur after B (like causality). In contrast, probabilities do NOT have a temporal direction.

  • Logical interpretation

    • Developed by Keynes and Carnap

    • Often call epistemic probability

    • Probability is a logical relation between a hypothesis and the evidence supporting it.

• Chapter 8: Why should we accept preference axioms?

  • RISK AVERSION

    • Against actuarial risks — Risk averter is defined as one who, starting from a position of certainty, is unwilling to take a bet which is actuarially fair.. Prefers (smaller) certainty even over lotteries that may bring more significant gain.

    • Against utility risks —The risk averter should not only maximise utility but also, at least sometimes, apply some decision rule that puts emphasis on avoiding bad outcomes. E.g. maximin rule

    • Against epistemic risks — Example: The maximin criterion for expected utilities (MMEU) - the alternative with the largest minimal utility ought to be selected. Encourages decision makers to expect worst

  • Utility function — Is able to represent those preferences if it is possible to assign a real number to each alternative, in such a way that alternative a is assigned a number greater than alternative b if, and only if, the individual prefers alternative a to alternative b. A decision maker with a concave utility function will always prefer a smaller prize for certain over actuarially fair lottery over larger and smaller prizes. The flatter the utility function is in a given interval the more actuarially risk averse the decision maker is in that interval

• Chapter 9: Causal vs. Evidential decision theory

  • Newcomb's problem — Shows that Dominance principle and maximising expected utility theory yield conflicting recommendations

  • Causal decision theory — Decision maker should keep all her beliefs about causal processes fixed during the decision making process and always choose an alternative that is optimal according to these beliefs. Causal structure is forward looking and completely insensitive to the past. Recommends to do what is likely going to bring the best possible result, while holding fixed all views about the likely causal structure of the world.

  • Evidential Decision Theory — Instead of asking "If I do X will Y happen" we should ask "What is the probability that if I were to do X, then Y would be the case, given that I do X?"

• Chapter 10: Bayesian vs. non-Bayesian decision theory

  • Bayesianism

    • Epistemic component — What rational agents ought to believe and which combinations of beliefs and desires are rationally permisable.

    • Deliberative component — Tells us what action is rational for the agent to perform given his or her present state of mind.

    • Ordering axiom — Most fundamental axiom of Beysianism. For any two uncertain prospects, the decision maker must be able to state a clear and unambiguous preference and all such preferences must be asymmetric and transitive.

  • Non-Bayesian theories

    • Externalist — An act is rational not merely by virtue of what the decision maker believes and desires. Rejects Humean belief-desire account which is considered too narrow

    • Internalist — Decision maker's beliefs and desires is all that matters when adjudicating whether an act is rational or not. In contrast to Bayesianism, preferences over risky acts are not just tools to measure degrees of belief and desires but constitute reasons to preferring one risky act over another

• Chapter 11: Game Theory

  • Studies decisions where outcomes depend partly on what other people/decision makers do

  • Prisoner's dilemma

    • What is optimal for each individual need NOT coincide with what is optimal for the group.

    • Individual rationality sometimes comes into conflict with group rationality.

    • Arise whenever a game is symmetrical i.e. everyone is facing the same strategies and outcomes and the ranking of outcomes is 2,2 4,1, 1,4 3,3

  • Non-cooperative, simultaneous-strategy, symmetric, nonzero-sum and finite game

  • Taxonomy

    • zero sum — you win as much as your opponent(s) lose (casino games, chess). The total amount of money/units of utility is fixed no matter what happens.

    • Non-cooperative — Players are not able to form binding agreements (but players CAN cooperate).

    • Simultaneous vs sequential

      • Simultaneous move — Players decide on their strategies without knowing what the other player(s) will do.

      • Sequential move — players have some (or full) information about the strategies played by other players in earlier rounds

    • Symmetric — All players face the same strategies and outcomes. Identity of players does not matter

    • Equilibrium

      • A pair of strategies is in equilibrium if and only if it holds that once this pair of strategies is chosen none of the players can reach a better outcome by UNILATERALLY switching to another strategy

      • To find equilibrium strategies look for strategies that fulfil the minimax condition. The outcome is determined by the minimal value of the row and the maximal value of the column

    • Finitely vs infinitely iterated

      • Finitely iterated — Games iterated finite number of times. In finitely iterated game rational players will behave in exactly the same way as in the one-shot version of the game

      • Infinitely iterated — Players do not know in advance whether they are about to play the last round of the game. The games does NOT actually need to be iterated ad infinitum. Each player may adjust his next move to what the opponent did in the previous round. Tit-for-tat is a best strategy: always cooperate in the first round and thereafter adjust your behaviour to whatever your opponent did in the previous round.

• Chapter 12: Non zero-sum and cooperative games

  • Nash equilibrium

    • Rational players will do whatever they can to ensure that they do not feel unnecessarily unhappy about their decision.

    • An equilibrium point is a set of strategies such that each player's strategy maximises his pay-off if the strategies of the others are held fixed. Thus each player's strategy is optimal against those of the others. [Nash 1950]

    • Many non zero-sum games have more than one Nash equilibrium which means such games can't be "solved" in a sense that we can't exactly figure out what rational players would do by just applying Nash equilibrium concept.

  • Pareto efficient state —A state is Pareto efficient if and only if no one's utility level can be increased unless the utility level for someone else is decreased.

• Chapter 13: Social Choice Theory

  • Analyses collective decision problems and how a set of individual preference orderings G can be aggregated in a systematic manner into a Social preference ordering S.

  • Social choice problem — Any decision problem faced by a group in which each individual is wiling to state at least ordinal preferences over outcomes. How to translate INDIVIDUAL preference ordering into SOCIAL preference ordering.

  • Social state — The state of the world that includes everything that individuals care about.

  • Social Welfare Function (SWF) — any decision rule that aggregates a set of individual preference orderings over social states into a social preference ordering over those states. (e.g. Majority rule). Any normatively reasonable SWF should be non-dictatorial i.e. S must not always coincide with the preference ordering of a particular individual. For every possible combination of individual preference orderings, an SWF must produce a social preference ordering that is complete, asymmetric and transitive. (i.e. Majority rule does not meet this condition)

• Chapter 14: Descriptive decision theory

  • People often act in ways that are irrational

  • Certainty effect — People value certain gain more that equally large expected gain. People reason incorrectly about small probabilities

  • Loss aversion is often stronger than gain preference

  • Prospect Theory — Descriptive theory of choice under risk. Expected utility principle should be modified by introducing two weighing functions: one for value and one for probability. The value of Loss or Gain is NOT linear as prescribed by expected utility theory.