Decisions and Expected Utility

Probability times value

Knowing probabilities is not enough; an agent must act. Decision theory assigns a utility (how good) to each outcome and chooses the action with the highest expected utility: the sum over outcomes of probability times utility. The best action depends on both the probabilities and the values: with costly mistakes, even unlikely bad outcomes can decide. This principle underlies rational agents, medical decision support, insurance and the reward signals of reinforcement learning.

Should I take an umbrella? run

I ran this with plain Python 3 (standard library only), with fixed random seeds where randomness is used. With a 30% chance of rain, taking an umbrella has expected utility 77.0 versus 70.0 without. At 10% rain the best action switches to no umbrella; at 30% and 50% it is umbrella.

p_rain = 0.3
utility = {  # (action, weather) -> how good the outcome is
    ("umbrella", "rain"): 70, ("umbrella", "dry"): 80,
    ("no umbrella", "rain"): 0, ("no umbrella", "dry"): 100}
for a in ["umbrella", "no umbrella"]:
    eu = p_rain * utility[(a, "rain")] + (1 - p_rain) * utility[(a, "dry")]
    print(f"expected utility of {a:<11}: {eu:.1f}")
for p in [0.1, 0.3, 0.5]:
    best = max(["umbrella", "no umbrella"], key=lambda a: p * utility[(a, "rain")] + (1 - p) * utility[(a, "dry")])
    print(f"P(rain) = {p}: best action {best}")

Output:

expected utility of umbrella   : 77.0
expected utility of no umbrella: 70.0
P(rain) = 0.1: best action no umbrella
P(rain) = 0.3: best action umbrella
P(rain) = 0.5: best action umbrella

Make utilities explicit

Writing down the value of each outcome exposes disagreements (how bad is a false alarm?) that otherwise stay hidden.

Quick check: How does a rational agent choose an action under uncertainty?

• At random
• By always choosing the most likely outcome
• By maximising expected utility
• By minimising the number of outcomes