How Surprised Would You Be?
Nerd games for calibrating nerds
(Follow-up to How Surprised and How Upset?)
It can be useful to try to gauge how surprised you would be by some event. One reason is that it can be a useful way to see whether you are well-calibrated to the probability of various outcomes, but it's also just a fun game to play with fellow nerds. I like to ask, "How surprised would you be on a scale from 0-10?" and see if we agree or disagree, and why. Another reason is that intuitions are strange and hard to pin down, and any form of mental scaffolding can help.
You could make the game more quantitative if you wanted to, by playing the subjective "How surprised would you be from 0-10?" game while also estimating the probability of the same events, and see how they line up. If e.g. you'd be very surprised (say, a 5/10) by X but on reflection X has a 70% change of occurring, then something is off. (In this post I'll sometimes move from the subjective 0-10 scale to the language of probability, which will be based on the assumption of a well-calibrated person (not me).)
Playing the game is still a necessary step that allows you to go from a subjective feeling to at least a ranking, so let me continue to explain it. The ends of the scale are of particular importance.
The way I play, 0 Surprised corresponds to anything you actually expect to happen, e.g. the sun will rise tomorrow or I will eat dinner today. This rule implies that much of the probability weight in this scale is concentrated at the base: a 70%, 90%, 51%, or 99.999% probability all correspond to 0 Surprised. You could try to develop a variant where you can be negative surprised, like -10 Surprised is ~100% confidence or something, but in practice I think this can't work, because Not Surprised isn't a feeling that comes with gradation in the way that Surprised is (at least not for me, let me know if I'm the weird one here).
Similarly, 10 Surprised corresponds to anything that is physically or logically impossible. So if I found out that 2+2=5, that'd be a 10 Surprised. Similarly if I found out that the moon is made of cheese, or if I woke up tomorrow in my wife's body. None of those things are possible, and if I encountered extremely strong evidence that they were true I would assume I was hallucinating or that I had been insane all along (all good markers of 10 Surprised).
Once you hit a 10 here, further gradation doesn't seem useful: would it be more surprising to find that that 2+2=5, or that electricity is made of tiny elves that dance through the wires to power your computer? You could argue that there are differences in your confidence of these things, like 0.999 or 0.99999, but I think in either case the subjective feeling grounds out in "I need to see a doctor right now because I've lost my mind." So, 10 Surprised.
The middle points of the scale are much more arbitrary, but I took some time to give each level a description and work out a few examples for each point that I personally find helpful.
Surprise Level:
0: No surprise
Anything I already expect to happen
e.g. the sun will rise tomorrow
e.g. the hamburger I ordered will not be cold
1: Minor surprise
I know that this kind of thing happens frequently
"Huh, ok then, it happens I guess"
e.g. my wife drinks a whole carton of milk in one day
e.g. the restaurant I frequent gets my order wrong
2: Somewhat surprised
It's specifically unlikely but in a category of things that happens frequently
"Wow who knew" or "interesting"
e.g. the restaurant throws in something "on the house"
e.g. my favorite character is killed off suddenly in a show (that isn't known for this kind of thing)
3: Kinda surprised
Not expected on any given day but might happen from time to time
e.g. a random friend with no known athletic training does a backfip
e.g. I get in a fender-bender (if I owned a car and drove regularly)
4: Pretty surprised
Should expect it eventually but very rarely
e.g. I run into an old friend while traveling
e.g. I hear that an acquaintance from college dies
5: Very surprised
These things do occur in principle, but I know the probability to be very low
I never expect to see such things
e.g. I make 20 free throws in a row
e.g. I have a major house fire tomorrow
6: Caught totally off-guard
"What the hell!?" (in a good or bad way)
This is roughly the highest level where I'm likely still assessing probabilities for the event, explicitly or implicitly
e.g. I win $1000+ in a city-wide raffle
e.g. I (a healthy 32 year-old) have a sudden heart attack tomorrow
7: Nearly can't believe it
Such things likely never even occurred to me
Some large secret has been kept from me or some systems are suppressing this fact for unknown reasons (at least that's how it feels)
e.g. my house happens to sit above an untapped gold mine (if I owned a house in the American suburbs)
e.g. my brother (who I talk to often) has a secret love-child / second family
8: Perspective is shattered
It's highly implausible that I haven't found out about this somehow before now
I must be wrong about many other things for this to be true
e.g. I'm adopted (not the biological son of either of my parents)
e.g. there are currently human colonists on Mars
9: World-changing revelation
Strictly possible and consistent with physics but throws many base-level beliefs into doubt
e.g. New Zealand isn't a real place
e.g. alcohol has no real effect, only placebo
10: Maximum surprise
Anything deemed "impossible"
e.g. from the window I see a chicken the size of a house, speaking perfect French
e.g. I wake up tomorrow inside my wife's body
I use the above as guideposts to be able to ask, "Would X be as surprising as a major house fire or closer to a major heart attack?" (i.e. is X a 5 or a 6 Surprised?) Also of course some of these will shift as time passes, e.g. as I get older a heart attack becomes less surprising, or if I discover my brother to be lying about lots of other important life events then I will be less surprised if he has a secret love child. Etc etc. I've argued with people about a few of the placements, but I think it tracks my subjective reactions; your results may vary (and it'd be very interesting to see other people marking similar scales with their own examples!).
Note finally that Surprised doesn't directly imply Upset; these are at least somewhat independent axes. In the examples I tried to include one that was negative-valence and one positive-or-neutral-valence, but the degree of correlation is not zero and it's hard to imagine being 9 Surprised by something and 0 Upset by it; the fact of being very surprised is at least a little bit upsetting in most cases. As you might have guessed, therefore, I do play a related game of How Upset, but it has proven much harder to calibrate (and also more dark: 10 Upset is some pretty bad stuff you guys). If I get around to it and it seems interesting then it'll appear as a follow-up to this at some point.