Retrospective Predictions

I haven't had a chance to read Nassim Nicholas Taleb's book "The Black Swan: The Impact of The Highly Improbable" yet, but I thought that Niall Ferguson's piece in the Telegraph last week, on it and how it relates to the coverage of the Virgina Tech massacre, was pretty eye-opening. The key insight is that there is a human cognitive bias which prevents us from appreciating improbable events - we tend to conflate improbable with impossible - and we are also naturally predisposed towards creating narratives or "retrospective predictions" for such events. It's easy (or rather facile) to see *now* the sequence of events leading to Cho's rampage and it's an easy trap to fall into to (incorrectly) assume that this sequence should have been obvious before the massacre.

By chance a similar discussion about a "family annihilation" has been taking place here in Ireland: Adrian Dunne killed his two young daughters and his wife (the official line is that she wasn't herself involved with the planning and implementation of these killings) before hanging himself. Most of the debate centres around what could and should have been done by the authorities to prevent this tragedy: Dunne had visited a funeral home shortly before and had ordered four coffins and given detailed instructions for the funeral in the event that an "accident" took place. The popular, and in my view incorrect, assumption seems to be that this event was utterly predictable given the (now) compelling narrative leading up to it.

Share this

Case in point

The arrest, in the wake of the VT massacre, of 18 year old close-cropped asian male Allen Lee for having written an essay containing disturbing statements. It appears to me that the authorities behind the arrest have build up a Compelling Narrative Predicting the VT Massacre and then applied it to Allen Lee.

(It could also be that college-age asian males with close-cropped hair have been arrested for disturbing essays on a monthly basis for years, decades, and that the media have merely been sensitized to the arrests by the VT massacre. But at the moment this appears to be a new thing, a post-VT thing.)

And what's especially

And what's especially regrettable is that as such events are improbable "pro-active" action like this will - as I noted in my comment at orcid.net - tend to have an overwhelming skew towards false positives. Let's say that there's a one in one hundred chance (and it's got to be even less probable than that) that such "disturbing-essay-writing" close-cropped asian males will go and carry out a massacre, you're then looking at 99 times out of one hundred, you've unjustly arrested an innocent guy.

Rare events

We don't underestimate rare events, we are just very bad at estimating probabilities... there is a list on wikipedia of the cognitive bias associated to this. We tend to have an internal probability we associate with an event that we modify as we receive information and which is very susceptible to hysteresis, we tend to estimate probabilities bases on "how easy it is" to image a specific scenario happening. For example we are generally more afraid of sharks than falling airplane pieces although the later are more likely to kill us... similarly, we buy lottery tickets because it's easy to imagine what it'd be like winning the lottery.

This is correct, but for me

This is correct, but for me what's eye-opening about Taleb's thesis is how he puts together this "mis-estimating" with hindsight bias: we use this to "explain away" the error to delude ourselves that we are not error-prone

The key insight is that

The key insight is that there is a human cognitive bias which prevents us from appreciating improbable events - we tend to conflate improbable with impossible - and we are also naturally predisposed towards creating narratives or "retrospective predictions" for such events.

I'm not sure if this is the same thing you're saying above, but having talked to a lot of people close to VT in the aftermath of the tragedy, I think the worst policy outcome for such improbable events is people trying to make sure they don't happen again. As far as I can see, VT shouldn't change a thing. No new policies about guns, counseling, locking doors, metal detectors, etc. This won't happen again, at least not at VT. It was a "singularity". Don't ruin a good thing.

I'm guessing nearly everyone associated with the university would disagree with me.

Yeah, I think the

Yeah, I think the implication of Black Swan that a) this is extremely unlikely to happen again and that b) there will indeed be some other completely unpredictable tragedy somewhere that we won't be able to prevent. It's hard for people to accept this - as Ferguson puts it, it's easier to live without this insight than with it - which is why the "Never again" meme is so widespread. This is the first reason why we shouldn't introduce new policies aimed at making sure it doesn't happen again - it's going to be futile. The second reason why not is back to Bastiat's seen and unseen - all the myriad costs and downsides, inconveniences and outright injustices just don't tend to enter into our calculations.

Goes both ways

I would be wary of drawing hasty lessons. On the other hand, when I read that we can't draw any lessons - as some in the media have stated or implied (e.g. The Daily Show) - that itself seems hasty to me.

I don't think it is possible

I don't think it is possible to draw useful lessons from "Black Swan" events except the lesson that we shouldn't seduce ourselves into thinking future events will be predictable and preventable. I also don't think it's appropriate to be angry at anyone (other than the perpetrator of an event such as this or the family annihilation) which is, unfortunately, the most common reaction.

I think it's worth drawing a distinction between these improbable tragedies and the repeatable, predictable tragedies such as, say, those associated with the Drug War. Now those are worth getting angry about.

It is not a "black swan"

It is not a "black swan" event in every respect. It is not the first murder. Therefore it is a data point in a vast sea of data points as a murder. It is a mass murder, and it is not alone as that either. It is a mass murder on a campus, a mass murder among a population known in advance by the murderer to be disarmed, and not the only one. It is a mass murder in which the murderer intended to send a message, and not the only one. It is not the first or last murder in which the murderer sent the particular message that he sent.

In fact it is predictable. It is [edit: not] a one-shot deal but rather it is a kind of event that does occur very rarely. There is no difference between this event and the deaths from the drug war, but a difference of degree. Campus killings are infrequent. Mass murders are infrequent. That is not a difference of type but of degree. [edit: Campus mass murders are not individually predictable but then neither are individual drug war killings. The difference is not predictability versus non-predictability of individual events, but a difference in frequency, a difference of degree.]

It is part of many patterns that have occurred and will recur. The difference is not that it is a one-shot event, but that it belongs to various sets of events many of which are rare but not one-shot, merely infrequent (campus killings). Some of which are not rare at all (gun killings).

Now, if you want to argue that campus killings are too infrequent for it to be cost-effective to guard against them, that's fine, but then you aren't reasoning from its black-swanness, but reasoning from its non-black-swanness, since you are appealing to the kind of event it is and appealing to the very low frequency of such events (campus killings). You are looking at the pattern and reasoning from the pattern. Even in that reasoning it is not a black swan event, one from which no lessons should be drawn.

Now where I agree with you is that we need to be careful about probabilities. We need to be careful about the conclusions that we draw from narratives, even valid ones.

A narrative is a chain of events A followed by B followed by C followed by D, etc.

What makes a narrative comprehensible and interesting and not a random collection of events is that there is a real causal relationship between A and B and C and so forth. Typically, A makes B more probable than otherwise, B makes C more probable, and so on. However, even in a valid narrative, the probability of Z given A is easily far too low for us to be scared about A. So it would be a horrendous mistake to draw the conclusion, "A, therefore Z". The wicked queen attempted to murder Snow White because Snow White was more beautiful than the wicked queen, in the opinion of the mirror on the wall at least. A (Snow White is beautiful) increased the probability of Z (the wicked queen attempted to murder her). It is a narrative that makes sense. But it would be the height of folly to conclude that if one woman is beautiful, then less beautiful women will try to murder her and should be arrested.

That I agree with. However I do not agree with the blanket judgment of the recent mass murder as holding no lessons for us at all. That seems too hasty a conclusion.

No, it is indeed a "black

No, it is indeed a "black swan" event, an improbable (not impossible) event - the point being that out of all the alienated close-cropped students only an extremely tiny minority do go ahead and start a massacre. Our hindsight bias picks up on all these salient points leading up to the massacre - Cho's personality, his writings. But this sort of stuff happens everyday and doesn't lead to massacres.

It is pointless to draw some sort of equivalence with the side effects of the drug war because they are predictable: Sentence a young man on a drugs charge and there is a high probability that he will get raped. Engage in military style no-knock raids and there will be deaths, the probability might be lower than those for prison rape but it will be significantly more probable than a campus massacre and as a direct result of an active and tolerated policy rather than as a consequence of neglect of some hitherto unknown possibility.

In theory some lessons might be drawn, in practice the notion that "lessons can be drawn" will overweight the salient points of the narrative and draw incorrect conclusions about the likelihood of future events and the ability to predict and forestall them.

If it is a black swan event

If it is a black swan event then your reasoning about black swan events is invalid. I was basing my tentative definition of 'black swan' on your reasoning, which is that "I don't think it is possible to draw useful lessons from "Black Swan" events except the lesson that we shouldn't seduce ourselves into thinking future events will be predictable and preventable."

My tentative definition of 'black swan' event is one which merits that conclusion. If in fact 'black swan' merely means 'improbable' then it does not merit that conclusion. For, the proper reasoning about the college shooting is identical to proper reasoning that we can do about killings from the drug war. For example, I could reduce my probability of dying from a stray drug war bullet if I wore a bullet-proof vest every day. However, in my estimation the frequency of killing in the drug war is insufficiently high to merit the significant cost to my comfort of wearing a bullet proof vest every day. And exactly the same form of reasoning applies to college campus killings: they are insufficiently frequent to merit any preventive measures that have a significant cost. The frequency is perhaps orders of magnitude less, but the form of reasoning is identical.

You contrasted killings in the drug war with the college campus killings, calling the the former "repeatable, predictable tragedies" and the latter "black swan", later explaining that "black swan" means only "improbable", which is no contrast at all. The form of reasoning that applies to both is identical. In both cases we can reason the same way to rightly conclude that certain proposed preventive measures are not worth the cost.

It doesn't mean "merely"

It doesn't mean "merely" improbable. I think the formulation Taleb uses is "highly improbable" and I'd stand over the assertion that this was indeed a highly improbable event - there are thousands of Cho-type guys on campuses all over America who don't go out and shoot a bunch of people.

The pertinent contrast with drug war deaths and prison rapes is that these directly result from a policy which is tolerated knowing full well that these deaths are probable outcomes. Everybody knows that prisoners are raped, everyone knows that a heavyhanded military action against drugs results in such deaths - they don't just come out of the blue like the VT massacre or, say, 9/11 the classic black swan event.

I restate that no particular

I restate that no particular jail death is individually predictable (unless e.g. you get inside the head of the particular killer - but the same would apply to Cho). What is (at least in principle) predictable is the overall change in probability of killing that results from a change in drug law, which will show up in a change in the frequency of death.

The same can be said of killings like the recent campus killing. Even here you can in principle predict (and over time see) the effect of various laws - not on whether a particular person will commit a particular crime on a particular day, but on the frequency. It will take a really, really long time if you restrict your attention to Virginia Tech campus and further restrict your attention to quiet Asians, but if you include it into a larger class of events (and there are many larger classes into which the Virginia Tech massacre can be fit), you may be able to see an effect on frequency fairly soon.

I like your blog entries by the way, including this one.