Predicting violence: the Seung-Hui Cho case

[Bill Glasheen]

I found this article in the Health Blog of the WSJ. Very interesting... It took a long time to provide all the hyperlinks in the original article, but they are there.

The bottom line from this nice piece of work is that mental health per se isn't a good predictor of violent behavior, or at least to the extent that information is stored in typical paper mental health records. Other factors appear to be more predictive. Only combining mental health conditions with recreation substance abuse is there an increased risk for violent behavior (a statistically significant interaction).

I'll wager there's a chance that the conclusions reached suggest that the field if Mental Health has a long, long way to go. If they can't be a part of identifying the propensity for violent behavior and modifying said risk, then do we perhaps have serious holes in the field of behavior health and mental health care? Food for thought.

- Bill

August 20, 2009, 9:43 AM ET

Violence Prediction and the Records of the Virginia Tech Shooter

By Shirley S. Wang

The mental health records of the Virginia Tech gunman unveiled yesterday don’t reveal much at all about why he killed 32 fellow students and himself. The records (online here and here) depict Seung-Hui Cho as a fairly ordinary depressed and anxious student and don’t give any clue about the rampage he would go one just a year and a half later.


Photo via Associated Press

That’s not surprising because predicting who might commit violence is really hard. After studying the subject extensively, researchers have identified certain signs associated with violent behavior. But even so, the ability to predict whether any one person is likely to be criminally violent is very low.

One review of the literature, now 10 years old but still referred to frequently, found that the best predictor of violence were the same for people with and without mental illness: whether they had a criminal history. Factors related to mental illness were less important to the equation.

A large study published in February in the Archives of General Psychiatry also found that mental illness by itself doesn’t predict violence. A past history of violence or substance use, or current stressful event like divorce or job loss, were more helpful in predicting violence than just knowing whether people had a mental illness, according to the examination of data from more than 30,000 people. But those who had a mental illness and used recreational substances had a greater likelihood of committing violence compared to those with a mental illness and used no substances, researchers found.

Of course that doesn’t mean that every person who fits those characteristics is going to be violent.

For more on the Virginia Tech gunman’s records, read coverage from the WSJ, Washington Post, and New York Times, ABC News and the Roanoke Times, among others.

[Bill Glasheen]

One review of the literature, now 10 years old but still referred to frequently, found that the best predictor of violence were the same for people with and without mental illness: whether they had a criminal history. Factors related to mental illness were less important to the equation.

As a person who professionally does predictive modeling and risk assessment, I got to thinking about this. And then suddenly it hit me. It's like saying the best predictor of future health care costs (from a single variable point of view) is prior costs. Duh!

While this speaks to recidivism and such, it also to some extent is polluted by the fact that the definition of violent behavior may be closely related to a conviction of violent behavior. So let's just say that this is to some extent an "uninteresting" result. What we'd like is a series of indicators which - when put into the right statistical model - will explain a good deal of the variance vis-a-vis a propensity for violence. And you'd like (if possible) for that definition to be independent of the likelihood that this person will get caught by the legal system.

On the other hand, this is to some extent an example of Pareto distribution behavior. Centuries ago, Vilfredo Pareto observed that a small number of people (in Italy) owned most of the wealth. His measurement point was 20/80, but the distribution is fairly predictable throughout the spectrum. So in this case, we see that a small number of people are responsible for most of the violent behavior. I'm thinking that the 80-20 rule holds here as well.

- Bill

[IJ]

Bill, this is no news at all to anyone in psychiatry who's actually thought about the issue. When they are asked to predict suicide (a far more common situation), they do ask about plans, if any, whether the desire is active or passive, or vague ("I just wish I could go to sleep"), gun ownership and other means, a history of attempts, how serious those were (eg shot self in head is more concerning than took 2 advil and scratched wrist with plastic knife), underlying condition, acute stressors, and all sorts of other stuff down to pet ownership.

Does that actually work? If you look at any one of those factors, there may be some odds ratio in the literature that supports its use, and these features are likely to impact the psychiatrist's call, but the only thing it comes down to when you have to decide to deprive a patient of their freedom and forcibly hospitalize them or let them go is your overall clinical judgment. And it's not great. I've seen people hospitalized who had a long history of WORSENING in house who were on a do-not-hospitalize list because of it (the attention provokes crazy behaviors). Psych occasionally releases someone for a cigarette break or discharge whose first act is a suicide within hours. One dove from the parking garage while I was on call for the MICU on Christmas (no, we cannot revive people after 10 stories of freefall) and another died an hour after discharge in Charlottesville, after being cleared by the trainee shrinks and their attending and showing steady progress and stability. We're not that good.

In retrospect, if Alan Greenspan can sign on, even partly, to the idea that housing prices could legitimately skyrocket after decades of slow increases, and the result of the bubble collapsing is the Great Recession, well, how good could any one doctor be predicting the behavior of individuals they can only spend 15-60 minutes with maybe once maybe a few times, acting only on those thoughts and concerns the patient is willing to divulge, or unable to conceal?? It's a heckuva lot like our inability to really know what WMD Saddam had, and how many experts were weighing in on that one??

This all reminds me of the books "Fooled by Randomness" and "The Black Swan." Occasional disasters can't all be foreseen or prevented.

[Bill Glasheen]

Ian

These kinds of hard questions need to be asked. And I'm glad someone published this. If the king isn't wearing clothes, well then let's call it for what it is and throw a blanket over him.

I don't know about you. But in my business world we wouldn't sell squat if our models were this pi$$ poor.

I'm not damning the field of mental health here, Ian. But I am saying that - in spite of their best efforts - this research shows that psychologists and psychiatrists aren't getting the kind of information they need to make informed decisions when it comes to protecting the patient and society. So what does it mean? I don't know yet. It could mean that they need to start asking different questions and collecting different kinds of data. It could also mean that the information just isn't there in the first place. It may be that such violent events are mathematical chaos in real life.

If I was the czar of the psychiatric field, I'd start looking to see if there were certain INDIVIDUALS who were good at predicting violence. If you can identify such people, then maybe you have leads to pattern recognition processes yet to be discovered. And if not... then maybe we should stop letting said people have power over who gets released and who doesn't. Keep it concrete, like...

Do this and you spend the rest of your life in jail. That is if you don't get shot before we catch you.

It doesn't need to be any more complicated if you can't improve on that. Personally I think they can do better, but so far the field has nothing to show for their research on violence prediction. If it comes down to recidivism happens and drugs are bad, well... any LEO with an IQ of 100 can tell you that.

Food for thought.

- Bill

[IJ]

Let's not lump in "the field of mental health" and "violence prediction." The future is tough. Psychiatrists--you know, good ones, like my partner--can tell you what percent of people with what symptoms will improve with medications vs placebo or stay the same no matter what, but they can't tell you if a drug WILL work in any one patient. That's the best you can get right now.

Are there any individuals good at predicting violence? I wonder how one would know. There's very little of it, and there's lots of psychiatrists running around. I mean, we have blood tests for prostate cancer and we can still tell very little about which men with higher PSA actually have prostate cancer, much less prostate cancer in need of treatment. Thus, the positive predictive value is quite low. What would be the case for psychiatrists all of whom occasionally do homicidality or suicidality evaluations on a patient or two, when the risk of a violent event is very low? A very few people will do something, the psychiatrists will probably miss a good deal of them, the ones they do keep will probably stay a few days and be released when their hold expires anyway, and then may do their thing anyway. Meanwhile many others will be locked up short term and would never have presented a risk--but it will be hard to prove, because their brief hospitalizations may have prevented something--conceivably. It'll be near impossible to do a trial using algorithms that exclude judgment in the ER, thus difficult to study systematically. The only way to know someone guessed right that someone was a real violence risk is if their call is to keep em and they're released and shoot up another college. How often will that happen? Meanwhile any attempt to increase hold rates will lead to outcries about the abuse of power and prejudice against the mentally ill, pointing out, rightly, that they're not really much more prone to violence. And after a program succeeds in locking up a bunch of people and the legal battles are ongoing, there will be no way to measure success because the odds of violence are so low the actual rate to expect is zero and will probably remain so. Imagine if we had an antiasteroid program in place for the last 10 years-did it help? Now imagine guessing if you didn't even know if there had been asteroids headed your way.

After all that fuss, then some new George Sodini will shoot up a gym because he's been a loser his whole life, and we'll wonder how we could possibly have intervened there without threats, seeking self help, or prior events.

At that point, the NRA will suggest that all public venues ought to have a few trained people with concealed weapons to handle the unforeseeable

[Bill Glasheen]

Ian

I'm not sure you're disagreeing with me here.

Indeed the PPV for violence with the present models suk. We know more outside the mental health arena about factors contributing to the propensity for violence (prior conviction, recreational drug use) than we do from what a mental health professional adds to that body of information, no? So... you prove my point. Why are they uniquely qualified to recommend against release on the basis of someone being a danger to others? Right now we see they aren't.

So do mental health researchers say "Gee, we can't do this!" or do they perhaps advance their field a bit more? Like... a lot maybe?

I'm just saying.

And we're not supposed to even try to make predictions on an individual level? Why not? What use are these models if you can't? I certainly am involved in other predictions (based upon health care data) on an individual patient level. It isn't easy, but then I guess that's why they pay us to do it.

Sure these are rare events, and so by definition more difficult to develop models for. Sure we can't do controlled studies. Etc., etc. But I build models on experiential data all the time. You take what you can get and run with it.

No need to defend a field in infancy. Do you think your friend would defend Freudian psychoanalysis? Why make excuses for similarly embryonic work? It is what it is.

- Bill