Saturday, April 16, 2005

Expertise Matters

A few weeks ago I proposed a theory as to why so many continue to cling to the idea that Saddam Hussein possessed weapons of mass destruction immediately preceding the American-led invasion. While this was far from the best thing I had ever written for this site, I felt that it was worth at least a little attention. However, I admit that there was some hesitation on my part. I knew that many might find the substance of that post to be rather irritating and that it was quite likely that I would experience some blowback. After a few minutes considering the pros and cons, I decided to enter it in the Carnival of the Vanities. I mean, really. How bad could it be?

As it turns out, not that bad. A lot of people read the post and I didn't get any death threats. But, at least one gentleman let it get under his skin and he laid into me in the comments and own his own blog. Alas, rather than address the subject of the post, the discussion largely digressed into a debate over the existence of WMDs. But, beggars can't be choosers. If that's what a reader wants to discuss, who am I to argue?

Anyway, during this little debate, my adversary (unintentionally, I think) revealed that he was unfamiliar with the conclusions of the Duelfer report. Since this document is essentially the final word on the subject, I didn't see how we could have a rational discussion until he was up to speed. In light of this fact, I recommended that he read this article, published in the Washington Post.

I realize this is a lot of prelude, but here comes the interesting part. Since this gentleman is of the conservative persuasion (he considers FOX News to be the most evenhanded major news source), he wasn't about to let the Washington Post be the final word on the issue. Therefore, he tracked down the actual Duelfer report and read it. Then, he posted an extensive summary of the report, arguing that the Washington Post had "shamelessly cherry-picked" quotes from the report in order to distort its meaning.

Now, if you didn't actually click through to the extensive summary, you should because that post is going to provide us with some context moving forward. As you can see, my friend has constructed a very compelling rejoinder to my claim that the WMD issue is settled. Moreover, he has quotes directly from the Duelfer report backing his position. I'm sure that many would read that post and come away believing that the jury is still out. Hell, after reading that post, I almost believe it.

In the end, though, I don't. And neither should you. Here's why.

The Duelfer Report is not exactly what you would call a thrifty read. It tops out at approximately 1000 pages (the complete PDF files total 200 MB). So, as you can see, when they refer to it as a comprehensive report, they aren't kidding. It's an extremely thorough examination of the WMD issue, truly leaving no stone unturned. This is, of course, exactly as it should be.

Unfortunately, when documents get to be this size, it starts to become difficult to summarize their meaning. When so much is said, how does one decide what's really important? Moreover, how easy is it for someone with absolutely no expertise in the subject to interpret the document's findings?

Not easy. Not easy at all.

This is a problem that I am starting to notice more and more in the blogosphere. In this modern era, we have access to enormous amounts of raw data. We also have at our disposal incredibly powerful tools for finding and evaluating this data. And finally, we have a medium in which to publish our conclusions. But, with all this power at our fingertips, is there any meaning to what we are producing? Or are we just a bunch of nimrods with cable modems and free time. Sometimes, it's difficult to tell.

On the one hand, I have to commend my conservative friend for actually getting into the trenches and doing some real work to back his position. Far too many people are willing to hold convictions on faith, sometimes even refusing to consider evidence brought to their doorstep. But, at the same time, you have to know your limits. A few weeks ago, when I was writing about the Lancet study regarding excess Iraqi deaths postinvasion, I faced a similar problem. I read the report (only about 10 pages) and I know more than the average bear about statistics. However, I don't know much about epidemiology or about cluster sampling. Therefore, I spent a lot of time reading what experts had to say about it before I presented my conclusions. To paraphrase Rumsfield, there are things that I know I know, things that I know I don't know, and things that I don't know that I don't know. Or something.

The bottom line is that, in the blogosphere, with the knowledge and tools at our disposal, almost anyone can seem like they know what they're talking about. However, the world is an incredibly complicated place and access to information does not directly correlate with comprehension. Often times it takes years of training and experience to extract meaning from a given data set. There was a time not long ago when that training and expertise would be a prerequisite for seeing the data in the first place. The fact that that is no longer true is mostly a good thing. I'm all for the widest distribution of information possible. But it means that we can no longer assume that the analyst knows more than we do. The presentation can be sharp, appealing, and loaded with raw data -- but still be completely wrong. Unless you're the expert, there's almost no way to know for sure.

With respect to the Duelfer report, it turns out that the authors anticipated this problem. As part of the full report, they published a short summary of key findings for each section (which, conveniently, can be downloaded separately (PDF)). It's sort of a Duelfer Report for Dummies, an ideal companion for nonexpert bloggers like myself. It's only 19 pages and is much easier to understand. And it shows how far off my conservative friend’s interpretation of the full document actually was.

But, I don't want this to come across as a slam against this gentleman. Sure, on the one hand, as consumers of blog-product -- buyer beware. On the other, though, blogger beware. This gentleman is far from the only one to get in over his head (I'm looking at you Powerline and Daily Kos). I've tried to avoid this particular pitfall, but if I have so far succeeded, I am sure it is only a matter of time before my intellectual hubris gets the better of me. No doubt, my downfall is quickly approaching.

So, remember: expertise matters. We've all got megaphones now, but all that means is that we're really fucking loud. Unless we really know what we're talking about, we're doing no more than broadcasting our ignorance. And that might make us feel good, but in the end, is that really why we're out here?
Weblog Commenting and Trackback by