Some unspecified quasi-Darwinian process will assure that those writings and editings by contributors of greatest expertise will survive; articles will eventually reach a steady state that corresponds to the highest degree of accuracy.

He then goes on to say:

Does someone actually believe this? Evidently so. Why? It’s very hard to say.

Actually, I don’t believe it is so hard to say, and will go into detail on the matter shortly.

McHenry then goes on to theorize that the currently-vogue educational technique of “journaling” is responsible for corrupting the thinking of today’s youth, consequently leading them to believe in something as ridiculous as the success of collaborative commons-based projects. I can only surmise that this bizarre tangent is due to a pet peeve of his, and believe it can be safely discarded.

Our friendly author then takes a stab at empirics. His method is to sample a number of Wikipedia entries, inspecting their previous versions and revision history to ascertain whether the quality has increased or decreased. The size of McHenry’s sample set is: 1.

At this point, it is worth noting that Wikipedia recently reached the 1 million article mark.

About this dubious empirical method, McHenry says:

…I chose a single article, the biography of Alexander Hamilton. I chose that topic because I happen to know that there is a problem with his birth date, and how a reference work deals with that problem tells me something about its standards.

So in other words, he is quite cognizant of the fact that he is about to make an induction from one article to one million, based on a degenerate case. I will temporarily leave the reader to make their own value judgment of this policy, and proceed within the bounds of McHenry’s game.

McHenry finds two problems with the Hamilton article. The first is that it gets a date wrong. The second is that it has declined in quality over time, at least, according to his standards. McHenry’s definition of quality seems to consist solely of presentational matters such as spelling, grammar, and text flow. These are of course important considerations, but I propose that there are other important facets of quality—for example, coverage. In a later section, I will attempt to sort out some of the confusion on this topic. For now, let us again take McHenry’s claims at face value, and proceed to the finale.

He concludes with a metaphor. I will reproduce it here in full (emphasis mine again):

The user who visits Wikipedia to learn about some subject, to confirm some matter of fact, is rather in the position of a visitor to a public restroom. It may be obviously dirty, so that he knows to exercise great care, or it may seem fairly clean, so that he may be lulled into a false sense of security. What he certainly does not know is who has used the facilities before him.

“Historical reviewability akin to a public restroom” describes traditional encyclopedias, not CBPP ones like Wikipedia

McHenry is essentially asking us to suspend all higher brain function at this point: in the above metaphor he simply pretends that the reviewability features he just based his entire analysis on do not exist.

A reminder is perhaps in order: to determine that his not-so-randomly chosen article declined in quality, McHenry used Wikipedia’s revision history feature to look at how it had changed over time. This feature of Wikipedia, which is a hallmark of open content production systems, makes it precisely the opposite of a public restroom. You can in fact see everything that “came before you” with Wikipedia.

Public reviewability would be _embarrassing_ to traditional content creators

What would McHenry’s metaphor apply more fittingly to?

Why, a traditional print encyclopedia, of course. If I wanted to analyze an arbitrary Britannica article’s evolution over time (for example), I’d have to somehow acquire the entire back catalog of the Britannica (assuming older editions can even be purchased), presumably reserve a sizeable warehouse to store them all, and block out a few days or so of my time to manually make the comparison.

Even the electronic forms of traditional encyclopedias are sure to be lacking such reviewability features. This makes sense, as public reviewability would be embarrassing to traditional content creators.

Viewing past corrections and changes are not exactly facilitated by this style of encyclopedia (but Wikipedia contributors have, admirably, not slandered them in spite of this)

So, in an artistic twist of doublespeak, McHenry has attempted to convince the reader that one of the key failings of traditional, closed media is actually the main problem with open, collaborative content. Presumably, love is also hate, war is peace, and so forth.

As if this was not enough, there were some other asides and general themes of his article that scream “FUD,” or simply boggle the mind by virtue of their illogic.

For example, McHenry makes a point that seems like it should, by all rights, completely discredit his own article. He says (emphasis added):

I know as well as anyone and better than most what is involved in assessing an encyclopedia. I know, to begin with, that it can’t be done in any thoroughgoing way. The job is just too big. Professional reviewers content themselves with some statistics—so many articles, so many of those newly added, so many index entries, so many pictures, and so forth—and a quick look at a short list of representative topics. Journalists are less stringent.

So in other words, no one can conclusively assess an encyclopedia. This is an odd thing to say before proceeding to assess an encyclopedia. McHenry also fails to meet even the approximate standard of his “professional reviewers,” unless you seriously consider examining one article to be statistically significant. Personally, I think routine use, amortized over millions of people, can actually shed some light on quality.