Wikipedia considered harmful ... or not

In an old post on Wikipedia, I said that

[I]t's easy to spot a backwater article that hasn't seen a lot of editing. This is not necessarily a bad thing. Obscure math articles, for example, tend to read like someone's first draft of a textbook, full of "Let x ..." and "it then clearly follows that ..." The prose may be a bit chewy, but whoever wrote it almost certainly cared enough to get the details right.

My feeling was that if you were really interested in, say, the functoriality of singular homology groups,  you'd probably have enough context to chew through prose like "This generality implies that singular homology theory can be recast in the language of category theory."*

In a recent ars technica article, John Timmer argues that impenetrable technical articles are actively harmful: "The problematic entries reinforce the popular impression that science is impossible to understand and isn't for most people—they make science seem elitist. And that's an impression that we as a society really can't afford."

I think that's a good point, but I'm not sure how bad the problem really is in practice.  A hard-to-read article is most likely to be harmful if a lot of people are seeing it, which also makes it more likely that someone will be able to improve it.  This is a fundamental assumption of Wikipedia in general, I think.  As such, it would be interesting to see some data behind it -- is there a strong correlation between the number of times a page is landed on and the number of edits (or edits per word, or such)?

Assuming that correlation holds, then someone coming to Wikipedia to learn about, say, physics should have a good chance at a gentle introduction.  Let's try:

• The main article on Physics seems like a perfectly good Wikipedia page.  It starts with a general introduction, goes into history, core theories, relation to other fields and so forth.  Let's look at one of those fields:
• Condensed matter physics still seems to be in good shape.  The first sentence doesn't seem completely useful at first: "Condensed matter physics is a branch of physics that deals with the physical properties of condensed phases of matter," but the next paragraph goes on to explain nicely what a condensed phase of matter is.  The rest of the article continues in a well-structured way to give an outline of the field.  Let's look at one of the theoretical aspects:
• Symmetry breaking "needs attention from an expert in Physics".  I'd agree with that assessment.  The general idea is still there, but we're definitely getting technical: "In physics, symmetry breaking is a phenomenon in which (infinitesimally) small fluctuations acting on a system crossing a critical point decide the system's fate, by determining which branch of a bifurcation is taken."  For example, what's a bifurcation?  Well, we can at least chase the link and find out:
• Bifurcation theory is actually a better-structured article, but no less technical.  It's actually a math article, not a physics article.

I would say that either of the last two would be intimidating to a non-physicist/mathematician.  I don't know if you could say the same about the first two.  Yes, there are still technical terms and concepts, but it's pretty hard to get away from that and still cover the material.  I would also say that a non-physicist interested in physics in general would be far more likely to land on the Physics article than the other three.

I also noticed that, while I have run across a few really impenetrable technical articles in Wikipedia, it didn't seem -- in this particular random walk, at least -- that the quality of the articles dropped off steadily as one went off the beaten path.  Fields intersect, and perhaps you're never too far from someone's beaten path.  I did chase a link from Bifurcation Theory to Stationary point, which was marked "The verifiability of all or part of this article is disputed", not something one expects in math articles, but it didn't seem particularly better or worse than the previous two, the warning notwithstanding.

Let's say that the random walk above is fairly representative -- and I think it is, based on other experience browsing Wikipedia.  What of Timmer's claim that general interest articles such as The Battle of the Wilderness are accessible, while technical articles such as Reproducing kernel Hilbert space are hostile?

I suspect Selection bias.  Timmer (and myself, and anyone who browses a lot of technical articles) sees a lot more technical articles than the average reader.  In fact, we should broaden that a bit and say "specialized" instead of "technical".  Just as math geeks might read a lot of math articles, history geeks will read a lot of history articles, sports geeks a lot of sports articles and so forth.  Should we really expect Reproducing kernel Hilbert space to be as newbie-friendly as The Battle of the Wilderness when the battle site has a plaque on a public highway talking about it?

Let's try following History like we did for Physics above, since Timmer's example of an excellent article is from that field.  The first area of study is Periodization, and already we see a notice that the article's sources remain unclear.  A large chunk of the article covers Marxian periodisation, giving the almost certainly false impression that this is the only significant way that historians divide history into periods.  This section seems to be a copy of the corresponding section in Marx's theory of history, suggesting that one of the most basic Wikipedia cleanups -- making sure that the bulk of the information is in one definitive place and everything else links to that place -- hasn't been done.

Just as impenetrable math or physics articles may give the impression that scientists are a bunch of elitists, the article on Periodization -- the second history article I tried -- may give the impression that historians are a bunch of leftists.  So maybe Wikipedia as a whole is unfriendly to academics as a whole?  That would be a sad irony indeed, but I doubt that's really what's going on.  Wikipedia is filling more than one function, after all.  It's a general introduction in some places, and a specialized reference work in others.  That seems fine.

There's a lot more to be said here.  Maybe survey articles like the ones on physics and history are the wrong place to look -- not that we've really defined what we're looking for.  I tried putting "The battle of" into the search bar and clicking on a few of the battles that came up.  All the articles seemed quite well written.  Perhaps the Wikipedia process works better for documenting specific events?  Or perhaps the search bar was just showing me the most popular articles, which in turn would be popular for being well written and well-written for being popular?

Overall, the proposition that Wikipedia articles on the sciences are bad for the sciences seems like a testable hypothesis, at least in principle, but properly testing it requires a lot more machinery -- methodology, statistical models, surveys, etc. -- than you'll find in a blog post or op-ed.  Without a more thorough study, we should take assertions like Timmer's as good starting points, not conclusions.

* I should point out that the article I linked to is not necessarily the kind of article that Timmer is complaining about.  I just picked a handy example of a technical article that would probably not be too familiar to most readers.  Yep, I ... just happened ... to be reading up on singular homology theory over my winter break.  What can I say?

Real science, hot off the web

A while ago I commented on an Economist article claiming that Web 2.0 tools "were beginning to change the shape of the scientific debate."  My contention was that the web wasn't so much changing the debate as changing the means of publication.  In particular, there had always been a trade-off between speed of publication and thoroughness of review, and the web was becoming a publishing mechanism of choice on the lightly-reviewed end of that continuum.

More recently, looking for something I no longer recall, I ran across Cornell's arxiv.org (I assume the x is meant to represent a Greek χ), a repository for "Open access to 703,281 e-prints in Physics, Mathematics, Computer Science, Quantitative Biology, Quantitative Finance and Statistics."  The number 703,281 was current when I scraped it. It's probably higher by now.

That's a lot of articles, but does anyone use it for anything important?  Well, one recent entry is Superluminal neutrinos in long baseline experiments and SN1987a (Cacciapaglia, Deandrea, Panizzi et. al., yes, those neutrinos).  Indeed, there seems to be a lot of activity in the experimental high-energy physics section overall, which makes sense.  It's useful to have experimental results available quickly, bearing in mind that there can be quite a bit of calibration, number-crunching and checking before an experimental paper is ready for public consumption (months, in the case of the neutrino paper).

Submissions to arxiv.org "must conform to Cornell University academic standards".  It's not immediately clear to me what process is in place to ensure this, but from a little browsing it's clear that these are serious academic papers.  It also seems reasonable to assume that most of the papers have not been through the full process of peer review required for publishing in a major journal.  Indeed, the published version is almost certainly not going to appear on such a site, if only for reasons of copyright.

It seems like a good niche to fill.  If you have a significant result that you're comfortable sharing with the world and staking your reputation on, there should be some way to make it immediately available, with the implied tradeoff between speed and thoroughly careful vetting.  Publishing under the aegis of a major university gives everyone some assurance that you're at least doing real research.  I notice from random sampling that the reference sections generally don't cite arxiv.org, giving some indication of the preliminary nature of the publications.

With that in mind, arxiv.org looks like a great resource not only for working academics but for the curious layperson as well.

Will web 2.0 "change the shape of scientific debate"?

The subtitle of this piece in The Economist set my spidey-sense tingling: "Web 2.0 tools are beginning to change the shape of scientific debate."

"Which web 2.0 tools are they talking about," I wondered, "and how are those tools changing scientific debate?" I also picked up notes of "it's all different now." (I'm a technologist by training and practice and a moderate technophile by inclination, but I'm also convinced that technology use reflects human nature at least as much as it changes it.)

Come to find out, they mean that people are -- in some cases -- bypassing journal publication as a means for publicizing results, and there is now a meta-blog, Research Blogging, aggregating research blogs. Cool stuff, and well worth a browse, but is it all that big a change?

As I said, I'm a technologist, not a scientist. The scientists I've known, I've known on a personal, non-professional basis. I won't claim to be an expert on the workings of the research community, but as far as I can tell there have always been several different levels of scientific information exchange, ranging from beer-soaked discussions at conferences through personal correspondence to publication in peer-reviewed journals.

My experience as a former member of the ACM is that there are also various flavors of publication: some are lightly reviewed and aimed at speedy publication of interesting ideas (e.g., SIGPLAN Notices) while some have heavyweight formal review processes and are aimed at publishing major developments in definitive form (e.g., Journal of the ACM). Now, one can argue whether computer science is a proper science, but as I understand it computer science research has much the same form as other areas of academic research.

From this point of view, a site like Blogging Research probably fits in between personal correspondence and publication in an unreviewed journal. More a filling in of a niche than a major shift in the landscape.

It also seems good to bear in mind two general trends: First, the overall volume and speed of communication has been steadily increasing for centuries, as has the number of people one can potentially communicate with easily. Second, the scientific community has been growing and becoming more specialized.

In Marin Mersenne's day, one could argue that the (Western) scientific community numbered in the dozens or hundreds, and Mersenne personally corresponded with a significant portion of them. Over time, communication has improved, the community has grown and the overall volume of scientific writing has increased dramatically. From this point of view, the adoption of email, newsgroups and now blogs is just part of a natural progression. For that matter, so is the development of the peer-reviewed journal.

The shape of scientific debate has more to do with the process of research and collaboration, I think, than with the particular means of communicating findings and theories. In that sense, I wouldn't expect the shape of debate to change much at all in the face of web 2.0 or web x.0 in general.

The article does point out an amusing example of the cobbler's children going barefoot, though:

[T]he internet was created for and by scientists, yet they have been slow to embrace its more useful features ... 35% of researchers surveyed say they use blogs. This figure may seem underwhelming, but it was almost nought just a few years ago.

(On the other hand, there were a lot fewer blogs of any kind a few years ago ... now how would one properly account for that statistically?)

[Not long after this was posted, ArXiv came along -- D.H. Dec 2018]