atw: Re: Testing Wikipedia
- From: "Anthony Self" <ASelf@xxxxxxxxxxxxxxxxxxxxx>
- To: <austechwriter@xxxxxxxxxxxxx>
- Date: Wed, 31 Aug 2011 14:57:34 +1000
Hi Graeme
Your suggestion that the only way to "test" Wikipedia reliability is to
vandalise it and see how long it takes for long-suffering volunteers to waste
their own valuable time (that could be spent on improving the quality of
content) to fix up the mess. (This reminds me of the Japanese approach to whale
research.)
I would suggest that Wikipedia's degree of reliability is reasonably well
established, not "unknown" as you suggest, because there have been sufficient
studies done to establish this. The links I provided are just two of many
conducted over many years. I tried to help you out by suggesting you try
"wikipedia reliability studies" in Google Scholar.
Wikipedia can test its own anti-vandalism protocols without you "helping" by
vandalising. According to a ProjectVandalism study from 2006, 4.61% of edits
was identified as vandalism, and the average time before reversion was 2.69
hours.
The study I pointed you too did indeed use the unethical vandalism approach.
However, it produced some statistics, which happily you have now read. You now
know that a third to a half of introduced "fibs" were corrected in 48 hours.
There you go, no need for the austechwriter list to repeat the same "study". (I
presume you were planning to help Wikipedia by passing on the "reliability
data" collected, or publishing it in an academic paper so that it could be used
as a reliable source? Perhaps not?)
How was my direction to the Counter Vandalism Unit patronising? You were
proposing vandalising Wikipedia. I was pointing out that you risk being caught
conducting this abuse, and the Counter Vandalism Unit provides some information
as to how you might get caught. There are over 2,500 people in this unit. They
use dozens of different software tools to help them locate vandalism, and
identify and block (ostracise) the culprit. As to your assertion that what you
were proposing was not vandalism, because you might delete the page after a
month, is nonsense. If I smash a few windows, wait for a month to see if
someone fixes them, and then fix up any outstanding damage, is that vandalism?
Wikipedia's definition of vandalism in the context of what you were proposing
(from Wikipedia editorial policies) is as follows.
Vandalism is any addition, removal, or change of content in a deliberate
attempt to compromise the integrity of Wikipedia. Examples of typical vandalism
are adding irrelevant obscenities and crude humor to a page, illegitimately
blanking pages, and inserting patent nonsense into a page. Vandalism is
prohibited. While editors are encouraged to warn and educate vandals, warnings
are by no means necessary for an administrator to block.
The only loophole is "good faith", and maybe that's how the Magnus study got
ethics approval. I can't say.
I am very happy to assume that you were acting in good faith, but that doesn't
preclude me from pointing out that what you were proposing fits the Wikipedia
definition of vandalism. And continuing with that assumption of good faith, can
I recommend another good read, namely "User Contribution and Trust in
Wikipedia" at www.ics.uci.edu/~sjavanma/CollabCom.
Tony Self
>>> Graeme Worth 31/08/11 2:45 AM >>>
Hi Tony
Date: Mon, 29 Aug 2011 17:37:12 +1000
From: "Anthony Self"
Subject: Re: Wikipedia - can we test it?
|Hi Graeme
|What you are suggesting (creating Wikipedia articles with deliberate errors)
is unethical. You would possibly be breaching the terms of use, and you would
certainly risk being ostracised. It is |known as vandalism or abuse.
So, taken in context with previous remarks, this roughly paraphrases as:
Wikipedia is a source of unknown reliability.
Wikipedia itself apparently has no interest in testing its own protocols for
reliability.
Anyone who suggests running such tests is classified as a 'vandal' and is
ostracised.
Brilliant - and this helps Wikipedia how?
|You may think you are anonymous, and therefore won't be caught. (And low risk
of getting caught shouldn't be a justification for commiting anti-social acts.)
It is important to realise that there is not |absolute anonymity on the
Internet. The staff of the Prime Minister's Department discovered this when
they were caught out making "bad faith" edits to the Wikipedia articles on
Peter Costello, |immigration policy, etc. Wikipedia uses a number of software
"agents" to identify vandalism, and third party sites, such as Wikipedia
Scanner, also serve as a kind of "sheriff". There is also a group |of
volunteers within Wikipedia called CVU - Counter Vandalism Unit.
|http://en.wikipedia.org/wiki/Wikipedia:CVU
|For an idea about how abuses of Wikipedia are reported and handled:
|http://en.wikipedia.org/wiki/Wikipedia:Abuse_reports
Thank you for your somewhat patronising direction to the Wikipedia Counter
Vandalism Unit. I'm really not sure if you actually read my post for what it
was, or for what you thought it might be - there again, this is the Austech
Writers forum, so that is probably a stupid question. What I suggested should
be done was so far removed from 'vandalism' that your response astonishes me.
Did you seriously consider that I would suggest these article would be left
there? Doesn't the phrase 'for a reasonable period of time' suggest that after
that time the article would be removed?
|There have already been a number of studies into vandalism and the reliability
of Wikipedia content. To locate these, try using the phrase "wikipedia
reliabilty studies" in Google Scholar. As a first |port of call, I'd suggest
you read "Early response to false claims in Wikipedia",
http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/viewArticle/2115.
Another good one is "Epistemology |and the Wikipedia"
http://dspace.sunyconnect.suny.edu/handle/1951/42589.
Have you actually read these articles you refer to? The first details a study
where the author deliberately introduced "fibs" into existing articles to see
how long it would take to have them corrected. So how, exactly, is that
different from what I proposed? Also of interest are his conclusions - only 1/3
to 1/2 of the errors were removed or corrected within 48 hours - hardly an
example of the success of the system you so vigorously promoted earlier in this
thread.
The second article, as you say, is a good one.
Having read both of these I cannot believe you made the comments you did about
the reliability of Wikipedia and the systems put in place to eradicate errors.
These systems are largely untested in any rigorous way, and what tests have
been done suggest, if anything, that Wikipedia, overall, is far from reliable.
Having read this forum for some considerable time I really should have known
better. I doubt that I'll bother contributing again.
|Please don't go smashing windows to see how long it takes for them to be
repaired.
But it would be a very quick repair wouldn't it - that's what you told us
originally!
Graeme
|Tony Self
Other related posts:
- » atw: Re: Testing Wikipedia - Anthony Self
