[obol] Re: Credibility
- From: Joel Geier <joel.geier@xxxxxxxx>
- To: Oregon Birders OnLine <obol@xxxxxxxxxxxxx>
- Date: Sun, 06 Jul 2014 22:41:38 -0700
Hi all,
I see that no one has yet responded to Clay Crofton's question, after
two full days, so I'll give it my best shot.
> Could someone please discuss how defining common birds ( Bald Eagles,
> Common Yellowthroats) and readily distinguishable birds like Northern
> Mockers as 'rare' advances the accuracy of eBird?
Folks on this list who have me pegged as an "eBird basher" might be a
little surprised to hear that I actually have some sympathy for eBird's
criteria for flagging regional rarities. A bird like Common Yellowthroat
might be very rare in the John Day River Valley, even if it's common in
western Oregon where most of us live.
This is an issue that I tried to come to grips with for the BirdNotes
web database (www.birdnotes.net, a home-grown, open-source,
all-volunteer project here in Oregon) some years before eBird launched
its first demo version. I dealt with it by building up a "likelihood"
matrix of bird species according to county and season.
Like today's volunteer eBird reviewers, I did this in my spare time, so
I also have some sympathy for their predicament as unpaid contributors
to this project. I certainly accept this on a human level, even if
sometimes it's a bit hard to swallow when I see more expansive claims
about the scientific merits of this process. You will never see serious
scientists in other fields defend shabby results based on the idea that
they were having too much fun pursuing their hobbies, on a particular
weekend.
I certainly had to field a lot of criticism whenever my system flagged a
seasonal/regional rarity that someone thought should be considered more
common. My response was to take this as good criticism, and try to
refine my matrix.
As an aside I should note that the review system in BirdNotes was very
different from eBird (I use the past tense here because, even though
BirdNotes is still fully functional, with the expansion of eBird in
recent years and consequent drop in BirdNotes usage, I not kept up the
quarterly reviews).
In BirdNotes, the guiding principles have always been that (1) the
purpose of the database is to provide a more structured way of storing
*casual* birding observations that birders were obtaining anyway, and
(2) every BirdNotes user is a peer of every other user, and everyone is
entitled to claim a bird on their list if that's what they truly believe
it was.
The system was set up to facilitate peer-to-peer discussions among
birders. Each list included a link to the observer's e-mail, so if you
had questions, you could e-mail them directly. My experience was that
this peer-to-peer system was very effective in catching potentially
erroneous identifications. In most cases, the observers corrected their
own lists if someone could convince them that they were in error.
However, they were also free to stick with their original
identification.
In eBird, we see a much more hierarchical model for the review process.
The model is basically the same as for Christmas Bird Counts -- partly
based on merit, partly based on people who happened to be in the right
place and with the right connections, when the eBird folks came casting
about for state- and county-level reviewers.
If I were to report a Northern Mockingbird (another of Clay's examples)
at my house next week, I guess that my report would be reviewed by the
same birder who was asking me, just a few years back, "What are the best
places to look for a mockingbird in Benton County?"
I'm not sure what would happen if I were to submit my past observations
of Northern Mockingbirds in Benton County (I have seen two over the past
16 years) -- would they be reviewed by the same guy who was asking me
where to find mockingbirds? Or would they just be accepted at face
value?
The word "credibility" in Clay's posting is an important and revealing
one. Credibility is VERY important in the world of competitive birding,
where people get very uptight about "proving" first state records, first
county records, big-day records, and that kind of thing.
The issue of "credibility" should be much less important in a purely
scientific endeavor where (one hopes) there are not such big ego risks
at stake. You go out as an observer, you record what you see/hear, and
then you submit your checklist. The guiding principle, one hopes, is
that every observer is doing his/her best to submit an accurate record
of their observations. If there is no competition, then it would seem
that there is no incentive to falsify or otherwise "stretch" results.
A new observer could be *wrong*, of course, but this is not a
"credibility" issue. It's simply a matter of making mistakes. With a
good database and appropriate statistical analysis, it should be
possible to sort most of these mistakes out, without a heavy-handed
subjective "review" process, at least to the point where they don't
affect any significant findings.
Where "credibility" really comes into play is the credibility of the
eBird database, as viewed by competitive listers/birders. The
hierarchical review process of eBird seems to be aimed mainly at
securing "credibility" among competitive birders, who tend to focus on
state or county lists.
If the aim were strictly scientific credibility, we would expect much
less of a focus on political boundaries such as counties. As Mike
Patterson has explained, on the most fundamental level eBird lists
fundamentally are recorded for geographic *points* rather than
geographic *areas*. Even if you record a county-wide list, eBird will
pick a singular point within that county to record as the location of
that list.
Given that data are recorded for *points*, it should be possible to
compare observations directly with published range maps (which could
easily digitized as polygons). Instead, observations are compared with
polygons that represent political subdivisions of the continent
(counties).
This makes sense if you understand that competitive birders are highly
sensitive to state and county lists, but less sensitive to geographic
occurrence within a given county.
For example, the recent report of multiple Yellow-breasted Chats in
Morrow County set off alarm bells, sufficient even to get me wound up.
On the other hand, a breeding-season Vesper Sparrow anywhere more than
two miles north of the Benton-Polk county line would be very significant
from a scientific and conservation standpoint, but it would probably not
be flagged by eBird filters because there are usually still a few Vesper
Sparrows in southernmost Polk County (most reliably, within a half mile
of the county line).
Having gone through this experience with BirdNotes, I'm not surprised
that the eBird administrators have made it a priority to ensure that any
rare-bird sightings that affect state or county lists are fully vetted.
However, this has very little to do with science, and much more to do
with public relations among a small sector of the birding public that
could potentially raise questions about the CREDIBILITY of eBird.
Nothing impacts impressions of "credibility" so much as a single oddball
report that slipped through the filters, even if it has no statistical
significance.
Personally, I think that eBird would be much more valuable without this
kind of data censoring -- if nothing else, at least for the sake of
appearances!
I remember bringing this up with BirdNotes founder Don Baccus in the
early days of BirdNotes in the last 1990s. His view -- which I agree
with -- was that it's better to rely on the statistical tendency of the
bulk of observations, to eventually swamp the oddball reports. Don't
worry so much about "credibility" of individual observations, as this
leads into the swamps of subjectivity.
Good birding,
Joel
--
Joel Geier
Camp Adair area north of Corvallis
Good Morning Learned Gentlemen of the OBOL community, I accept the
consensus that I don't have sufficient 'cred' to report a rarity like a
Brown Booby or a Wrentit at TR NWR. Never the less, I feel my question went
unanswered last week. Many years ago I sat through many lectures about the
scientific method. If your data is edited your not conducting science.
--
OBOL archives: www.freelists.org/archive/obol
Manage your account or unsubscribe: //www.freelists.org/list/obol
Contact moderators: obol-moderators@xxxxxxxxxxxxx
Other related posts:
