[TN-Bird] Pulliam BBS contribution
- From: kbreault <kbreault@xxxxxxxxxxxxx>
- To: tn-bird@xxxxxxxxxxxxx
- Date: Mon, 17 Oct 2011 18:36:52 -0700 (PDT)
Bill Pulliam is to be congratulated for sending his Breeding Bird Survey (BBS)
graphs to tn-bird. It is not often that we get to consider science on tn-bird
and I for one would like to see more of it. Also impressive was the overall
look of the presentation. All in all a fine contribution, well worth the
effort.
I have spent a few days reading BBS materials and research acticles and these
are some of my more general thoughts. First, BBS samples are essentially
"convenient" samples and the observations are not randomly distributed. Those
responsible for the BBS say that in targeted habitats suitable roads on which
roadside observations take place were chosen randomly. In other words,
roadside
observations in selected habitats are privileged. Hypothetically, compared to
other places some birds may increase, others decrease in these putatively
dynamic roadside environments. Those who do spatial analyses, geographers,
economists, sociologists, epidemiologists, and medical researchers, know that
convenient samples and others that are not randomly distributed are not without
serious problems. For example, and having used these methods quite a bit in
the
past, I know that different spatial units often produce markedly different
results. What this means is that if we had another data set using different
methods with which to compare the BBS data, it is likely that we would find
different results for at least some species. Scientists who do not use
comparable data sets may not fully appreciate the degree to which results can
differ. That is what Roger Applegate is saying. Another way of looking at
this
is that if the BBS had a larger sample, and with convenient samples that is
quite important, the results would have been more convincing.
Second, this is not, or should not be, the end of the story. Knowing that the
data are flawed should be the beginning of our investigation. Note that flawed
data may not produce results that are systematically biased. If possible the
next step is to determine if there is bias. Depending on the situation there
may be empirical ways of detecting and controlling for bias. Auguably, in the
case at hand BBS roadside observations may be biasing the data. The best way
of
determining that is to survey areas in the same habitats using the very same
BBS
methodology (e.g., size of spatial unit, number of observations per route), but
without the roadside observations. If we find that these alternative areas
produce findings similar to that of the BBS we can be more confident in the BBS
results. If there are significant differences we can either control for the
differences in the BBS findings or alter the BBS methodology.
Third, given that the BBS data are flawed assigning probability levels to
differences is not without difficulty and I would be uncomfortable with any but
the most conservative probability standard. Even with high quality data our
interest in science is mainly with differences that are "important," not ones
that are merely significant. Flawed data for which differences are only
significant at the .05 level are the most likely to be overturned. And of
course, any suggestion a trend may exist even though the difference is not
significant at the .05 level is spurious by definition. So what I think we can
take away from the BBS results are findings that show important differences, as
in the case of birds such as Bewick's Wren, Loggerhead Shrike, Golden-winged
Warbler, Yellow Warbler, etc.. Beyond important differences we are asking more
than the data will allow. Note that Bill makes a point of highlighting the
most
important findings, and he may want to give us more confidence in the results
by
using more conservative summary statistics in developing the lists of
increasing
and declining species.
Fourth, the size and "selection" of the study period may be a concern for
some.
Forty-five years may seem like a lot but with these low-sample time-series data
it is very difficult to generalize beyond the period of study. Thus, for
example, if we had BBS data from 1956 instead of 1966 it is possible that the
findings would have been different. (For some declining species the 1966
inauguration date may seriously underestimate the degree of decline.)
Similarly, the findings might well be different in 2020. Note that these data
can be "convenient" or ad hoc in another way. What we would really like are
start-and-stop times that are meaningful to the substantive issues under
discussion. For example, one useful moment to begin analyses of climate change
is the historical development of industriatization. Meaningful temporal
periods provide for focused theoretical discussion, predictive modeling and
causal attribution. Especially if we are interested in bird populations as a
whole, I would much rather see--not that I will get the chance--BBS data from
1966 to 2066 or even 2100, 100 and 134 years with which we will have more
information about the success of bird populations.
Fifth, disaggregate, disaggregate, disaggregate! (Or analyze, analyze, analyze
to use the Greek derivation.) Studies like BBS give us a great deal of
information but if there is any methodological prescription we should have
learned by now it is that we produce the best science by "disaggregating."
Thorough research on individual species will always trump more superficial
research on large numbers of species. After all, and those who use BBS data
uniformly agree, we really want to know much more than which species are
increasing or decreasing.
Finally, if we believe that our methods and theories are set in concrete we are
not doing science. Our obligation, our "calling" (to borrow a term from The
Big
Year), is to try to produce better science in the future. There are few areas
in which the last word has been said. In science last words are for those who
have nothing else to say.
Thanks again to Bill for his strong contribution, and to the others who added
to
this most interesting discussion, the best kind of midterm break possible. And
with a first Eurasian Wigeon in Arkansas just a few miles from Memphis, and a
major movie on birding what's not to like?
Kevin Breault
Brentwood, TN
Other related posts:
