[lit-ideas] Re: The nature of Media Bias
- From: "Lawrence Helm" <lawrencehelm@xxxxxxxxxxxx>
- To: <lit-ideas@xxxxxxxxxxxxx>
- Date: Sun, 12 Dec 2004 10:10:41 -0800
Judy:
I'm glad you've taken an interest in this. After discovering the study, I
was anxious to decide whether I could depend upon G&M?s methodology. I went
to sleep last night reading their original analysis and then finished it
when this morning. The analysis has been out there (somewhere) since
September 2003. Barro reviewed it in June of 2004 for Business Week and
then in December 2004 for The Weekly Standard. From June to December his
confidence in the G&M methodology seems to have increased. [By the way, in
TWS article he credits the Pew Research Center for providing corroborating
evidence, i.e., ?the strong tendency of journalists to describe themselves
as liberal.?]
My impression of the G&M approach is that they start with the ADA data on
house members (they use the house rather than the Senate because the latter
disproportionately represents small states). They then assume the House
members to be as they are represented by the ADA and proceed to read a fixed
number of lines or sentences of their house speeches. They look for
think-tank references and rate them in terms of who cites them.
Conservative house members would be assumed to cite Conservative think
tanks. Liberal house members would be assumed to cite Liberal think tanks.
Once they determined which category to put 200 think tanks in, they turned
to Media Outlets and determined the nature of their citing of these same
think tanks. The ADA data permitted G&M to rate House Members on a scale of
0 to 100. By using the same procedure on Think Tanks, they were able to
rate them in a similar manner.
G&M did indicate that their 2003 analysis was a work in progress. Their
?back of the envelope ADA Estimates? (See pages 23 and 24 of their
analysis?) may not indicate doubt as to the procedure, but there is some
doubt about these particular results. I note that the data in Barro?s Dec
2004 article is based upon G&M ?Calculations as of November 2004,? and they
are different from the 2003 analysis calculations. For one thing the House
Median has moved from 39 to 50.1 which more closely approximates the way the
country votes. Also, all of the calculations have been updated. Barro
continues to refer to the G&M study as ?ongoing.?
One element that causes is me doubt is that in the 2003 analysis, the ACLU
citations when weighted by citation scored 42.66. When weighted by average
score it rated 34.99; which was to the right of the house Median of 39.
?The primary reason that the ACLU appears so conservative is that it opposed
the McCain-Feingold Campaign Finance Bill. Consequently, conservatives
tended to cite this fact often. Indeed, slightly more than half of the ACLU
sentences cited in Congress were due to one person, Mitch McConnell
(R.-Kt.), who strongly opposed the McCain-Feingold bill. If we omit ACLU
citations that are due to McConnell, then the average score, weighted by
sentences, increases to 70.12.? This implies that someone will have to be
subjectively modifying the raw data to throw out elements considered to be
aberrations.
Another element that causes me a problem is that ?for each media outlet we
selected an observation period for the data that we estimated would yield at
least 1200 sentences of data. Because there is less data to collect for
magazines and television shows (e.g. a transcript for a 30-minute show
contains only a small fraction of the sentences that are contained in a
daily newspaper), we collected all the dates that were available in
Lexis-Nexis for these two forms of Media.? This data period at first seemed
too short to me, but after thinking about it, the Q&M procedure isn?t
intended to determine bias once and for all but only bias during a limited
period. So if they did an analysis of a data period every month for a year
and discovered that the results showed the House Representatives, the Think
Tanks, and the Media Outlets to continue to have roughly the same bias that
they had during previous data periods, their approach would inspire more
confidence. Perhaps that has happened, and perhaps that is why Barro seems
to be more enthusiastic about the G&M procedure in his Dec 2004 article than
he was in his Jun 2004 article.
Another concern I had pertained to the Think Tanks. These are the 200 Think
Tanks that G&M used in their analysis: HYPERLINK
"http://www.wheretodoresearch.com/Think_Tanks.htm"http://www.wheretodoresear
ch.com/Think_Tanks.htm What precisely do G&M do with these Think Tanks? Do
they all fit nicely on the Left and the Right? Or does it matter? Who is
more likely to cite the National Audubon Society, for example? If Liberals
are more likely to be concerned about environmental issues than
conservatives, then they will probably be more likely to cite the Audubon
Society. So I suppose G&M would put the Audubon Society on the Liberal
side. I suppose that works.
Lawrence Helm
San Jacinto
-----Original Message-----
From: Judy Evans
Sunday, December 12, 2004, 8:12:28 AM, Lawrence Helm wrote:
LH> As to the criteria for the Measure of Media Bias analysis, I cited the
LH> original data from the original analysis in my previous note. There is
LH> indication in it of how the ADA items are used. However, Groseclose &
Milyo
LH> refer to ?back of the envelope? conclusions. They feel their theory is
good
LH> and their conclusions broadly sound, but the research and data aren?t as
LH> thorough as they will eventually be (if I understand them)
Their "back of the envelope" refers to their simplified explanation of their
method. In their reply to Nunberg they say he's criticising that and not
their real method (which involves some complex stats). I think they're
happy with their conclusions!
Their paper's a UCLA Working Paper (so, not published; it wouldn't count as
a publication here, anyway, and I don't think they'd say it had been
published). They gave the paper at UCLA (Groseclose holds a "by courtesy"
appointment in Pol Sci there), it was apparently very badly received. More
than one problem's been pointed out, and there are points their reply to
Nunberg can't really overcome, one being the very basic point their "ADA
score" method of proving bias is just not very good, another, that it's
absurd to measure bias by citation of liberal or conservative sources
(that's shorthand; but citation is what they used).
The paper's cited by 5 others; one (the only one I've read) has
misinterpreted Groseclose and Milyo, suggesting they claim really rather
more than they do (certainly more than they should): that's a Stanford
Research Paper, by a man who holds a name chair in Political Economy at
Stanford. So it goes....
BTW here's a more informative piece by Barro:
http://post.economics.harvard.edu/faculty/barro/bw/bw04_0614.pdf
--
Judy Evans, Cardiff, UK
mailto:judithevans001@xxxxxxxxxxxxxx
--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.296 / Virus Database: 265.5.0 - Release Date: 12/9/2004
------------------------------------------------------------------
To change your Lit-Ideas settings (subscribe/unsub, vacation on/off,
digest on/off), visit www.andreas.com/faq-lit-ideas.html
Other related posts:
