[lit-ideas] Re: The nature of Media Bias

  • From: Judith Evans <judithevans001@xxxxxxxxxxxxxx>
  • To: lit-ideas@xxxxxxxxxxxxx
  • Date: Tue, 14 Dec 2004 02:30:25 +0000 (GMT)

Lawrence,
 
First my apologies to anyone who doesn't like .html mail.  My usual computer 
has more or or less collapsed and I haven't yet set this one up fully, so I'm 
relying on web mail I can't control Next, it's late here now, I'll reply 
briefly.
 
>>>>>>>>.
The analysis has been out there (somewhere) since
September 2003. Barro reviewed it in June of 2004 for Business Week and
then in December 2004 for The Weekly Standard. From June to December his
confidence in the G&M methodology seems to have increased
>>>>>>>>>>>>>>>
 
But I see no evidence that people in political science and media studies have 
any confidence in it at all -- I haven't checked the account of the criticism 
it met at UCLA (from, I read, everyone from senior faculty to graduate 
students), I'll try. 
 
>My impression of the G&M approach is that they start with the ADA data on
>house members 
 
They take the ADA ranking but are unhappy with it so use the median score 
instead of the mean.  I've just read something that purports to be their paper 
-- the .pdf is on my other computer -- it doesn't reference the ADA source.  
(The references are few, they suggest G/M are as ill-informed re media analysis 
as I assumed from their cv's.)
 
>>>>>>>
Conservative house members would be assumed to cite Conservative think
tanks. Liberal house members would be assumed to cite Liberal think tanks.
>>>>>
 
And so think tanks can be classified that way and then the media can be 
classified by the think tanks they cite (except that Rand is a problem -- see 
G/M -- so part of Rand is omitted -- yes). To be fair, they aren't quite as 
crass as this suggests: they omit citations that carry an obvious hostile 
labelling.  (Actually "they" is 20 research assistants, unless the  RAs simply 
gathered the data.) And they do find an explanation for the ranking of The 
Drudge Report. But still, this is rather like someone assessing the members of 
an e-mail list by counting the number of times they mention Marx.
 
>>>>>>>.
This implies that someone will have to be
subjectively modifying the raw data to throw out elements considered to be
aberrations.
>>>>>
Yes. This is not an objective piece of work, and it would be better if they 
didn't suggest it was. These men aren't I'd say right-wing ideologues, but they 
do seem to believe rather blindly in the power and integrity of econometric 
techniques.
 
>>>>>>>>>
This data period at first seemed
too short to me, but after thinking about it, the Q&M procedure isn?t
intended to determine bias once and for all but only bias during a limited
period. 
>>>>>
 
But that isn't how it's been reported in the media that have taken the study up.
 
>>>>>>>>>.
What precisely do G&M do with these Think Tanks? Do
they all fit nicely on the Left and the Right? Or does it matter?
>>>>>>>
 
That's interesting too - yes it is a problem -- they seem to say it doesn't 
matter yet as the Rand is a problem (its reports, apparently, vary 
ideologically by topic) they omitted some  of Rand.  
 
>>>>>>>>>
 [By the way, in
TWS article he credits the Pew Research Center for providing corroborating
evidence, i.e., ?the strong tendency of journalists to describe themselves
as liberal.?] 
>>>>>
 
that isn't really evidence, Lawrence.
 
If this paper -- OK, it does seem to be just a working paper, so it shouldn't 
really be greeted too harshly but otoh, nor should it be received as though it 
were a magnum opus!
 
But really the main reason for scepticism is the result.  If they really 
believe the media are as "liberal" as they say, well, there's something wrong 
with them.  
 
Judy
 

 


Lawrence Helm <lawrencehelm@xxxxxxxxxxxx> wrote:
Judy: 


I'm glad you've taken an interest in this. After discovering the study, I
was anxious to decide whether I could depend upon G&M?s methodology. I went
to sleep last night reading their original analysis and then finished it
when this morning. The analysis has been out there (somewhere) since
September 2003. Barro reviewed it in June of 2004 for Business Week and
then in December 2004 for The Weekly Standard. From June to December his
confidence in the G&M methodology seems to have increased. [By the way, in
TWS article he credits the Pew Research Center for providing corroborating
evidence, i.e., ?the strong tendency of journalists to describe themselves
as liberal.?] 



My impression of the G&M approach is that they start with the ADA data on
house members (they use the house rather than the Senate because the latter
disproportionately represents small states). They then assume the House
members to be as they are represented by the ADA and proceed to read a fixed
number of lines or sentences of their house speeches. They look for
think-tank references and rate them in terms of who cites them.
Conservative house members would be assumed to cite Conservative think
tanks. Liberal house members would be assumed to cite Liberal think tanks.
Once they determined which category to put 200 think tanks in, they turned
to Media Outlets and determined the nature of their citing of these same
think tanks. The ADA data permitted G&M to rate House Members on a scale of
0 to 100. By using the same procedure on Think Tanks, they were able to
rate them in a similar manner. 



G&M did indicate that their 2003 analysis was a work in progress. Their
?back of the envelope ADA Estimates? (See pages 23 and 24 of their
analysis?) may not indicate doubt as to the procedure, but there is some
doubt about these particular results. I note that the data in Barro?s Dec
2004 article is based upon G&M ?Calculations as of November 2004,? and they
are different from the 2003 analysis calculations. For one thing the House
Median has moved from 39 to 50.1 which more closely approximates the way the
country votes. Also, all of the calculations have been updated. Barro
continues to refer to the G&M study as ?ongoing.?



One element that causes is me doubt is that in the 2003 analysis, the ACLU
citations when weighted by citation scored 42.66. When weighted by average
score it rated 34.99; which was to the right of the house Median of 39.
?The primary reason that the ACLU appears so conservative is that it opposed
the McCain-Feingold Campaign Finance Bill. Consequently, conservatives
tended to cite this fact often. Indeed, slightly more than half of the ACLU
sentences cited in Congress were due to one person, Mitch McConnell
(R.-Kt.), who strongly opposed the McCain-Feingold bill. If we omit ACLU
citations that are due to McConnell, then the average score, weighted by
sentences, increases to 70.12.? This implies that someone will have to be
subjectively modifying the raw data to throw out elements considered to be
aberrations.



Another element that causes me a problem is that ?for each media outlet we
selected an observation period for the data that we estimated would yield at
least 1200 sentences of data. Because there is less data to collect for
magazines and television shows (e.g. a transcript for a 30-minute show
contains only a small fraction of the sentences that are contained in a
daily newspaper), we collected all the dates that were available in
Lexis-Nexis for these two forms of Media.? This data period at first seemed
too short to me, but after thinking about it, the Q&M procedure isn?t
intended to determine bias once and for all but only bias during a limited
period. So if they did an analysis of a data period every month for a year
and discovered that the results showed the House Representatives, the Think
Tanks, and the Media Outlets to continue to have roughly the same bias that
they had during previous data periods, their approach would inspire more
confidence. Perhaps that has happened, and perhaps that is why Barro seems
to be more enthusiastic about the G&M procedure in his Dec 2004 article than
he was in his Jun 2004 article.



Another concern I had pertained to the Think Tanks. These are the 200 Think
Tanks that G&M used in their analysis: HYPERLINK
"http://www.wheretodoresearch.com/Think_Tanks.htm"http://www.wheretodoresear
ch.com/Think_Tanks.htm What precisely do G&M do with these Think Tanks? Do
they all fit nicely on the Left and the Right? Or does it matter? Who is
more likely to cite the National Audubon Society, for example? If Liberals
are more likely to be concerned about environmental issues than
conservatives, then they will probably be more likely to cite the Audubon
Society. So I suppose G&M would put the Audubon Society on the Liberal
side. I suppose that works.



Lawrence Helm

San Jacinto











-----Original Message-----
From: Judy Evans





Sunday, December 12, 2004, 8:12:28 AM, Lawrence Helm wrote:







LH> As to the criteria for the Measure of Media Bias analysis, I cited the

LH> original data from the original analysis in my previous note. There is

LH> indication in it of how the ADA items are used. However, Groseclose &
Milyo

LH> refer to ?back of the envelope? conclusions. They feel their theory is
good

LH> and their conclusions broadly sound, but the research and data aren?t as

LH> thorough as they will eventually be (if I understand them)





Their "back of the envelope" refers to their simplified explanation of their
method. In their reply to Nunberg they say he's criticising that and not
their real method (which involves some complex stats). I think they're
happy with their conclusions!



Their paper's a UCLA Working Paper (so, not published; it wouldn't count as
a publication here, anyway, and I don't think they'd say it had been
published). They gave the paper at UCLA (Groseclose holds a "by courtesy"
appointment in Pol Sci there), it was apparently very badly received. More
than one problem's been pointed out, and there are points their reply to
Nunberg can't really overcome, one being the very basic point their "ADA
score" method of proving bias is just not very good, another, that it's
absurd to measure bias by citation of liberal or conservative sources
(that's shorthand; but citation is what they used).



The paper's cited by 5 others; one (the only one I've read) has
misinterpreted Groseclose and Milyo, suggesting they claim really rather
more than they do (certainly more than they should): that's a Stanford
Research Paper, by a man who holds a name chair in Political Economy at
Stanford. So it goes....







BTW here's a more informative piece by Barro:



http://post.economics.harvard.edu/faculty/barro/bw/bw04_0614.pdf













-- 

Judy Evans, Cardiff, UK 

mailto:judithevans001@xxxxxxxxxxxxxx












-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.296 / Virus Database: 265.5.0 - Release Date: 12/9/2004


------------------------------------------------------------------
To change your Lit-Ideas settings (subscribe/unsub, vacation on/off,
digest on/off), visit www.andreas.com/faq-lit-ideas.html



------------------------------------------------------------------
To change your Lit-Ideas settings (subscribe/unsub, vacation on/off,
digest on/off), visit www.andreas.com/faq-lit-ideas.html

Other related posts: