[Wittrs] Wittgenstein, Statistics and Judgment

Sean Wilson <whoooo26505@xxxxxxxxx> · Thu, 26 Jan 2012 21:27:34 -0800 (PST)

... I don't have time to do this now. I've got too much on my plate. I'd like 
to maybe help next week. There's no guarantee of that either. But it is such a 
good conversation that I do hope to say something helpful. But I do want to 
quickly note a couple of obvious problems with your view in passing:

1. It really doesn't surprise me that many number crunchers would fail to see 
how Wittgenstein could help people when their "science claims." There's not a 
subject in the world that Wittgenstein couldn't help. It's really doesn't 
benefit our field if scholars harbor such an attitude.

2. The comments about "what expert opinion is" and how you could replicate it 
formally is PRECISELY the kind of thing that Wittgenstein is concerned with in 
several areas of his later work. In saying things like "how does one select 
among connoisseurs," etc., -- aren't we really admitting that we haven't a firm 
grasp on what aesthetical judgments are to Wittgenstein, and how they fit into 
his other ideas? I wonder, have you read this stuff -- or are you just going 
off what you intuitively feel that the word "connoisseur" is doing in the 
sentence?  

A connoisseur is not a from-the-seat prognosticator. It's not asking Terry 
Bradshaw who will win the football game and comparing that to a computer model. 
This is the thing that is not understood. When you poll the law professors 
about who will win the case, they aren't giving you a "connoisseur judgment," 
at least not in a Wittgensteinian sense. If any student would have written that 
in a paper, it would be near the F range. 

To have a connoisseur, you must first have a specific behavior that becomes 
celebrated in the culture. There's a learning curve with the appreciation of 
the behavior. Only after the person goes through the learning curve (training) 
does he or she come to fully understand its context. The connoisseurs are what 
keep the context properly appreciated. So if the behavior is predicting how a 
judge would vote, I suppose the only true connoisseurs of that activity would 
be the judge's wife (spouse). And there would have to be a learned appreciation 
for this specific behavior. 

Try to relate this to Wittgenstein's views on aspect seeing and imponderable 
evidence. The connoisseur understands the behavior the way that the mother can 
understand a child just by looking at the eyes.  The connoisseur doesn't use or 
need "statistics" because he or she already has superior information. He or she 
has developed the ability to "aspect-see." The rest of us, of course, have no 
choice but to rely on whatever information we can. We're are neither the 
judge's spouse nor the child's parent. The difficulty comes when some of us 
throw away the stats to make our own hunches. The question of whether this is 
good or bad is ultimately a function of how good the information is versus how 
insightful the person might be at "seeing something." I threw away a linear 
regression about approval ratings because I didn't think its picture of account 
more helpful than the one I had formed by watching and studying presidential 
elections. The fallacy here is to
 say that I threw away "science" for something inferior. Surely I did not.  

Note also that if the behavior in question is what constitutes good legal 
casuistry, there are connoisseurs here -- Ronald Dworkin probably being the 
best. Legal judging is clearly an "aesthetic" in a Wittgensteinian sense, as my 
book will shortly show. We must be careful about one thing: not all 
supposed connoisseurs are good at their craft. Just as you can get a bad 
haircut or a bad tailor, so too can you get a bad judge or philosophy-of-law 
scholar. (Or a bad parent or poor spouse). It's not enough that a person be in 
the position of connoisseur; to do the job right, they must be good at it.  

So, if you ever do build a true connoisseur simulator (with AI), it would have 
absolutely nothing to do with the fact that connoisseur judgment plays the role 
that it does in our life. Wittgenstein isn't against you building anything. 
He's mostly concerned that, when you make claims, you aren't confused.

I've got to go to bed and I can't do any more of this for a while. Let's talk 
about it next month. It would really be a good panel! 

(P.S. Sent to Wittrs -- careful when hitting reply all)

Regards and thanks. 

Dr. Sean Wilson, Esq. 
[spoiler]Assistant Professor 
Wright State University 
Personal Website: http://seanwilson.org
SSRN papers: http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=596860
Wittgenstein 
Discussion: http://seanwilson.org/wittgenstein.discussion.html[/spoiler]

________________________________
From: Daniel Katz <katzd@xxxxxxxxxxx>
To: LAWCOURT-L@xxxxxxxxxxxxxxxxxxx 
Sent: Thursday, January 26, 2012 2:26 PM
Subject: Re: On Prediction ...

My prior thoughts were aimed at trying to put to this list an honest discussion 
point as to where we believe the next major advance in going to come in 
judicial politics.  

Turns out that Wittgenstein is really NOT likely to be the answer to this 
question.    

For example, how does one select among connoisseurs, etc.  What is the 
switching rule to decide whether it is time to now longer believe a 
connoisseur, etc.   The challenge has been put forth before and I will put it 
forth again 
---  

Can you leverage any insight from Wittgenstein to develop a model 
(qualitative, quantitative, etc. ) that can outperform EITHER  crowd 
sourced   AND/OR   the best algorithmic approach available.    

If you can, than Wittgenstein has something to contribute to this 
conversation.  Indeed, if Team Wittgenstein would like to enter the 
tourney -  I am sure that Josh Blackman over at FantasySCOTUS would be 
happy to take your entry: http://www.fantasyscotus.net/

Returning now the thrust of my thoughts: 
Crowd sourced prediction offers lots of potential with the caveat that we need 
some second order rule that looks to the specific contours of the information 
environment, etc.  In other words, crowds can be stupid (particularly at 
pricing rare events.)  

I just want to say that I really do believe that expert opinion is a very 
interesting topic ...  IF we are going actually analyze the specific data 
streams and weighting scheme over data that allows them to succeed than we have 
something to add to a much broader conversation.   Indeed, that is what has 
been happening in many other fields during the new "Soft AI Revolution."   

On the flip side, there are a number of lab groups across the academy that have 
been working for several years at developing / testing a variety of different 
prediction algorithms including but not limited to the classification trees, 
neural networks, etc.   (Note the class of techniques are a little bit "up 
market" from sort of simple linear regression model).    

Best,
Dan 

>>> "Peppers, Todd" <peppers@xxxxxxxxxxx> 01/26/12 1:42 PM >>>
Folks:

I truly enjoy this listserv, and find most of the postings to be truly 
informative, but I cannot stomach another discussion/debate about ole Ludwig 
Wittgenstein.  Could we perhaps hold the discussion about "connoisseur 
judgment" off-line?

Respectfully,

Todd Peppers

Dr. Todd C. Peppers 
Henry H. & Trudye H. Fowler Chair
Associate Professor
Public Affairs
P: 540-375-2417
F: 540-375-2405
peppers@xxxxxxxxxxx

-----Original Message-----
From: Sean Wilson [mailto:whoooo26505@xxxxxxxxx] 
Sent: Thursday, January 26, 2012 1:34 PM
To: LAWCOURT-L@xxxxxxxxxxxxxxxxxxx
Subject: Wittgenstein, Judgment & Statistics

Greetings Jeff.

First, thank you for those references. It will take me a few days to digest 
them. I'm trying to finish up my book, for good, because I have manuscript 
submission deadlines approaching. But I would enjoy a discussion in here about 
Wittgenstein's idea of "connoisseur judgment" and how it compares with those 
who base judgment purely upon statistical inference -- and what, in fact, that 
even means. My guess is that each camp has information deficits relative to one 
another. It would be a cutting edge discussion: one that could deserve its own 
conference panel. Let me digest these sources of yours and offer some thoughts 
about how a proper understanding of connoisseur judgment could help these 
conversations, if it can.

I'll try to get some thoughts together by next week. 

But thank you once again for the references. 

(P.S. Sent to Wittrs)

Regards and thanks. 

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
Personal Website: http://seanwilson.org
SSRN papers: http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=596860
Wittgenstein Discussion: http://seanwilson.org/wittgenstein.discussion.html

________________________________
From: Jeffrey Segal <jeffrey.segal@xxxxxxxxxxxxxx>
To: LAWCOURT-L@xxxxxxxxxxxxxxxxxxx 
Sent: Thursday, January 26, 2012 8:46 AM
Subject: polls and elections

Sean Wilson wrote yesterday that he prefers "connoisseur judgments" over linear 
regression models.  Unfortunately for Sean's argument, a substantial volume of 
literature shows that quantitative assessments clearly outperform qualitative 
judgments.  See most notably, Philip Tetlock's "Expert Political Judgment: How 
Good is it? How Can we Know?" demonstrating across the board that simple 
algorithms outperform expert judgments and Daniel Kahneman's "Thinking, Fast 
and Slow" on the same point. 

Closer to home, Robyn Dawes demonstrates that even improper linear models 
outperform qualitative faculty judgments on prospective graduate students 
(Dawes, Robyn M. "The robust beauty of improper linear models in decision 
making,  34 American Psychologist 571 (1979) (showing that even improperly 
(i.e., evenly) weighted linear models outperform expert judgments of an 
admissions committee over the eventual quality of graduate students), and 
really close to home, Andrew Martin's computer out-predicted law professors in 
the law professors' areas of expertise in a series of Supreme Court 
decisions.  Andrew D. Martin, Kevin M. Quinn, Pauline T. Kim, and Theodore W. 
Ruger. 2004. "Competing Approaches to Predicting Supreme Court 
Decisionmaking."2 Perspectives on Politics761. 
On the merits of Sean's point, my Stony Brook colleague Helmut Norpoth has a 
linear prediction model based on the New Hampshire primary showing that 
incumbents with challengers lose--Truman '52 (who dropped out after a weak New 
Hampshire showing), Johnson '68 (ditto), Ford '76, Carter '80, and Bush '92)-- 
whereas incumbents without strong primary challengers win (Ike 56, LBJ 64, RMN 
72, RWR 84, GWB 04, and a prediction for BHO in 12). See 
http://www.huffingtonpost.com/helmut-norpoth/new-hampshire-primary-for_b_1200199.html

Sincerely,

Jeff

Jeffrey Segal
Distinguished Professor and Chair
Department of Political Science
Stony Brook University
Stony Brook, NY 11794
phone 631-632-7662
fax 631-632-4116
jeffrey.segal@xxxxxxxxxxxxxx
http://www.sunysb.edu/polsci/jsegal/

2011-2012 Contact Information
Senior Visiting Research Scholar
Center for the Study of Democratic Politics
314 Robertson Hall
Princeton University 08544
phone 609-258-7941          

_______________________________________________
Wittrs mailing list
Wittrs@xxxxxxxxxxxxxxxxxxx
http://undergroundwiki.org/mailman/listinfo/wittrs_undergroundwiki.org

[Wittrs] Wittgenstein, Statistics and Judgment

Other related posts: