[net-gold] Metastudy on Impact of Inquiry in K-12 - Response to Wurman
- From: "David P. Dillard" <jwne@xxxxxxxxxx>
- To: Temple University Net-Gold Archive <net-gold@xxxxxxxxxxxxxxxxxxx>, Temple Gold Discussion Group <TEMPLE-GOLD@xxxxxxxxxxxxxxxxxxx>, Net-Gold <net-gold@xxxxxxxxxxxxxxxx>, Educator Gold <Educator-Gold@xxxxxxxxxxxxxxx>, Educator Gold <Educator-Gold@xxxxxxxxxxxxxxxx>, K12AdminLIFE <K12AdminLIFE@xxxxxxxxxxxxxxx>, Net-Platinum <net-platinum@xxxxxxxxxxxxxxx>, NetGold <netgold@xxxxxxxxxxxxxxx>, "Net-Gold @ Nabble" <ml-node+3172864-337556105@xxxxxxxxxxxxx>, K-12ADMINLIFE <K12ADMIN@xxxxxxxxxxxxxxxxxxx>, net-gold@xxxxxxxxxxxxx
- Date: Sat, 6 Feb 2010 12:06:44 -0500 (EST)
.
Date: Fri, 5 Feb 2010 16:33:26 -0800
From: Richard Hake <rrhake@xxxxxxxxxxxxx>
Reply-To: Net-Gold@xxxxxxxxxxxxxxx
To: PHYSLRNR@xxxxxxxxxxxxxxxxxxxxxxx
Cc: AERA-L@xxxxxxxxxxxxxxxxx, Net-Gold@xxxxxxxxxxxxxxx
Subject: [Net-Gold] Metastudy on impact of inquiry in k-12 - Response to
Wurman
If you reply to this long (26 kB) post please don't hit the reply
button unless you prune the copy of this post that may appear in your
reply down to a few relevant lines, otherwise the entire already
archived post may be needlessly resent to subscribers.
*************************************************
ABSTRACT: In the abstract of my post "Re: Metastudy on impact of
inquiry in k-12" [Hake (2010a)], I wrote: "Joe Bellina (2010), in a
post 'Metastudy on impact of inquiry in k-12' ALERTED subscribers to
"Inquiry-Based Science Instruction - What Is It and Does It Matter?
Results from a Research Synthesis Years 1984 to 2002" [Minner, Levy,
& Century (2009)]."
Ze'ev Wurman (2010), evidently misunderstanding the above, responded
(paraphrasing) "Would Hake care to speculate as to the reasons Joe
BELLINA RESTRICTED HIS RESEARCH to 1984-2002 and ignored the last 7
years of rather fruitful studies in this area?"
Ze'ev apparently did not scan the abstract of Minner et al. (2009) in
which the authors give the following reasons for restricting *their*
research (*not* Bellina's) to data from 1984 to 2002 (paraphrasing):
"[That timeframe] was selected to continue a line of synthesis work
last completed in 1983 by Bredderman (1983) and by Shymansky et al.
(1983), and to accommodate a practicable cutoff date given the
research project timeline, which ran from 2001 to 2006."
If Ze'ev and others *suspect* that Minner et al. may have cherry
picked 1984-2002 so as to "focus on data from periods that suited
their theses," then to make a case they would need to provide data
outside the 1984-2002 period that *conflicts* with Minner et al.'s
indication of "a clear, positive trend favoring inquiry-based
instructional practices."
My survey of data in "Direct Science Instruction Suffers a Setback in
California - Or Does It?" [Hake (2004)] showed that ALL the data,
including that outside the 1984-2002 period, was generally
consistent with the pro-inquiry assessment of Minner et al. (2009).
Not surveyed was "Why Minimal Guidance During Instruction Does Not
Work: An Analysis of the Failure of Constructivist, Discovery,
Problem-Based, Experiential, and Inquiry-Based Teaching" [Kirschner,
Sweller, & Clark (2006)]. But that paper, despite its misleading
title, does *not* counter the theses of Minner et al. (2009), as
explained in e,g., "Language Ambiguities in Education Research"
[Hake (2008b)].
*******************************************
In response to my post "Re: Metastudy on impact of inquiry in k-12"
[Hake (2010a)], Ze'ev Wurman wrote [bracketed by lines "WWWW. . . .",
my insert at ". . . . .[[insert]]. . . . . . ."]
WWWWWWWWWWWWWWWWWWWWW
Does prof. Hake care to speculate as to the reasons Joe Bellina
restricted his research to 1984-2002 and ignored the last 7 years of
rather fruitful studies in this area? After all, the "choristers" he
mentions did publish important studies after 2002 in this area.
Perhaps I wouldn't be so questioning were it not for the recent
findings how CRU. . . . .[[ CRU = Climate Research Unit (at the U. of
East Anglia)]]. . . . . researchers attempted to focus on data from
periods that suited their theses.
WWWWWWWWWWWWWWWWWWWWW
The first paragraph of the *abstract* of "Re: Metastudy on impact of
inquiry in k-12" transmitted to various discussion lists, read as
follows:
"ABSTRACT: Joe Bellina (2010), in a post 'Metastudy on impact of
inquiry in k-12' alerted subscribers to 'Inquiry-Based Science
Instruction - What Is It and Does It Matter? Results from a Research
Synthesis Years 1984 to 2002' [Minner, Levy, & Century (2009)]. . . .
. . . . . . . . . . . . . . "
Thus Bellina merely *alerted* subscribers to the report by Minner,
Levy, & Century (2009). BELLINA HIMSELF DID *NOT* DO THE RESEARCH.
Thus Ze'ev's question *should* have been:
"Does prof. Hake care to speculate as to the reasons Minner, Levy, &
Century (2009) restricted their research to 1984-2002 and ignored the
last 7 years of rather fruitful studies in this area?"
Ze'ev may or may not have bothered to click on
<http://groups.yahoo.com/group/Net-Gold/message/31779> so to access
my *entire* 33 kB post on the open archives of Net-Gold. If he did,
he may have overlooked the reference to Minner et al. (2009), copied
into the present REFERENCE list below and/or failed to scan the
Minner et al. abstract, giving Minner et al.'s explanation of why
they restricted their analysis of data to the 1984-2002 timeframe.
Minner, Levy, & Century (2009) wrote [bracketed by lines
"MLC-MLC-MLC-. . . . . . "; slightly edited]:
MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC
Abstract: The goal of the Inquiry Synthesis Project was to synthesize
findings from research conducted between 1984 and 2002 to address the
research question, What is the impact of inquiry science instruction
on K-12 student outcomes? THE TIMEFRAME OF 1984 TO 2002 WAS SELECTED
TO CONTINUE A LINE OF SYNTHESIS WORK LAST COMPLETED IN 1983 [my CAPS]
by Bredderman (1983) in Review of Educational Research 53: in
Journal of Research in Science Teaching 20: 387-404], AND TO
ACCOMMODATE A PRACTICABLE CUTOFF DATE GIVEN THE RESEARCH PROJECT
TIMELINE, WHICH RAN FROM 2001 TO 2006 [my CAPS]. . . . . . . . .
Various findings across 138 analyzed studies indicate a clear,
positive trend favoring inquiry-based instructional practices,
particularly instruction that emphasizes student active thinking and
drawing conclusions from data.
MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC
Of course, I realize that Ze'ev and others may *suspect* that the
above explanation by Minner et al. of their data timeframe 1984 to
2002 is bogus, and that Minner et al. actually cherry picked that
period so as to "focus on data from periods that suited their theses."
But to make that case, Ze'ev and others would need to provide data
outside the 1984 to 2002 period that *conflicts* with Minner et al.'s
indication of "a clear, positive trend favoring inquiry-based
instructional practices."
In the passage below from "Direct Science Instruction Suffers a
Setback in California - Or Does It?" [Hake (2004)] and bracketed by
lines "HHHHHHH. . . . .", I discuss data, including some *outside*
the 1984 to 2002 timeframe,that is *consistent* with the positive
assessment of inquiry methods by Minner et al. (2009) [see Hake
(2004) for references other than Alberts (2000)l, Arons (1983, 1997),
and Hake (1998a,b; 2008a)]:
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
There is a substantial amount of scientific research evidence [for
discussions of what constitutes "scientific research evidence" in
education see Shavelson & Towne (2000) & Burkhardt & Schoenfeld
(2003)] that "hands-on guided-inquiry methods" [commonly called
"inquiry" or "interactive engagement" methods] are far more effective
than "direct instruction" for promoting student learning *in
conceptually difficult areas* [for reviews see e.g., Hake (2004j);
Doss- Hammel (2004); Lowery (2003); and the literature references in
AAAS (1993, 2004), NRC (1996; 1997a,b; 1999, 2000, 2001, 2003),
Bransford et al. (1999), and Donovan et al. (1999).
In Hake (2004j) I wrote [bracketed by lines "hhhhhh. . . ."]:
hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
[The California Curriculum Commission (CCC)] appears to inhabit a
"private universe" [Schneps & Sadler (1985)], seemingly oblivious of
the literature of cognitive science [see, e.g. Bransford et al.
(1999)] and three decades of science-education research showing the
superiority of hands- and minds-on pedagogy to direct instruction in
conceptually difficult areas [see e.g., Karplus (1974, 1977, 1981);
Arons (1960, 1972, 1974, 1983, 1985, 1997, 1998); Shymansky et. al.
(1983, 1989, 1990); Halloun & Hestenes (1985a,b); McDermott & Redish
(1999); Hake (1998a,b; 2002a,b); Lopez & Schultz (2001); FOSS (2001);
Pelligrino et al. (2001); Crouch & Mazur (2001); Fagen et al. (2002);
Fuller (2002)]; Redish (2003); and Belcher (2003). Note that none of
the above research concerns unguided "discovery learning," an evident
bugaboo of CCC's Stan Metzenberg and executive director Thomas Adams
(2004).
hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
Still other references showing the superior effectiveness of hands-on
guided inquiry methods over direct instruction are Bredderman (1982,
1983, 1985), Kyle et al. (1988), Jorgenson & Vanosdall (2002), GLEF
(2001), and Anderson (2002).
In addition, the eleven K-12 science-education studies listed in
Table 1 of Lipsey & Wilson (1993) (where the test group is
characterized by reform methods) yield a total N = 888 students and
average effect size <d> = 0.36 [Cohen (1988)]. Most of these studies
include grades 4 or 6 to 12 with the effect size control group being
traditional direct instruction and the measurement unit being
"achievement" or "learning" (presumably as measured by tests).
Cohen's rule of thumb - based on typical results in social science
research - that d = 0.2, 0.5, 0.8 imply respectively "small,"
"medium," and "large" effects, but Cohen cautions that the adjectives
"are relative, not only to each other, but to the area of behavioral
science or even more particularly to the specific content and
research method being employed in any given investigation."
My own survey [Hake (1998a,b)] yielded a much larger effect size of d
= 2.43 [Hake (2002a)] in favor of the effectiveness of interactive
engagement over direct instruction and such large differences have
been corroborated by many other physics education researchers as
discussed in Hake (2002a,b) . . . .[[and more recently in
"Design-Based Research in Physics Education Research: A Review" [Hake
(2008a)] ]]. . . . ..
In sharp contrast there is, as far as I am aware, *ZERO scientific
evidence* for the superiority (in conceptually difficult areas of
science education) of "direct instruction" [in any of its many guises
[see Sec. III (8) below and Hake (2004p)] to "inquiry" [operationally
defined by Alberts (2000)] or "interactive engagement" [operationally
defined by Hake (1998a,b)]. Of course, neither "inquiry" nor
"interactive engagement" methods should be confused with the extreme
"discovery learning" mode, researched by Klahr & Nigam (2004). Their
research suggests that, not surprisingly, an *extreme* mode of
"discovery learning, in which there is almost no teacher guidance, is
inferior to "direct instruction" for increasing third and fourth
grade children's effective use of the control of variables strategy,
a so-called "process skill." It might be interesting for Klahr &
Nigam to extend their study to more guided forms of "discovery
learning" and to children's acquisition of "operative knowledge"
[Arons (1983)].
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Not cited in the above quote from Hake (2004) is "Why Minimal
Guidance During Instruction Does Not Work: An Analysis of the Failure
of Constructivist, Discovery, Problem-Based, Experiential, and
Inquiry-Based Teaching" [Kirschner, Sweller, & Clark (2006)]. But
that paper, despite it misleading title, does *not* counter the
theses of Minner et al. (2009) as explained most incisively in
"Language Ambiguities in Education Research," [Hake (2008b)]; and
also by Hmelo-Silver et al. (2007), Kuhn (2007), Minner et al.
(2009), Schmidt et al. (2007), and authors of the pro-constructivist
chapters in Tobias & Duffy (2009)].
Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands.
<rrhake@xxxxxxxxxxxxx>
<http://www.physics.indiana.edu/~hake/>
<http://www.physics.indiana.edu/~sdi/>
<http://HakesEdStuff.blogspot.com/>
<http://iub.academia.edu/RichardHake>
"Above all things we must be aware of what I will call 'inert ideas'
- that is to say, ideas that are merely received into the mind
without being utilized, or tested, or thrown into fresh combinations."
Alfred North Whitehead (1929, 1965) in "The Aims of Education"
REFERENCES [Tiny URL's courtesy <http://tinyurl.com/create.php>.]
Alberts, B. 2000. "Forward: A Scientists Perspective on Inquiry" in
NRC (2000). Alberts defines "inquiry activities" as those that allow
"students to conceptualize a question and then seek possible
explanations that respond to that question."
Arons, A.B. 1983. "Achieving Wider Scientific Literacy," Daedalus,
Spring; reprinted as Chapter 12 in Arons (1997). Arons wrote:
"Researchers in cognitive development describe two principle classes
of knowledge: figurative (or declarative) and operative (or
procedural). 'Declarative knowledge' consists of knowing 'facts,' for
example, that the moon shines by reflected sunlight, that the earth
and planets revolve around the sun . . . . 'operative knowledge', on
the other hand, involves understanding the source of such declarative
knowledge (How do we know the moon shines by reflected sunlight? Why
do we believe the earth and planets revolve around the sun when
appearances suggest that everything revolves around the earth? . . .
.) and the capacity to use, apply, transform, or recognize the
relevance of the declarative knowledge to new or unfamiliar
situations.
Arons, A.B. 1997. "Teaching Introductory Physics." Wiley. Amazon.com
information at <http://tinyurl.com/yzzlgs3>. Note the "look Inside"
feature.
Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66: 64-74; online at
<http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> (84 kB).
Hake, R.R. 1998b. "Interactive-engagement methods in introductory
mechanics courses," online at
<http://www.physics.indiana.edu/~sdi/IEM-2b.pdf> (108 kB). A crucial
companion paper to Hake (1998a).
Hake, R.R. 2004. "Direct Science Instruction Suffers a Setback in
California - Or Does It?" AAPT Announcer 34(2): 177; online at
<http://www.physics.indiana.edu/~hake/DirInstSetback-041104f.pdf>
(420 KB). A pdf version of the slides shown at the meeting is also
available at <http://www.physics.indiana.edu/~hake/AAPT-Slides.pdf>
(132 kB). See also Hake (2005).
Hake, R.R. 2005. "Will the No Child Left Behind Act Promote Direct
Instruction of Science?" Am. Phys. Soc. 50: 851 (2005); online at
<http://tinyurl.com/3x85l5> (256 kB).
Hake, R.R. 2008a. "Design-Based Research in Physics Education
Research: A Review," in Kelly, Lesh, & Baek (2008)]. A
pre-publication version of that chapter is online at
<http://www.physics.indiana.edu/~hake/DBR-Physics3.pdf> (1.1 MB).
Hake, R.R. 2008b. "Language Ambiguities in Education Research,"
submitted to the Journal of Learning Sciences on 21 August but
mindlessly rejected; online at
<http://www.physics.indiana.edu/~hake/LangAmbigEdResC.pdf> (1.2 MB)
and as ref. 54 at <http://www.physics.indiana.edu/~hake>. David Klahr
<http://en.wikipedia.org/wiki/David_Klahr> wrote to me privately
(quoted by permission): "I liked the paper. I think it's very
thoughtful and nuanced. However it is tough going, even for someone
as familiar with the issues (and as favorably cited by you) as I am.
It's a shame that it was rejected, but I wonder if the reviewer just
wasn't up to the very careful reading necessary to really follow your
arguments all the way through. Even though I know this area quite
well, obviously, I did have to really focus to fully understand the
distinctions you were making."
Hake, R.R. 2010a. "Re: Metastudy on impact of inquiry in k-12" online
at <http://groups.yahoo.com/group/Net-Gold/message/31779>. Post of 3
Feb 2010 8:40 am EST to AERA-L, Net-Gold, and PhysLrnR. The abstract
was transmitted to various discussion lists and appears at
<http://hakesedstuff.blogspot.com/2010/02/
re-metastudy-on-impact-of-inquiry-in-k.html>
with a provision for comments.
Hake, R.R. 2010b. "Re: Constructivist Instruction: Success or
Failure?," online on the OPEN! AERA-L archives at
<http://tinyurl.com/yb9443e>. Post of 17 Jan 2010 17:10:41-0800 to
AERA-L, Net-Gold, and PhysLrnR. The abstract only was sent to various
discussion lists. The abstract is also online at
<http://hakesedstuff.blogspot.com/2010/01/
re-constructivist-instruction-success.html>
with a provision for comments.
Hmelo-Silver, C.E., R.G. Duncan, and C.A. Chinn. 2007. "Scaffolding
and Achievement in Problem-Based and Inquiry Learning: A Response to
Kirschner, Sweller, and Clark (2006)," Educational Psychologist
42(2): 99-107; online as a 96 kB pdf at <http://tinyurl.com/2zy783>.
Kelly, A.E., R.A. Lesh, & J.Y. Baek. 2008. "Handbook of Design
Research Methods in Education: Innovations in Science, Technology,
Engineering, and Mathematics Learning and Teaching." Routledge.
Publisher's information at <http://tinyurl.com/4eazqs>; Amazon.com
information at <http://tinyurl.com/5n4vvo>.
Kirschner, P.A., J. Sweller, & R.E. Clark. 2006. "Why Minimal
Guidance During Instruction Does Not Work: An Analysis of the Failure
of Constructivist, Discovery, Problem-Based, Experiential, and
Inquiry-Based Teaching." Educational Psychologist 41(2): 75-86;
online at <http://tinyurl.com/3xmp2m> (176 kB). See also Sweller et
al. (2007) for a response to Hmelo-Silver et al. (2007), Kuhn (2007),
and Schmidt et al. (2007).
Klahr, D. & M. Nigam. 2004. "The equivalence of learning paths in
early science instruction: effects of direct instruction and
discovery learning," Psychological Science 15(10): 661-667; online at
<http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>. See also
Klahr & Li (2005), Strand-Cary & Klahr (2008), and Klahr (2009).
Klahr, D. & J. Li. 2005. "Cognitive Research and Elementary Science
Instruction: From the Laboratory, to the Classroom, and Back,"
Journal of Science Education and Technology 14(2): 217-238; online at
<http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>.
Klahr, D. 2009. "To Every Thing There is a Season, and a Time to
Every Purpose Under the Heavens: What About Direct Instruction" In
Tobias & Duffy (2009); online at
<http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>.
Kuhn. D. 2007. "Is Direct Instruction an Answer to the Right
Question?" Educational Psychologist 42(2): 109-113; online at
<http://www.usc.edu/dept/education/cogtech/publications/kuhn_ep_07.pdf>
(56 kB).
Lipsey, M.W. & D.B. Wilson. 1993. "The Efficacy of Psychological,
Educational, and Behavioral Treatment: Confirmation From
Meta-Analysis," American Psychologist 48(12): 1181-1209; *formerly*
online at at <http://www.vanderbilt.edu/cerm/Amer_Psych_paper.pdf>
(3.4 MB). Reprinted in D. Weisburd & S. Bushway, eds. "Quantitative
Methods in Criminology." Hampshire, UK: Ashgate, 2005. See also
Wilson & Lipsey
(2001).
Minner, D.D. , A.J. Levy, & J. Century. 2009. "Inquiry-Based Science
Instruction - What Is It and Does It Matter? Results from a Research
Synthesis Years 1984 to 2002," Journal of Research in Science
Teaching, Early View (Articles online in advance of print); online at
<http://www3.interscience.wiley.com/cgi-bin/fulltext/123205106/PDFSTART>.
Some Chemed-L and Phys-L subscribers have reported troubles in
downloading this report, but others (including myself) have had no
problem.
NRC. 2000. "Inquiry and the National Science Education Standards: A
Guide for Teaching and Learning," National Academy Press; online at
<http://books.nap.edu/catalog/9596.html>.
Schmidt, H.G., S.M.M. Loyens, T. van Gog, & F. Paas. 2007.
"Problem-Based Learning is Compatible with Human Cognitive
Architecture: Commentary on Kirschner, Sweller, and Clark (2006),"
Educational Psychologist 42(2): 91-97; online as a 72 kB pdf at
<http://tinyurl.com/2uxf6z>.
Strand-Cary, M. & D. Klahr. 2008. "Developing elementary science
skills: Instructional effectiveness and path independence," in
Cognitive Development 23(4), a special issue on "Scientific reasoning
- where are we now?" Guest editors Beate Sodian and Merry Bullock;
online at <http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>.
Sweller, J. , P.A. Kirschner, & R.E. Clark. 2007. "Why Minimally
Guided Teaching Techniques Do Not Work: A Reply to Commentaries,"
Educational Psychologist 42(2): 115-121; online as a 76 kB pdf at
<http://tinyurl.com/2v4led>.
Tobias, Sigmund & T.M. Duffy. 2009. "Constructivist Instruction:
Success or Failure?" Routledge; forward by Robert J. Sternberg,
publisher's information at <http://tinyurl.com/y9xpear>. Amazon.com
information at <http://tinyurl.com/ye8y5xp>. For a *severely*
truncated version see the Google Book preview at
<http://tinyurl.com/yaffdma>. See also the commentary by Hake (2010b).
Whitehead, A. N. 1967. "Aims of Education and other essays. " Free
Press. Amazon.com information at <http://tinyurl.com/yepksuj>.
First published in 1929. Note the "Look Inside" feature.
Wilson, D.B. & M.W. Lipsey. 2001. "The Role of Method in Treatment
Effectiveness Research: Evidence from Meta-Analysis." Psychological
Methods 6(4): 413-429.
Wurman, Z. 2010. "Re: Metastudy on impact of inquiry in k-12," post
of 4 Feb 2010 13:23:52 -0800 to AP-Physics, Biopi-L, EvalTalk,
Physhare, and Physoc; online on the PHYSOC archives at
<http://tinyurl.com/ycfzdlp>. To access the archives of PHYSOC one
needs to subscribe, but that takes only a few minutes by clicking on
<http://listserv.uark.edu/archives/physoc.html> and then clicking on
"Join or leave the list (or change settings)." If you're busy, then
subscribe using the "NOMAIL" option under "Miscellaneous." Then, as a
subscriber, you may access the archives and/or post messages at any
time, while receiving NO MAIL from the list!
.
Other related posts:
- » [net-gold] Metastudy on Impact of Inquiry in K-12 - Response to Wurman - David P. Dillard
