. Date: Wed, 3 Feb 2010 08:40:19 -0800 From: Richard Hake <rrhake@xxxxxxxxxxxxx> Reply-To: Net-Gold@xxxxxxxxxxxxxxx To: PHYSLRNR@xxxxxxxxxxxxxxxxxxxxxxx Cc: AERA-L@xxxxxxxxxxxxxxxxx, Net-Gold@xxxxxxxxxxxxxxx Subject: [Net-Gold] Re: Metastudy on impact of inquiry in k-12 If you reply to this very long (33 kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers. *************************************** ABSTRACT: Joe Bellina (2010), in a post "Metastudy on impact of inquiry in k-12" alerted subscribers to "Inquiry-Based Science Instruction-What Is It and Does It Matter? Results from a Research Synthesis Years 1984 to 2002" [Minner, Levy, & Century (2009)]. Their abstract reads in part (slightly edited): " The goal of the Inquiry Synthesis Project was to synthesize findings from research conducted between 1984 and 2002 to address the research question, "What Is The Impact Of Inquiry Science Instruction On K-12 Student Outcomes?". . . . . Various findings across 138 analyzed studies INDICATE A CLEAR, POSITIVE TREND FAVORING INQUIRY-BASED INSTRUCTIONAL PRACTICES [my CAPS], particularly instruction that emphasizes student active thinking and drawing conclusions from data. Teaching strategies that actively engage students in the learning process through scientific investigations are more likely to increase conceptual understanding than are strategies that rely on more passive techniques, which are often necessary in the current standardized-assessment laden educational environment." Leaving aside my own niggling criticism of their monumental mixed-methods-research effort, Minner et al., even despite the "antipositivist vigilantes," rightfully add another voice to the chorus bemoaning the lack of *operational definitions* for various pedagogical approaches. Among other choristers are: Century (2004), Klahr & Li (2005), Anderson (2007), Hake (2008), Strand-Cary & Klahr (2008), and Klahr (2009). *************************************** Joe Bellina (2010) in his post "Metastudy on impact of inquiry in k-12" wrote [my insert at ". . . .[[insert]]. . . . ]: 'This paper. . . . .[[Minner, Levy, & Century (2009) plus its seven auxiliary Technical Reports, referenced in this post (but not in Minner et al.) as EDC (2006a,b,c,d,e; 2009a,b)]]. . . ., should be of interest to all who are concerned about the effectiveness of student centered guided inquiry instruction." The abstract of Minner et al. (2009) reads in part (slightly edited): " The goal of the Inquiry Synthesis Project was to synthesize findings from research conducted between 1984 and 2002 to address the research question: "What Is The Impact Of Inquiry Science Instruction on K-12 Student Outcomes?". . . . . Various findings across 138 analyzed studies INDICATE A CLEAR, POSITIVE TREND FAVORING INQUIRY-BASED INSTRUCTIONAL PRACTICES [my CAPS], particularly instruction that emphasizes student active thinking and drawing conclusions from data. Teaching strategies that actively engage students in the learning process through scientific investigations are more likely to increase conceptual understanding than are strategies that rely on more passive techniques, which are often necessary in the current standardized-assessment laden educational environment." In this post I shall forego ;-) curmudgeonly comments on the advisability, as I see it, for K-12 education researchers to follow the lead of physicists Halloun & Hestenes (1985a,b) and *directly* measure students' learning through pre/post testing [even despite the rampant pre/post paranoia (Hake, 2006a, 2010a) that plagues some Psychologists, Education specialists, and Psychometricians (PEP's)], using (a) valid and consistently reliable "Conceptual Inventories"; and (b) traditional courses as controls. Conceptual Inventories developed through arduous quantitative and qualitative research by disciplinary experts are currently being used to improve undergraduate - and some high-school - courses in science, technology, engineering, and math (STEM) disciplines (but not psychology!). For references see, e.g.: (1) Hake (2008a,b; 2010a,b; (2) "Workshop on Linking Evidence and Promising Practices in STEM Undergraduate Education" [National Academies (2008)]; and (3) the Wikipedia Entry on "Concept Inventories" at <http://en.wikipedia.org/wiki/Concept_inventory>. [I note that Arizona State University's exemplary "Modeling Instruction Program" <http://modeling.asu.edu/> in which student learning has been rigorously evaluated by pre/post testing, and which "was designated in 2001 by the U.S. Department of Education as one of two exemplary programs in K-12 Science Education out of 27 programs evaluated," is *not* included :-( in the "Bibliography of Studies Included in the Final Inquiry Synthesis Project Analyses" [The Inquiry Synthesis Project (2009b)]. Instead of the above niggling criticism of the monumental mixed-methods-research effort of Minner et al. (2009), I'll focus on a crucial question that Minner et al. sought to answer: "WHAT *IS* "STUDENT-CENTERED GUIDED-INQUIRY INSTRUCTION"?? ........................................ (1) in order to address their self-imposed and very difficult research question: "WHAT IS THE IMPACT OF INQUIRY SCIENCE INSTRUCTION ON K-12 STUDENT OUTCOMES?" ..................(2) Minner, Levy, & Century (2009), after working towards an answer to "(1)", rightfully add another voice to the chorus bemoaning the importance of OPERATIONAL DEFINITIONS in educational research, even despite the "antipositivist vigilantes" [Phillips (2000); Phillips & Burbules (2000), Hake (2006b)]. Minner, Levy, & Century (2009, page 3) cogently explain [bracketed by lines "MLC-MLC-MLC-. . . . . . "; see that article for the references other than Anderson (2007); EDC (2006b) - called by Minner et al. "The Inquiry Synthesis Project (2006b)"; and Kirschner, Sweller, and Clark (2006); my insert at ". . . . [[insert]]. . . ."]: MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC The term inquiry has figured prominently in science education, yet it refers to at least three distinct categories of activities - what scientists do (e.g., conducting investigations using scientific methods), how students learn (e.g., actively inquiring through thinking and doing into a phenomenon or problem, often mirroring the processes used by scientists), and a pedagogical approach that teachers employ (e.g., designing or using curricula that allow for extended investigations). However, whether it is the scientist, student, or teacher who is doing or supporting inquiry, the act itself has some core components. The NRC describes these core components from the learner's perspective as ''essential features of classroom inquiry'' (NRC, 2000, p. 25) including: (1) Learners are engaged by scientifically oriented questions. (2) Learners give priority to evidence, which allows them to develop and evaluate explanations that address scientifically oriented questions. (3) Learners formulate explanations from evidence to address scientifically oriented questions. (4) Learners evaluate their explanations in light of alternative explanations, particularly those reflecting scientific understanding. (5) Learners communicate and justify their proposed explanations. The National Science Education Standards would add one more to this list: learners design and conduct investigations (NRC, 1996). There is more consensus regarding what students should learn about scientific inquiry than how teachers should instruct students (Anderson, 2007). For example, within each of the features of classroom inquiry listed above, there can be varying degrees of direction from the teacher. The amount of direction and decision-making done by the teacher versus the student has produced distinctions such as open and guided inquiry (NRC, 2000). However, these distinctions are often poorly articulated by scholars and practitioners alike. For example, the way in which ''minimally guided instruction' '- said. . . . [by Kirschner, Sweller, and Clark (2006)]]. . . . . to be synonymous with constructivist, discovery, problem-based, experiential, and inquiry-based instruction - was defined by Kirschner, Sweller, and Clark (2006) IS NOT THE WAY THAT MOST INQUIRY-ORIENTED PRACTITIONERS OR RESEARCHERS WOULD DESCRIBE THESE KINDS OF INSTRUCTIONAL APPROACHES - which do have instructional guidance throughout the learning process (Hmelo-Silver, Duncan, & Chinn, 2007). However, IT IS PRECISELY THE LACK OF A SHARED UNDERSTANDING OF THE DEFINING FEATURES OF VARIOUS INSTRUCTIONAL APPROACHES THAT HAS HINDERED SIGNIFICANT ADVANCEMENT IN THE RESEARCH COMMUNITY ON DETERMINING EFFECTS OF DISTINCT PEDAGOGICAL PRACTICES. . . . . [[My CAPS.]]. . . . . Therefore, in the Classifying Inquiry Science Instruction subsection, we will present a conceptual framework and definition of inquiry instruction used in this research. This framework was built upon the findings from reviewing several resources (EDC (2006b), one of which was the National Science Education Standards. A diversity of resources were necessary since there was not consensus in the literature about the specific components of inquiry science instruction; therefore, looking for similarities across existing ''definitions'' of inquiry was necessary to develop the OPERATIONAL DEFINITION. . . . . [[My CAPS.]]. . . . . articulated in our framework. This framework . . . . [[see page 6, Table 1 "Inquiry science instruction conceptual framework"]]. . . . . integrates all of the aspects of inquiry noted above but is more functionally specified. MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC-MLC The "lack of a shared understanding of the defining features of various instructional approaches" and the lack of *operational definitions* for various pedagogical approaches has also been bemoaned by, e.g. : 1. Century (2004) in "Better Science Instruction Requires Clear Definitions." Century wrote [bracketed by lines "CCCCCC. . . . . ."; my insert at ". . . .[[insert]]. . . ."]: CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC Thanks go to Sara L. Ford, the teacher quoted in the final paragraphs of your front-page article on science instruction "NCLB Could Alter Science Teaching" . . . .[[Cavenaugh, 2004)]]. . . , for articulating a fundamental flaw in the debate between direct instruction and discovery learning. Her suggestion that individuals are too quick to label these instructional approaches cuts to the heart of one of the most critical challenges for science education researchers and their practitioner colleagues today: WE HAVE NO SHARED UNDERSTANDING OF THE TERMS WE USE-"DIRECT INSTRUCTION," "INQUIRY," "DISCOVERY LEARNING," AND "HANDS-ON"-IN OUR DEBATES OF EFFECTIVE SCIENCE INSTRUCTION. . . . . [My CAPS.]. . . .. FOR THE PAST THREE YEARS, WE AT THE CENTER FOR SCIENCE EDUCATION, IN NEWTON, MASS., HAVE BEEN CONDUCTING A SYNTHESIS OF RESEARCH THAT SEEKS TO ANSWER THE QUESTION, "WHAT IS THE EFFECT OF INQUIRY SCIENCE INSTRUCTION ON STUDENT OUTCOMES?" . . . . [My CAPS.] . . . . We are synthesizing all research conducted on this topic since 1984, and have so far reviewed more than 1,000 documents. One of the greatest challenges of our work has been finding a way to describe "inquiry" that accommodates the many ways it has historically been, and continues to be, used and understood. Even when those in the field acknowledge that there are varied understandings of terms, as was done in your recent article. . . . [[[Cavenaugh, 2004)]]. . . . ., we continue to debate the relative merits of these instructional approaches as though our failure to clearly articulate terms did not matter. Our group has seen first hand what one would hope would be obvious: Clarifying terms does matter. WITHOUT CLEAR DEFINITIONS, WE WILL JUST TALK PAST EACH OTHER AND WILL NEVER UNDERSTAND WHAT IS AND ISN'T EFFECTIVE INSTRUCTION. [My CAPS] At best, DEBATES USING UNDEFINED TERMS ARE UNPRODUCTIVE . . . . [My CAPS.] . . . . ,. At worst, they undermine our ability to build a body of knowledge in the field. CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC 2. Klahr & Li (2005) in "Cognitive Research and Elementary Science Instruction: From the Laboratory, to the Classroom, and Back." They wrote: "ONE THING IS CLEAR FROM ALL OF THIS: IT IS ESSENTIAL FOR THE FIELD OF EDUCATION TO MAKE MUCH MORE PRECISE USE OF TERMINOLOGY BEFORE MOVING ON TO PUBLIC DEBATES AND POLICY DECISIONS. [My CAPS.] Indeed, it is surprising that when education researchers and science educators join in heated debates about discovery learning, direct instruction, inquiry, hands-on, or minds-on, they usually abandon one of the foundations of science-the operational definition. The field of science cannot advance without clear, unambiguous, operationally defined, and replicable procedures. Education science is no exception. 3. Anderson (2007) in "Inquiry as an organizing theme for science curricula." Anderson wrote: "Such conversation. . . . [[about 'inquiry']] . . . . will be much more profitable. . . .if we recognize that 'INQUIRY' IS AN IMPRECISE WORD. [My CAPS.] Using the word in a conversation about science education is a bit like using the word 'romance' in a conversation about human relationships. It has different meanings in varied contexts, and is hard to guess what particular meaning a given speaker has in mind when the word is used. If the word is to continue to be useful we will have to press for clarity when the word enters a conversation and not assume we know the intended meaning." 4. Hake (2008a) in "Language Ambiguities in Education Research." I wrote [: "BUT WAIT! What do Klahr & Nigam (2004); Kirschner, Sweller, and Clark (2006); and other education researchers, instructors, administrators, activists, and policy makers *mean* by 'discovery learning,' and what do they *mean* by 'direct instruction'? In criticisms [Hake (2004, 2005) of the California Curriculum Commission's anti-hands-on 'Criteria For Evaluating K-8 Science Instructional Materials In Preparation for the 2006 Adoption,' I OPINED THAT POPULAR PEDAGOGIC TERMS SUCH AS 'DISCOVERY LEARNING,' 'DIRECT INSTRUCTION,' 'HANDS-ON ACTIVITIES,' 'ACTIVE LEARNING,' 'COOPERATIVE LEARNING,' 'INQUIRY,' AND 'INTERACTIVE ENGAGEMENT,' SHOULD BE *OPERATIONALLY DEFINED* [see, e.g. Holton & Brush (2001), Phillips (2000)], even despite the 'antipositivistic vigilantes' [Phillips (2000), Phillips & Burbules (2000)]. More generally, rigorous operations should be defined for distinguishing pedagogic method X from other methods Y, Z, A, B, C, . . ." 5. Strand-Cary & Klahr (2008) in "Developing elementary science skills: Instructional effectiveness and path independence." They wrote: "CLEAR OPERATIONAL DEFINITIONS ARE USEFUL [my CAPS] within studies (i.e., one can go beyond arbitrary labels to determine exactly what kinds of procedures and measures were used), but also facilitate cross study comparisons and help determine whether studies should be compared to each other at all." 6. Klahr (2009) in "To Every Thing There is a Season, and a Time to Every Purpose Under the Heavens: What About Direct Instruction." Klahr wrote [bracketed by lines "KKKKK. . . . "; see Klahr's article for references other than Kirschner, Sweller, and Clark (2006) and EDC (2006b); my insert at ". . . .[[insert]]. . . ."]: KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK Over the past 20 years or so, and culminating in the critique (Kirschner, Sweller, & Clark, 2006) and debate at the 2007 AERA meeting that motivated this volume, there have been extensive and heated exchanges among education researchers, learning scientists, and science educators about "discovery learning," "direct instruction," "authentic inquiry," and "hands-on science" (Adelson, 2004; Begley, 2004; EDC, 2006b; Hmelo-Silver, Duncan, & Chinn, 2007; Janulaw, 2004; Klahr, Triona, & Williams, 2007; Kuhn, 2007; Ruby, 2001; Strauss, 2004; Tweed, 2004; Schmidt, Loyens, van Gog, & Paas, 2007). However, these arguments typically fail to establish a common vocabulary to define the essential aspects of the types of instruction being compared. I believe that in order to advance our ability to create effective instructional procedures, our field needs to become much more precise in the terminology it uses to describe instructional contexts and procedures, before moving on to advocacy about curriculum design. IN THE AREA OF SCIENCE EDUCATION, MORE THAN OTHERS, IT IS PARTICULARLY TROUBLING-AND IRONIC-THAT THESE DEBATES OFTEN ABANDON ONE OF THE FOUNDATIONS OF SCIENCE: THE OPERATIONAL DEFINITION. But a scientific field cannot advance without clear, unambiguous, and replicable procedures. [My CAPS] KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK Richard Hake, Emeritus Professor of Physics, Indiana University 24245 Hatteras Street, Woodland Hills, CA 91367 Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands. <rrhake@xxxxxxxxxxxxx> <http://www.physics.indiana.edu/~hake/> <http://www.physics.indiana.edu/~sdi/> <http://HakesEdStuff.blogspot.com/> <http://iub.academia.edu/RichardHake> The true meaning of a term is found by observing what a man does with it, not what he says about it. P.W. Bridgman (1927, 1960) "In the course of coming into contact with empirical material, physicists have gradually learned how to pose a question properly. Now proper questioning often means that one is more than half the way towards solving the problem." Werner Heisenberg (1958). REFERENCES [Tiny URL's courtesy <http://tinyurl.com/create.php>. All URL's were accessed on 28-30 January 2010.] Abel, S. & N. Lederman, eds. 2007. "Handbook of Research on Science Education." Lawrence Erlbaum; publisher's information at <http://tinyurl.com/yehgwls>. Amazon.com information at <http://tinyurl.com/ycyadtv>. A severely truncated version is online as a Google book preview at <http://tinyurl.com/yc9lylx>. Anderson, R.D. 2007. "Inquiry as an organizing theme for science curricula," in Abel & Lederman (2007, pp. 807-830). A severely expurgated version of Anderson's chapter is online in as Google book preview at <http://tinyurl.com/yc29324> (click on "Page 807 >"). Bellina, J. 2010."Metastudy on impact of inquiry in k-12," online on the OPEN Phys-L archives at <https://carnot.physics.buffalo.edu/archives/2010/1_2010/msg00255.html>. Post of 17 Jan 2010 22:10:00-0500 to Chemed-L, generalscience@xxxxxxxxx, physicalscience@xxxxxxxxx, Phys-L, PhysLrnR, & science@xxxxxxxxxxxxxxxxxxxxxxxxxxx Bridgman, P.W. 1960. "Logic of Modern Physics." McMillan. First published in 1927. Amazon.com information at <http://tinyurl.com/y8tnz3s>. Cavanagh, S. 2004. "NCLB Could Alter Science Teaching," Education Week 24(11): 12-13, 10 November; online at <http://tinyurl.com/2n4mrd>, scroll to the APPENDIX. Century, J.R. 2004. "Better Science Instruction Requires Clear Definitions," Education Week 24(15): 32, 8 December; online to subscribers at <http://tinyurl.com/ydq25fj>. EDC. 2006a. " The Inquiry Synthesis Project, Center for Science Education, Technical Report #1: Generating the Synthesis Sample of Studies; online at <http://cse.edc.org/products/inquirysynth/pdfs/technicalReport1.pdf> (156 kB). Here and below in the EDC Technical Report references, I have corrected Minner et al.'s (2009) URL's by replacing their "/ /" with "//". (Inclusion of the former in the URL yields a "not found" panel on my computer.) EDC. 2006b. " The Inquiry Synthesis Project, Center for Science Education, Technical Report #2: Conceptualizing Inquiry Science Instruction; online at <http://cse.edc.org/products/inquirysynth/pdfs/technicalReport2.pdf> (132 kB). EDC. 2006c. " The Inquiry Synthesis Project, Center for Science Education, Technical Report #3: Operationalizing the Inclusion/Exclusion Coding Process; online at <http://cse.edc.org/products/inquirysynth/pdfs/technicalReport3.pdf> (112 kB). EDC. 2006d. " The Inquiry Synthesis Project, Center for Science Education, Technical Report #4: Report-Study Reconciliation Process; online at <http://cse.edc.org/products/inquirysynth/pdfs/technicalReport4.pdf> (108 kB). EDC. 2006e. " The Inquiry Synthesis Project, Center for Science Education, Technical Report #5: Operationalizing the Inquiry Science Instruction Coding Process; online at <http://cse.edc.org/products/inquirysynth/pdfs/technicalReport5.pdf> (116 kB). EDC. 2009a. " The Inquiry Synthesis Project, Center for Science Education, Technical Report #6: Operationalizing the Coding of Research Rigor, Context, and Study Findings; online at <http://cse.edc.org/products/inquirysynth/pdfs/technicalReport6.pdf> (164 kB). EDC. 2009b. " The Inquiry Synthesis Project, Center for Science Education, Technical Report #7: Bibliography of Studies Included in Final Inquiry Synthesis Project Analyses; online at <http:/ /cse.edc.org/products/inquirysynth/pdfs/technicalReport7.pdf> (136 kB). Hake, R.R. 2004. "Direct Science Instruction Suffers a Setback in California - Or Does It?" AAPT Announcer 34(2): 177; online at <http://www.physics.indiana.edu/~hake/DirInstSetback-041104f.pdf> (420 KB). A pdf version of the slides shown at the meeting is also available at <http://www.physics.indiana.edu/~hake/AAPT-Slides.pdf> (132 kB). Hake, R.R. 2005. "Will the No Child Left Behind Act Promote Direct Instruction of Science?" Am. Phys. Soc. 50: 851 (2005); online at <http://tinyurl.com/3x85l5> (256 kB). Hake, R.R. 2006a. "Possible Palliatives for the Paralyzing Pre/Post Paranoia that Plagues Some PEP's," Journal of MultiDisciplinary Evaluation, Number 6, November, online at <http://survey.ate.wmich.edu/jmde/index.php/jmde_1/article/view/41/50> [PEP's = psychologists, education specialists, and psychometricians.] This even despite the admirable anti-alliteration advice at psychologist Donald Zimmerman's site <http://mypage.direct.ca/z/zimmerma/> to "Always assiduously and attentively avoid awful, awkward, atrocious, appalling, artificial, affected alliteration." Hake, R.R. 2006b. "Is the 'Scientific Method' the Same as 'Positivism' ?" online on the OPEN! AERA-D archives at <http://tinyurl.com/yzxyl9v>. Post of 26 Sep 2006 12:35:55-0700 to AERA-D, HOPOS-L, IFETS, PhysLrnR, and POD. Hake, R.R. 2008a. "Language Ambiguities in Education Research," submitted to the Journal of Learning Sciences on 21 August but mindlessly rejected; online at <http://www.physics.indiana.edu/~hake/LangAmbigEdResC.pdf> (1.2 MB) and as ref. 54 at <http://www.physics.indiana.edu/~hake>. Klahr wrote to me privately (quoted by permission): "I liked the paper. I think it's very thoughtful and nuanced. However it is tough going, even for someone as familiar with the issues (and as favorably cited by you) as I am. It's a shame that it was rejected, but I wonder if the reviewer just wasn't up to the very careful reading necessary to really follow your arguments all the way through. Even though I know this area quite well, obviously, I did have to really focus to fully understand the distinctions you were making." Hake, R.R. 2008b. "Design-Based Research in Physics Education Research: A Review," in Kelly, Lesh, & Baek (2008)]. A pre-publication version of that chapter is online at <http://www.physics.indiana.edu/~hake/DBR-Physics3.pdf> (1.1 MB). Hake, R.R. 2010a. "Should We Measure Change? Yes!" online at <http://www.physics.indiana.edu/~hake/MeasChangeS.pdf> (2.5 MB) and as ref. 43 at <http://www.physics.indiana.edu/~hake>. To appear as a chapter in "Evaluation of Teaching and Student Learning in Higher Education" [Hake (in preparation)], a Monograph of the American Evaluation Association <http://www.eval.org/>. A severely truncated version is online at Hake (2006a). Hake, R.R. 2010b. "Re: Constructivist Instruction: Success or Failure?," online on the OPEN! AERA-L archives at <http://tinyurl.com/yb9443e>. Post of 17 Jan 2010 17:10:41-0800 to AERA-L, Net-Gold, and PhysLrnR. The abstract only was sent to various discussion lists. The abstract is also online at <http://hakesedstuff.blogspot.com/2010/01/re-constructivist-instruction-success.html> with a provision for comments. Halloun, I. & Hestenes, D. 1985a. "The initial knowledge state of college physics," Am. J. Phys. 53(11): 1043-1055; online at <http://modeling.asu.edu/R&E/Research.html>. Contains the "Mechanics Diagnostic" test (omitted from the online version), precursor to the widely used "Force Concept Inventory" [Hestenes et al. (1992)]. Halloun, I. & D. Hestenes. 1985b. "Common sense concepts about motion," Am. J. Phys. 53(11): 1056-1065; online at <http://modeling.asu.edu/R&E/Research.html>. Hestenes, D., M. Wells, & G. Swackhamer. (1992). "Force Concept Inventory," Phys. Teach. 30(3): 141-158; online (except for the test itself) at <http://modeling.asu.edu/R&E/Research.html>. The 1995 revision by Halloun, Hake, Mosca, & Hestenes is online (password protected) at the same URL, and is currently available in 16 languages: Chinese, Czech, English, Finnish, French, German, Greek, Italian, Malaysian, Persian, Portuguese, Russian, Spanish, Slovak, Swedish, & Turkish. Heisenberg, W. 1958, 2007. "Physics & Philosophy: The Revolution in Modern Sciences." Harper Perennial Modern Classics. (Evidently first published in 1958 - originally published as "Physik und Philosophie.") Amazon.com information at <http://tinyurl.com/yd45csz>. Note the searchable "Look Inside" feature. Holton, G. & S.G. Brush. 2001. "Physics the Human Adventure: From Copernicus to Einstein and Beyond." Rutgers University Press, pp. 161-162. Amazon.com information at <http://tinyurl.com/yhz6xqa>. A minimally useful Google "book preview" is online at <http://tinyurl.com/2nfts6>. Operational definitions are discussed in Chapter 12 "On the Nature of Concepts." Kelly, A.E., R.A. Lesh, & J.Y. Baek. 2008. "Handbook of Design Research Methods in Education: Innovations in Science, Technology, Engineering, and Mathematics Learning and Teaching." Routledge. Publisher's information at <http://tinyurl.com/4eazqs>; Amazon.com information at <http://tinyurl.com/5n4vvo>. Kirschner, P.A., J. Sweller, & R.E. Clark. 2006. "Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching." Educational Psychologist 41(2): 75-86; online at <http://tinyurl.com/3xmp2m> (176 kB). Klahr, D. & M. Nigam. 2004. "The equivalence of learning paths in early science instruction: effects of direct instruction and discovery learning," Psychological Science 15(10): 661-667; online at <http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>. Klahr, D. & J. Li. 2005. "Cognitive Research and Elementary Science Instruction: From the Laboratory, to the Classroom, and Back," Journal of Science Education and Technology 14(2): 217-238; online at <http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>. Klahr, D. 2009. "To Every Thing There is a Season, and a Time to Every Purpose Under the Heavens: What About Direct Instruction" In Tobias & Duffy (2009); online at <http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>. Minner, D.D. , A.J. Levy, & J. Century. 2009. "Inquiry-Based Science Instruction - What Is It and Does It Matter? Results from a Research Synthesis Years 1984 to 2002," Journal of Research in Science Teaching, Early View (Articles online in advance of print); online at <http://www3.interscience.wiley.com/cgi-bin/fulltext/123205106/PDFSTART>. Some Chemed-L and Phys-L subscribers have reported troubles in downloading this report, but others (including myself) have had no problem. National Academies. 2008. "Workshop on Linking Evidence and Promising Practices in STEM Undergraduate Education": Commissioned Papers at <http://www7.nationalacademies.org/bose/PP_Commissioned_Papers.html>. Phillips, D.C. 2000. "Expanded Social Scientist's Bestiary: a guide to fabled threats to, and defenses of, naturalistic social science." Rowman & Littlefield - information at <http://tinyurl.com/ycmlvy>. Amazon.com information at <http://tinyurl.com/ydm5utt>. See especially Chapter 9 on "Positivism." Phillips, D.C. & N.C. Burbules. 2000. "Postpositivism and Educational Research." Rowman & Littlefield; publisher's information at <http://tinyurl.com/yncvls >. Amazon.com information <http://tinyurl.com/yelju39>. See especially "Mistaken accounts of positivism," pp. 11-14. Strand-Cary, M. & D. Klahr. 2008. "Developing elementary science skills: Instructional effectiveness and path independence," in Cognitive Development 23(4): 488-511, a special issue on "Scientific reasoning - where are we now?" Guest editors Beate Sodian and Merry Bullock; online at <http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>. Tobias, Sigmund & T.M. Duffy. 2009. "Constructivist Instruction: Success or Failure?" Routledge; forward by Robert J. Sternberg, publisher's information at <http://tinyurl.com/y9xpear>. Amazon.com information at <http://tinyurl.com/ye8y5xp>. For a *severely* truncated version see the Google Book preview at <http://tinyurl.com/yaffdma>. See also the commentary by Hake (2010b). .