. . Date: Sat, 16 Apr 2011 12:52:49 -0700 From: Richard Hake <rrhake@xxxxxxxxxxxxx> Reply-To: Net-Gold@xxxxxxxxxxxxxxx To: AERA-L@xxxxxxxxxxxxxxxxx Cc: Net-Gold@xxxxxxxxxxxxxxx Subject: [Net-Gold] The 'Teacher Effect' - Response to Hansen . . If you reply to this long (27 kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers. . **************************************** . ABSTRACT: In response to "Is the 'Teacher Effect' the Dominant Factor in Students 'Academic Gain?" [Hake (2011a)], Math-Teach's Robert Hansen wrote (paraphrasing): . "Is there a complete high-school assessment endorsed by PER [Physics Education Research]? The FCI [Force Concept Inventory] assesses only one component of physics and I think that keeps people, especially physicists, from getting too excited over normalized gains on the FCI." . Hansen is either dismissive or oblivious of the fact that PER is concerned with (a) students' conceptual understanding, (b) students' ability to solve non-algorithmic problems, and (c) at least 10 other capabilities listed in this post. I suggest over 30 references to the PER literature that might reduce Hansen's confusion. . **************************************** . In response to "Is the 'Teacher Effect' the Dominant Factor in Students 'Academic Gain?" [Hake (2011a)], Math-Teach's Robert Hansen (2011a) in a post "Re: Is the 'Teacher Effect' the Dominant Factor in Students 'Academic Gain?" wrote (with typos corrected): . ". . . . do you have an example of a full high school physics assessment endorsed by the PER group? The FCI assesses only a component of physics and I think that keeps people, especially people involved in physics, from getting too excited over normalized gains on the FCI. " . Unfortunately, although Robert Hansen often criticizes Physics Education Research (PER), he's evidently never bothered to find out what's going on that field. . For example, Hansen appears to be dismissive or oblivious of the fact that PER is concerned with not only: . (a) students' conceptual understanding, but also . (b) students' ability to solve non-algorithmic problems, as well as . (c) many other student capabilities (see the section below bracketed by "H2-H2-H2-H2-"). . For example, the last paragraph of "Interactive-engagement vs traditional methods: A six thousand- student survey of mechanics test data for introductory physics courses" [Hake (1998a)] is [my CAPS]: . "Results for 30 (N = 3259) of the above 62 courses ON THE PROBLEM-SOLVING MECHANICS BASELINE TEST of Hestenes-Wells . . . .[[Hestenes & Wells (1992)]]. . . . imply that IE strategies enhance problem-solving ability. The conceptual AND PROBLEM-SOLVING TEST RESULTS strongly suggest that the classroom use of Interactive Engagement (IE) E methods can increase mechanics-course effectiveness well beyond that obtained in traditional practice." . Regarding the Mechanics Baseline Test, in Hake (1998a) I wrote [bracketed by lines "H1-H1-H1-. . . .": . H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1 . The Mechanics Baseline test is designed to measure more quantitative aspects of student understanding than the FCI. It is usually given only as a posttest. Figure 4 shows a plot of the average percentage score on the problem-solving Mechanics Baseline (MB) posttest vs the average percentage score on the FCI posttest for all the available data [Hake (1998b)]. The solid line is a least-squares fit to the data points. THE TWO SCORES SHOW AN EXTREMELY STRONG POSITIVE CORRELATION WITH COEFFICIENT r = + 0.91. . Such a relationship is not unreasonable because the MB test (unlike most traditional algorithmic-problem physics exams) requires conceptual understanding in addition to some mathematical skill and critical thinking. Thus the MB test is more difficult for the average student, as is also indicated by the fact that MB averages tend to be about 15% below FCI averages, i.e., the least-squares-fit line is nearly parallel to the diagonal (%MB = %FCI) and about 15% points below it. . It is sometimes objected that the problems on the MB test do not sufficiently probe more advanced abilities such as those required for problems known as: "context rich" [Heller et al. (1992), Heller & Hollabaugh (1992), UMinnPhysEd (2011); "experiment" [Van Heuvelen (1995); "goal-less" [D'Alessandris (1994)]); "out-of-lab" [Hake (2011)]; or Fermi [Morrison (1963), UMaryFPS (2011)]. . On the other hand, some instructors object that neither the MB problems nor those indicated above are "real" problems because they are somewhat different from "Halliday-Resnick back-of- chapter problems." Considering the differences in outlook, it may be some time before a more widely accepted problem-solving test becomes available. . H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1-H1 . In addition to conceptual understanding and non-algorithmic problem solving, in Hake (2002) I wrote: . H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2 . Does the class average normalized gain <g> for the FCI, MD, or FMCE. . . . [[Force Concept Inventory, Mechanics Diagnostic, Force Motion Concept Evaluation]]. . . . provide a definitive assessment of the overall effectiveness of an introductory physics class? NO! It assesses *only the attainment of a MINIMAL conceptual understanding of mechanics.* In some first-semester or first-quarter introductory physics courses, subjects other than mechanics are often covered. The effectiveness of the course in promoting student understanding of those topics would not, of course, be assessed by the normalized gain on the FCI, MD, or FMCE. . Furthermore, as indicated in. . . . [[ the editor suppressed :-( ]]. . . . Hake (1998b), among desirable outcomes of the introductory course that <g> DOES NOT measure directly are students': . 1. satisfaction with and interest in physics; . 2. understanding of the nature, methods, and limitations of science; . 3. understanding of the processes of scientific inquiry such as experimental design, control of variables dimensional analysis, order-of-magnitude estimation, thought experiments, hypothetical reasoning, graphing, and error analysis; . 4. ability to articulate their knowledge and learning processes; . 5. ability to collaborate and work in groups; . 6. communication skills; . 7. ability to solve real-world problems; . 8. understanding of the history of science and the relationship of science to society and other disciplines; . 9. understanding of, or at least appreciation for, "modern" physics; . 10. ability to participate in authentic research. H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2-H2 . Despite all the above, recent Hansen "PERclamations" are: . A. "If the PER is to be taken seriously they need to show what their intended result is and the best way to show that is with what their final exams look like. This is exactly, among other things, what did the NCTM in and relegated them to the back of the standards bus. They would never finalize their theories and state what they expected of the student in each of the courses. Constant talk with no clear goal." Hansen (2011b) . B. "I realize your confusion Hake, in all these blurbs and quotations. I run across the same problem when I immerse myself into a curriculum or pedagogy that spends more time with the theory of education than with education. I found it best at that point to take a step back and examine the nature and complexity of the problems the students are expected to solve at it's conclusion. It is so much easier to define the expertise we expect in terms of problems than it is to define what expertise is or how it works. This is why a curriculum has to be married to its work and results more than to its pedagogical theory or it isn't much of a curriculum, it would just be theory." . Hansen (2011c) . Regarding "A": PER, as is education research generally, is not a monolith, and no agreement among all PER's as to "what is expected of the student in each of the courses" has been achieved. . Regarding "B": I realize Hansen's confusion. In fact I know of few people in the ADLsphere [Academic Discussion List Sphere- for a guide see Hake (2010)] who are more confused than Hansen. Before indulging in more of his vacuous PERclamations, Hansen might do well to spend a few hours familiarizing himself with the PER literature. Over thirty suggestions on where he might start are preceded by asterisks * in the REFERENCE list below. . . . Richard Hake, Emeritus Professor of Physics, Indiana University Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands President, PEdants for Definitive Academic References which Recognize the Invention of the Internet (PEDARRII) <rrhake@xxxxxxxxxxxxx> <http://www.physics.indiana.edu/~hake> <http://www.physics.indiana.edu/~sdi> <http://HakesEdStuff.blogspot.com> <http://iub.academia.edu/RichardHake> . . . "The premium so often put in schools upon external 'discipline,' and upon marks and rewards, upon promotion and keeping back, are the obverse of the lack of attention given to life situations in which the meaning of facts, ideas, principles, and problems is vitally brought home." John Dewey (1916) . . . REFERENCES [All URL's accessed on 16 April 2011; some shortened by <http://bit.ly/>.] . . D'Alessandris, P. 1994. "The Development of Conceptual Understanding and Problem-Solving Skills through Multiple Representations and Goal-less Problems," AAPT Announcer 24(4): 47. . Dewey, J. 1916. "The Nature of Realization or Appreciation"in "Democracy and Education : an Introduction to the Philosophy of Education," online at <http://bit.ly/evICPb>, Section 1.1. . Dizikes, P. 2010. "Rough calculations: Sanjoy Mahajan's new book, Street-Fighting Mathematics, lays out practical tools for educated guessing and down-and-dirty problem-solving,' MITnews, 29 March; online at <http://bit.ly/dVxLUY>. . Epstein, J. 2007 "Development and Validation of the Calculus Concept Inventory," in "Proceedings of the Ninth International Conference on Mathematics Education in a Global Community," 7-12 September, edited by Pugalee, Rogerson, & Schinck; online as a 48 kB pdf at <http://bit.ly/bqKSWJ>. . *Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A six thousand- student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66(1), 64-74 (1998); online at <http://bit.ly/d16ne6>. See also Hake (1998b). . *Hake, R.R. 1998b. "Interactive- engagement methods in introductory mechanics courses," online at <http://bit.ly/aH2JQN>. Submitted on 6/19/98 to the "Physics Education Research Supplement to AJP" (PERS), but rejected :-( by its editor on the grounds that the very transparent, well-organized, and crystal-clear Physical-Review-type data tables were "impenetrable"! . *Hake, R.R. 2001. "The math education research community (LONG!!), on the PhysLrnR <http://bit.ly/heo5j9>. Post of 3 Dec 2001 14:02:08-0800. To access the archives of PhysLnR one needs to subscribe :-(, but that takes only a few minutes by clicking on <http://bit.ly/beuikb> and then clicking on "Join or leave the list (or change settings)." If you're busy, then subscribe using the "NOMAIL" option under "Miscellaneous." Then, as a subscriber, you may access the archives and/or post messages at any time, while receiving NO MAIL from the list! [Contrasts PER with MER (Mathematics Education Research).] . *Hake, R.R. 2003a. "Can Mathematicians Learn Anything from Physics/Astronomy Education Research? online on the OPEN! Math-Teach archives at <http://bit.ly/f3jA7k>. Post of 3 Sep 2003 12:26:34 -0700 to ASSESS, Biopi-L, Chemed-L, EvalTalk, FYA, Math-Teach, Phys-L, PhysLrnR, and POD. For a Galilean dialogue on the differences between PER and MER see Hake (2003b). ! [Contrasts PER with MER (Mathematics Education Research).] . *Hake, R.R. 2003b. "Re: Math Education Research," online on the OPEN! Math-Teach archives at <http://bit.ly/gwFiGw>. Post of 12 Feb 2003bv21:01:38-0800 Math-Teach, Phys-L, and PhysLrnR. ! [Gives a Galilean dialogue on the differences between PER and MER.] . Hake, R.R. 2010. "A Guide to the ADLsphere: Over Eighty Academic Discussion Lists With Links To Their Archives & Search Engines," online as a 3.9 MB pdf at <http://bit.ly/970OZr>. . Hake, R.R. 2011a. "Is the 'Teacher Effect' the Dominant Factor in Students' Academic Gain?" online on the OPEN! AERA-L archives at <http://bit.ly/g6UWUZ>. Post of 7 Apr 2011 17:51:59-0700 to AERA-L and Net-Gold. The abstract and link to the complete post were transmitted to various discussion lists and are also on my blog "Hake'sEdStuff" at <http://bit.ly/ifvkSz>. . *Hake, R.R. 2011b. Socratic Dialog-Inducing (SDI) Lab Web Site <http://www.physics.indiana.edu/~sdi>." Out-of-lab problems OLP's are contained in SDI Lab #1 (p. 27), SDI Lab #3 (pp. 18-19, 37-38), and SDI Lab #4 (pp.14, 19, 21, 25). . Hansen, R. 2011a. "Re: Is the 'Teacher Effect' the Dominant Factor in Students 'Academic Gain?" on the OPEN! Math-Teach archives at <http://bit.ly/gCGm8X>. Post of 9 Apr 2011 3:30 PM (the Math Forum doesn't specify the time zone :-( ) . . Hansen, R. 2011b. "Re: Is the 'Teacher Effect' the Dominant Factor in Students 'Academic Gain?" on the OPEN! Math-Teach archives at <http://bit.ly/eqWA52>. Post 10 Apr 10:51 PM (the Math Forum doesn't specify the time zone :-( ). . Hansen, R. 2011c. "Re: Question About AP #2" on the OPEN! Math-Teach archives at <http://bit.ly/gd4XBm>. Post 14 Apr 143:27 PM (the Math Forum doesn't specify the time zone :-( ). . *Heller, P., R. Keith, S. Anderson. 1992. "Teaching problem solving through cooperative grouping, Part 1: Group vs individual problem solving," Am. J. Phys. 60(7): 627-636; online to subscribers at <http://bit.ly/gKqbYl>. . *Heller, P. & M. Hollabaugh. 1992. "Teaching problem solving through cooperative grouping, Part 2: Designing problems and structuring groups," Am. J. Phys. 60(7): 637-644; online to subscribers at <http://bit.ly/gKqbYl>. See also UMinnPhysEd (2011). . *Henderson, C., E. Yerushalmi, V.H. Kuo, P. Heller, and K. Heller. 2004. "Grading student problem solutions: The challenge of sending a consistent message," Am. J. Phys. 72(2):, 164-169; online at <http://bit.ly/e7siYJ>. . *Heron, P.R.L. & D. Meltzer. 2005. "The future of physics education research: Intellectual challenges and practical concerns," Am. J. Phys. 73(5): 390-394; online as a 56 kB pdf at <http://bit.ly/axznvY>. They wrote: "Helping students to approach novel problems in a systematic fashion is a major goal of physics instruction. It also is one of the most difficult goals to achieve, although significant success has been reported [Van Heuvelen (1991), Van Heuvelen & Zou (2001), Heller et al. (1992), Heller & Hollabaugh (1992), Reif & Scott (1999)]. However, much remains unknown. Efforts to understand the interrelationships among conceptual knowledge, mathematical skills, and logical reasoning ability should significantly enhance our progress toward helping students become better problem solvers [Leonard et al. (1996), Henderson et al. (2004), Reif & Scott (1999)]. . *Hestenes, D. and M. Wells. 1992. "A Mechanics Baseline Test," Phys. Teach 30(3): 159-166; online to subscribers at <http://bit.ly/ePQlY6>. . *Hsu, L., E. Brewe, T.M. Foster, & K.A. Harper. 2004. "Resource Letter RPS-1: Research in problem solving," Am. J. Phys. 72(9): 1147-1156; online to subscribers at <http://bit.ly/dJvsAm>. . *Kim, E. & S-J Pak. 2002. "Students do not overcome conceptual difficulties after solving 1000 traditional problems," Am. J. Phys. 70(7): 759-765; online to subscribers at <http://bit.ly/eaHjo6>. . *Labov, J.B., S.R. Singer, M.D. George, H.A. Schweingruber, & M.L. Hilton. 2009. "Effective Practices in Undergraduate STEM Education Part 1: Examining the Evidence," CBE Life Sci Educ 8(3): 157-161; online at <http://bit.ly/cRc0JC>. This article includes a discussion of the "Workshop on Linking Evidence and Promising Practices in STEM Undergraduate Education" [National Academies (2008)]. . *Leonard, W. J., R.J. Dufresne, & J.P. Mestre. 1996. ''Using qualitative problem-solving strategies to highlight the role of conceptual knowledge in solving problems,'' Am. J. Phys. 64(12): 1495-1503; online to subscribers at <http://bit.ly/esJQjp >. . Morrison, P. 1963. "Fermi Questions," Am. J. Phys. 31(8): 626- 627; online to subscribers at <http://bit.ly/fsx9Jj>. Morrison wrote: "There is a kind of power over the theoretical and experimental studies in which [the prospective physics graduate student] is engaged which is difficult to define, but whose presence is perhaps more important than the knowledge which is more formal and complete. There is one test of such power which is at the same time a remarkably apt method for its development. The method was the common and frequently amusing practice of Enrico Fermi, perhaps the most widely creative physicist of our times. Fermi delighted to think up and a once to discuss and answer questions which drew upon everyday experience, and upon the ability to make rough approximations, inspired guesses, and statistical estimates from very little data. A few samples are indispensable: How much does a watch gain or lose when carried up a mountain? How many piano tuners are there in the city of Chicago?" The Fermi approach to problems is exploited by Sanjoy Mahajan (2010) in "Street-Fighting Mathematics." . Mahajan, S. 2010. "Street-Fighting Mathematics: The Art of Educated Guessing and Opportunistic Problem Solving." Forward by Carver Mead <http://bit.ly/dVmIHu>. MIT Press, publisher's information at <http://bit.ly/ghF5XY>. Amazon.com information at <http://amzn.to/ggP2Sv>. See also Dizikes (2010). . *Mazur, E. 2009. "Confessions of a Converted Lecturer" talk at the University of Maryland on 11 November 2009. That talk is now on UTube at <http://bit.ly/dBYsXh>, and the abstract, slides, and references - sometimes obscured in the UTube talk - are at <http://bit.ly/9qzDIq> as a 4 MB pdf. As of 16 April 2011 12:33:00-0700 Eric's talk had been viewed 35,494 times. In contrast, serious articles in the education literature (or even exciting posts such as this one) are often read only by the author and a few cloistered specialists, creating tsunamis in educational practice equivalent to those produced by a pebble dropped into the Pacific Ocean. . *McDermott, L. C., & Redish, E. F. 1999. RL-PER1: Resource letter on physics education research. Am. J. Phys. 67, 755-767; online as a 279 kB <http://bit.ly/dJ44qB>. . *Meltzer, D.E. 2005. "Relation between students' problem-solving performance and representational format," Am. J. Phys. 73(5): 463-478; online at <http://bit.ly/fa53tz>. . *National Academies. 2008. "Workshop on Linking Evidence and Promising Practices in STEM Undergraduate Education": (a) introductory sessions are online at <http://bit.ly/ciNwjQ>; (b) commissioned Papers are online at <http://bit.ly/ceg1Bx>. Papers related to Concept Inventories <http://en.wikipedia.org/wiki/Concept_inventory> for mathematics are conspicuously absent, but Epstein's (2005) "Calculus Concept Inventory" is a step in the right direction. See also the commentary on this workshop by Labov et al. (2009). . *NCSU. 2010. "Assessment Instrument Information Page," Physics Education R & D Group, North Carolina State University," online at <http://bit.ly/9gfUpY>. . *Redish, E.F. 2003 "Teaching Physics With the Physics Suite" (TPWPS), John Wiley, TPWPS is online at <http://bit.ly/gdE3Tu>. Note the crucial correction of Fig. 5.2 and its caption on page 100. . *Reif, F. 1995. "Millikan Lecture 1994: Understanding and teaching important scientific thought processes," Am. J. Phys. 63(1): 17-32; online to subscribers at <http://bit.ly/asI9Rk >. See esp. Section V. "Problem Solving." . *Reif, F. & L.A. Scott. 1999. ''Teaching scientific thinking skills: Students and computers coaching each other,''Am. J. Phys. 67(9): 819-831; online to subscribers at <http://bit.ly/hIlpT3>. . *Sabella, M.S. & E.F. Redish. 2007. "Knowledge organization and activation in physics problem solving," Am. J. Phys. 75(11): 1017-1029; online to subscribers at <http://bit.ly/dMoJao>. . *Singh, C. 2009. "Categorization of problems to assess and improve proficiency as teachers and learners,"Am. J. Phys. 77(1): 73-80; online at <http://bit.ly/fNEx3T>. . *Thacker, B., E. Kim, K. Trefz, and S.M. Lea. 1994. "Comparing problem solving performance of physics students in inquiry-based and traditional introductory physics courses," Am. J. Phys. 62(7): 627-633; online to subscribers at <http://bit.ly/dPaCf9>. . *UMaryFPS. 2011. University of Maryland Fermi Problems Site, online at <http://bit.ly/ezoFK5>. . *UMinnPhysEd. 2011. Univ. of Minnesota (a) "Cooperative Group Problem Solving" <http://bit.ly/guUtRP>, (b) Context-Rich Problems <http://bit.ly/ewghAC>, and (c) "Problem Solving Labs" <http://bit.ly/iiX6JR>. . *Van Heuvelen, A. 1991. Learning to think like a physicist: A review of research-based instructional strategies. Am. J. Phys. 59(10): 891-897; an abstract is online at <http://bit.ly/e2rhlX >. . *Van Heuvelen, A. 1995. "Experiment Problems for Mechanics," Phys. Teach. 33(5): 176-180; online to subscribers at <http://bit.ly/hCULSM>. . *Van Heuvelen, A & X. Zou. 2001. ''Multiple representations of work-energy processes,'' Am. J. Phys. 69(2): 184-194; online at <http://bit.ly/dEJZQc>. . . .