[net-gold] Two Different Meanings of "Formative Evaluation" #2
- From: "David P. Dillard" <jwne@xxxxxxxxxx>
- To: Other Net-Gold Lists -- Educator Gold <Educator-Gold@xxxxxxxxxxxxxxx>, net-gold@xxxxxxxxxxxxx, NetGold <netgold@xxxxxxxxxxxxxxx>, K-12ADMINLIFE <K12ADMIN@xxxxxxxxxxxxxxxxxxx>, K12AdminLIFE <K12AdminLIFE@xxxxxxxxxxxxxxx>, Nabble Groups Net-Gold <ml-node+s3172864n3172864h56@xxxxxxxxxxxxx>, Net-Platinum <net-platinum@xxxxxxxxxxxxxxx>, Net-Gold <NetGold_general@xxxxxxxxxxxxxxxxx>, Temple Gold Discussion Group <TEMPLE-GOLD@xxxxxxxxxxxxxxxxxxx>, Temple University Net-Gold Archive <net-gold@xxxxxxxxxxxxxxxxxxx>, "Net-Gold @ Wiggio.com" <netgold@xxxxxxxxxxxxxx>, Health Lists -- Health Diet Fitness Recreation Sports <healthrecsport@xxxxxxxxxxxxxxx>, HEALTH-RECREATION-SPORTS-TOURISM@xxxxxxxxxxxxxxxxxxx
- Date: Thu, 30 Jan 2014 05:50:10 -0500 (EST)
.
Date: Wed, 29 Jan 2014 19:06:18 -0800
From: Richard Hake <rrhake@xxxxxxxxxxxxx>
Reply-To: Net-Gold@xxxxxxxxxxxxxxx
To: AERA-L@xxxxxxxxxxxxxxxxx, Net-Gold@xxxxxxxxxxxxxxx
Subject: [Net-Gold] Two Different Meanings of "Formative Evaluation" #2
.
This is an expanded and improved version of my post of 28 Jan 2014. Sorry
for the bother.
.
If you reply to this long (90 kB) post please don't hit the reply button,
bane of discussion lists, unless you prune the copy of this post that may
appear in your reply down to a few relevant lines, otherwise the entire
already archived post may be needlessly resent to subscribers. The
abstract reads:
.
*******************************************************
.
ABSTRACT: In response to my post "FairTest Appears To Be Uninformed on
'Formative Evaluation' In The JCSEE Sense" at <http://bit.ly/1dL3c5K>,
Michael Paul Goldenberg and FairTest's Monty Neil posted what appeared to
be non sequitures on EDDRA2, symptomatic of the general failure of
educators to recognize the existence of two very different meanings of
"Formative Evaluation":
.
(1) "Evaluation designed and used to improve an intervention, especially
when it is still being developed" – JCSEE (1994) ascopied on p. 132 of
Frechtling et al. (2010) at <http://bit.ly/1aYcgYn>. . . . . . . . . . . .
. . . . (FE-JCSEE)
.
(2) "All those activities undertaken to provide information to be used
asfeedback so as to adapt the teaching to meet student needs" –
paraphrasedfrom Black & Wiliam (1998, p. 2) at <http://bit.ly/1jTqiwK>.. .
. . . . . . (FE-B&W)
.
An example of FE-JCSEE is zero-stakes pre/post testing utilizing Concept
Inventories <http://bit.ly/dARkDY> which are constructed by disciplinary
experts through arduous qualitative and quantitative research– see e.g.:
(a) "The Impact of Concept Inventories on Physics Education and Its
Relevance for Engineering Education" [Hake (2011)] at
<http://bit.ly/nmPY8F> (8.7 MB), and (b) "Can the Cognitive Impact of
Calculus Courses be Enhanced?" [Hake (2013)] at <http://bit.ly/1loHgC4>
(2.7 MB).
.
An example of FE-B&W is its use in the "interactive engagement" (IE)
methods that have been shown –Hake (1998a) at <http://bit.ly/9484DG> and
many others to achieve average normalized gains <g> on Concept Inventories
<http://bit.ly/dARkDY> that are about two standard deviations above those
of traditional passive-student lecture courses. Here IE methods are
defined [Hake 1998a)] as: "methods designed to promote conceptual
understanding through the active engagement of students in heads-on
(always) and hands-on (usually) activities that yield *immediate feedback*
through discussion with peers and/or instructors."
.
******************************************************
.
Thus far (29 Jan 2014 15:53-0800) my post "FairTest Appears To Be
Uninformed on 'Formative Evaluation' In The JCSEE Sense" [Hake (2014b)]
has stimulated three responses on EDDRA2, all of which appear to be non
sequitures, symptomatic of the general failure to recognize the existence
of two very different meanings of "Formative Evaluation."
.
1111111111111111111111111111111111111111
.
1. Michael Paul Goldenberg (2014a) wrote: ". . . . when was the 'Joint
Committee on the Standards for Educational Evaluation' made the definitive
authority on the nature of formative assessment?"
.
Nobody that I know of has either stated or implied that the JCSEE is "the
definitive authority on the nature of formative assessment."
.
Nevertheless, according to information at <http://bit.ly/1jBIDRs>
[updated; my inserts at ". . . . . [[insert]]. . . . . "]:
.
"Created in 1975, the Joint Committee is a coalition of major professional
associations concerned with the quality of evaluation. The Joint Committee
is housed at the Center for Evaluation and Assessment, University of Iowa,
Iowa City, IA.. . . . .[[<http://bit.ly/1bthSXd>]]. . . . . [It] has
published three sets of standards for evaluations that are now widely
recognized. The 'Personnel Evaluation Standards' <http://bit.ly/1jXQQgz>
was published in 1988; 'The Student Evaluations Standards'
<http://bit.ly/1jFHad0> was published in 2003; and 'The Program Evaluation
Standards (3rd edition) by Yarbrough et al. <http://bit.ly/1aD4sNQ> was
published in 2011 by Sage Publications. The Joint Committee is accredited
by the 'American National Standards Institute' (ANSI). . . . .
.[[<http://bit.ly/1mPkILv>]]. . . . Standards approved by ANSI become
American National Standards."
.
The Glossary on pp. 129-135 of Frechtling et al. (2010) gives the
following JCSEE definitions:
.
(a) ASSESSMENT: Often used as a synonym for evaluation. The term is
sometimes recommended for restriction to processesthat are focused on
quantitative and/or testing approaches. . . . . . . . . . . . . . . . . .
. .(A-JCSEE)
.
(b) FORMATIVE EVALUATION: "Evaluation designed and usedto improve an
intervention, especially when it is still being developed" – JCSEE (1994)
as copied on p. 132 of Frechtling et al. (2010). . . . . . . . . . . . .
.(FE-JCSEE)
.
(c) SUMMATIVE EVALUATION: Evaluation designed to present conclusions about
the merit or worth of an intervention andrecommendations about whether it
should be retained, altered, or eliminated.. . . . . . . . . .(SE-JCSEE)
.
For recent reviews of evaluation/assessment in education see e.g.,
(a)"Informing the Practice of Teaching Using Formative and Interim
Assessment: A Systems Approach" [Lissitz (2013)]; (b) "Can We Reverse the
Wrong Course on Data and Accountability?" [NEPC (2014)]; and (c) "Testing
Resistance and Reform News" [Schaeffer (2014)].
.
Regarding the distinction sometimes made between "assessment" and
"evaluation" in the above (A-JCSEE) - the late Bob Leamnson (2003) opined
"To the several who suggested that 'we all get together' (somehow) and
agree on how 'evaluation' and 'assessment' are to be defined and
thereafter used by one and all, I suggest that you're trying to swim up
Niagara Falls." Accordingly, in this post I make no distinction between
the two terms.
.
Regarding the JCSEE definition of "formative evaluation" in FE-JCSEE
above, note that it is quite different from that Black & Wiliam (1998, p.
2, paraphrased):
.
"All those activities undertaken to provide information to be used as
feedback so as to adapt the teaching to meet student needs" – paraphrased
from Black & Wiliam (1998, p. 2), . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . (FE-B&W)
.
which I characterized in Hake (2014a) as assessment done "on the fly" by
teachers so as to immediately adapt their teaching to meet student needs.
.
An example of FE-B&W is its use in the method of the *historical* Socrates
– see [Hake (2007a)] - as employed, for example in "Socratic Dialogue
Inducing Labs" [Hake (1992, 2008b, 2012)].
.
More generally, examples of FE-B&W are its use in the "interactive
engagement" (IE) methods that have been shown by Hake (1998a,b)] [and
about 25 other groups as listed in Hake (2008b)] to produce average
pre-to-posttest normalized gains <g> that are about two standard
deviations greater than those produced by traditional passive-student
lecture methods. Here IE methods are defined [Hake 1998a)] as: methods
designed to promote conceptual understanding through the active engagement
of students in heads-on (always) and hands-on (usually) activities that
yield *immediate feedback* through discussion with peers and/or
instructors."
.
2222222222222222222222222222222222222222222
.
2. Michael Paul Goldenberg (2014b) wrote: "Can we assume that Prof. Hake
is ignorant of the rest of the literature on formative assessment simply
because he fails to cite or mention it? Probably that would be unfair."
.
If MPG would actually take a few minutes to scan my post "FairTest
Appears To Be Uninformed on 'Formative Evaluation' In The JCSEE Sense"
[Hake (2014)] before pontificating, he would discover that I referenced
"Inside the Black Box: Raising Standards Through Classroom Assessment"
[Black & Wiliam (1998)] and several articles by Richard Shavelson (2008)
<http://bit.ly/VvmqKa> and his group at Stanford, articles that extol
formative assessment in the sense used by Black & Wiliam (1998).
.
3333333333333333333333333333333333333333333
.
3. FairTest's Monty Neill (2014) wrote: "To say that we are uninformed
about formative assessment is more than misleading, conflating one
particular approach with the general approach. Pre-post testing is one
form of formative assessment."
.
I did *not* write that FairTest was uninformed about formative assessment!
The title of my post is "FairTest Appears To Be Uninformed on 'Formative
Evaluation' IN THE JCSEE SENSE."
.
Furthermore, in the abstract of my post I wrote: ". . .both Ravitch and
"FairTest" appear to be uninformed regarding the virtues of rigorous
measurement of students' higher-order learning by means of zero-stakes
*FORMATIVE* [i.e., "designed and used to improve an object, especially
when it is still being developed" (JCSEE, 1994)] pre/post testing
utilizing Concept Inventories <http://bit.ly/dARkDY> which are constructed
by disciplinary experts through arduous qualitative and quantitative
research– see e.g.: (a) "The Impact of Concept Inventories on Physics
Education and Its Relevance for Engineering Education" [Hake (2011)] and
(b) "Can the Cognitive Impact of Calculus Courses be Enhanced?" [Hake
(2013)].
.
Regarding Neill's statement that "Pre-post testing is one form of
formative assessment," more accurately "Pre-post testing *can* be a form
of formative assessment IF the results are not used for summative
assessment." The use of pre-post testing used for summative assessment of
teachers or courses runs afoul of the laws of Dunkenfield and Campbell –
see the signature quotes.
.
Judging from Goldenberg's (2014a,b) posts, he seems to miss the point that
people or groups are free to define terms in any way they choose as long
as the definitions are useful and *operationally* meaningful – see e.g.,
Language Ambiguities in Education Research" [Hake (2008c)].
.
.
.
Richard Hake, Emeritus Professor of Physics, Indiana University; Honorary
Member, Curmudgeon Lodge of Deventer, The Netherlands; President, PEdants
for Definitive Academic References which Recognize the Invention of the
Internet (PEDARRII); LINKS TO: Academia <http://bit.ly/a8ixxm>; Articles
<http://bit.ly/a6M5y0>; Blog <http://bit.ly/9yGsXh>; Facebook
<http://on.fb.me/XI7EKm>; GooglePlus <http://bit.ly/KwZ6mE>; Google
Scholar <http://bit.ly/Wz2FP3>; Linked In <http://linkd.in/14uycpW>;
Research Gate <http://bit.ly/1fJiSwB>; Socratic Dialogue Inducing (SDI)
Labs <http://bit.ly/9nGd3M>; Twitter <http://bit.ly/juvd52>.
.
.
.
CAMPBELL'S LAW [Campbell (1976)] at <http://bit.ly/hMsyUr>: "The more any
quantitative social indicator is used for social decision making, the more
subject it will be to corruption pressures and the more apt it will be to
distort and corrupt the social processes it is intended to monitor."
.
DeHAAN (2005) at <http://bit.ly/ncAuQa>: "There is substantial evidence
that scientific teaching in the sciences, i.e., teaching that employs
instructional strategies that encourage undergraduates to become actively
engaged in their own learning, can produce levels of understanding,
retention and transfer of knowledge that are greater than those resulting
from traditional lecture/lab classes. But widespread acceptance by
university faculty of new pedagogies and curricular materials still lies
in the future."
.
DUKENFIELD'S LAW – [Kleiman (2010)] at <http://bit.ly/bsRokM>: "If a thing
is worth winning, it's worth cheating for."
.
LATOUR (1987) at <http://bit.ly/Reg9Ra>: "A paper that does not have
references is like a child without an escort walking at night in a big
city it does not know: isolated, lost, anything may happen to it."
.
STOKSTAD (2001) at <http://bit.ly/1dOtxQj>: "Physicists are out in front
in measuring how well students learn the basics, as science educators
incorporate hands-on activities in hopes of making the introductory course
a beginning rather than a finale."
.
WOOD & GENTILE (2003) at <http://bit.ly/SyhOvL>: "Physics educators have
led the way in developing and using objective tests to compare student
learning gains in different types of courses, and chemists, biologists,
and others are now developing similar instruments. These tests provide
convincing evidence that students assimilate new knowledge more
effectively in courses including active, inquiry-based, and collaborative
learning, assisted by information technology, than in traditional
courses."
.
.
.
REFERENCES [URL shortened by <http://bit.ly/> and accessed on 29 Jan
2014.]
.
.
.
Black, P. & D. Wiliam. 1998. "Inside the Black Box: Raising Standards
Through Classroom Assessment," Phi Delta Kappan 80(2): 139-144, 146-148;
online as a 397 kB pdf at <http://bit.ly/1jTqiwK>.
.
Campbell, D.T. 1976. "Assessing the Impact of Planned Social Change,"
Paper #8 Occasional Paper Series, Dartmouth College; online as a 201 kB
pdf at <http://bit.ly/hMsyUr>.
.
DeHaan, R.L. 2005. "The Impending Revolution in Undergraduate Science
Education," Journal of Science Education and Technology 14(2): 253-269;
online as a 152 kB pdf at <http://bit.ly/ncAuQa>.
.
Frechtling J., M.M. Mark, D.J. Rog, V. Thomas, H. Frierson, S.Hood, & G.
Hughes. 2010. NSF Report, "The 2010 User-Friendly Handbook for Project
Evaluation," online as a 815 kB pdf at <http://bit.ly/1aYcgYn>.
.
Goldenberg, M.P. 2014a. Re: FairTest Appears To Be Uninformed on
'Formative Evaluation' In The JCSEE Sense," on the CLOSED EDDRA2 archives
at <http://yhoo.it/1aDxTze>. Post of 26 January 12:32 pm to EDDRA2.
[EDDRA2 fails to specify the time zone.]
.
Goldenberg, M.P. 2014b. Re: FairTest Appears To Be Uninformed on
'Formative Evaluation' In The JCSEE Sense," on the CLOSED EDDRA2 archives
at <http://yhoo.it/1lkDHlG>. Post of 26 January 12:58 pm to EDDRA2.
[EDDRA2 fails to specify the time zone.]
.
Hake, R.R. 1992. "Socratic pedagogy in the introductory physics lab,"
Phys. Teach. 30: 546-552; updated version (4/27/98) online as an 88 kB pdf
at <http://bit.ly/9tSTdB>.
.
Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66: 64-74; online as an 84 kB pdf at
<http://bit.ly/9484DG> . As pointed out in "The NRC Finally Comes to Its
Senses on Improving STEM Education" [Hake (2013a)] in "Adapting to a
Changing World - Challenges and Opportunities in Undergraduate Physics
Education" [NRC (2013, p. 35)] it's stated: "Hake's (1998a) seminal report
on the effectiveness of interactive engagement methods remains an
important contribution to undergraduate physics education. The article
presents results from the Mechanics Diagnostic (MD) [Halloun & Hestenes
(1985) and the Force Concept Inventory (FCI) [Hestenes et al. (1992),
given before and after instruction on Newtonian mechanics in a variety of
courses taught using different approaches. . . . . . . the conclusion,
that more effective instructional approaches involve active learning, has
been supported by many other studies using different methodology: e.g.,
Meltzer and Thornton (2012) and Hoellwarth et al. (2005). See also the
crucial but generally ignored companion paper Hake (1998b)
.
Hake, R.R. 1998b. “Interactive-engagement methods in introductory
mechanics courses,” online as a 108 kB pdf at <http://bit.ly/aH2JQN>. A
crucial companion paper to Hake (1998a). Submitted on 6/19/98 to the
“Physics Education Research Supplement” (PERS) of the American Journal of
Physics, but rejected by its editor on the grounds that the very
transparent, well organized, and crystal clear Physical-Review-type data
tables were "impenetrable"!
.
Hake, R.R. 2003a. "Re: A taxonomy," POD posts of 9 Jul 2003, online at
<http://bit.ly/1b2aoOb> (a diagram is shown).
.
Hake, R.R. 2003b. "Re: A taxonomy," POD posts of 12 Jul 2003, online at
<http://bit.ly/bAAkMH>.
.
Hake, R. R. 2005. "The Physics Education Reform Effort: A Possible Model
for Higher Education," online as a 100 kB pdf at <http://bit.ly/9aicfh> .
This is a slightly edited version of an article that was (a) published in
the "National Teaching and Learning Forum" 15(1), December 2005, online to
subscribers at <http://bit.ly/1hGv92V> (if your institution doesn't
subscribe to NTLF, then it should), and (b) disseminated by the Tomorrow's
Professor list <http://bit.ly/d09Y8r> as Msg. 698 on 14 Feb 2006. Type
"698" into the slot at the top of the page.
.
Hake, R.R. 2006. "Possible Palliatives for the Paralyzing Pre/Post
Paranoia that Plagues Some PEP's" [PEP's = Psychometricians, Education
specialists, and Psychologists], Journal of MultiDisciplinary Evaluation,
Number 6, November, online at <http://bit.ly/12Fwr81>.
.
Hake, R.R. 2007a. "The Socratic Method of the Historical Socrates, Plato's
Socrates, and the Law School's Socrates"; online on the OPEN! AERA-L
archives at <http://bit.ly/b5v58m>. Post of 21 Jun 2007 13:43:05 -0700 to
AERA-L and Net-Gold. The abstract and link to the complete post were
transmitted to various discussion lists and are also on my blog
"Hake'sEdStuff" at <http://bit.ly/hb8kKD> with a provision for comments.
.
Hake, R.R. 2007b. "Should We Measure Change? Yes!" online as a 2.5 MB pdf
at <http://bit.ly/d6WVKO>. A severely truncated version appears at Hake
(2006).
.
Hake, R.R. 2008a. "Re: pre-to-post tests as measures of
learning/teaching," online on the OPEN! AERA-L archives at
<http://bit.ly/MmPxwp>. Post of 28 Jan 2008 17:33:48-0800 to AERA-L and
Net-Gold.
.
Hake, R.R. 2008b. "Design-Based Research in Physics Education Research: A
Review," in Kelly, Lesh, & Baek (2008)]. A pre-publication version of that
chapter is online as a 1.1 MB pdf at <http://bit.ly/9kORMZ>.
.
Hake, R.R. 2008c. "Language Ambiguities in Education Research," submitted
to the "Journal of Learning Sciences" on 21 August but REJECTED! :-( -
proof positive of an exemplary article!; online as a 1.2 MB pdf at
<http://bit.ly/bHTebD> and as ref. 54 at <http://bit.ly/a6M5y0>.
.
Hake, R.R. 2009. "Re: evaluation & assessment," online on the OPEN! POD
archives at <http://bit.ly/1eeuRRc>. Post of 27 Jan 2009 10:27:35-0800 to
POD.
.
Hake, R.R. 2011. "The Impact of Concept Inventories On Physics Education
and Its Relevance For Engineering Education," invited talk, 8 August,
second annual NSF-sponsored "National Meeting on STEM Concept
Inventories," Washington, D.C., online as a 8.7 MB pdf at
<http://bit.ly/nmPY8F>. See also (a) "The Physics Education Reform Effort:
A Possible Model for Higher Education" (Hake, 2005); (b) "Should We
Measure Change? Yes!" (Hake, 2007b); (c) "Re: pre-to-post tests as
measures of learning/teaching" (Hake, 2008a); (d) "Design-Based Research
in Physics Education Research: A Review" (Hake, 2008b)]; and (e) "The NRC
Finally Comes to Its Senses on Improving STEM Education" (Hake, 2013a).
But can multiple-choice tests such as the FCI and CCI measure students'
high-order learning? Wilson & Bertenthal (2011, p. 95) think so, writing:
"Although many people recognize that multiple-choice items are an
efficient and effective means of determining how well students have
acquired basic content knowledge, many do not recognize that they also can
be used to measure complex cognitive processes. For example, the 'Force
Concept Inventory' (Hestenes, Wells, and Swackhamer, 1992) is an
assessment that uses multiple-choice items but taps into higher-level
cognitive processes."
.
Hake, R.R. 2012. "Helping Students to Think Like Scientists in Socratic
Dialogue Inducing Labs," Phys. Teach. 50(1): 48-52; online to subscribers
at <http://bit.ly/wLy3En>. A version identical to the Physics Teacher
article except for (a) minor formatting changes, and (b) the addition of
a few *hot-linked* URL's is online as a 299 kB pdf at
<http://bit.ly/x5ruYF>.
.
Hake, R.R. 2013a. "The NRC Finally Comes to Its Senses on Improving STEM
Education," online on the OPEN! AERA-L archives at
<http://bit.ly/154M5yf>. Post of 18 Sep 2013 20:19:57-0400 to AERA-L and
Net-Gold. The abstract and link to the complete post are being transmitted
to various discussion lists and also on my blog "Hake'sEdStuff" at
<http://bit.ly/19hfX6n> with a provision for comments.
.
Hake, R.R. 2013b. "Can the Cognitive Impact of Calculus Courses be
Enhanced?" An update of 26 Dec 2013 of an invited talk of 24 April 2012,
Department of Mathematics, University of Southern California, online as a
2.7 MB pdf at <http://bit.ly/1loHgC4>.
.
Hake, R.R. 2014a. "Re: The Defiant Parents: Testing’s Discontents" online
on the OPEN! AERA-L archives at <http://bit.ly/1mYwWoa>.The abstract and
link to the complete post are being transmitted to several discussion
lists and are also on my blog "Hake'sEdStuff" at <http://bit.ly/1dEZzOD>.
.
Hake, R.R. 2014b. "FairTest Appears To Be Uninformed on 'Formative
Evaluation' In The JCSEE Sense," on the OPEN! AERA-L archives at
<http://bit.ly/1dL3c5K>. Post of 26 Jan 2014 16:00:18-0800 to AERA-L,
Net-Gold. Transmitted on 25 Jan 2014 to EDDRA2 and ARN-L.
.
Halloun, I. & Hestenes, D. 1985a. "The initial knowledge state of college
physics," Am. J. Phys. 53(11): 1043-1055; online at
<http://bit.ly/b1488v>, scroll down to "Evaluation Instruments." See also
Halloun & Hestenes (1985b).
.
Halloun, I. & D. Hestenes. 1985b. "Common sense concepts about motion."
Am. J. Phys. 53(11) 1056-1065; online at <http://bit.ly/b1488v>, scroll
down to "Evaluation Instruments."
.
Halloun, I., R.R. Hake, E.P. Mosca, & D. Hestenes. 1995. "Force Concept
Inventory (1995 Revision),” online (password protected) at
<http://bit.ly/b1488v>, scroll down to "Evaluation Instruments." Currently
available in 20! languages: Arabic, Chinese, Croatian, Czech, English,
Finnish, French, French (Canadian), German, Greek, Italian, Japanese,
Malaysian, Persian, Portuguese, Russian, Spanish, Slovak, Swedish, &
Turkish.
.
Hargreaves, A, & and H. Braun. 2013. "Data-Driven Improvement and
Accountability," National Education Policy Center (NEPC); online as a 823
kB pdf at <http://bit.ly/1b8yKCS>. The Executive Summary states [my
CAPS]: "The drive to enhance learning outcomes has assumed increasing
salience over the last three decades. These outcomes include both high
levels of tested achievement for all students and eliminating gaps in
achievement among different sub-populations (raising the bar and closing
the gap). This policy brief examines policies and practices concerning the
use of data to inform school improvement strategies and to provide
information for accountability. We term this twin-pronged movement,
data-driven improvement and accountability (DDIA). . . . . . . . In
general, we find that over more than two decades, through accumulating
statewide initiatives in DDIA and then in the successive Federal
initiatives of the No Child Left Behind Act and Race to the Top, DDIA IN
THE U.S. HAS COME TO EXERT INCREASINGLY ADVERSE EFFECTS ON PUBLIC
EDUCATION, because high-stakes and high-threat accountability, rather than
improvement alone, or improvement and accountability together, have become
the prime drivers of educational change. This, in turn, has exerted
adverse and perverse effects on attempts to secure improvement in
educational quality and equity. The result is that, in the U.S.,
Data-Driven Improvement and Accountability has often turned out to be Data
Driven Accountability at the cost of authentic and sustainable
improvement."
.
Hestenes, D., M. Wells, & G. Swackhamer. 1992. “Force Concept Inventory,”
Phys. Teach. 30(3): 141-158; online as a 100 kB pdf at
<http://bit.ly/foWmEb> [but without the test itself]. For the 1995
revision see Halloun et al. (1995).
.
Hoellwarth, C., Moelter, M.J., & Knight, R.D. 2005. "A direct comparison
of conceptual learning and problem solving ability in traditional and
studio style classrooms," Am. J. Phys. 73(5): 459-463; online to
subscribers at <http://bit.ly/156hHhF>.
.
JCSEE. 1994. Joint Committee on Standards for Educational Evaluation, The
Program Evaluation Standards: How to Assess Evaluations of Educational
Programs," 2nd ed., Sage. See the glossary of evaluation terms on pp.
203-210 of this publication, repeated on pp. 129-135 of Frechtling et al.
(2010). Amazon.com information at <http://amzn.to/L3MvvY>, note the
searchable "Look Inside" feature. An expurgated Google book preview is
online at <http://bit.ly/1mI2Sdl>. This 2nd edition has evidently now
been superseded by the 3rd edition [Yarbrough et al. (2011)]. For a
four-quadrant delineation of the formative/summative and public/private
dimensions of assessment/evaluation see Hake (2003a,b; 2009).
.
Kleiman, M. 2010. "Dukenfield's Law of Incentive Management," The
Atlantic, 13 Aug; online at <http://bit.ly/bsRokM>.
.
Latour. B. 1987. "Science in Action - How to Follow Scientists and
Engineers Through Society," Harvard University Press, publisher's
information at <http://bit.ly/Reg9Ra>. Amazon.com information at
<http://amzn.to/r4bXPN>, note the searchable "Look Inside" feature.
.
Leamnson, B. 2003. "Taxonomy revisited," online on the OPEN! POD archives
at <http://bit.ly/g2i7Oe>. Post of 11 Jul 2003 16:52:46-0400 to POD.
.
Lissitz, R.W. ed. 2013. "Informing the Practice of Teaching Using
Formative and Interim Assessment: A Systems Approach." Information Age
Publishing, publisher's information at <http://bit.ly/1aFf4M9>.
Amazon.com information at <http://amzn.to/1aFfwdk>. For a review see
Preston (2013).
.
Meltzer, D.E. & R. K. Thornton. 2012. "Resource Letter ALIP-1:
Active-Learning Instruction in Physics," Am. J. Phys. 80(6): 478-496,
online to subscribers at <http://bit.ly/O35gtB>.
.
NEPC. 2014. National Education Policy Center, Univ. of Colorado, "Can We
Reverse the Wrong Course on Data and Accountability?" online at
<http://bit.ly/1dLS7Bk>. I thank Jane Jackson for alerting me to this
report. It states: "In their report 'Data-Driven Improvement and
Accountability' . . . . . . .[[Hargreaves & Brennan (2013)]] . . . . . .
authors Andy Hargreaves, the Thomas More Brennan Professor of Education in
the Lynch School of Education, and Henry Braun, the Boisi Professor of
Education and Public Policy in the Lynch School of Education, find that
the use of data in the U.S. is too often limited to simply measuring
short-term gains or placing blame, rather than focusing on achieving the
primary goals of education. The report is published by the National
Education Policy Center (NEPC), which is housed at the University of
Colorado Boulder."
.
NRC. 2013. National Research Council, "Adapting to a
Changing
World—Challenges and Opportunities in
Undergraduate Physics Education,"
online at <http://bit.ly/126os6j>. The description reads: " 'Adapting to a
Changing World' was commissioned by the National Science Foundation to
examine the present status of undergraduate physics education, including
the state of physics education research, and, most importantly, to develop
a series of recommendations for improving physics education that draws
from the knowledge we have about learning and effective teaching. Our
committee has endeavored to do so, with great interest and more than a
little passion. The Committee on Undergraduate Physics Education Research
and Implementation was established in 2010 by the Board on Physics and
Astronomy of the National Research Council. This report summarizes the
committee's response to its statement of task, which requires the
committee to produce a report that identifies the goals and challenges
facing undergraduate physics education and identifies how best practices
for undergraduate physics education can be implemented on a widespread and
sustained basis, assess the status of physics education research (PER) and
discuss how PER can assist in accomplishing the goal of improving
undergraduate physics education best practices and education policy."
.
Preston, B. 2013. "Moving Forward with Caution: Assessment Research and
Practice. An Essay Review of Lissitz's 'Informing the Practice of Teaching
Using Formative and Interim Assessment,' Education Review 16(5); online as
a 442 kB pdf at <http://bit.ly/1fismxi>.
.
Schaeffer, B. 2014. "Testing Resistance and Reform News," online on the
CLOSED! EDDRA2 archives at <http://yhoo.it/1nft2XU>. With luck this may
eventually appear on the OPEN! but DERELICT!! ARN-L archives at
<http://bit.ly/jeiTPm>, but as of 29 Jan 16:15-0800 posts for January 2014
had not appeared! :-(
.
Shavelson, R.J. 2008. "Formative Assessment," Guest Editor's Introduction,
special issue of "Applied Measurement in Education" online at
<http://bit.ly/nn2Rcz>. Also free online at the same URL are five articles
on formative assessment that appear in Applied Measurement in Education
21(4) of 4 October; also online to subscribers at <http://bit.ly/phtxnG>.
.
Stokstad, E. 2001. "Reintroducing the Intro Course." Science
293:1608-1610, 31 August 2001: online to subscribers at
<http://bit.ly/1dOtxQj>.
.
Wilson, M.R. & M.W. Bertenthal, eds. 2005. "Systems for State Science
Assessment," Nat. Acad. Press; online at <http://bit.ly/f6WFeg>. On page
94 they write: "Performance assessment is one approach that offers great
potential for assessing complex thinking and reasoning abilities, but
multiple-choice items also have their strengths. Although many people
recognize that multiple-choice items are an efficient and effective means
of determining how well students have acquired basic content knowledge,
many do not recognize that they also can be used to measure complex
cognitive processes. For example, the Force Concept Inventory (Hestenes,
Wells, and Swackhamer, 1992) is an assessment that uses multiple-choice
items but taps into higher-level cognitive processes."
.
Wood, W.B., & J.M. Gentile. 2003. "Teaching in a research context,"
Science 302: 1510; 28 November; online as a 213 kB pdf
<http://bit.ly/SyhOvL>, thanks to Ecoplexity <http://bit.ly/152aFQ9>.
.
Yarbrough, D.B., L.M. Shula, R,K. Hopson, & F.A. Caruthers. 2011. "The
Program Evaluation Standards: A Guide for Evaluators and Evaluation
Users," 3rd ed. Sage, publisher's information at <http://bit.ly/l98gg2>.
Amazon.com information at <http://amzn.to/1mPovZg>, note the search able
"Look Inside" inside feature. An expurgated Google book preview is online
at <http://bit.ly/1aDkYNK>.
.
.
Other related posts:
- » [net-gold] Two Different Meanings of "Formative Evaluation" #2 - David P. Dillard
