Embed Size (px)
Citation preview
On
Form
ativ
e (E
mbe
dded
) Sc
ienc
e A
sses
smen
t
Ric
hard
J. S
have
lson
Stan
ford
Uni
vers
ity
Gre
at M
inds
Pro
gram
Dia
logu
eSc
ienc
e Ed
ucat
ion
in th
e 21
st C
entu
ry: P
ushi
ng th
e En
velo
pe o
n St
uden
t Ass
essm
ent
Apr
il 11
, 200
1
Apr
il 11
, 200
1IM
SA G
reat
Min
ds2
Ove
rvie
w
•N
atio
nal p
olic
y tre
nds a
nd a
sses
smen
t•
New
and
con
tinui
ng d
evel
opm
ents
in
asse
ssm
ent o
f sci
ence
ach
ieve
men
t•
New
focu
s on
clas
sroo
m a
sses
smen
t•
Con
clud
ing
com
men
ts
Apr
il 11
, 200
1IM
SA G
reat
Min
ds3
Nat
iona
l Pol
icy
Tren
ds in
Sc
ienc
e Ed
ucat
ion
•C
ontin
ued
com
mitm
ent t
o in
quiry
-bas
ed sc
ienc
e-ed
ucat
ion
refo
rm fo
r all
stud
ents
from
AA
AS,
NA
S an
d N
SF•
Incr
ease
d pr
essu
re fo
r im
prov
emen
t in
scie
nce
teac
hers
’ ped
agog
ical
con
tent
know
ledg
e (G
lenn
) •
Incr
ease
d pr
essu
re fo
r “ha
rd”
evid
ence
of t
he p
ositi
ve
impa
ct a
nd to
“sc
ale
up”
inno
vatio
ns th
at w
ork
•In
crea
sed
acco
unta
bilit
y w
ith tr
aditi
onal
test
s
Apr
il 11
, 200
1IM
SA G
reat
Min
ds4
Incr
ease
d A
ccou
ntab
ility
With
Tr
aditi
onal
Tes
ts
Form
ativ
e
Sum
mat
ive
Typ
ePu
rpos
eA
genc
y
Lear
ning
Cer
tific
atio
n
Acc
ount
abili
ty
stud
ent
teac
her
indi
vidu
alex
tern
al te
sts
sam
ple
surv
eys
teac
her
exte
rnal
tests
exte
rnal
tests
indi
vidu
al
Paul
Bla
ck 3
/98N
atio
nal
Focu
s
Mat
ch-M
ism
atch
Apr
il 11
, 200
1IM
SA G
reat
Min
ds5
Dev
elop
men
ts in
Ass
essm
ent:
Con
cept
ual F
ram
ewor
kD
ecla
rativ
e
Pro
cedu
ral
Sch
emat
icSt
rate
gic
Kno
wle
dge
K
now
ledg
e
K
now
ledg
e
K
now
ledg
e(K
now
ing
the
(K
now
ing
the
(Kno
win
g th
e(K
now
ing
the
“whi
ch”
“that”
)
“how”
) “w
hy)
and
“when
”)Pr
ofic
ienc
yLo
wH
igh
Exte
nt(H
ow m
uch?
)
Stru
ctur
e(H
ow is
it o
rgan
ized
?)
Oth
ers
(Pre
cisi
on?
Effic
ienc
y?A
utom
atic
ity?)
Cog
nitiv
eC
ogni
tive
Tool
s:To
ols:
Plan
ning
Plan
ning
Mon
itorin
gM
onito
ring
Dom
ain-
spec
ific
cont
ent:
• fac
ts• c
once
pts
• prin
cipl
es
Dom
ain-
spec
ific
prod
uctio
n sy
stem
s
Dom
ain-
spec
ific
stra
tegi
es,
heur
istic
s,
cont
exts
Prob
lem
sc
hem
ata,
m
enta
l m
odel
s
Apr
il 11
, 200
1IM
SA G
reat
Min
ds6
Dev
elop
men
ts in
Ass
essm
ent:
Fram
ewor
k Li
nks t
o Te
sts
Dec
lara
tive
Proc
edur
alSc
hem
atic
Kno
wle
dge
Kno
wle
dge
Kno
wle
dge
•Pe
rfor
man
ce A
sses
smen
ts•
Jour
nals
•C
once
pt M
aps
•Se
man
tic M
aps
•Pe
rfor
man
ce
Ass
essm
ents
•In
terv
iew
s•
M-C
Tes
ts
•M
ultip
le-C
hoic
e•
Fill-
in•
Jour
nals
Proc
edur
e M
aps
Mod
els/
Men
tal M
aps
Exte
nt
Stru
ctur
e
Oth
ers
Apr
il 11
, 200
1IM
SA G
reat
Min
ds7
Dev
elop
men
ts in
Ass
essm
ent:
Exam
ple
Mul
tiple
-Cho
ice
Air
is m
ade
up o
f man
y ga
ses.
Whi
ch g
as is
foun
d in
the
grea
test
am
ount
?
A. N
itrog
enB
. Oxy
gen
C. C
arbo
n D
ioxi
deD
. Hyd
roge
n
TIM
SS P
opul
atio
n 2
Apr
il 11
, 200
1IM
SA G
reat
Min
ds8
Dev
elop
men
ts in
Ass
essm
ent:
Exam
ple
Con
cept
Map
wat
erra
in
river
scl
ouds
soil
sun
ocea
ns
is fa
lling
wat
er
com
esfr
om
cont
ain
goes
into
rive
rs
flow
to
cont
ain
shin
es o
n
From
Whi
te &
Gun
ston
e: “
Prob
ing
Und
erst
andi
ng”
(199
2, p
. 16)
Apr
il 11
, 200
1IM
SA G
reat
Min
ds9
Dev
elop
men
ts in
Ass
essm
ent:
Exam
ple
Perf
orm
ance
Ass
essm
ent
Flas
hlig
htSt
icky
To
wer
s
Stud
ent
Not
eboo
ks
and
Penc
ils
Stud
ents
are
ask
ed to
mod
el th
e pa
th o
f the
sun
from
su
nris
e to
sun
set a
nd u
se d
irec
tion,
leng
th, a
nd
angl
es o
f sha
dow
s to
sol
ve lo
catio
n pr
oble
ms.
Apr
il 11
, 200
1IM
SA G
reat
Min
ds10
Dev
elop
men
ts in
Ass
essm
ent:
Exam
ple
Men
tal M
odel
Pro
blem
•(A
) A ro
cket
is m
ovin
g al
ong
side
way
s in
deep
spac
e, w
ith it
s eng
ine
off,
from
poi
nt
A to
poi
nt B
. It
is n
ot n
ear a
ny p
lane
ts o
r oth
er o
utsi
de fo
rces
. Its
eng
ine
is fi
red
at
poin
t B a
nd le
ft on
for 2
sec
whi
le th
e ro
cket
trav
els f
rom
poi
nt B
to p
oint
C.
Dra
w in
th
e sh
ape
of th
e pa
th fr
om B
to C
. (S
how
you
r bes
t gue
ss fo
r thi
s pro
blem
eve
n if
you
are
unsu
re o
f the
ans
wer
.)•
(B) S
how
the
path
from
C a
fter t
he e
ngin
e is
turn
ed o
ff o
n th
e sa
me
draw
ing.
B
C
A
C
BA
Cor
rect
Inco
rrec
t
Sour
ce: C
lem
ent,
J. (1
982)
. St
uden
ts’ p
reco
ncep
tions
in in
trodu
ctor
y m
echa
nics
. Am
eric
an Jo
urna
l of P
hysi
cs,
50(1
), 66
-71.
Apr
il 11
, 200
1IM
SA G
reat
Min
ds11
Dev
elop
men
ts in
Ass
essm
ent:
Val
idity
Evi
denc
e fr
om T
IMSS
-RK
now
ledg
eTy
pePe
rcen
tM
ultip
le C
hoic
ePe
rcen
tO
pen
Ende
dPe
rcen
tTo
tal
Ana
lytic
(Dec
lara
tive
&10
1222
Sche
mat
ic)
Proc
edur
al6
28
Con
cept
ual
(Dec
lara
tive)
169
25
Fact
ual
(Dec
lara
tive)
381
38
“Not
Sci
ence
”5
27
Tota
l73
2710
0
Apr
il 11
, 200
1IM
SA G
reat
Min
ds12
Dev
elop
men
ts in
Ass
essm
ent:
Val
idity
Evi
denc
eC
orre
latio
ns fr
om S
hultz
’s D
isse
rtatio
n
(N
=109
6th
Gra
ders
Stu
dyin
g Ec
olog
y)–
Rea
ding
and
Mul
tiple
-Cho
ice:
0.6
9–
Rea
ding
and
Con
cept
Map
: 0.
53–
M-C
and
CM
: 0.
60–
Rea
ding
and
Per
form
ance
As
sess
men
t: 0
.25
–M
-C a
nd P
A:
0.33
–C
M a
nd P
A:
0.43
Dec
lara
tive
Kno
wle
dge
Dec
lara
tive
vs.
Proc
edur
al
Kno
wle
dge
Apr
il 11
, 200
1IM
SA G
reat
Min
ds13
New
Foc
us o
n C
lass
room
Ass
essm
ent
Form
ativ
e
Sum
mat
ive
Typ
ePu
rpos
eA
genc
y
Lear
ning
Cer
tific
atio
n
Acc
ount
abili
ty
stud
ent
teac
her
indi
vidu
alex
tern
al te
sts
sam
ple
surv
eys
teac
her
exte
rnal
tests
exte
rnal
tests
indi
vidu
al
Paul
Bla
ck 3
/98C
alifo
rnia
fo
cus
Mat
ch-M
ism
atch
Apr
il 11
, 200
1IM
SA G
reat
Min
ds14
The
“Rev
erse
” Lo
gic
of D
evel
opin
g A
sses
smen
ts fo
r a U
nit o
f Ins
truct
ion
Impl
emen
tatio
nof
Ass
essm
ents
Dev
elop
men
tof
Ass
essm
ents
65
4
1
3
2
End-
of-U
nit
Embe
dded
Embe
dded
End-
of-U
nit
Apr
il 11
, 200
1IM
SA G
reat
Min
ds15
Ass
essm
ent T
ask
Stru
ctur
e
Con
tent
Kno
wle
dge
Con
tent
Kno
wle
dge
Ric
h
Mor
e C
ogni
tive
Dem
andi
ng
Proc
ess S
kills
Proc
ess S
kills
Con
stra
ined
Ope
n
Less
Cog
nitiv
eD
eman
ding
Lean
Baxt
er &
Gla
ser,
98
Apr
il 11
, 200
1IM
SA G
reat
Min
ds16
Cla
ssro
om (F
orm
ativ
e) A
sses
smen
t •
Ever
yday
teac
hing
pra
ctic
e co
ncei
ved
as in
tegr
al in
as
sess
men
t•
Ass
essm
ent u
sed
to d
eter
min
e ga
pbe
twee
n w
hat
stud
ent k
now
s and
kno
wle
dge
goal
•Te
ache
r, pe
er, a
nd se
lfas
sess
men
ts c
ompr
ise
clas
sroo
m a
sses
smen
t•
Feed
back
crit
ical
to c
lose
the
gap
–G
rade
s?–
Qua
litat
ive
feed
back
use
ful t
o cl
osin
g ga
p?–
Bot
h?
Apr
il 11
, 200
1IM
SA G
reat
Min
ds17
Cla
ssro
om A
sses
smen
t: Ex
ampl
es•
Teac
her
–O
bser
vatio
ns–
Que
stio
ns–
Inte
rvie
ws
–Jo
urna
ls–
Cur
ricul
um-p
rovi
ded
and/
or te
ache
r ass
essm
ents
•Se
lf an
d Pe
er (C
lear
goa
ls a
bsol
utel
y es
sent
ial)
–R
evie
w/g
rade
eac
h ot
her’
s wor
k–
Rev
iew
eac
h ot
her’
s jou
rnal
s–
Ref
lect
on
lear
ning
Apr
il 11
, 200
1IM
SA G
reat
Min
ds18
Type
s of E
mbe
dded
Ass
essm
ents
•PO
E•
Inte
rvie
w•
Portf
olio
•D
raw
ings
•O
ther
s
•M
ultip
le-C
hoic
e•
Shor
t Ans
wer
•C
ompl
etio
n•
Exer
cise
s•
Ven
n di
agra
ms
Apr
il 11
, 200
1IM
SA G
reat
Min
ds19
The
Gap
Use
form
ativ
e as
sess
men
t inf
orm
atio
n to
:
•D
eter
min
e th
e ga
p be
twee
n st
uden
t pe
rfor
man
ce a
nd th
e le
arni
ng g
oals
•Pr
ovid
e qualitative
feed
back
to st
uden
t on
how
to c
lose
the
gap
Apr
il 11
, 200
1IM
SA G
reat
Min
ds20
Jour
nals
: An
Ass
essm
ent T
ool
for T
each
ers a
nd S
tude
nts
Journals
Journals:
•A
re a
writ
ten
acco
unt o
f wha
t stu
dent
s do
in th
eir s
cien
ce c
lass
, and
po
ssib
ly, o
f wha
t the
y le
arn
•M
ay p
rovi
de a
n un
obtr
usiv
ein
dica
tor o
f cla
ss e
xper
ienc
es
•A
re se
en a
s an
imm
edia
teas
sess
men
t --i
n ve
ry c
lose
pro
xim
ity to
the
curr
icul
um
•A
re v
iew
ed a
s ass
essm
ents
at t
wo
leve
ls:
–at
the
indi
vidu
al le
vela
re c
onsi
dere
d a
sour
ce o
f evi
denc
e be
arin
g on
st
uden
t’s p
erfo
rman
ce o
ver
a co
urse
of i
nstr
uctio
n–
at th
e cl
assr
oom
leve
l are
a so
urce
of e
vide
nce
of o
ppor
tuni
ties s
tude
nts
had
to le
arn
scie
nce
Apr
il 11
, 200
1IM
SA G
reat
Min
ds21
Som
e Fi
ndin
gs A
bout
Stu
dent
s’
Jour
nals
as A
sses
smen
t Too
lsR
elia
bilit
y:R
ater
s can
con
sist
ently
iden
tify
jour
nal e
ntrie
sSt
uden
ts’ s
cien
ce jo
urna
ls c
an b
e re
liabl
y sc
ored
Val
idity
Infe
renc
es a
bout
impl
emen
tatio
n us
ing
jour
nal s
core
s wer
e ju
stifi
edIn
fere
nces
abo
ut st
uden
ts’ p
erfo
rman
ce w
ere
also
enc
oura
ging
Use
fuln
ess
Uni
t im
plem
enta
tion
and
teac
her f
eedb
ack
scor
es h
elpe
d to
ex
plai
n di
ffer
ence
s in
the
perf
orm
ance
acr
oss c
lass
room
s
Apr
il 11
, 200
1IM
SA G
reat
Min
ds22
Con
clus
ions
•C
lass
room
form
ativ
e as
sess
men
t may
be
a cr
itica
l key
to im
prov
ing
stud
ents
’ ac
hiev
emen
t in
inqu
iry sc
ienc
e••
Ass
essm
ent r
efor
m w
ill re
quire
ext
ensi
ve
Ass
essm
ent r
efor
m w
ill re
quire
ext
ensi
ve
use
of te
chno
logy
to m
anag
e tim
e an
d co
st*
use
of te
chno
logy
to m
anag
e tim
e an
d co
st*
•C
olla
bora
tion
betw
een
rese
arch
ers a
nd
teac
hers
abs
olut
ely
esse
ntia
l*
Apr
il 11
, 200
1IM
SA G
reat
Min
ds23
Tech
nolo
gy &
Our
Col
labo
ratio
n:
Cos
t Red
uctio
n an
d Sc
orin
g Ef
ficie
ncy
•C
ompu
ter a
dapt
ive
test
ing-
-test
ing
exte
nt o
f de
clar
ativ
e an
d (p
erha
ps) s
trate
gic
know
ledg
e•
Com
pute
r con
cept
and
cog
nitiv
e m
appi
ng--
test
ing
stru
ctur
al k
now
ledg
e•
Com
pute
r sim
ulat
ion
of a
n in
vest
igat
ion-
-te
stin
g pr
oced
ural
and
/or s
trat
egic
kn
owle
dge
Apr
il 11
, 200
1IM
SA G
reat
Min
ds24
Col
labo
ratio
n--C
onve
rgen
ce o
f H
ands
-On
and
Com
pute
r Si
mul
atio
n PA
s
H1
H2
C1
C 2
r H1H
2 = .5
3
r C1C2
= ?
r H1C
1 = .5
2r H
2C2 =
?
r H1C
2 = ?
r C1H2
= .4
5
Apr
il 11
, 200
1IM
SA G
reat
Min
ds25
Than
ks fo
r Inv
iting
Me!
