Transcript
Page 1: Accountability Testing Without Accountability?

Interchange, Vol. 32/2, 191-202, 2001.©Kluwer Academic Publishers. Printed in the Netherlands.

$FFRXQWDELOLW\�7HVWLQJ�:LWKRXW�$FFRXQWDELOLW\"

721<�&�0��/$0�DQG&$7+(5,1(�%25',*121

2QWDULR�,QVWLWXWH�IRU�6WXGLHV�LQ�(GXFDWLRQRI�WKH�8QLYHUVLW\�RI�7RURQWR

&DVDV� DQG� 0HDJKDQ� ������� DQG� )URHVH�*HUPDLQ� ������� SURYLGH� DFRPSUHKHQVLYH�UHYLHZ�RI�WKH�OLWHUDWXUH�RQ�WKH�SUREOHPV�DQG�GLOHPPDVLQ�LPSOHPHQWLQJ�WKH�YDULRXV�W\SHV�RI�KLJK�VWDNHV�VWDQGDUGL]HG�WHVWLQJIRU�VHOHFWLRQ��SODFHPHQW��DFFRXQWDELOLW\��HGXFDWLRQDO�LPSURYHPHQW��DQGJUDGXDWLRQ�RU�H[LWLQJ��,Q�RXU�́ ILQDO�ZRUG�µ�ZH�OLPLW�RXU�FRPPHQWV�WR�WKH2QWDULR�3URYLQFLDO�:LGH�$FKLHYHPHQW�7HVWLQJ�3URJUDP��HVSHFLDOO\�LQUHJDUG�WR�LWV�LPSDFW�RQ�FXUULFXOXP�DQG�LQVWUXFWLRQ��

7KH�DXWKRUV�RI�DOO�WKUHH�SDSHUV�LQ�WKLV�LVVXH�KDYH�SUHVHQWHG�PDQ\FULWLFLVPV�RI�VWDQGDUGL]HG�DFKLHYHPHQW�WHVWLQJ��LQFOXGLQJ�WKH�2QWDULRSURYLQFLDO�WHVWV��DQG�UHMHFWHG�WKH�GHIHQVH�WKDW�KLJK�VWDNHV�DFKLHYHPHQWWHVWV�SRVLWLYHO\�DIIHFW�VWXGHQW�OHDUQLQJ��,Q�RXU�SDSHU��ZH�H[SRVH�´WKH$FKLOOHV·� KHHO� RI� HGXFDWLRQ� UHIRUPµ� �&DVDV� � 0HDJKDQ�� ������ E\UHSRUWLQJ�WHDFKHUV·�GLVSOHDVXUH�ZLWK�WKH�2QWDULR�*UDGH���WHVW��:H�KDYHQRW�\HW�KDG�WKH�RSSRUWXQLW\�IRU�SURSRQHQWV�RI�VWDQGDUGL]HG�DFKLHYHPHQWWHVWLQJ�WR�UHVSRQG�DQG�GHIHQG�WKH�PHULWV�DQG�YDOXHV�RI�VWDQGDUGL]HGWHVWLQJ��:H·G�OLNH�LQ�RXU�́ ILQDO�ZRUGµ�WR�UDLVH�D�QXPEHU�RI�TXHVWLRQV�DQGFKDOOHQJHV�IRU�WKH�(GXFDWLRQ�4XDOLW\�DQG�$VVHVVPHQW�2IILFH��(4$2�DQG�WKH�0LQLVWU\�RI�(GXFDWLRQ�DQG�7UDLQLQJ��0(7��WR�DGGUHVV�DQG�IRUWKH� SXEOLF� WR� GHEDWH�� :H� KRSH� WKLV� GLDORJXH� EHWZHHQ� WKH� 2QWDULRJRYHUQPHQW��WKH�HGXFDWLRQ�FRPPXQLW\��DQG�WKH�SXEOLF�ZLOO�KDSSHQ�DQGLW�ZLOO�UHO\�QRW�RQ�SROLWLFDO�DJHQGDV��EXW�UDWKHU�RQ�ORJLFDO�DUJXPHQWV�OLWHUDWXUH�DERXW�UHVHDUFK�ILQGLQJV�RQ�ODUJH�VFDOH�DFKLHYHPHQW�WHVWLQJ�DQG� ILQGLQJV� IURP� UHVHDUFK� RQ� WKH� 2QWDULR� 3URYLQFH�:LGH� 7HVWLQJ3URJUDP��

Page 2: Accountability Testing Without Accountability?

��� 721<�&�0��/$0�DQG�&$7+(5,1(�%25',*121

+DV�WKH�3URYLQFH�:LGH�7HVWLQJ�3URJUDP$FFRPSOLVKHG�LWV�6WDWHG�3XUSRVHV"

$FFRUGLQJ� WR� WKH������6WDQGDUGV� IRU�(GXFDWLRQDO� DQG�3V\FKRORJLFDO7HVWLQJ��

/DUJH�VFDOH�WHVWLQJ�LV�LQFUHDVLQJO\�YLHZHG�DV�D�WRRO�RI�HGXFDWLRQDOSROLF\�«�DV�D�PHDQV�WR�LQIOXHQFH�FXUULFXOXP�DQG�LQVWUXFWLRQ�� WRKROG�WHDFKHUV�DQG�VFKRRO�DGPLQLVWUDWRUV�DFFRXQWDEOH��WR�LQFUHDVHVWXGHQW�PRWLYDWLRQ��DQG�WR�FRPPXQLFDWH�SHUIRUPDQFH�H[SHFWDWLRQVWR�VWXGHQWV��WR�WHDFKHUV��DQG�WR�WKH�SXEOLF��,I�VXFK�JRDOV�DUH�VHW�IRUWKDV�SDUW�RI�WKH�UDWLRQDOH�IRU�D�WHVWLQJ�SURJUDP��WKH�YDOLGLW\�RI�WKHWHVWLQJ�SURJUDP�QHHGV�WR�EH�H[DPLQHG�ZLWK�UHVSHFW�WR�WKHVH�JRDOV��$3$��$(5$���1&0(��������S�������

6KHSDUG� ������� HFKRHV� WKLV� VHQWLPHQW� ZKHQ� VKH� GHFODUHV� WKDW� WHVWGHYHORSHUV�´DUH�UHVSRQVLEOH�IRU�WKH�XVHV�WKDW�WKH\�DGYHUWLVHµ��S�������,QRWKHU�ZRUGV��WKH\�DUH�DFFRXQWDEOH�IRU�WKH�WHVWV�WKH\�GHYHORS�

7KH�2QWDULR�SURYLQFLDO�WHVWV�DUH�XVHG�IRU�ERWK�DFFRXQWDELOLW\�DQGHGXFDWLRQDO� LPSURYHPHQW� �VHH�/DP��%RUGLJQRQ��������� ,Q�RUGHU� WROHJLWLPDWHO\�XVH�WKHVH�WHVW�UHVXOWV�WR�KROG�VFKRRO�ERDUGV��VFKRROV��DQGWHDFKHUV� DFFRXQWDEOH�� (4$2� VKRXOG� SURYLGH� HYLGHQFH� WKDW� WKH� WHVWUHVXOWV�DUH�LQGHHG�UHOLDEOH��YDOLG��DQG�ELDV�IUHH��DFFXUDWHO\�UHIOHFWLQJVWXGHQW�DFKLHYHPHQW�OHYHOV��:H�DOVR�QHHG�HYLGHQFH�WKDW�WKHVH�UHVXOWVDUH�DSSURSULDWH�IRU�UDQNLQJ�VFKRRO�ERDUGV�DQG�VFKRROV�DQG�IRU�FRPSDULQJDFKLHYHPHQW� DFURVV� WLPH� WR� GHWHUPLQH� FKDQJHV� LQ� HGXFDWLRQDOFRQGLWLRQV��

,Q�UHJDUG�WR�WKH�ILUVW�FRQFHUQ��VRXQGQHVV�RI�WKH�WHVW�UHVXOWV��LQ�WKHZDNH�RI�WKH�SXEOLFDWLRQ�RI�WKH�SURYLQFLDO�WHVW�UHVXOWV��WKH�JRYHUQPHQWDQG� SXEOLF·V� LQWHUHVW� DSSHDUV� WR� IRFXV� RQ� WKH� VFRUHV� ZLWK� PLQLPDODWWHQWLRQ�GLUHFWHG� WR� WKH�TXDOLW\�RI� WKH� WHVWV��2FFDVLRQDOO\�� WHDFKHUVKDYH�H[SUHVVHG�FRQFHUQV�DERXW�WKH�GLVFUHSDQFLHV�EHWZHHQ�SURYLQFLDOWHVW�UHVXOWV�DQG�WKHLU�RZQ�VWXGHQW�DVVHVVPHQWV��VHH�/DP��%RUGLJQRQ��������HGXFDWRUV�KDYH�TXHVWLRQHG�DSSURSULDWHQHVV�RI�WHVW�GLIILFXOW\�DQGFRQWHQW� UHOHYDQFH� �2(&7$�� ������� DQG� QHZV� DUWLFOHV� KDYH� UHIOHFWHGWHDFKHUV�DQG�SDUHQWV�FRQFHUQV��+RZHYHU��WKXV�IDU��WKH�SXEOLF�DQG�WKHHGXFDWLRQ�FRPPXQLW\�KDYH�QRW�FROOHFWLYHO\�GHPDQGHG�(4$2�SURGXFHILQGLQJV� IURP� WHVW� YDOLGDWLRQ� DQG� TXDOLW\� DVVXUDQFH� VWXGLHV�� 7KHSHUYDVLYH�SXEOLF�LPSUHVVLRQ�DQG�WKH�(4$2�SRVLWLRQ�WKDW�WKH�WHVW�UHVXOWVWUXO\�LQGLFDWH�VWXGHQWV·�NQRZOHGJH�DFTXLVLWLRQ�KDV�QR�HYLGHQWLDO�VXSSRUW�WKDW�ZH�DUH�DZDUH�RI���6XFK�XQVXSSRUWHG�LQIHUHQFHV�SRVH�D�GLVWXUELQJGDQJHU� WR� WKH� KHDOWK� RI� WKH� HGXFDWLRQ� V\VWHP�� 7KLV� SUHGLFDPHQW� LVUHIOHFWHG�LQ�ZKDW�+DHUWHO��������ZURWH��

Page 3: Accountability Testing Without Accountability?

$&&2817$%,/,7<�7(67,1*�:,7+287�$&&2817$%,/,7<" ���

7KH�XVH�RI� WHVW� VFRUHV� WR� LQGH[�HGXFDWLRQDO� VXFFHVV� RU� IDLOXUH� LVDOPRVW�QHYHU�TXHVWLRQHG��/RZ�VFRUHV�DUH�EDG�QHZV��KLJK�VFRUHV�DUHJRRG�QHZV��,Q�WKH�UKHWRULF�RI�HGXFDWLRQ�UHIRUP��LW�RIWHQ�VRXQGV�DV�LILPSURYLQJ�WKH�HGXFDWLRQ�V\VWHP�LV�V\QRQ\PRXV�ZLWK�LPSURYLQJ�WHVWVFRUHV���S�������

0DNLQJ�LPSURYHPHQW�RI�WHVW�VFRUHV�DQ�HQG�LQ�LWVHOI�LQ�HGXFDWLRQ�LV�DFRQVHTXHQFH� RI� WHDFKHUV� WHDFKLQJ� WR� WKH� WHVW� DQG� QDUURZLQJ� WKHFXUULFXOXP� WR� WKH� FRQWHQW� WKH� WHVW� FRYHUV��ZKLFK� LV� D� VRXUFH� RI� WHVWLQYDOLGLW\� DQG� D� FHUWDLQW\� LI� WKH� VWDNHV� LQ� WHVWLQJ� DUH� KLJK�� $PSOHHYLGHQFH�LQ�WKH�OLWHUDWXUH�VXEVWDQWLDWHV�WKLV�HIIHFW�RI�KLJK�VWDNHV�WHVWLQJLQ�VFKRROV��,Q�WKLV�LQWHUFKDQJH�H[FKDQJH��&DVDV�DQG�0HDJKDQ�������QRWH� WKH� GLYHUVLRQ� RI� LQVWUXFWLRQDO� WLPH� IURP� RWKHU� DUHDV� RI� WKHFXUULFXOXP��VXFK�DV�DUW��PXVLF��SK\VLFDO�HGXFDWLRQ��WR�IRFXV�RQ�D�QDUURZEDQG� RI� OHDUQLQJ� WDVNV� RU� VNLOOV� WKDW� GRPLQDWH� WKH� DVVHVVPHQWLQVWUXPHQW��6XFK�D�ELDV�DOVR�PLQLPL]HV��LQ�VSHFLILF�VXEMHFWV��IRU�H[DPSOHPDWKHPDWLFV� RU� ODQJXDJH�� DVSHFWV� RI� WKH� FXUULFXOXP�QRW� FRQVLGHUHGSULPDU\� FRPSRQHQWV� RI� WKH� DVVHVVPHQW� LQ� WKDW� SDUWLFXODU� VXEMHFW�)URHVH�*HUPDLQ��������DOVR�QRWHV�WKLV�SUHVVXUH�WR�DOLJQ�WKH�FXUULFXOXPZLWK�WKH�FRQWHQW�RI�WKH�WHVW��ZLWK�LWV�HQVXLQJ�GLPLQLVKPHQW�RI�RWKHUVXEMHFW� DUHDV��� %HVLGHV� GHFU\LQJ� WKH� WUHQG� WRZDUG� QDUURZLQJ� WKHFXUULFXOXP��KH�FLWHV�HYLGHQFH�WKDW�WKHVH�WHVWV�IUHTXHQWO\�H[KLELW�D�ELDVDJDLQVW�PLQRULW\�DQG�ORZ�LQFRPH�VWXGHQWV��DQ�LVVXH�ZKLFK�&DVDV�DQG0HDJKDQ��������DOVR�GLVFXVV�

7HDFKHUV� FDQ� WHDFK� WR� WKH� WHVW� LQ� WZR� GLVWLQFW� ZD\V�� HLWKHU� E\WHDFKLQJ�H[FOXVLYHO\�WKH�WHVW�GRPDLQ��RU�E\�GULOOLQJ�WKH�VWXGHQWV�ZLWKWHVW�WDNLQJ�VNLOOV�DQG�SUDFWLFH�WHVWV��FRDFKLQJ�WKHP�GXULQJ�WHVWLQJ��DQGWHDFKLQJ�WKHP�DQVZHUV�WR�WKH�DFWXDO�WHVW�LWHPV��$OWKRXJK�WKH�ILUVW�WDFWLFFDQ�OHDG�WR�UHDO�OHDUQLQJ�DQG�KHQFH�IRUPV�DQ�DFFHSWDEOH�SUDFWLFH��WKHVHFRQG� LV� XQDFFHSWDEOH� EHFDXVH� LW� ORZHUV� WKH� VFRUHV·� YDOLGLW\� E\DUWLILFLDOO\�LQIODWLQJ�WHVW�SHUIRUPDQFH��KHQFH�UDLVLQJ�WHVW�VFRUHV�ZLWKRXWDQ�LQFUHDVH�LQ�OHDUQLQJ��6KHSDUG���������

(YHQ�DFFHSWDEOH�WHDFKLQJ�WR�WKH�WHVW�KDV�GLVDGYDQWDJHV�ZKHQ�WKHWHVWV�XVH�PXOWLSOH�FKRLFH�LWHPV�DQG�WHVW�GLVFUHWH�NQRZOHGJH�DQG�VNLOOV�HYHQ� LI� VWXGHQWV�DUH� LQ� IDFW� OHDUQLQJ� IURP� LW��7KHVH�VWXGHQWV�GR�QRWDWWDLQ�WKH�W\SH�RI�OHDUQLQJ�HVSRXVHG�E\�WKH�FXUUHQW�HGXFDWLRQ�UHIRUPPRYHPHQW��DFTXLULQJ�LQ�GHSWK�XQGHUVWDQGLQJ�DQG�KLJKHU�RUGHU�WKLQNLQJDQG� UHDVRQLQJ� VNLOOV��+HQFH�� HGXFDWRUV� UHSODFH�PXOWLSOH�FKRLFH� WHVWVZLWK�SHUIRUPDQFH�DVVHVVPHQW��LQFOXGLQJ�SRUWIROLR�DVVHVVPHQW��LQ�ODUJH�VFDOH�WHVWLQJ�SURJUDPV��

Page 4: Accountability Testing Without Accountability?

��� 721<�&�0��/$0�DQG�&$7+(5,1(�%25',*121

3UHVXPDEO\�� SHUIRUPDQFH� DVVHVVPHQWV�PHDVXUH� DFKLHYHPHQW� RIZRUWK\� HGXFDWLRQDO� RXWFRPHV�� KHQFH�� WHDFKHUV� WHDFK� WR� WHVWV� ZRUWKWHDFKLQJ�WR��:LJJLQV���������+RZHYHU��´DV�WHDFKHUV�WHDFK�WR�HYHQ�WKHEHVW�RI�WHVWV��WKH�PHDQLQJ�RI�WHVW�VFRUHV�FDQ�FKDQJH��DQG�YDOLGLW\�FDQHURGHµ��+DHUWHO��������S��������7HDFKLQJ�WR�WKH�WHVW�WKDW�LV�SHUIRUPDQFHEDVHG�UHSUHVHQWV�D�WDVN�GULYHQ�DSSURDFK�WR�SHUIRUPDQFH�DVVHVVPHQW�EXW�RQO\�D�FRQVWUXFW�GULYHQ�DSSURDFK��QRW�D�WDVN�GULYHQ�DSSURDFK��FDQSURYLGH�PHDQLQJ� RI� WKH� WDUJHW� FRQVWUXFW�ZKLFK� VHUYHV� WR� JXLGH� ´WKHVHOHFWLRQ� RU� FRQVWUXFWLRQ� RI� UHOHYDQW� WDVNVµ� �0HVVLFN�� ������ S�� ����%HVLGHV��DSDUW�IURP�WKH�REYLRXV�GLIILFXOWLHV�UHODWHG�WR�FRVWV�DQG�ORJLVWLFV�SHUIRUPDQFH�DVVHVVPHQWV�DUH�QRWRULRXV�IRU�KLJK�VXEMHFWLYLW\�DQG�ORZJHQHUDOL]DELOLW\��/LQQ��������6KDYHOVRQ��%D[WHU���3LQH���������

$V� )URHVH�*HUPDLQ� ������� SRLQWV� RXW�� RQH� FDQQRW� HOLPLQDWHSUREOHPV� DVVRFLDWHG� ZLWK� WKH� WUDGLWLRQDO� DVVHVVPHQWV� PHUHO\� E\FKDQJLQJ�WKH�IRUP�RI�DVVHVVPHQW��+HQFH��HGXFDWRUV�VKRXOG�QRW�WUHDWSHUIRUPDQFH� RU� DXWKHQWLF� DVVHVVPHQWV� DV� V\QRQ\PRXV� ZLWKSV\FKRPHWULFDOO\�VRXQG�DVVHVVPHQWV��HYHQ�WKRXJK�WKH�WHUP�́ DXWKHQWLFµLPSOLHV�MXVW�WKDW���,W�LV�MXVW�DV�LPSHUDWLYH�WR�YDOLGDWH�D�SHUIRUPDQFHDVVHVVPHQW�DV�D�PXOWLSOH�FKRLFH�WHVW��EHFDXVH�SHUIRUPDQFH�UDWLQJV�DUHOLDEOH�WR�HUURU��,W�IROORZV�WKDW�WKH�2QWDULR�SURYLQFLDO�WHVWLQJ�SURJUDP�HYHQ� WKRXJK� LW� XVHV� SHUIRUPDQFH� DVVHVVPHQW� SURFHGXUHV�� UHTXLUHVSURSHU�YDOLGDWLRQ�

$SDUW�IURP�WKH�IDFW�WKDW�SHUIRUPDQFH�DVVHVVPHQWV�DUH�IDOOLEOH��OLNHDOO�DVVHVVPHQWV��WKH\�FDQ�DOVR�KDYH�D�FRUUXSWLYH�LQIOXHQFH�RQ�LQVWUXFWLRQLQ�KLJK�VWDNHV�HQYLURQPHQWV��:KLWIRUG��-RQHV���������DQ�H[DPSOH�RIVXFK�LQIOXHQFH�LV�WHDFKLQJ�WR�WKH�WHVW�DV�GLVFXVVHG�DERYH��+HQFH��HYHQWKRXJK�SHUIRUPDQFH� DVVHVVPHQWV� FDQ� LPSDFW� WHDFKLQJ� DQG� OHDUQLQJSRVLWLYHO\�� WKHVH�HIIHFWV�FDQ�HDVLO\�EH�RYHUVKDGRZHG�E\�WKH�QHJDWLYHHIIHFWV�RI�KLJK�VWDNHV�WHVWLQJ�SUHVHQWHG�LQ�WKH�WKUHH�SDSHUV�KHUH��DQGVXEVHTXHQWO\�DGG�QRWKLQJ�WR�WKH�HGXFDWLRQ�UHIRUP�PRYHPHQW��+DHUWHO�������DVVHUWV�WKDW�́ UHJDUGOHVV�RI�WKH�YDOXH�RI�SHUIRUPDQFH�DVVHVVPHQWVLQ�WKH�FODVVURRP��D�PHDVXUHPHQW�GULYHQ�UHIRUP�VWUDWHJ\�WKDW�UHOLHV�RQSHUIRUPDQFH�DVVHVVPHQWV�WR�GULYH�FXUULFXOXP�DQG�LQVWUXFWLRQ�VHHPVERXQG�WR�IDLOµ��S�������

2QH�RI�WKH�PHFKDQLVPV�RI�DFFRXQWDELOLW\�WHVWLQJ�LV�XVLQJ�WHVW�UHVXOWV�IURP�HLWKHU�PXOWLSOH�FKRLFH�RU�SHUIRUPDQFH�WHVWV��WR�MXGJH�DQG�FRPSDUHSHUIRUPDQFH�DPRQJ�VFKRRO�ERDUGV��DPRQJ�VFKRROV��DPRQJ�WHDFKHUV��DQGDFURVV� \HDUV�� $� QXPEHU� RI� IDFWRUV� FDQ� FRQWULEXWH� WR� D� VWXGHQW·VSHUIRUPDQFH� RQ� WKH� SURYLQFLDO� WHVWV�� LQFOXGLQJ� FRJQLWLYH� DELOLWLHV�

Page 5: Accountability Testing Without Accountability?

$&&2817$%,/,7<�7(67,1*�:,7+287�$&&2817$%,/,7<" ���

OLQJXLVWLF� EDFNJURXQG�� SULRU� OHDUQLQJ�� H[SHULHQFHV� RXWVLGH� RI� VFKRRO�DFKLHYHPHQW� DVSLUDWLRQ�� IDPLO\� VRFLDO� HFRQRPLF� VWDWXV�� SDUHQWDOLQYROYHPHQW� LQ� WKH� HGXFDWLRQ� SURFHVV�� VFKRRO� UHVRXUFHV�� DQG� VR� RQ�%HFDXVH�LQVWUXFWLRQ�LV�RQO\�RQH�FRQWULEXWLQJ�IDFWRU��XVLQJ�SURYLQFLDO�WHVWUHVXOWV�WR�MXGJH�WHDFKHUV·�DELOLW\�WR�WHDFK�LV�XQIDLU�WR�ERWK�WHDFKHUV�DQGVFKRROV��)RU� LQVWDQFH�� VKRXOG�ZH� FRPSDUH� VWXGHQW�SHUIRUPDQFH� LQ�DVFKRRO�ZLWK�ORZ�DELOLW\��XQPRWLYDWHG�VWXGHQWV�DQG�OLPLWHG�UHVRXUFHV�WRWKDW�LQ�DQRWKHU�VFKRRO�ZLWK�KLJK�DELOLW\��PRWLYDWHG�VWXGHQWV�DQG�DPSOHUHVRXUFHV"� $V� 3RSKDP� ������� QRWHG�� ´&XUUHQW� VWDQGDUGL]HG� WHVWV�EHFDXVH�WKH\�FRQWDLQ�WRR�PDQ\�LWHPV�WKDW�UHDOO\�GR�QRW�WDS�D�WDOHQWHGWHDFKHUV·�LQVWUXFWLRQDO�VNLOO��VKRXOG�QRW�EH�XVHG�WR�HYDOXDWH�WKH�TXDOLW\RI�LQVWUXFWLRQDO�SURJUDPV�ZKHWKHU�DW�WKH�QDWLRQDO��VWDWH��GLVWULFW��VFKRRO�RU�WHDFKHU�OHYHOµ��S�������

,Q�2QWDULR�� HYHQ� WKRXJK�VFKRRO� FKDUDFWHULVWLFV�DUH�SXEOLVKHG� �RUPDGH�DYDLODEOH��DORQJ�ZLWK�VFKRRO�UDQNLQJV��WKH\�DUH�HDVLO\�DQG�RIWHQLJQRUHG��7KH�XVH�RI�SURYLQFLDO�WHVW�UHVXOWV�WR�FRPSDUH�VFKRROV�LV�WKH�PRVWSRZHUIXO�PHDQV�WR�IRUFH�WHDFKHUV�WHDFK�WR�WKH�WHVW��$V�)URHVH�*HUPDLQ�������VWDWHV��LI�VDQFWLRQV�WR�VFKRROV�DQG�WHDFKHUV�DUH�WLHG�WR�ORZ�WHVWVFRUHV�DQG�ERQXVHV�WR�LPSURYHG�WHVW�VFRUHV��WKH�GLVDGYDQWDJHG�ZLWKLQWKH� 2QWDULR� VFKRRO� V\VWHP� ZLOO� EHFRPH� IXUWKHU� GLVHQIUDQFKLVHG�6SHFLDOL]HG�SURJUDPV��VXFK�DV�WKRVH�IRU�SK\VLFDOO\��PHQWDOO\��RU�VRFLDOO\FKDOOHQJHG� VWXGHQWV� FRXOG� IDFH� UHGXFWLRQV� DV� HGXFDWRUV� FRQFHQWUDWHUHVRXUFHV�RQ�FXUULFXOXP�UHODWHG�WR�WKH�DVVHVVPHQW��&RPSHWHQW�WHDFKHUVDYRLG�DQG�ZHDN�WHDFKHUV�HQG�XS�LQ�ORZ�VFRULQJ�VFKRROV��7KHVH�DQG�RWKHUQHJDWLYH�LPSDFWV�RI�KLJK�VWDNHV�WHVWLQJ�RQ�HTXLW\�RFFXU�LUUHVSHFWLYH�RIWKH�IRUP�RI�DVVHVVPHQW��7D\ORU��������+DHUWHO���������

(GXFDWRUV� QHHG� WR� GRFXPHQW� WKH�QHJDWLYH� HIIHFWV� RI� KLJK�VWDNHVWHVWLQJ�LQ�2QWDULR�LQ�RUGHU�WR�GHYHORS�VWUDWHJLHV�WR�UHPHG\�WKH�SUREOHPV�,Q�RIIHULQJ� ´ZD\V�RI�HQKDQFLQJ� WKH�YDOLGLW\�� FUHGLELOLW\��DQG�SRVLWLYHLPSDFW� RI� DVVHVVPHQW� DQG� DFFRXQWDELOLW\� V\VWHPV�ZKLOH�PLQLPL]LQJWKHLU� QHJDWLYH� HIIHFWV�µ� /LQQ� ������� UHFRPPHQGV� WR� ´SXW� LQ� SODFH� DV\VWHP�IRU�HYDOXDWLQJ�ERWK�WKH�LQWHQGHG�SRVLWLYH�HIIHFWV�DQG�WKH�PRUHOLNHO\�XQLQWHQGHG�QHJDWLYH�HIIHFWV�RI�WKH�V\VWHPµ��S������

7KH�GDQJHUV�RI�LJQRULQJ�WKH�SV\FKRPHWULF�SURSHUWLHV�RI�KLJK�VWDNHVWHVWV��FDVXDOO\�XVLQJ�WHVW�VFRUHV�WR�HYDOXDWH�HGXFDWLRQ�SURJUDPV�DQGWHDFKHUV�ZLWKRXW�FDUHIXO�GHOLEHUDWLRQ��DQG�IDLOLQJ�WR�PRQLWRU�LQWHQGHGDQG�XQLQWHQGHG�WHVWLQJ�FRQVHTXHQFHV�DUH�VXPPHG�LQ�RQH�SDUDJUDSK�LQWKH� $(5$� SRVLWLRQ� VWDWHPHQW� RQ� KLJK�VWDNHV� WHVWLQJ� LQ� 3UH� .���HGXFDWLRQ��������

Page 6: Accountability Testing Without Accountability?

��� 721<�&�0��/$0�DQG�&$7+(5,1(�%25',*121

,I�KLJK�VWDNHV�WHVWLQJ�SURJUDPV�DUH�LPSOHPHQWHG�LQ�FLUFXPVWDQFHVZKHUH�HGXFDWLRQDO�UHVRXUFHV�DUH� LQDGHTXDWH�RU�ZKHUH�WHVWV� ODFNVXIILFLHQW�UHOLDELOLW\�DQG�YDOLGLW\�IRU�WKHLU�LQWHQGHG�SXUSRVHV��WKHUHLV�SRWHQWLDO�IRU�VHULRXV�KDUP��3ROLF\�PDNHUV�DQG�WKH�SXEOLF�PD\�EHPLVOHG� E\� VSXULRXV� WHVW� VFRUH� LQFUHDVHV� XQUHODWHG� WR� DQ\IXQGDPHQWDO�HGXFDWLRQDO�LPSURYHPHQW��VWXGHQWV�PD\�EH�SODFHG�DWLQFUHDVHG�ULVN�RI�HGXFDWLRQDO� IDLOXUH�DQG�GURSSLQJ� RXW�� WHDFKHUVPD\�EH�EODPHG�RU�SXQLVKHG�IRU�LQHTXLWDEOH�UHVRXUFHV�RYHU�ZKLFKWKH\� KDYH� QR� FRQWURO�� DQG� FXUULFXOXP� DQG� LQVWUXFWLRQ� PD\� EHVHYHUHO\�GLVWRUWHG�LI�KLJK�WHVW�VFRUHV�SHU�VH��UDWKHU�WKDQ�OHDUQLQJ�EHFRPH�WKH�RYHUULGLQJ�JRDO�RI�FODVVURRP�LQVWUXFWLRQ���S����

7KH�PRVW�SXEOLFO\�DQQRXQFHG�UDWLRQDOH� IRU�DQG� LQWHQGHG�JRDO�RI� WKH2QWDULR�3URYLQFH�:LGH�7HVWLQJ�3URJUDP�LV�HGXFDWLRQDO�HQKDQFHPHQW�7KH�JUDGH���WHDFKHU�VXUYH\�UHSRUWHG�LQ�RXU�SDSHU��DQG�D�JUDGH���WHDFKHUVXUYH\�IXQGHG�E\�WKH�0(7��KDYH�EHJDQ�WR�H[SORUH��WKURXJK�WHDFKHU�VHOI�UHSRUWLQJ��WKH�SURYLQFLDO�WHVWV·�LPSDFW�RQ�LQVWUXFWLRQ��+RZHYHU��ZH�GRQRW�NQRZ�ZKDW�RWKHU�UHVHDUFK�VWXGLHV�KDYH�EHHQ�LQLWLDWHG�E\�(4$2�DQGWKH�0(7��)RU�LQVWDQFH��ZH�GR�QRW�NQRZ�ZKHWKHU�(4$2�KDV�GDWD�RQ�WKHLPSOHPHQWDWLRQ�RI�WKH�LPSURYHPHQW�SODQV�DW�WKH�VFKRRO�ERDUG��VFKRRO�WHDFKHU��DQG�VWXGHQW�OHYHOV��DQG�ZKDW�GDWD�LW�KDV�RQ�WKH�LPSDFW�RI�WKHVHLPSURYHPHQW�SODQV�RQ�LQVWUXFWLRQ�DQG�OHDUQLQJ��2XU�JUDGH���VXUYH\IRXQG�WKDW�����RI�WKH�WHDFKHUV�GLG�QRW�EHOLHYH�WKDW�VFKRRO�ERDUGV�XVHWHVW�UHVXOWV�WR�SODQ�DQG�LPSURYH�VFKRROV�DQG�SURJUDPV������UHSOLHG�WKDWWKH\�GRQ·W�NQRZ��1RW�RQO\�GRHV�(4$2�KDYH�WKH�UHVSRQVLELOLW\�WR�DVVXUHDQG�SURYLGH�HYLGHQFH�LQ�VXSSRUW�RI�SURSHU�XVH�RI�SURYLQFLDO�WHVW�VFRUHVWR�HYDOXDWH�VFKRRO�ERDUGV��VFKRROV��DQG�WHDFKHUV��DV�VXJJHVWHG�DERYH��LWDOVR� KDV� WKH� REOLJDWLRQ� WR� DVVXUH� WKDW� WHVW� UHVXOWV� DUH� LQGHHGVXFFHVVIXOO\�XVHG�WR�LPSURYH�LQVWUXFWLRQ��DQG�WR�VXEVWDQWLDWH�FODLPV�RIVXFK�WHVW�LPSDFW�ZLWK�HYLGHQFH��

$V�ZH�VHHN�HYLGHQFH�VXSSRUWLQJ�WKH�XVH�RI�SURYLQFLDO�WHVW�UHVXOWV�IRUDFFRXQWDELOLW\� DQG� HGXFDWLRQDO� LPSURYHPHQW�� ZH� QHHG� WR� UDLVH� WKHTXHVWLRQ� RI� ZKHWKHU� RU� QRW� WKH� SURYLQFLDO� WHVWLQJ� SURJUDP� FDQVLPXOWDQHRXVO\�VHUYH�ERWK�SXUSRVHV�ZHOO�

&DQ�WKH�2QWDULR�3URYLQFH�:LGH�7HVWLQJ�3URJUDP$FFRPSOLVK�LWV�'XDO�3XUSRVHV"

7HVWV� DUH� GHVLJQHG� IRU� GLIIHUHQW� SXUSRVHV�� DUH� FRQVWUXFWHG� DQGLPSOHPHQWHG�GLIIHUHQWO\��DQG�WKHLU�WHFKQLFDO�UHTXLUHPHQWV�YDU\��)RUH[DPSOH��DFFRXQWDELOLW\�WHVWV�KDYH�KLJKHU�WHFKQLFDO�UHTXLUHPHQWV�WKDQWHVWV�GHVLJQHG�WR�HQKDQFH�LQVWUXFWLRQ��0DWUL[�VDPSOLQJ��ZKLFK�VHOHFWV

Page 7: Accountability Testing Without Accountability?

$&&2817$%,/,7<�7(67,1*�:,7+287�$&&2817$%,/,7<" ���

D�VDPSOH�RI�VWXGHQWV�DQG�DVNV�HDFK�WR�UHVSRQG�WR�D�SRUWLRQ�RI�WKH�WHVW�LV�DSSURSULDWH�IRU�HYDOXDWLQJ�HGXFDWLRQ�SURJUDPV�DQG�PRQLWRULQJ�WUHQGVDW� WKH� VFKRRO� RU� SURYLQFH� OHYHO�� EXW� QRW� VXSSRUWLQJ� OHDUQLQJ� DW� WKHLQGLYLGXDO�OHYHO��8VLQJ�WKH�VDPH�WHVW�IRU�PXOWLSOH�SXUSRVHV�LQFUHDVHV�WKHOLNHOLKRRG�RI�WHVW�PLVXVH��

7KH�2QWDULR�SURYLQFLDO� WHVWV�DUH� LQWHQGHG� WR� HQKDQFH�TXDOLW\� RIHGXFDWLRQ�WKURXJK�DFFRXQWDELOLW\�DQG�LQVWUXFWLRQDO�LPSURYHPHQW��ZHTXHVWLRQ�ZKHWKHU�WKHVH�WHVWV�FDQ�VHUYH�ERWK�SXUSRVHV�ZHOO��)RU�H[DPSOH�IRU�DFFRXQWDELOLW\�SXUSRVHV��VFKRROV�DUH�UDQNHG��WKLV�DFWLRQ�GHPRUDOL]HVWKH� WHDFKHUV� DQG� FRUUXSWV� WKHLU� LQVWUXFWLRQDO� SUDFWLFHV�� QHJDWLYHO\DIIHFWLQJ�LQVWUXFWLRQ�DQG�OHDUQLQJ��2QWDULR�WHDFKHUV·�ELJJHVW�REMHFWLRQWR�WKH�SURYLQFLDO�WHVWLQJ�SURJUDP���7KH�FXUUHQW�OHYHO�RI�YDOLGDWLRQ�DQGTXDOLW\� FRQWURO� PD\� VXIILFH� LI� WKH� WHVWV� DUH� XVHG� RQO\� WR� LPSURYHLQVWUXFWLRQ�� EXW� QRW� ZKHQ� WKH� WHVW� UHVXOWV� DUH� DOVR� XVHG� IRUDFFRXQWDELOLW\�SXUSRVHV��)LQDOO\��WKH�PDWUL[�VDPSOLQJ�PD\�DGHTXDWHO\HYDOXDWH�VFKRROV�EXW�QRW�LQGLYLGXDO�VWXGHQWV·�OHDUQLQJ��

3RSKDP��������EHOLHYHV�WKDW�´WR�GHVLJQ�ODUJH�VFDOH�DVVHVVPHQWV�VRWKH\�FRQWULEXWH�PHDQLQJIXOO\�WR�ERWK�DFFRXQWDELOLW\�DQG�LQVWUXFWLRQ�ZLOOWDNH�ERWK�FUHDWLYLW\�DQG�DGGLWLRQDO�HIIRUWµ��S�������+H�JRHV�RQ�WR�RIIHUILYH�UHFRPPHQGDWLRQV�IRU�DFKLHYLQJ�WKLV�HQG�

5HFRPPHQGDWLRQ����,Q�D�IXQGDPHQWDO�IDVKLRQ��UHYLVH�ODUJH�VFDOHDVVHVVPHQW�SULRULWLHV�VR�WKDW�LQVWUXFWLRQDO�HQKDQFHPHQW�LV�HTXLYDOHQWLQ�LPSRUW�WR�HGXFDWLRQDO�DFFRXQWDELOLW\�

5HFRPPHQGDWLRQ� ���$XJPHQW� WKH� LQVWUXFWLRQDO� DFXPHQ�RI� WKRVHFKDUJHG� ZLWK� WKH� GHVLJQ�� GHYHORSPHQW�� DQG� GHOLYHU\� RI� ODUJH�VFDOHHGXFDWLRQDO�DVVHVVPHQWV�

5HFRPPHQGDWLRQ����&RQFHSWXDOL]H�DQ\�VNLOO�WR�EH�DVVHVVHG�QRW�RQO\VR� LW� LV� LPSRUWDQW� IRU� FKLOGUHQ� WR� DFTXLUH� EXW� DOVR� VR� WHDFKHUV� FDQHIIHFWLYHO\�DQG�HIILFLHQWO\�SURPRWH�LWV�DWWDLQPHQW�

5HFRPPHQGDWLRQ����,I�WKH�VWXGHQW·V�NQRZOHGJH�LV�WR�EH�DVVHVVHG�GHOLQHDWH�WKH�HQWLUH�NQRZOHGJH�GRPDLQ�SULRU�WR�LWHP�GHYHORSPHQW��DQGWKHQ�VDPSOH�IURP�WKDW�GRPDLQ�DV�UHSUHVHQWDWLYHO\�DV�SRVVLEOH�

5HFRPPHQGDWLRQ����4XLW�FRQWHQGLQJ�WKDW�QDWLRQDOO\�VWDQGDUGL]HGDFKLHYHPHQW�WHVWV��RU�VWDWH�GHYHORSHG�FORQHV�RI�WKRVH�WHVWV��VKRXOG�EHHPSOR\HG�WR�HYDOXDWH�HGXFDWLRQDO�TXDOLW\��,I� WKH\� LQWHQG� WR� FRQWLQXH� XVLQJ� WKH� SURYLQFLDO� WHVWV� WR� KROG� VFKRROERDUGV�� VFKRROV�� DQG� WHDFKHUV� DFFRXQWDEOH� DV�ZHOO� DV� WR� UHIRUP� DQGLPSURYH� LQVWUXFWLRQ�� (4$2� DQG� WKH�0(7� VKRXOG� UHYLHZ� 3RSKDP·VUHFRPPHQGDWLRQV��GHFLGH�ZKHWKHU�WKH�SURYLQFLDO�WHVWLQJ�SURJUDP�KDV

Page 8: Accountability Testing Without Accountability?

��� 721<�&�0��/$0�DQG�&$7+(5,1(�%25',*121

WKH�FDSDELOLW\�WR�VHUYH�ERWK�SXUSRVHV��DQG�FRQVLGHU�ZKHWKHU�WKH\�KDYHWKH� ´FUHDWLYLW\� DQG� DGGLWLRQDO� HIIRUWµ� WR� LPSOHPHQW� WKH� QHFHVVDU\FKDQJHV��

&DQ�/DUJH�6FDOH�7HVWLQJ�7UXO\�'ULYH�(GXFDWLRQDO�5HIRUP"7KH�JRYHUQPHQW��WHDFKHUV��HGXFDWRUV��VWXGHQWV��DQG�WKH�SXEOLF�DW�ODUJHDOO�DSSDUHQWO\�DJUHH�WKDW�WKH�HGXFDWLRQ�V\VWHP�QHHGV�LPSURYHPHQW��DQGWKDW�VRPH�IRUP�RI�UHVWUXFWXULQJ�LV�QHFHVVDU\�WR�PDNH�HGXFDWLRQ�XVHIXODQG�WR�UDLVH�LWV�VWDQGDUGV�RI�HGXFDWLRQ�VR�WKDW�LW�DGHTXDWHO\�SUHSDUHVJUDGXDWLQJ�VWXGHQWV�WR�IXQFWLRQ�HIIHFWLYHO\�LQ�WKH�VRFLHW\��+RZHYHU��WKHQDWXUH�RI�WKH�EHVW�DQG�PRVW�HIILFLHQW�HGXFDWLRQ�UHIRUP�VWUDWHJ\�UHPDLQVXQGHU�GHEDWH��

,Q�1RUWK�$PHULFD��ERWK�DW�WKH�IHGHUDO�DQG�VWDWH�SURYLQFLDO�OHYHOV�ODUJH�VFDOH�DFFRXQWDELOLW\�WHVWLQJ�V\VWHPV�KDYH�EHHQ�ZLGHO\�XVHG�DV�DYHKLFOH� WR� GULYH� HGXFDWLRQ� UHIRUP� �VHH� )URHVH�*HUPDLQ�� ������� ,Q2QWDULR��WKH�JRYHUQPHQW�KDV�DGRSWHG�WKLV�PHDVXUHPHQW�GULYHQ�VWUDWHJ\WR�UHIRUP�HGXFDWLRQ��E\�SXEOLVKLQJ�DQG�UDQNLQJ�VFKRROV�RQ�WHVW�UHVXOWV�SURYLGLQJ�PRGHOV� RI� GHVLUDEOH� LQVWUXFWLRQ� DQG� FRQWHQW� WKURXJK� WHVWDGPLQLVWUDWLRQ�DQG�VFRULQJ��DQG�E\�UHTXLULQJ�VFKRRO�ERDUGV��VFKRROV��DQGWHDFKHUV�WR�GHYHORS�DQG�LPSOHPHQW�LPSURYHPHQW�SODQV�EDVHG�RQ�WHVWUHVXOWV�� +RZHYHU�� ERWK� PHDVXUHPHQW� H[SHUWV� DQG� HGXFDWRUV� KDYHTXHVWLRQHG�WKH�HIIHFWLYHQHVV�RI�XVLQJ�KLJK�VWDNHV�WHVWLQJ�DV�DQ�DJHQW�RIFKDQJH�RQ�WKH�JURXQGV�WKDW�UHIRUPLQJ�HGXFDWLRQ�UHTXLUHV�PRUH�WKDQHVWDEOLVKLQJ�VWDQGDUGV�DQG�FRQVWUXFWLQJ�WHVWV�WR�PHDVXUH�DWWDLQPHQWRI�WKRVH�VWDQGDUGV��DQG�WKDW�WKH�XVH�RI�WKLV�VWUDWHJ\�PD\�EH�SROLWLFDOO\PRWLYDWHG�

0F/DXJKOLQ�DQG�6KHSDUG���������DUJXH�WKDW�UHOLDQFH�RQ�UHZDUG�DQGSXQLVKPHQW� WKURXJK� DFFRXQWDELOLW\� WHVWLQJ� DORQH�� ZLWKRXW� FDSDFLW\EXLOGLQJ�DQG�VWDII�GHYHORSPHQW��ZLOO�QRW�VXIILFH�WR�GULYH�JHQXLQH�FKDQJHLQ�LQVWUXFWLRQ���,Q�RXU�VXUYH\������RI�WKH�WHDFKHUV�VXUYH\HG�IHOW�WKDW�WKHPRQH\�VSHQW�RQ�SURYLQFLDO�WHVWLQJ�VKRXOG�EH�GLUHFWHG�WRZDUGV�WHDFKHUSURIHVVLRQDO�GHYHORSPHQW�LQVWHDG���5HFRJQL]LQJ�WKDW�FODVVURRP�EDVHGDVVHVVPHQW�LV�PXFK�PRUH�HIIHFWLYH�WKDQ�H[WHUQDOO\�LPSRVHG�DVVHVVPHQWLQ�VXSSRUWLQJ�OHDUQLQJ��6KHSDUG��������SURSRVHV�FUHDWLRQ�RI�WKH�UROH�RIDVVHVVPHQW�LQ�D�OHDUQLQJ�FXOWXUH�DQG�VXJJHVWV�WKDW�́ RXU�JRDO�VKRXOG�EHWR�ILQG�ZD\V�WR�IHQG�RII�WKH�QHJDWLYH�HIIHFWV�RI�H[WHUQDOO\�LPSRVHG�WHVWVDQG� WR� GHYHORS� LQVWHDG� FODVVURRP� DVVHVVPHQW� SUDFWLFHV� WKDW� FDQ� EHWUXVWHG�WR�KHOS�VWXGHQWV�WDNH�WKH�QH[W�VWHSV�LQ�OHDUQLQJµ��S������

Page 9: Accountability Testing Without Accountability?

$&&2817$%,/,7<�7(67,1*�:,7+287�$&&2817$%,/,7<" ���

+DHUWHO��������EHOLHYHV�WKDW�D�PHDVXUHPHQW�GULYHQ�DFFRXQWDELOLW\V\VWHP�ZLOO�ZRUN�WR�UHIRUP�HGXFDWLRQ�LI�

7KHUH�DUH� FOHDU� H[SHFWDWLRQV�� WHDFKHUV�ZLOO� NQRZ�ZKDW� WKH\�DUHVXSSRVHG�WR�WHDFK��VWXGHQWV�ZLOO�VHH�KRZ�KDUG�WKH\�PXVW�ZRUN�WRPDNH�WKH�JUDGH��DQG�WD[SD\HUV�ZLOO�NQRZ�ZKHWKHU�WKHLU�VFKRROV�DUHPHDVXULQJ�XS��,I�WKH�VWDQGDUGV�DUH�DSSURSULDWH�� LI�VWXGHQWV�DQGWHDFKHUV�DUH�SUHSDUHG�WR�DFFHSW�WKH�FKDOOHQJH�RI�PHHWLQJ�WKHP��LIWKH�SKDVH�LQ�SHULRG�IRU�DFFRXQWDELOLW\�LV�UHDOLVWLF�� LI�UHOLDEOH�DQGYDOLG� WHVWV� DUH� DYDLODEOH� WR� DVFHUWDLQ� WKH� H[WHQW� RI� VWXGHQWV·PDVWHU\��LI�WHDFKHUV�KDYH�WKH�UHTXLVLWH�NQRZOHGJH�DQG�WUDLQLQJ�WRKHOS�VWXGHQWV�PHHW�WKH�FKDOOHQJH�RI�QHZ�VWDQGDUGV��LI�VFKRROV�DUHQRW�KREEOHG�E\�H[WUDQHRXV�GHPDQGV�DQG�UHTXLUHPHQWV��LI�QHFHVVDU\LQVWUXFWLRQDO�PDWHULDOV�DQG�UHVRXUFHV�DUH�DYDLODEOH��LI�RXW�RI�VFKRROIDFWRUV�DUH�JLYHQ�DSSURSULDWH�FRQVLGHUDWLRQ���S������

6R�ZK\�GR�SROLF\PDNHUV�ILQG�DVVHVVPHQW�VR�DSSHDOLQJ�DV�DQ�DJHQW�RIHGXFDWLRQDO�UHIRUP"�/LQQ��������RIIHUV�IRXU�H[SODQDWLRQV��´)LUVW��WHVWVDQG� DVVHVVPHQWV� DUH� UHODWLYHO\� LQH[SHQVLYH� ����� 6HFRQG�� WHVWLQJ� DQGDVVHVVPHQW� FDQ� EH� H[WHUQDOO\� PDQGDWHG� ����� 7KLUG�� WHVWLQJ� DQGDVVHVVPHQW�FKDQJHV�FDQ�EH�UDSLGO\�LPSOHPHQWHG������)RXUWK��UHVXOWV�DUHYLVLEOH�«�DQG�FDQ�EH�UHSRUWHG�WR�WKH�SUHVVµ��S������)XUWKHUPRUH��/LQQFODLPV�DIWHU�WKH�LQWURGXFWLRQ�RI�D�QHZ�DVVHVVPHQW��WHVW�VFRUHV�LQFUHDVHRYHU�WKH�ILUVW�IHZ�\HDUV�RI�WKH�WHVWLQJ�SURJUDP��ZLWK�RU�ZLWKRXW�UHDOOHDUQLQJ��DV�ERWK�WHDFKHUV�DQG�VWXGHQWV�EHFRPH�JHDUHG�WR�WKH�WHVW�DQG�´WKH�UHVXOWLQJ�RYHUO\�URV\�SLFWXUH�WKDW�LV�SDLQWHG�E\�VKRUW�WHUP�JDLQVREVHUYHG� LQ� PRVW� QHZ� WHVWLQJ� SURJUDPV� JLYHV� WKH� LPSUHVVLRQ� RILPSURYHPHQW�ULJKW�RQ�VFKHGXOH�IRU�WKH�QH[W�HOHFWLRQµ��S������7KH�FRYHU�XSIRU�WKLV�SROLWLFDO�PDQRHXYHU�RI�XVLQJ�DQ�DFFRXQWDELOLW\�WHVWLQJ�V\VWHP�WRIDNH�VXFFHVV�LQ�HGXFDWLRQ�UHIRUP�UHVWV�RQ�D�GHDUWK�RI�TXDOLW\�FRQWUROLQLWLDWLYHV�WKDW�ZRXOG�H[SRVH�WKH�WHVWLQJ�SURJUDP·V�YDOLGLW\�DQG�HIIHFWV�

(SLORJXH7KH�$PHULFDQ�(GXFDWLRQDO�5HVHDUFK�$VVRFLDWLRQ�²�$(5$���-XO\�������UHFHQWO\�LVVXHG�D�SRVLWLRQ�VWDWHPHQW�FRQFHUQLQJ�KLJK�VWDNHV�WHVWLQJ�LQ3UH�.����HGXFDWLRQ��EDVHG�RQ�WKH������6WDQGDUGV�IRU�(GXFDWLRQDO�DQG3V\FKRORJLFDO�7HVWLQJ��$FFRUGLQJ�WR�$(5$��

7KLV� VWDWHPHQW� LV� LQWHQGHG� DV� D� JXLGH� DQG� D� FDXWLRQ� WR� SROLF\PDNHUV�� WHVWLQJ� SURIHVVLRQDOV�� DQG� WHVW� XVHUV� LQYROYHG� LQ� KLJK�VWDNHV� WHVWLQJ� SURJUDPV� ����� 7KLV� VWDWHPHQW� VHWV� IRUWK� D� VHW� RIFRQGLWLRQV� HVVHQWLDO� WR� VRXQG� LPSOHPHQWDWLRQ� RI� KLJK�VWDNHVHGXFDWLRQDO�WHVWLQJ�SURJUDPV�����WKDW�HYHU\�KLJK�VWDNHV�DFKLHYHPHQWWHVWLQJ�SURJUDP�LQ�HGXFDWLRQ�VKRXOG�PHHW���S����

Page 10: Accountability Testing Without Accountability?

��� 721<�&�0��/$0�DQG�&$7+(5,1(�%25',*121

7KH�IROORZLQJ�LV�D�OLVWLQJ�RI�WKHVH�FRQGLWLRQV��� 3URWHFWLRQ�DJDLQVW�KLJK�VWDNHV�GHFLVLRQV�EDVHG�RQ�D�VLQJOH�WHVW���� $GHTXDWH�UHVRXUFHV�DQG�RSSRUWXQLW\�WR�OHDUQ���� 9DOLGDWLRQ�IRU�HDFK�VHSDUDWH�LQWHQGHG�XVH����� )XOO� GLVFORVXUH� RI� OLNHO\� QHJDWLYH� FRQVHTXHQFHV� RI�KLJK�VWDNHVWHVWLQJ�SURJUDPV���� $OLJQPHQW�EHWZHHQ�WKH�WHVW�DQG�WKH�FXUULFXOXP���� 9DOLGLW\�RI�SDVVLQJ�VFRUHV�DQG�DFKLHYHPHQW�OHYHOV��� 2SSRUWXQLWLHV�IRU�PHDQLQJIXO�UHPHGLDWLRQ�IRU�H[DPLQHHV�ZKR�IDLOKLJK�VWDNHV�WHVWV���� $SSURSULDWH�DWWHQWLRQ�WR�ODQJXDJH�GLIIHUHQFHV�DPRQJ�H[DPLQHHV���� &DUHIXO�DGKHUHQFH�WR�H[SOLFLW�UXOHV�IRU�GHWHUPLQLQJ�ZKLFK�VWXGHQWVDUH�WR�EH�WHVWHG����� 6XIILFLHQW�UHOLDELOLW\�IRU�HDFK�LQWHQGHG�XVH����� 2QJRLQJ� HYDOXDWLRQ� RI� LQWHQGHG� DQG�XQLQWHQGHG� HIIHFWV� RI� KLJK�VWDNHV�WHVWLQJ�:H� VWURQJO\� XUJH� WKDW� (4$2� DQG� WKH� 0(7� DGGUHVV� QRW� RQO\� WKHFRQFHUQV�UDLVHG�LQ�WKLV�LQWHUFKDQJH�H[FKDQJH�E\�WKH�GLIIHUHQW�DXWKRUV�EXW�DOVR�WKH�DERYH����FRQGLWLRQV��7KHVH�ERGLHV�VKRXOG�VKDUH�ZLWK�WKHSXEOLF�WKH�H[WHQW�WR�ZKLFK�WKHVH�FRQGLWLRQV�KDYH�EHHQ�PHW��)RU�HDFKFRQGLWLRQ�QRW�PHW��WKH\�VKRXOG�VWDWH�ZKDW�DFWLRQV�KDYH�EHHQ�WDNHQ�RUZLOO�EH�WDNHQ��

,Q�DQ�DUWLFOH�DGYLVLQJ�SDUHQWV�RQ�KRZ�WR�XVH�2QWDULR�3URYLQFLDO�WHVWUHVXOWV�WR�KHOS�WKHLU�NLGV�VXFFHHG�LQ�VFKRRO��$LQVZRUWK�QRWHG�WKDW�´ORYHLW�RU�KDWH� LW�� WHVWLQJ� LV�KHUH�WR�VWD\� >LQ�2QWDULR@µ� �������S��������:HDJUHH��WKLV�LV�SUHFLVHO\�ZK\�LW�LV�VR�LPSRUWDQW�WKDW�2QWDULR�UHVLGHQWVKROG�(4$2�DQG�WKH�0(7�DFFRXQWDEOH�IRU�PHHWLQJ�WKH�LQWHQGHG�JRDO�RIWKH�3URYLQFH�:LGH�7HVWLQJ�3URJUDP��WR�LPSURYH�WKH�TXDOLW\�RI�HGXFDWLRQLQ�2QWDULR�ZLWK�QR�RU�PLQLPDO�KDUPIXO�HIIHFWV��DQG�KHQFH�MXVWLI\�WKHPLOOLRQV�RI�WD[�SD\HU�GROODUV�VSHQW�RQ�WKH�WHVWLQJ�SURJUDP��LQVWHDG�RIWHDFKHU�GHYHORSPHQW�DQG�FDSDFLW\�EXLOGLQJ���

7KH�2QWDULR�JRYHUQPHQW�VKRXOG�EH�SUHSDUHG�WR�GHIHQG�LWV�XVH�RI�WKH3URYLQFH�:LGH�7HVWLQJ�3URJUDP�DV�DQ�DJHQW�RI�HGXFDWLRQDO�FKDQJH��DQGWR�GR�VR�ZLWK�HYLGHQFH�FROOHFWHG�WKURXJK�VRXQG�UHVHDUFK�VWXGLHV��7KDWPHDQV�PRUH�UHVHDUFK��PRUH�WKDQ�ZKDW�LV�FXUUHQWO\�EHLQJ�IXQGHG�E\(4$2�DQG�WKH�0(7��VKRXOG�EH�FRQGXFWHG�WR�H[DPLQH�WKH�SV\FKRPHWULFSURSHUWLHV� RI� WKH� SURYLQFLDO� WHVW� UHVXOWV� DQG� WKH� LQWHQGHG� DQGXQLQWHQGHG�HIIHFWV�RI�WKH�WHVWLQJ�SURJUDP��)LQGLQJV�IURP�WKHVH�UHVHDUFK

Page 11: Accountability Testing Without Accountability?

$&&2817$%,/,7<�7(67,1*�:,7+287�$&&2817$%,/,7<" ���

VWXGLHV� VKRXOG� EH� XVHG� WR� HQDFW� SROLF\� WKDW� VDIHJXDUGV�PLVXVH� DQGJXLGHV� IXWXUH� GHYHORSPHQW� RI� WKH� SURYLQFLDO� WHVWV��2QO\� WKHQ� LV� WKHJRYHUQPHQW�DFFRXQWDEOH�IRU�LWV�WHVWLQJ�SURJUDP��,Q�WKLV�HUD�RI�HGXFDWLRQUHIRUP�� LI� WHDFKHUV� DQG� VFKRROV� DUH� KHOG� DFFRXQWDEOH�� VKRXOGQ·W� WKHJRYHUQPHQW�EH�KHOG�DFFRXQWDEOH�DV�ZHOO"

$XWKRUV�$GGUHVV�

2QWDULR�,QVWLWXWH�IRU�6WXGLHV�LQ�(GXFDWLRQ8QLYHUVLW\�RI�7RURQWR����%ORRU�6WUHHW�:HVW7RURQWR��2QWDULR��&$1$'$�0�6��9�(0$,/��WODP#RLVH�XWRURQWR�FD

5()(5(1&(6$LQVZRUWK��/���������1RYHPEHU���7HVW�3LORWV��&DQDGLDQ�/LYLQJ����������$PHULFDQ�(GXFDWLRQDO�5HVHDUFK�$VVRFLDWLRQ��������-XO\���3RVLWLRQ�VWDWHPHQW

FRQFHUQLQJ�KLJK�VWDNHV�WHVWLQJ�LQ�3UH�.����HGXFDWLRQ��$YDLODEOH�DW�$(5$ZHE�VLWH��KWWS���ZZZ�DHUD�QHW��

$PHULFDQ� 3V\FKRORJLFDO� $VVRFLDWLRQ�� $PHULFDQ� (GXFDWLRQDO� 5HVHDUFK$VVRFLDWLRQ�� DQG� 1DWLRQDO� &RXQFLO� RQ� 0HDVXUHPHQW� LQ� (GXFDWLRQ��������� 6WDQGDUGV� IRU� HGXFDWLRQDO� DQG� SV\FKRORJLFDO� WHVWLQJ�:DVKLQJWRQ��'&��$PHULFDQ�3V\FKRORJLFDO�$VVRFLDWLRQ�

&DVDV��)�5���0HDJKDQ��'�(����������5HQHZLQJ�WKH�GHEDWH�RYHU�WKH�XVH�RIVWDQGDUGL]HG� WHVWLQJ� LQ� WKH� HYDOXDWLRQ� RI� OHDUQLQJ� DQG� WHDFKLQJ�,QWHUFKDQJH��������

)URHVH�*HUPDLQ��%����������6WDQGDUGL]HG�WHVWLQJ���KLJK�VWDNHV�GHFLVLRQV� HGXFDWLRQDO�LQHTXLW\��,QWHUFKDQJH��������

+DHUWHO��(�+����������3HUIRUPDQFH�DVVHVVPHQW�DQG�HGXFDWLRQ�UHIRUP��3KL'HOWD�.DSSDQ�����������������

/DP��&�0���%RUGLJQRQ��&����������$Q�H[DPLQDWLRQ�RI�(QJOLVK�WHDFKHUV·RSLQLRQV� DERXW� WKH� 2QWDULR� *UDGH� �� UHDGLQJ� DQG� ZULWLQJ� WHVW�,QWHUFKDQJH��������

/LQQ��5�/����������$VVHVVPHQWV�DQG�DFFRXQWDELOLW\��(GXFDWLRQDO�5HVHDUFKHU�������������

/LQQ�� 5�/�� �������� (GXFDWLRQDO� DVVHVVPHQW�� ([SDQGHG� H[SHFWDWLRQV� DQGFKDOOHQJHV��(GXFDWLRQDO�(YDOXDWLRQ�DQG�3ROLF\�$QDO\VLV�����������

0F/DXJKOLQ��0�:���6KHSDUG��/�$����������,PSURYLQJ�HGXFDWLRQ�WKURXJKVWDQGDUGV�EDVHG�UHIRUP��$�UHSRUW�RI�WKH�1DWLRQDO�$FDGHP\�RI�(GXFDWLRQSDQHO� RQ� VWDQGDUGV�EDVHG� HGXFDWLRQ� UHIRUP��6WDQIRUG�� &$��1DWLRQDO$FDGHP\�RI�(GXFDWLRQ��

Page 12: Accountability Testing Without Accountability?

��� 721<�&�0��/$0�DQG�&$7+(5,1(�%25',*121

0HVVLFN�� 6�� �������� 7KH� LQWHUSOD\� RI� HYLGHQFH� DQG� FRQVHTXHQFHV� LQ� WKHYDOLGDWLRQ�RI�SHUIRUPDQFH�DVVHVVPHQWV��(GXFDWLRQDO�5HVHDUFKHU���������������

2(&7$�5HSRUW���������)DOO���([SHUW�DQDO\VLV�RI�JUDGH���DVVHVVPHQW�FRQILUPVWHDFKHU�FRQFHUQV��7KH�5HSRUWHU��������

3RSKDP��:�-����������:KHUH�ODUJH�VFDOH�HGXFDWLRQDO�DVVHVVPHQW�LV�KHDGLQJDQG�ZK\�LW�VKRXOGQ·W��(GXFDWLRQDO�0HDVXUHPHQW��,VVXHV�DQG�3UDFWLFH��������������

6KDYHOVRQ�� 5�-��� %D[WHU�� *�3�� � 3LQH�� -�� �0D\� ������� 3HUIRUPDQFHDVVHVVPHQWV��3ROLWLFDO�UKHWRULF�DQG�PHDVXUHPHQW�UHDOLW\��(GXFDWLRQDO5HVHDUFKHU���������������

7D\ORU��&�����������$VVHVVPHQW�IRU�PHDVXUHPHQW�RU�VWDQGDUGV��7KH�SHULO�DQGSURPLVH� RI� ODUJH�VFDOH� DVVHVVPHQW� UHIRUP�� $PHULFDQ� (GXFDWLRQDO5HVHDUFK�-RXUQDO�����������������

6KHSDUG�� /�$�� �������� :K\� ZH� QHHG� EHWWHU� DVVHVVPHQWV�� (GXFDWLRQDO/HDGHUVKLS�������������

6KHSDUG��/�$����������7KH�FHQWUDOLW\�RI�WHVW�XVH�DQG�FRQVHTXHQFHV�IRU�WHVWYDOLGLW\��(GXFDWLRQDO�0HDVXUHPHQW��,VVXHV�DQG�3UDFWLFH�������������

6KHSDUG�� /�$�� �������� 7KH� UROH� RI� DVVHVVPHQW� LQ� D� OHDUQLQJ� FXOWXUH�(GXFDWLRQDO�5HVHDUFKHU��������������

:KLWIRUG��%�/���-RQHV��.����������.HQWXFN\�OHVVRQ��+RZ�KLJK�VWDNHV�VFKRRODFFRXQWDELOLW\�XQGHUPLQHV�D�SHUIRUPDQFH�EDVHG�FXUULFXOXP�YLVLRQ��,Q%�/��:KLWIRUG��.��-RQHV��(GV����$FFRXQWDELOLW\�DVVHVVPHQW��DQG�WHDFKHUFRPPLWPHQW��/HVVRQV�IURP�.HQWXFN\·V�UHIRUP�HIIRUWV��SS����������$OEDQ\�1<��6WDWH�8QLYHUVLW\�RI�1HZ�<RUN�3UHVV�

:LJJLQV��*�� �������$SULO��� 7HDFKLQJ� WR� WKH� �DXWKHQWLF�� WHVW��(GXFDWLRQDO/HDGHUVKLS���������������


Recommended