ŎŬŤŰĶŀĿŪūĺĺĶŀľĿŀ�C:agÖQV2VĖĢbÅģ�

¸�QVQVģ¾GÒVÎÕĖ�

ÉÍżį��

�ĴŕœʼnZĴƞ£ö�

ËżÊȁ �·_äĘƞēƀƅ�ǁǫǐǥdžǐ�?ƞ¡$Ɯ�ƂƯpĭƲŕœʼnú�Ƙ­ƨƯǷ�

�ŶǏƶDžNJǧƘƟ·_äĘƞ-²ƅ;ìǮ�

Ăż�ȁ �äĘǹ�ƙĿĽŀ�ƞǺƖƞ�p��Ƙ�pƋưƒ�ǁǫǐǥdžǐ�?DZQ Dz�ƙ�?ƲEƥƛƀĨĬ�DZ@�p��ƜƖƀƗǵ�?ſƮǺǸ�ǵ�?ƛƌǺǸ�ǵAûŅľ�Dz�

āżYȁ �đãÏ�m¯�

ø]èȁ �7-è!H�

āYZĴ�

ø]Ă��

ƊƊƘŕœʼnZĴ¿džǥƷǑƜŴ&Ʈ�ƂƥƎǷ�

āYdžǁƵû³�

43 2

2

42

22

41

8

17

36

44

48

6

āYdžǁƵ%ij�

43

39

11

17

45

18

47

9

13

21

43

ƼǎǂǦ%ij�

45 47 26 11 36 49 30 23 17 46 33 7 41 45 19 41 45 26 31 48

5 7 16 18 31 4 42 12 25 6 14 11 18 10 22 6 5 17 21 8

œŨşũśūŢŦšĶŔŧŢŦūûÙ�

45 47

26 11

36 49

30

23 17

46 33 7 41

45 19

41

45

26

31

48

5

7 16

18 31

4 42 12

25 6 14

11 18 10

22 6 5

17

21

8

45 47 26 11 36 49 30 23 17 46 33 7 41 45 19 41 45 26 31 48

5 7 16 18 31 4 42 12 25 6 14 11 18 10 22 6 5 17 21 8

7 1

7 1

3 5

2 6

1 7

7 1

14 2

17 7

19 13

20 20

0.35 0.05

0.7 0.1

0.85 0.35

0.95 0.65

1.0 1.0

45 47

26 11

36 49

30

23 17

46 33 7 41

45 19

41

45

26

31

48

5

7 16

18 31

4 42 12

25 6 14

11 18 10

22 6 5

17

21

8

45 47 26 11 36 49 30 23 17 46 33 7 41 45 19 41 45 26 31 48

5 7 16 18 31 4 42 12 25 6 14 11 18 10 22 6 5 17 21 8

ŔŤŧūĶŕœʼnĶʼnŬũŭşŪ�

7 1

7 1

3 5

2 6

1 7

7 1

14 2

17 7

19 13

20 20

0.35 0.05

0.7 0.1

0.85 0.35

0.95 0.65

1.0 1.0

45 47

26 11

36 49

30

23 17

46 33 7 41

45 19

41

45

26

31

48

5

7 16

18 31

4 42 12

25 6 14

11 18 10

22 6 5

17

21

8

45 47 26 11 36 49 30 23 17 46 33 7 41 45 19 41 45 26 31 48

5 7 16 18 31 4 42 12 25 6 14 11 18 10 22 6 5 17 21 8

ŕœʼn�ä�ƞīÔƞûÙ�¯DZ<oĎ�¯Dz�

7 1

7 1

3 5

2 6

1 7

7 1

14 2

17 7

19 13

20 20

0.35 0.05

0.7 0.1

0.85 0.35

0.95 0.65

1.0 1.0

ŷ

Ÿ

Ź�ź�

Ż�

( ) ( )∑=

−− #$

%&'

( −×+=

5

1

11

2i

iiii FPFFPFTPFTPFAUC

ŎŬŤŰĶŀĿŪūĺĺĶŀľĿŀ�C:agÖQV2VĖĢbÅģ�

¸�QVQVģ¾GÒVÎÕĖ�

ÉÍżį��

ÿ�ƜƃƉƯ(�L´�

ÃjſƮ�Ãjƛƌ� ĆpDZÿ�Dz�

ªjƘſƯżŵż(�L´żŶżÃjƘſƯ�

0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 80-90 90-100

ıżm�

(�L´� Ãj�ªj�

õ�ƞÆ�Ɯ^ƎƯ(�L´ƞ%h�

Ãjƛƌ�ƞÆ��

ÃjſƮ�ƞÆ��

(�L´ƞ%hƙ�ª÷%hƞ�Y�

ªj� Ãj�(�L´�

ım�

ªjƛÆ�Ɯ^ƎƯ�(�L´ƞ%h�

ªjƛÆ�Ɯ^ƎƯ�(�L´ƞª÷%h�

ÃjƛÆ�Ɯ^ƎƯ�(�L´ƞ%h�

ÃjƛÆ�Ɯ^ƎƯ�(�L´ƞª÷%h�

1.0

0.5

0.0 1.0 0.5 0.0

“ÃjųƲªƌƈŲÃjųƙ(�ƌƒ+A�

“ªjųƲĞēƕƗŲÃjųƙ(�ƌƒ+A�

�ª÷%hƙROC�ä�

False Positive Fraction

True

Pos

itive

Fra

ctio

n

ªj� Ãj�(�L´�

ım�

�ª÷%h�

ROC�äƞ�Y�

Freq

uenc

y

Confidence Level

1)ż(�L´ƞ%hǏǭNJƞ9r�

2)ż�ª÷%hƞ�Y�

Prob

abili

ty

Function of Confidence Level

3)żROC�äƞ�Y�

True

Pos

itive

Fra

ctio

n

False Positive Fraction

1.0

0.5

0.0 1.0 0.5 0.0

ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZǹDz�

�ª÷%hƙ�ª÷ǖǥǢǭNJa�b�

Ĥv� Ħv�ƌƆƀ��

Æ�ƞım� σn

σs

mn ms

FP(x|n)

TP(x|s)

)( nss

ns mmmma ≥−

=σ

s

nbσσ

=

2

22ns

npdaσσ

µµ

+

+=

( )[ ]aFPFbTPF +Φ×Φ= −1

!"

#$%

&Φ= −

21 daAUC

ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZǺDzż ,ǘƴƷǧ�

ǹñËżŖūŬŞŰĶŒśťşDZZĴ"\×Dz�

ŀñËżŏōŗDZŕœʼnŏōŗ¿ǘƴƷǧƙƀƁxFDz�

ŁñËżŲĶĶĶĶŲżƘJƳƓǣNjǦǎƶC�

łñËżʼnŐŐDZđãÏ�m¯ǵĵƀ�ƅŔŧŪDz�

ŃñËȈ�ŒşšśūŢŭşƛĂ�ƞ�ƞ%ƓƉƞāYǏǭNJ�qƜdzDZƵdžNJǦdžƾDzƲ ƉƯǷ�

ŃǴŒşšśūŢŭşÆ��ǴǹñËȈ�ŔŧŪŢūŢŭşƛĂ�ƞ�ƞ%ƓƉƞāYǏǭNJ�qƜdzDZƵdžNJǦdžƾDzƲ ƉƯǷ�

Ĺ°ȁ ,ƟƎƣƗ3ùð�TǷdžǝǭdžƩ!ùƞdžǝǭdžƅſưƠƹǥǭƅ$Ɨ/�ƌƥƏƳǷ�

ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZǻDzżljǘǐƞĊ/� ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZǼDzżǘƴƷǧĕ~�

ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZǽDzżǘƴƷǧþY�

ǹDz�*Ɯ¿xƌƒ ,ǘƴƷǧƲ�¿ƎƯƄȅżØƂȁY

ǺDz�ǘƴƷǧĕ~DZ*džǥƷǑ7·Dz�ǻDz�!ĖDZFDzƥƒƟǵöÝDZCDzƌƒ$,ǘƴƷǧƲ�zƎƯƄȅżØƂȁC

ǼDz�$,ǘƴƷǧCƲ�Y�ǽDz�ÁīóÐƟ!ĖDZFDzƄǵƥƒƟöÝƌƒƩƞDZCDzƄȅżØƂȁC

dzdzdzǚǩƿǥǡdžNJǭǐdzdzdz��ǾDz�DZƩƌǵ ,�ƅǺƖƞǣNjǦǎƶƜ^ƎƯāY�ƞMADzǺƖƞǣNjǦǎƶƜƖƀƗǵǹÂËǵǺÂËǵƥƒƟ��ƞú�ƅtöƄȅżØƂȁall

ǿDz�áûÊ�xf¡YƟ�ª÷ǖǥǢǭNJDZBivariateDzǵROC�ä�ƞīÔDZAreaDzǵƥƒƟſƯ¹YƞTPFǵƞƚưƲ^ĈƙƌƗñƁƄȅż��ØƂȁArea

ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZǾDzż$,ǘƴƷǧǹ�

ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZǿDzż$,ǘƴƷǧǺ�

�¿ƎƯƞƟ�ƜƊƞǏǭNJ�

ŕœʼnŏōŗƜƬƯŕœʼnƼǭǙǘƶnjǎƶǫƿDZȀDzż$,ǘƴƷǧǻ�

ǣNjǦǎƶĞƞáûÊ�xf¡Yƞà��

ŕœʼn�äƞóÐżƹƾLjǧǘƴƷǧżŔŤŧūŕœʼnŀľĿľżDZǹDz� ŕœʼn�äƞóÐżƹƾLjǧǘƴƷǧżŔŤŧūŕœʼnŀľĿľżDZǺDz�

ROCKITƞ$,ǏǭNJƄƭaǵbǵAUCƲ ,ƎƯ�

ǏǭNJƟǣNjǦǎƶ)ƜƐưƑưƞǪǭƾDŽǭǐƘ ,ƎƯ�

ŕœʼn�äƞóÐżƹƾLjǧǘƴƷǧżŔŤŧūŕœʼnŀľĿľżDZǻDz�ŎŬŤŰĶŀĿŪūĺĺĶŀľĿŀ�

C:agÖQV2VĖĢbÅģ�

cĠQV�éVĖ�

ÑdżQČ�

ŕœʼnĶř޺ٺũĶƲ¿ƀƒŴŕœʼnø]èZĴ�

ǹDzËÊ�

�ëĖ4ÞŚä�Ƙ¡$ƋưƒêàÚpƞïwvě)ƜƃƀƗǵǏƶDžNJǧÁ�DŽdžǎǡƞǔƷLJ¹vƅǵÿ�ìƞD�DZƥƒƟ.0DzƜpĭƲ8ƤƎƞƄȅ�

ǺDz�¢�

�§�êàÚpƞïwvě)�

ǻDz�?�

����_ä}òV�DZŎŖŕŗDz¦´Á�ǏǭNJǜǭdžƞªj�Ɯ§�àÚp�DZwvȁdžǗƽǤǥſƮǵïvȁdžǗƽǤǥƛƌDzƲ -Ƿ�

��dzdžǗƽǤǥDZspicularDzȁŲ"ƄƭOƢDƀƒęºƞĤpųƘǵwvíÇƞ¹sƞ�Ɩ�

�LjǠǒǭƘ�¿ƎƯŎŖŕŗĻŊňƞ�¥�

ȃƻǦDžǒǧǘƺǭǟnjǐȄ�

ǟǐǦƾdžǃƷLJȁŀľłŅűŀľłŅżƿǨƷdžǀǭǧȁĿŀŜŢūDZ#̵mľļľȈŁļŃƜ^uDzż�

ǗƾLjǧǃƷLJȁľļĿńŃťťżǛnjNjƛƌǵ¾Á�ǏǭNJǘƺǭǟnjǐDZňŢšĶŋŦŞŢśŦĸ�

ȃ�LjǠǒǭ¿ǘƺǭǟnjǐȄ�

ǟǐǦƾdžǃƷLJȁĿľŀłűĿľŀłżƿǨƷdžǀǭǧȁŅŜŢūDZ#̵mƟî/ć�±ƦDzż�

ǗƾLjǧǃƷLJȁľļŁŃľťťżňőŔÁ�ǘƺǭǟnjǐ�

ø]èZĴ¿Ă�� with Truth (Gold Standard) Á��

§�êàÚpDZïvȁdžǗƽǤǥƛƌDzƙŴǏƶDžNJǧǔƷLJ�

§�êàÚp�DZǏƶDžNJǧǔƷLJĵDz�

§�êàÚp�DZǏƶDžNJǧǔƷLJ�Dz�

ïvDZdžǗƽǤǥƛƌDzƙwvDZdžǗƽǤǥſƮDzƞ§�êàÚpDZêªj¤ĐƛƌDz�

§�êàÚp�DZdžǗƽǤǥƛƌDz�

§�êàÚp�DZdžǗƽǤǥſƮDz�

ïvDZdžǗƽǤǥƛƌDzƙwvDZdžǗƽǤǥſƮDzƞ§�êàÚpDZêªj¤ĐſƮDz�

§�êàÚp�DZdžǗƽǤǥſƮDz�

§�êàÚp�DZdžǗƽǤǥſƮDz�

§�êàÚpDZwvȁdžǗƽǤǥſƮDzƙŴǏƶDžNJǧǔƷLJ�

§�êàÚp�DZǏƶDžNJǧǔƷLJĵDz�

§�êàÚp�DZǏƶDžNJǧǔƷLJ�Dz�

ŕœʼnĶř޺ٺũĶƲ¿ƀƒŴŕœʼnø]èZĴƞ|į�

ROC Viewer ƞLjnjǎƶǫƿ�

ø]èZĴ�

ROCFITƜƬƯƼǭǙǘƶnjǎƶǫƿ�

DBM-MRMCƜƬƯáûÊ�xf¡Y�

ǏǭNJħû�

ǐǨǭǓǫƿ¿Ă�ƞĕ~�

àÚpƟâ^Ɯïv� àÚpƟâ^Ɯwv�

�ZĴƘ¿ƀƯÆ��

àÚpƞUKƞ�¶ƞ�(�ƅĩƌƀÆ��

¡$ƅĪjƜĩƌƀ� ¡$ƅĪjƜÛ4�

ǐǨǭǓǫƿƜ��ĔƛÆ��

ŕœʼnø]èZĴ¿ljǘǐƸƹƵŴDZŕœʼnĶř޺ٺũĶŭşũĶŃļľļľļŁĸ�

kzŀŀǬŀŁlmżVòSH�żVòć�ÎÕ¼�

Žø]èZĴ��Ǎǭǧƞĝȼž�

�ÉÍį�żDZ¸�QVDzǵÑdQČżDZcĠQVDzǵ�

�6ż«=żDZcĠQVDzǵġĖô�żDZDŽƼǂQVDz�

ËÊ�

ŕœʼnĽŐŕœʼnĽŌŕœʼnø]èZĴƞZĴûÁƃƬơø]èZĴƞĒñƜƃƉƯZĴèƃƬơø]èƞ|ĞƲ�²ƌǵƄƖǵĆƦ9ƮąfƪǠdžƲƛƈƎƒƨƞ�¯ƜƖƀƗ¡üƌǵƐưƭƲZ½ƎƯƒƨƞø]èZĴ¿ƞljǘǐƸƹƵƲĝÈƎƯǷ�

ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZǹDzżZñǘƴƷǧƞM{�

ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZǺDzż'�Áī� ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZǻDzżZĴCƙø]èCƞ ,�

ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZǼDzżĂ�Á�ǘƴƷǧƞĆƦčƦ� ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZǽDzżÁīóÐǁǢǫǐƞ ,�

ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZǾDzżóÐǬ¨ìƞþY� ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZǿDzżŕśūŢŦšǕǭóÐƞþY�

ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZȀDzżNjƷƵǩƿƞþY� ŕœʼnĶř޺ٺũƞLjnjǎƶǫƿDZȀDzżZĴĝR�

ƊưƘ´�XƘƎǷZĴĝRǞNJǫƲ�ƌƗø]èZĴƲĝRƌƥƎǷ�

ŕœʼnĶř޺ٺũƜƬƯø]èZĴDZǹDzżǐǨǭǓǫƿ�

Ų©ƢųǞNJǫƟǵāYƲƌƛƀƙƵƾǎƶǙƜƛƮƥƏƳ�

ŕœʼnĶř޺ٺũƜƬƯø]èZĴDZǺDzżāY�¯�

ǟƸdžƲ�ƕƗƀƈƙǕǭƅe>ƜÓ/ƌƥƎǷÏ�mƜuƍƗ�æƲ®YƌƾǦnjƾƎƯƙǕǭƅĉƜNƱƮƥƎ�

ŕœʼnĶř޺ٺũƜƬƯø]èZĴDZǼDzż�ZĴĝR�

ǐǨǭǓǫƿƅßƱƯƙBx�ƅóÐƋưƥƎƞƘǵ"\ƲĆƳƘǵŲBxƌƥƎųƞǞNJǫƲ�ƌƗZĴƲĝRƌƥƎǷ�

ŕœʼnĶř޺ٺũƜƬƯø]èZĴDZǽDzż�ZĴ�

ǐǨǭǓǫƿƘñƕƒƞƙBƍ�¯ƘƎƣƗƞĂ�ƜƖƀƗāYƲñƀƥƎǷ�

ŕœʼnĶř޺ٺũƜƬƯø]èZĴDZǾDzż�ZĴß�

�ZĴƅßƱƯƙBx�ƅóÐƋưƥƎƞƘǵŲà�ƲóÐƎƯųƞǞNJǫƲ�ƌƗZĴƲßƌƥƎǷ�

ŕœʼnĶř޺ٺũƜƬƯø]èZĴDZǿDzżà�óÐ�

ŕœʼn�ä�YƵǧǂǦLJǡƛƌƞ¾ǏǭNJƜƬƯŕœʼn�äƅóÐƋưƥƎƞƘǵŲĜƍƯųƲ�ƌƗljǘǐƲßƌƥƎǷ�

ŕœʼnĶř޺ٺũƞ$,ǘƴƷǧĶżżŴŕœʼnŏōŗ¿ǏǭNJ�

1ñËȁ�NJƷǐǧǵø]èCǵDŽdžǎǡDZÜáDzC�2ñËȁ�KITżŵROCKIT¿ ,ǘƴƷǧƜtö�3ñËȁ�ǣNjǦǎƶCDZŲųƘJƧƊƙƅtİDz�4ñËȁ�CLLDZđãÏ�m¯DZC)Ƙ�ƅQƆƀ�

ƅPositive DZLDzƙƀƁxF�

5ñËȈ28ñËȁ ���Negative�Ɯ�ƂƭưƒāY��

29ñËȁƵdžNJǦdžƾżDZĹĹĹĹĹĹĹĹDz�

30ñËȈ65ñËȁ���Positive�Ɯ�ƂƭưƒāY��

66ñËȁƵdžNJǦdžƾżDZĹĹĹĹĹĹĹĹDz�

ŕœʼnĶř޺ٺũƞ$,ǘƴƷǧ�

ŕœʼnĶř޺ٺũƘƞø]èZĴqƜ�zƋưƯǘƴƷǧƟȂ�

ǹDz ,Á�ǘƴƷǧƜāYǏǭNJƲ -ƌƒŝŪŭǘƴƷǧ�

\Table\YYYYMMDDHHMM_DZZĴNJƷǐǧDz_DZø]èCDz_Table.csv ǺDzŕœʼnŏōŗ ,ǘƴƷǧ¿ƞūşůūǘƴƷǧ�

\RocKit\YYYYMMDDHHMM _ DZZĴNJƷǐǧDz_DZø]èCDzĶ_Rockit.txt�ǻDzø]�Ğ×ƲýĚƌƒZĴǩƿƞǎƽdžǐǘƴƷǧ�

\Log\YYYYMMDDHHMM _ DZZĴNJƷǐǧDz_DZø]èCDzĶ_log.txt �

ŕœʼnĶř޺ٺũƞ$,ǘƴƷǧĶżżŗśŜŤşǏǭNJ�

ŎŬŤŰĶŀĿŪūĺĺĶŀľĿŀ�C:agÖQV2VĖĢbÅģ�

¸�QV¾GÒVÎÕĖ�

ÉÍżį��

ŕœʼn�äĞƞáûÊ�xf¡Y�

ŕœʼn�äĞƞfƟǵáûÊƛ¡YƲñƱƛƉưƠxFƅƛƀǷ�

ŕœʼn�äĞƞfƞ¡Y¯ȁ�

�ŇŘʼnĞƞŖūŬŞşŦūĶū¡Y�

�ſƯ¹YƞŌŔŌ�ƜƃƉƯŗŔŌ�ƞfƞ¡Y�

��ª÷ǖǥǢǭNJśƙŜƞfƞ¡Y�

�ø]èĞƃƬơø]Ă�ĞƞN/Ʋçyƌƒ¡Y�

�ŶżőŬŤūŢĻŕşśŞşũĶőŬŤūŢĻʼnśŪş¯żDZŎśŝţţŦŢŠş¯Dz�

False Positive Fraction

True

Pos

itive

Fra

ctio

n

0.0 0.0

1.0 0.5

0.5

1.0

False Positive Fraction

True

Pos

itive

Fra

ctio

n

0.0 0.0

1.0 0.5

0.5

1.0 0.820 0.750

0.810 0.740

0.840 0.800

0.760 0.750

0.790 0.780

0.804 0.764

żp= 0.041

0.830 0.900

0.700 0.650

0.900 0.710

0.850 0.740

0.950 0.830

0.846 0.766

żp= 0.140

ŕœʼn�äĞƞfƞ¡YƙŴø]Ă�ƞĩ�mƙƞğ��

False Positive Fraction

True

Pos

itive

Fra

ctio

n

0.0 0.0

1.0 0.5

0.5

1.0

False Positive Fraction

True

Pos

itive

Fra

ctio

n

0.0 0.0

1.0 0.5

0.5

1.0

False Positive Fraction

True

Pos

itive

Fra

ctio

n

0.0 0.0

1.0 0.5

0.5

1.0

Ă�ƞĩ�mƅĵƎƇƯDZ¡$ƅĩƌƎƇƯDzƙǵƚƔƭƩAUCƅ`Ƌƈǵ�èƞfƅ`ƋƈƛƕƗáûÊ�xfƲĀ�ƎƯƊƙƅIĩƘſƯǷ�

Ă�ƞĩ�mƅ�ƎƇƯDZ¡$ƅ�ƌƎƇƯDzƙǵƚƔƭƩAUCƅQƆƈǵ�èƞfƅ`ƋƈƛƕƗáûÊ�xfƲĀ�ƎƯƊƙƅIĩƘſƯǷ�

Ă�ƞĩ�mƅĔnDZĩƌƀƅ�ƙƄ¡$;ìDzƓƙǵAUCƅ0.70Ȉ0.85ƙƛƮǵ�èƞfƅ¬ċÊQƆƈƛƕƗǵáûÊ�xfƲĀ�ƌƪƎƈƛƯǷ�

��ŨśŢũşŞĻȇ¡YƜƬƯŴŇŘʼnĞƞáûÊ�xf¡Y�

ø]è A B ΔAUC

ǹ 0.830 0.900 -0.070

2 0.700 0.650 0.050

3 0.900 0.710 0.190

4 0.850 0.740 0.110

5 0.950 0.830 0.120

Ave. 0.846 0.766 0.08

1

)(1

2

−

Δ−Δ=∑=

Δ r

AUCAUCSd

r

kk

AUC

835.1=Δ=

Δ

rSdAUCtAUC

t%hƘîÀmDZȆǶǹDzƞp=0.05ƞ�ƬƮƩtƅ`ƋƉưƠǵ5ĥ»5%ƘáûÊ�xfƅſƯǷ�

ņŃķ�IJ1ĞƜƬƯŴŇŘʼnĞƞáûÊ�xf¡Y�

AUCrdf

AUCrdf

r

kk

AUC

SdtAUCUB

SdtAUCLB

rr

AUCAUCSd

Δ−=

Δ−=

=Δ

•+Δ=

•−Δ=

−

Δ−Δ=∑

975.0;1

975.0;1

1

2

)1(

)(ø]è A B ΔAUC

ǹ 0.830 0.900 -0.070

2 0.700 0.650 0.050

3 0.900 0.710 0.190

4 0.850 0.740 0.110

5 0.950 0.830 0.120

Ave. 0.846 0.766 0.08

95% C.I.= (-0.041, 0.201)

ŕœʼn�äĞƞáûÊ�xf¡Yżőŕőʼn¯�

ŎśŝţţŦŢŠş¯Ʋ¿ƀƒőŬŤūŢĻŕşśŞşũĶőŬŤūŢĻʼnśŪşƞ%�%�DZŇŒœřŇDzƜƬƯáûÊ�xf¡Y�

!!!!!!

"

#

$$$$$$

%

&

•

••••

••••

•

•

!!!!!!

"

#

$$$$$$

%

&

•

••••

••••

•

•

2rc2r22r1

22c222221

21c212211

1rc1r21r1

12c122121

11c112111

XXX

XXXXXX

XXX

XXXXXX

)()1(ˆkijijijk AcAcY −−=

!!!!!!

"

#

$$$$$$

%

&

•

••••

••••

•

•

!!!!!!

"

#

$$$$$$

%

&

•

••••

••••

•

•

2rc2r22r1

22c222221

21c212211

1rc1r21r1

12c122121

11c112111

YYY

YYYYYY

YYY

YYYYYY

ǣNjǦǎƶ(t)×ø]è(r)×ø]Ă�(c)ƞāY�DZXtrcDzƞǏǭNJǟǐǦƾdž�

ǣNjǦǎƶ(t)×ø]è(r)×ø]Ă�(c)ƞÄ��DZYtrcDzƞǏǭNJǟǐǦƾdž�

ijA)(kijA

Ä��DZPseudo valueDzƞûÙ�

ȁŝ�ƞĂ�ƘûÙƌƒŇŘʼn�

ȁDZŝĻĿDz�ƞĂ�ƘûÙƌƒŇŘʼn�

ÉÍį�, Wė[ſƄƝ: ROCú�ƜƃƉƯÁ�DŽdžǎǡĞƞáûÊ�xfƞ¡Y�¯ǶJackknife¯ƙƐƞĔuǶ. ���_ä}òV�ă, 53(6), 691-698, (1997).

ŊňőĻőŕőʼnƞ ,ǘƴƷǧ�

1ñËȁ �NJƷǐǧ�2ñËȁ �Reader1 3ñËȁ �ǣNjǦǎƶCDZŲųƘJƧDz�4ñËȁ �Lȁ�ƅQƆƀ�ƅ

Positive

Readerƞ�ƓƉǵNegative CaseƞāY�ǵdzǵPositive CaseƞāY�ǵdzǵReaderǯǵƲåƮďƎǷ�

ǏǭNJƞ�qƟŲǯųƲ ,��qƜǏǭNJ!ĖƲĕƳƘǵǢǣi×ƜǁǗǭǝǭdžǐƌƗǵǎƽdžǐǘƴƷǧƘ�UƎƯ�

ŊňőĻőŕőʼnƜƬƯŕœʼn�äĞƞáûÊ�xf¡YDZǹDz�

ŊňőĻőŕőʼnƜƬƯŕœʼn�äĞƞáûÊ�xf¡YDZǺDz� ŊňőĻőŕőʼnƜƬƯŕœʼn�äĞƞáûÊ�xf¡YDZǻDz�

ƊưƅZñǘƴƷǧ�

Ċ/Áī�

ŊňőĻőŕőʼnƜƬƯŕœʼn�äĞƞáûÊ�xf¡YDZǼDz� ŊňőĻőŕőʼnƜƬƯŕœʼn�äĞƞáûÊ�xf¡YDZǽDz�

ŊňőĻőŕőʼnƜƬƯŕœʼn�äĞƞáûÊ�xf¡YDZǾDz� ŊňőĻőŕőʼnƜƬƯŕœʼn�äĞƞáûÊ�xf¡YDZǿDz�

ŊňőĻőŕőʼnƞ$,ǘƴƷǧżŇŒœřŇǘƴƷǧDZǹDz� ŊňőĻőŕőʼnƞ$,ǘƴƷǧżŇŒœřŇǘƴƷǧDZǺDz�

ŊňőĻőŕőʼnƞ$,ǘƴƷǧżŇŒœřŇǘƴƷǧDZǻDz� ŊňőĻőŕőʼnƞ$,ǘƴƷǧżŇŒœřŇǘƴƷǧDZǼDz�

p�<0.05

95ǰ�IJ1Ğ�

ŊňőĻőŕőʼnƞ$,ǘƴƷǧżŔŧŧŤǏǭNJ� ŔŕœŔŕœʼnƜƬƯƼǭǙǘƶnjǎƶǫƿDZǹDz�

True

Pos

itive

Fra

ctio

n

False Positive Fraction

1.0

0.5

0.0 1.0 0.5 0.0

Prob

abili

ty

Function of Confidence Level

ROC�äƜ�ǘnjƾƅ¾ƍƯ�

ŔŕœŔŕœʼnƜƬƯƼǭǙǘƶnjǎƶǫƿDZǺDz�

�ª÷ǖǥǢǭNJaǵbƙŔũŧŨŕœʼnƞ�ƌƀǖǥǢǭNJcǵda

!"#

$%&

−()

*+,

- +++−Φ+

()

*+,

- +++−Φ=

!"#

$%&

−()

*+,

- ++−−Φ+

()

*+,

- +−−−Φ=

+

−≡

+≡

)(12

)1(

12

)1()(

)(12

)1(

12

)1()(

11

12

2

2

2

2

2

cHccdavc

cdavcvTPF

cHccdavc

cdavcvFPF

bbc

bada

Pan X, Metz CE: The “proper” binormal model: Parametric receiver operating characteristic curve estimation with degenerate data. Acad Radiol 2005; 12:1293-1300

ŔŕœŔŕœʼnƜƬƯƼǭǙǘƶnjǎƶǫƿDZǻDzżŔŤŧūŔũŧŨŕœʼnDZǹDz�

ŔŕœŔŕœʼnƜƬƯƼǭǙǘƶnjǎƶǫƿDZǼDzżŔŤŧūŔũŧŨŕœʼnDZǺDz�

DBM-MRMCƞ$,ǏǭNJƲƊƊƜ ,�

ŔŕœŔŕœʼnƜƬƯƼǭǙǘƶnjǎƶǫƿDZǽDzżŔŤŧūŔũŧŨŕœʼnDZǻDz�

ŕœʼnú�à�ƞ¡üĮËżŴDZƦƳƛƘçƂƗƦƥƌƫƁǮDz�

ǹDz�ǔƷLJſƮƙǔƷLJƛƌƜƃƉƯïwvě)ƞŕœʼn�äĞƜƟáûÊ�xfƅĄƨƭưƒƄȅ�

ǺDz�ŇƿǧǭǚƙňƿǧǭǚƘà�ƜēƀƅſƕƒƄȅ�

ǻDz�ŇƿǧǭǚƙňƿǧǭǚƘø]èĞƞN/ƜēƀƅſƕƒƄȅ�

ǽDz�ƩƕƙƩǠdžƞPƄƕƒÆ�ƟƚưƄȅ�

ǾDz�ƩƕƙƩǔƷLJƅwpĭƲ8ƤƌƒÆ�ƟƚưƄȅ�

ǿDz�Ă�ƞĩ�mƙǔƷLJƞpĭƜƟğ�ƅſƕƒƄȅ�

ȀDz�ø]è�ƟáûÊ�xf¡Yƞà�ƜpĭƌƒƄȅ�