Dartmouth Legal Track

Wei-Ming Chen

Sphurti Saraph

Paul Thompson

Plan for Participation

• Combine the results of several open source search engines using the combination of expert opinion ( CEO) algorithm

• Train CEO algorithm using relevance feedback from year 1 of the Legal Track

• Incorporate word sense disambiguation

Actual Participation

• Only had time to implement Lucene / Indri

• Results in paper

Unofficial Subsequent Results

• Implemented Lucene

• Combined results of Lemur / Indri and Lucene using CEO algorithm

• Results from Lucene slightly better than either Lemur / Indri or CEO

• No training of CEO algorithm

Figure 1.a: Estimated Recall Table (Indri) Figure 1.b: Estimated Precision Table (Indri)

Figure 2.a: Estimated Recall Table (Lucene) Figure 2.b: Estimated Precision Table (Lucene)

Figure 3.a: Estimated Recall Table (CEO) Figure 3.b: Estimated Precision Table (CEO)

Estimated Precision At 5 docs 0.2849 At 10 docs 0.2946 At 100 docs 0.2299 At 1000 docs 0.1495 At 5000 docs 0.1312 At 10000 docs 0.1144 At 15000 docs 0.1092 At 20000 docs 0.1065 At 25000 docs 0.1019

Estimated Recall At 5 docs 0.0024 At 10 docs 0.0031 At 100 docs 0.0078 At 1000 docs 0.0355 At 5000 docs 0.0958 At 10000 docs 0.1530 At 15000 docs 0.2085 At 20000 docs 0.2466 At 25000 docs 0.2749

Estimated Precision At 5 docs 0.3105 At 10 docs 0.3472 At 100 docs 0.2603 At 1000 docs 0.1949 At 5000 docs 0.1432 At 10000 docs 0.1337 At 15000 docs 0.1307 At 20000 docs 0.1252 At 25000 docs 0.1194

Estimated Recall At 5 docs 0.0019 At 10 docs 0.0032 At 100 docs 0.0129 At 1000 docs 0.0502 At 5000 docs 0.1285 At 10000 docs 0.2224 At 15000 docs 0.2696 At 20000 docs 0.2952 At 25000 docs 0.3223

Estimated Precision At 5 docs 0.2929 At 10 docs 0.3266 At 100 docs 0.2684 At 1000 docs 0.1924 At 5000 docs 0.1459 At 10000 docs 0.1376 At 15000 docs 0.1230 At 20000 docs 0.1246 At 25000 docs 0.1194

Estimated Recall At 5 docs 0.0023 At 10 docs 0.0030 At 100 docs 0.0137 At 1000 docs 0.0514 At 5000 docs 0.1138 At 10000 docs 0.2021 At 15000 docs 0.2380 At 20000 docs 0.2915 At 25000 docs 0.3223

Future

• Train CEO algorithm

• Incorporate linguistic features