ImageCLEFmed 2007 Medical Image Retrieval Results

Last updated - 8 October 2007, William Hersh

A total of 13 groups participated in ImageCLEFmed 2007. These groups collectively submitted 147 runs.

The original results presented at the CLEF meeting have been updated based on revision of relevance judgments that were found to be incorrect. New qrels have been derived from the revised relevance judgments and are available on this Web site in the file qrels2007.txt. Another problem we found in our analysis of the data was that different runs used different cases for their outputs. As trec_eval is case-sensitive, this impacted some groups' results. For that reason, we have lower-cased the qrels as well as all official runs. Any additional runs that groups perform should keep this in mind. The lower-cased runs and the output from trec_eval for them are in the protected area of the track Web site, for which all participants have a login and password.

The new results are in the spreadsheet on this site, 2007revised.xls. The table below lists the worksheets of the spreadsheet and their contents.

Worksheet	Contents
Revised	Output from all runs from trec_eval
Summary-Type	MAP of all runs with a column for run type added
Summary-SortedMAP	Data from Summary-Type sorted by
Chart	Graph of Summary-SortedMAP, with x-axis sorted by MAP score

You should use these official results in your final papers for the CLEF proceedings. We will be using them in the track overview paper.