01961nas a2200169   4500008004100000245006900041210006700110260000900177300001200186490000600198520140100204653001801605653002301623100001401646700001401660856011701674       2010                        eng d00aContent-based image retrieval systems-reviewing and benchmarking0 aContentbased image retrieval systemsreviewing and benchmarking  c2010  a54 - 640 v83 aThe last detailed review of Content-based Image Retrieval Systems (CBIRS) is that from Veltkamp and Tanase [VT02], updated in 2002. Since then, many new systems emerged, other systems were improved, but many systems are no longer supported. This paper reconsiders the systems described by Veltkamp and Tanase and proposes in addition for a selection of existing CBIRS a quantitative comparison. For this purpose we developed a benchmarking system for CBIRS based on how accurate they could match ideal (ground truth) results. As measure we introduced the RankWRN , which we developed based on the Normalized Avarage Rank. Our measure allows fair and accurate comparisons (due to multiple similarity values) of different CBIRS with respect to different queries. None of the other benchmarks allow this comparison. The choice for a system to be compared quantitatively was motivated by availability of the source or runtime code and the liveness (i.e., active development on the CBIRS). The benchmarked systems were SIMBA, SIMPLIcity, PictureFinder, ImageFinder, Caliph&amp;, VIPER/GIFT and Oracle Intermedia. Results show that VIPER/GIFT performs best in the settings of our benchmark, Caliph&amp; is second and SIMBA third. The default parameter settings of the systems tend to show the best results. MPEG-7 descriptors, which are used in Caliph&amp;, show a good performance in the benchmarking.10aImage ranking10aMPEG-7 descriptors1 aKosch, H.1 aMaier, P.  uhttp://www.scopus.com/inward/record.url?eid=2-s2.0-77958035095&partnerID=40&md5=3332d95bebdc7fdabeda9f57902ea608