Office of Undergraduate Research

Study of image similarity perception

The research group of information retrieval (RGIR) of the School of Information Studies (SOIS) is currently performing research about the perception of image similarity in the context of image retrieval systems. Image retrieval systems aim at providing tools to browse and retrieve large images collections (a typical example of such systems is Google images). The major idea of these systems is to organize images according to their degree of similarity, computed via image processing techniques (use of color, texture, shape, identification of specific objects and concepts) or via textual information provided with the images (tags/annotations or any other text resource available). This central notion of similarity as it is perceived by humans is poorly studied and understood, raising issues in the development of effective image retrieval systems and evaluation benchmarks. We are currently performing a study involving human subjects to better understand what people can expect from image retrieval systems. During these experiments, subjects are presented images and asked to provide some feedback about their similarity (rating the similarity and ranking images). The comments of the subjects during these experiments are recorded and constitute a rich source of qualitative and quantitative information about similarity perception that we want to exploit in addition to the purely quantitative data gathered by our experimental software.

Tasks and responsibilities:

The students will have to perform a first analysis of the audio recordings obtained during the experiments. One (major) part of the work is to transcribe the audio tracks to allow further analysis of their content by the principal investigator. A second part is to identify some specific events (hesitations of participants, comments about specific aspects of the images) during the recordings and annotate the tracks to allow further statistical analysis of the data. This second part is performed using Morae, a software dedicated to behavioral research, and will be performed in close interaction with the principal investigator to refine the analysis protocol when required.