CALL FOR PAPERS

ECIR 2009 Workshop on
Contextual Information Access, Seeking and Retrieval Evaluation

April 6th, 2009, Toulouse, France

AIMS

The main purpose of this workshop is to bring together IR researchers working on or interested in the evaluation of approaches to contextual information access, seeking and retrieval, and let them to share their latest research results, to express their opinions on the related issues, and to promote discussion on the future directions of evaluation.

BACKGROUND

Since the 1990s, the interest in the notion of context in Information Access, Seeking and Retrieval increased. Many researchers have been concerning with the use of context in adaptive, interactive, personalized or collaborative systems, the design of explicit and implicit feedback techniques, the investigation of relevance, the application of a notion of context to problems like advertising or mobile search.

Previous workshops and conferences, e.g. IR in Context (IRiX, 2005), Adaptive IR (AIR, 2006, 2008), Contex-based IR (CIR, 2005, 2007) and Information Interaction in Context (IIiX, 2006, 2008) gathered researchers exploring theoretical frameworks and applications which have focused on contextual IR systems.

An important issue which gave raise to discussion has been Evaluation.

It is commonly accepted that the traditional evaluation methodologies used in TREC, CLEF, NTCIR and INEX campaigns are not always suitable for considering the contextual dimensions in the information seeking/access process. Indeed, laboratory-based or system oriented evaluation is challenged by the presence of contextual dimensions such as user interaction, profile or environment which significantly impact on the relevance judgments or usefulness ratings made by the end user.

Therefore, new research is needed to understand how to overcome the challenge of user-oriented evaluation and to design novel evaluation methodologies and criteria for contextual information retrieval evaluation.  Until now, the experiments and results have been often technology or application and no standard evaluation methodology or standard test collections emerged from contextual information access, seeking and retrieval research.

TOPICS

Original and unpublished papers are welcome on any aspect including:

• User, system and context modelling for information access seeking and retrieval evaluation.
• Evaluation of implicit or explicit feedback techniques.
• Evaluation of personal information retrieval systems.
• Social media and networking based search.
• Learning algorithms that use non-traditional relevance judgments.
• Novel or extension of traditional evaluation measures.
• Novel techniques for collecting document relevance.
• Contextual and user simulation algorithms.
• Design of novel test collections.
• Accuracy evaluation of personal profiles built using implicit set-level responses.
• Merging ranking from collaborative system outputs.
• Application and evaluation of context-based systems for distributed retrieval, personal search, digital libraries, archives and museums.
• Application and evaluation of context-based access to television broadcasted recordings, image, video and music collections.