A Probabilistic Automaton for the Dynamic Relevance Judgement of Users
http://data.open.ac.uk/oro/35133
is a Article , Academic article

Outgoing links

Property Object
Dataset Open Research Online
Creator
At ACM SIGIR2010 Workshop on Simulation of Interaction (SemInt)
Date 2010-07
Is part of repository
Status Peer reviewed
URI
  • http://data.open.ac.uk/oro/document/100683
  • http://data.open.ac.uk/oro/document/99906
  • http://data.open.ac.uk/oro/document/99912
  • http://data.open.ac.uk/oro/document/99926
  • http://data.open.ac.uk/oro/document/99927
  • http://data.open.ac.uk/oro/document/99928
  • http://data.open.ac.uk/oro/document/99929
Abstract Conventional information retrieval (IR) evaluation relies on static relevance judgements in test collections. These, however, are insufficient for the evaluation of interactive IR (IIR) systems. When users browse search results, their decisions on whether to keep a document may be infuenced by several factors including previously seen documents. This makes user-centred relevance judgements not only dynamic but also dependent on previous judgements. In this paper, we propose to use a probabilistic automaton (PA) to model the dynamics of users' relevance judgements. Based on the initial judgement data that can be collected in a proposed user study, the estimated PA can further simulate more dynamic relevance judgements, which are of potential usefulness for the evaluation of IIR systems.
Authors authors
Type
Label Zhang, Peng ; Cervino Beresi, Ulises; Song, Dawei and Hou, Yuexian (2010). A Probabilistic Automaton for the Dynamic Relevance Judgement of Users. In: ACM SIGIR2010 Workshop on Simulation of Interaction (SemInt), 19-23 Jul 2010, Geneva, Switzerland.
Title A Probabilistic Automaton for the Dynamic Relevance Judgement of Users