Search results

1 – 2 of 2
Per page
102050
Citations:
Loading...
Access Restricted. View access options
Article
Publication date: 13 May 2020

Jo Bates, Paul Clough, Robert Jäschke, Jahna Otterbacher and Kris Unsworth

117

Abstract

Details

Online Information Review, vol. 44 no. 2
Type: Research Article
ISSN: 1468-4527

Access Restricted. View access options
Article
Publication date: 25 July 2008

Jahna Otterbacher and Dragomir Radev

Automated sentence‐level relevance and novelty detection would be of direct benefit to many information retrieval systems. However, the low level of agreement between human judges…

460

Abstract

Purpose

Automated sentence‐level relevance and novelty detection would be of direct benefit to many information retrieval systems. However, the low level of agreement between human judges performing the task is an issue of concern. In previous approaches, annotators were asked to identify sentences in a document set that are relevant to a given topic, and then to eliminate sentences that do not provide novel information. This paper aims to explore a new approach in which relevance and novelty judgments are made within the context of specific, factual information needs, rather than with respect to a broad topic.

Design/methodology/approach

An experiment is conducted in which annotators perform the novelty detection task in both the topic‐focused and fact‐focused settings.

Findings

Higher levels of agreement between judges are found on the task of identifying relevant sentences in the fact‐focused approach. However, the new approach does not improve agreement on novelty judgments.

Originality/value

The analysis confirms the intuition that making sentence‐level relevance judgments is likely to be the more difficult of the two tasks in the novelty detection framework.

Details

Journal of Documentation, vol. 64 no. 4
Type: Research Article
ISSN: 0022-0418

Keywords

1 – 2 of 2
Per page
102050