Markus Kattenbeck and David Elsweiler
It is well known that information behaviour can be biased in countless ways and that users of web search engines have difficulty in assessing the credibility of results. Yet…
Abstract
Purpose
It is well known that information behaviour can be biased in countless ways and that users of web search engines have difficulty in assessing the credibility of results. Yet, little is known about how search engine result page (SERP) listings are used to judge credibility and in which if any way such judgements are biased. The paper aims to discuss these issues.
Design/methodology/approach
Two studies are presented. The first collects data by means of a controlled, web-based user study (N=105). Studying judgements for three controversial topics, the paper examines the extent to which users agree on credibility, the extent to which judgements relate to those applied by objective assessors and to what extent judgements can be predicted by the users’ position on and prior knowledge of the topic. A second, qualitative study (N=9) utilises the same setup; however, transcribed think-aloud protocols provide an understanding of the cues participants use to estimate credibility.
Findings
The first study reveals that users are very uncertain when assessing credibility and their impressions often diverge from objective judges who have fact checked the sources. Little evidence is found indicating that judgements are biased by prior beliefs or knowledge, but differences are observed in the accuracy of judgements across topics. Qualitatively analysing think-aloud transcripts from participants think-aloud reveals ten categories of cues, which participants used to determine the credibility of results. Despite short listings, participants utilised diverse cues for the same listings. Even when the same cues were identified and utilised, different participants often interpreted these differently. Example transcripts show how participants reach varying conclusions, illustrate common mistakes made and highlight problems with existing SERP listings.
Originality/value
This study offers a novel perspective on how the credibility of SERP listings is interpreted when assessing search results. Especially striking is how the same short snippets provide diverse informational cues and how these cues can be interpreted differently depending on the user and his or her background. This finding is significant in terms of how search engine results should be presented and opens up the new challenge of discovering technological solutions, which allow users to better judge the credibility of information sources on the web.
Details
Keywords
Amanda Spink and Jannica Heinström
Ever since our cognitive make-up allowed it, human beings have used their information behaviour abilities to help them survive. Information behaviour evolved in response to the…
Abstract
Ever since our cognitive make-up allowed it, human beings have used their information behaviour abilities to help them survive. Information behaviour evolved in response to the need by early humans to benefit from information that could not be immediately accessible in the nearby environment or obtained through communication. Humans developed an information behaviour ability, including processes of information sense making, foraging, seeking, organising and using. Information behaviour brought several benefits to early humans, including greater influence and control over their environment, and the degree in which they could use the environment for their own gain and survival. Information behaviour thus brought several advantages for the survival of early humans, and consequently emerged as a genetically favoured trait (Spink, 2010).
David Elsweiler, Max L. Wilson and Brian Kirkegaard Lunn
Originally grounded in library and information science, the majority of information behaviour and information-seeking theories focus on task-based scenarios where users try to…
Abstract
Originally grounded in library and information science, the majority of information behaviour and information-seeking theories focus on task-based scenarios where users try to resolve information needs. While other theories exist, such as how people unexpectedly encounter information, for example, they are typically related back to tasks, motivated by work or personal goals. This chapter, however, focuses on casual-leisure scenarios that are typically motivated by hedonistic needs rather than information needs, where people engage in searching behaviours for pleasure rather than to find information. This chapter describes two studies on (1) television information behaviour and (2) the casual information behaviours described by users of Twitter. The first study focuses on a specific casual-leisure domain that is familiar to many, while the second indicates that our findings generalise to many other casual-leisure scenarios. The results of these two studies are then used to define an initial model of casual-leisure information behaviour, which highlights the key differences between casual-leisure scenarios and typical information behaviour theory. The chapter concludes by discussing how this new model of casual-leisure information behaviour challenges the way we design information systems, measure their value and consequently evaluate their support for users.
David Bawden is professor of information science at City University London, UK. He has a first degree in organic chemistry (Liverpool University) and masters and doctoral degrees…
Abstract
David Bawden is professor of information science at City University London, UK. He has a first degree in organic chemistry (Liverpool University) and masters and doctoral degrees in information science (Sheffield University). He worked in research information services in the pharmaceutical industry before joining City University in 1990. His academic interests include the history and philosophy of the information sciences, information-related behaviour, knowledge organisation, scientific information, digital literacy and academic-practitioner research collaboration. He is editor of the Journal of Documentation, the leading European journal of library/information science, and is a member of the board of EUCLID, the European Association for Library and Information Teaching and Research. His interests in individual differences in information behaviour stem from studies of ‘information for creativity’ in the 1980s, and he has a particularly interest in ways of understanding individual attitudes and preferences as a way of improving information provision. His email address is db@soi.city.ac.uk.
Ian Ruthven, Mark Baillie and David Elsweiler
The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and…
Abstract
Purpose
The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and their confidence in assessing relevance for a topic, affect the relevance judgements made and the assessor's ability to predict which documents they will assess as being relevant.
Design/methodology/approach
The study was conducted as part of the Text REtrieval Conference (TREC) HARD track. Using a specially constructed questionnaire information was sought on TREC assessors' personal context and, using the TREC assessments gathered, the responses were correlated to the questionnaire questions and the final relevance decisions.
Findings
This study found that each of the three factors (interest, knowledge and confidence) had an affect on how many documents were assessed as relevant and the balance between how many documents were marked as marginally or highly relevant. Also these factors are shown to affect an assessors' ability to predict what information they will finally mark as being relevant.
Research limitations/implications
The major limitation is that the research is conducted within the TREC initiative. This means that we can report on results but cannot report on discussions with the assessors. The research implications are numerous but mainly on the effect of personal context on the outcomes of a user study.
Practical implications
One major consequence is that we should take more account of how we construct search tasks for IIR evaluation to create tasks that are interesting and relevant to experimental subjects.
Originality/value
Examining different search variables within one study to compare the relative effects on these variables on the search outcomes.
Details
Keywords
Melanie A. Kilian, Markus Kattenbeck, Matthias Ferstl, Bernd Ludwig and Florian Alt
Performing tasks in public spaces can be demanding due to task complexity. Systems that can keep track of the current task state may help their users to successfully fulfill a…
Abstract
Purpose
Performing tasks in public spaces can be demanding due to task complexity. Systems that can keep track of the current task state may help their users to successfully fulfill a task. These systems, however, require major implementation effort. The purpose of this paper is to investigate if and how a mobile information assistant which has only basic task-tracking capabilities can support users by employing a least effort approach. This means, we are interested in whether such a system is able to have an impact on the way a workflow in public space is perceived.
Design/methodology/approach
The authors implement and test AIRBOT, a mobile chatbot application that can assist air passengers in successfully boarding a plane. The authors apply a three-tier approach and, first, conduct expert and passenger interviews to understand the workflow and the information needs occurring therein; second, the authors implement a mobile chatbot application providing minimum task-tracking capabilities to support travelers by providing boarding-relevant information in a proactive manner. Finally, the authors evaluate this application by means of an in situ study (n = 101 passengers) at a major European airport.
Findings
The authors provide evidence that basic task-tracking capabilities are sufficient to affect the users’ task perception. AIRBOT is able to decrease the perceived workload airport services impose on users. It has a negative impact on satisfaction with non-personalized information offered by the airport, though.
Originality/value
The study shows that the number of features is not the most important means to successfully provide assistance in public space workflows. The study can, moreover, serve as a blueprint to design task-based assistants for other contexts.
Details
Keywords
Sarah Ahmed A. Albassam and Ian Ruthven
The purpose of this paper is to understand how typical users of YouTube judge the relevance of videos in leisure contexts; what are the reasons users give when judging video…
Abstract
Purpose
The purpose of this paper is to understand how typical users of YouTube judge the relevance of videos in leisure contexts; what are the reasons users give when judging video material as relevant or not relevant?
Design/methodology/approach
A naturalistic diary was performed in which 30 participants completed diaries providing details on their video relevance criteria. The analysis revealed 28 relevance criteria grouped into eight categories.
Findings
In total, 28 relevance criteria were identified through the analyses of the diaries’ content and they were grouped into eight categories. The findings revealed that criteria related to the content of the video are the most dominant group of criteria with topicality being the most dominant criterion. There is a considerable overlap between leisure relevance criteria and previous relevance criteria studies, but the importance of these criteria varies among different contexts. New criteria, e.g. habit emerged from the data which tend to be more related to leisure contexts.
Research limitations/implications
The decision to follow a naturalistic approach reduced the level of control on the study. A further limitation can be found in the participants’ sample used in this study, all the participants of the main study were university or college students.
Practical implications
This study attempted to enrich the current literature by investigating users’ video relevance criteria in leisure contexts. This investigation might have implications on the design of video search systems.
Originality/value
Previous relevance criteria studies focussed on work contexts and the information judged was mainly in text format. This paper outlines new insights by investigating video relevance criteria in leisure context.