Search results

This paper seeks to discuss reliability problems associated with questionnaires, commonly employed in library and information science. It aims to focus on the effects of “common method variance” (CMV), which is a form of bias, and ways of countering these effects.

The paper critically reviews the use of existing tools for demonstrating reliability in questionnaire‐based studies. In particular, it focuses on Cronbach's alpha, “Harman's single factor test” and Lindell and Whitney's “marker variable” approach. The paper introduces an illustrative case study based on the work on metacognition and web searching. It goes on to make recommendations for researchers considering using a questionnaire‐based approach.

CMV is a problem affecting questionnaire‐based studies in different disciplines across social and information science. Where questionnaire items are more abstract, CMV has been found to be more of a problem. The widely used Cronbach alpha measure, of the reliability of a questionnaire, may often be affected by CMV. Where method bias dominates the results, a high alpha score may do no more than indicate that the questionnaire consistently allows participants to accede to their biases. “Harman's single factor test” is criticised as lacking sufficient foundation, and it is concluded that it should not be used. The marker variable approach is a useful option but must be applied with caution.

A number of practical implications for researchers are drawn. Questionnaire‐based work investigating abstract constructs should be assumed to carry a high likelihood of CMV, and therefore should attempt to avoid it and demonstrate the degree of success in this regard. Otherwise, interpretation of the results should assume CMV. A number of approaches to assessing and isolating CMV are discussed.

The paper draws attention to a problem that is arguably often overlooked in questionnaire‐based studies, namely method bias. It discusses a number of approaches whereby it may be identified and controlled.

The research reported here generated a list of criteria adopted by postgraduate students when evaluating websites. The analysis presented aims to determine whether metacognition played any part in the evaluation of websites by volunteers.

Forty‐eight students participated in the study. They carried out a series of searches designed to bring them into contact with a range of websites, from forums to electronic books. The students were encouraged to “think aloud” as they searched, and to explain their actions and strategies. Search sessions were recorded, transcribed, and subjected to ethnographic content analysis.

A range of evaluation criteria is presented. The criteria were applied at different stages of the search process and demonstrate varying degrees of metacognition. Observations on evaluation processes are also presented. Factors affecting evaluation included the purpose of the search, advice received from lecturers, and the perceived nature of the website.

Some volunteers encountered difficulties because of an overly simplistic approach to evaluation. Any training in information literacy skills should aim at equipping students with guidelines to help them assess the credibility of websites, while encouraging them to reflect on the nature of their search task and to apply the guidelines accordingly.

The evaluation criteria and processes described in this paper are induced from detailed observations of a large cohort of students. The qualitative nature of the study meant that these observations could be put into the context of the students' own reflections on their choice of website.

The purpose of this paper is to describe: a new taxonomy of metacognitive skills designed to support the study of metacognition in the context of web searching; a data collection instrument based on the taxonomy; and the results of testing the instrument on a sample of university students and staff.

The taxonomy is based on a review of the literature, and is extended to cover web searching. This forms the basis for the design of the data collection instrument, which is tested with 405 students and staff of Sheffield University.

Subjects regard the range of metacognitive skills focused on as broadly similar. However, a number of significant differences in reported metacognition usage relating to age, gender and discipline.

These findings contribute to the long‐term aims of the research which are to: develop a model of the actual and potential role of metacognition in web searching, and identify strategic “metacognitive interventions” that can be built into an intelligent information retrieval system, driven by the model, capable of enhancing retrieval effectiveness by compensating for metacognitive weaknesses on the part of the searcher.

The value of the paper lies in: the consideration of metacognition in the context of web searching, the presentation of an extensible taxonomy of metacognitive skills, development and testing of a prototype metacognitive inventory, finding of significant differences in reported metacognition usage according to age, gender and discipline, and reflection of the implications of the results for future research into web searching.

The authors investigate how COVID-19 has influenced the amount, type or topics of abuse that UK politicians receive when engaging with the public.

This work covers the first year of COVID-19 in the UK, from March 2020 to March 2021 and analyses Twitter abuse in replies to UK MPs. The authors collected and analysed 17.9 million reply tweets to the MPs. The authors present overall abuse levels during different key moments of the pandemic, analysing reactions to MPs by gender and the relationship between online abuse and topics such as Brexit, the government’s COVID-19 response and policies, and social issues.

The authors have found that abuse levels towards UK MPs were at an all-time high in December 2020. Women (particularly those from non-White backgrounds) receive unusual amounts of abuse, targeting their credibility and capacity to do their jobs. Similar to other large events like general elections and Brexit, COVID-19 has elevated abuse levels, at least temporarily.

Previous studies analysed abuse levels towards MPs in the run-up to the 2017 and 2019 UK General Elections and during the first four months of the COVID-19 pandemic in the UK. The authors compare previous findings with those of the first year of COVID-19, as the pandemic persisted, and Brexit was forthcoming. This research not only contributes to the longitudinal comparison of abuse trends against UK politicians but also presents new findings, corroborates, further clarifies and raises questions about the previous findings.

The peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-07-2022-0392