SOAP and the Web of Science: how to bulk download query results

Abstract

Purpose

This paper aims to give step-by-step guidance for downloading data from any site that uses a simple object access protocol (SOAP) interface such as Web of Science (WOS)/Web of Knowledge. The SOAP interface is a method that is used widely by Web sites to allow automatic download of data. Using a case study approach, we also discuss some problems that can happen and ways to avoid them. The WOS is used as a demonstration.

Design/methodology/approach

This case study explores ways to download large amounts of data from a Web site. As pitfalls were encountered, techniques were developed to circumnavigate them. The result is a specific procedure for accessing these types of databases.

Findings

The paper provides a transferrable procedure that non-technical people can use to access WOS and other SOAP data sources.

Practical implications

The set of scripts are presented in the paper that will allow others to download data from any site that uses a SOAP interface. Explanations are given so that the procedure can be adapted for other tasks.

Originality/value

This paper clearly illustrates how to interact with a SOAP interface with explicit instructions for downloading from WOS. It is written for the non- technical with illustrations at each step. Instructions are given for modifying the scripts so that librarians can search using other keywords and dates as well as downloading from other Web sites.

Citation

Johnson, E.O. (2016), "SOAP and the Web of Science: how to bulk download query results", Library Hi Tech News, Vol. 33 No. 1, pp. 11-20. https://doi.org/10.1108/LHTN-08-2015-0059