Web scraping for food price research

Abstract

Purpose

The purpose of this paper is to discuss web scraping as a method for extracting large amounts of data from online sources. The author wants to raise awareness of the method’s potential in the field of food price research, hoping to enable fellow researchers to apply this method.

Design/methodology/approach

The author explains the technical procedure of web scraping, reviews the existing literature, and identifies areas of application and limitations for food price research.

Findings

The author finds that web scraping is a promising method to collect customised, high-frequency data in real time, overcoming several limitations of currently used food price data sources. With today’s applications mostly focussing on (online) consumer prices, the scope of applications for web scraping broadens as more and more price data are published online.

Research limitations/implications

To better deal with the technical and legal challenges of web scraping and to exploit its scalability, joint data collection projects in the field of agricultural and food economics should be considered.

Originality/value

In agricultural and food economics, web scraping as a data collection technique has received little attention. This is one of the first articles to address this topic with particular focus on food price analysis.

Citation

Hillen, J. (2019), "Web scraping for food price research", British Food Journal, Vol. 121 No. 12, pp. 3350-3361. https://doi.org/10.1108/BFJ-02-2019-0081