Fields of Gold: Scraping Web Data for Marketing Insights

作     者：Boegershausen, Johannes Datta, Hannes Borah, Abhishek Stephen, Andrew T. 

作者机构：Erasmus Univ Rotterdam Sch Management Mkt Rotterdam Netherlands Tilburg Univ Mkt Tilburg Netherlands INSEAD Mkt Fontainebleau France Univ Oxford Said Business Sch Mkt Oxford England Univ Oxford Said Business Sch Res Oxford England 

出 版 物：《JOURNAL OF MARKETING》 (市场营销杂志)

年 卷 期：2022年第86卷第5期

页      面：1-20页

核心收录：

学科分类：12[管理学] 120202[管理学-企业管理（含：财务管理、市场营销、人力资源管理）] 0202[经济学-应用经济学] 02[经济学] 1202[管理学-工商管理] 020205[经济学-产业经济学] 

基　　金：Marketing Science Institute Dutch Research Council (NWO) [451-17-028] 

主　　题：web scraping application programming interface API crawling validity user-generated content social media big data 

摘      要：Marketing scholars increasingly use web scraping and application programming interfaces (APIs) to collect data from the internet. Yet, despite the widespread use of such web data, the idiosyncratic and sometimes insidious challenges in its collection have received limited attention. How can researchers ensure that the data sets generated via web scraping and APIs are valid? While existing resources emphasize technical details of extracting web data, the authors propose a novel methodological framework focused on enhancing its validity. In particular, the framework highlights how addressing validity concerns requires the joint consideration of idiosyncratic technical and legal/ethical questions along the three stages of collecting web data: selecting data sources, designing the data collection, and extracting the data. The authors further review more than 300 articles using web data published in the top five marketing journals and offer a typology of how web data have advanced marketing thought. The article concludes with directions for future research to identify promising web data sources and embrace novel approaches for using web data to capture and describe evolving marketplace realities.