Defoe: A Spark-Based Toolbox for Analysing Digital Historical Textual Data
Name
Nyhan_AAM_eScience_2019_paper_28_easychair.pdf
Description
visibility:open
Size
2.56 MB
Format
Adobe PDF
Checksum (CRC64NVME)
ktj4fVas5jk=
Resource type
Conference paper (unpublished)
Date published
2019
Abstract
This work presents defoe, a new scalable and portable digital eScience toolbox that enables historical research. It allows for running text mining queries across large datasets, such as historical newspapers and books in parallel via Apache Spark. It handles queries against collections that comprise several XML schemas and physical representations. The proposed tool has been successfully evaluated using five different large-scale historical text datasets and two HPC environments, as well as on desktops. Results shows that defoe allows researchers to query multiple datasets in parallel from a single command-line interface and in a consistent way, without any HPC environment-specific requirements.
Contributor (organisation)
British Library
Living with Machines
Project(s)
Living with Machines
Funder
| Funder name | Awards |
Arts and Humanities Research Council | AH/S01179X/1 |
Event title
2019 IEEE 15th International Conference on e-Science (e-Science) - San Diego, United States
Publisher
IEEE
Place of publication
San Diego, CA, USA
Official URL
Related URL
Additional information
The attached file is the authors' accepted manuscript of this paper published by IEEE.
Collection