An Open Data Approach to Revealing Indigenous Texts in Large-Scale Digital Repositories:  A Case-Study of Locating Pages of Māori Text in the HathiTrust

1. Abstract

In this case study we report on our experiences in locating pages of M?ori text in the HathiTrust Digital Library (HTDL). Using traditional biographic metadata, i.e., the language field, only 182 items were returned out of HTDL’s 17.1 million volumes. Our Open Data approach is based on the freely available HathiTrust Extracted Features Dataset. We establish a collection of high frequency terms in Te Reo Māori, which we iteratively use as search terms to identify a group of candidate texts. We then apply NLP analysis to verify those texts that contain substantial amounts of the Māori language. Using this approach we were able to increase the number of volume returned to 598. This positive result suggests that scholars who want to analyse other low-resourced languages should be able to adopt our workflow to reveal otherwise hidden texts in their desired languages.