News Search Using Discourse Analytics

Abstract

The vast numbers of digitised documents containing historical data constitute a rich research data repository. However, computational methods and tools available to explore this data are still limited in functionality. Research on historical archives is still largely carried out manually. Text mining technologies offer novel methods to analyse digital content to identify various types of semantic information in these documents and to extract them as semantic metadata. Methods range from the automatic identification of named entities (e.g., people, places, organisations, etc.) to more sophisticated methods to extract information about events (e.g., births, deaths, arrests, etc.), allowing users to greatly increase the specificity of their search. We have created an extended model of event interpretation to allow searches to be refined based on various discourse facets, including isolating definite information about events from more speculative details, distinguishing positive and negative opinions and categorising events according to information source. We present ISHER as an example of a multi-faceted, semantically oriented system for searching news articles from the New York Times, dating back to 1987. We explain how our extended event interpretation model can enhance search capabilities in systems such as ISHER, including the identification of contrasting and contradictory information in news articles.