GRAFLOG: Understanding Drawings through Natural Language

Abstract

This paper describes an experimental interactive graphics interface, GRAFLOG, in which drawings receive linguistic interpretations. It is possible to emulate linguistic interaction in situations where graphics is thought to be necessary. The paper presents examples of such a kind of dialogue and the architecture of the implementation. The paper explains how representations of drawings can be constructed by treating graphical symbols as"objects", and how a parallel linguistic interpretation for these symbols can be constructed. It highlights the relevance of"deictic expressions" and"spatial prepositions" in building the interface mechanisms between these two kinds of representations. Lastly, it shows how a reasoning component is constructed for making deductions from premises that are found in both the graphical and linguistic domains. Using GRAFLOG, it is possible to represent knowledge through words and pictures. GRAFLOG is implemented, using an object oriented programming style, in PROLOG and GKS.