SVG Pictograms with Natural Language Based and Semantic Information

Kazunari Ito*, Motohiro Matsuda*, Martin J. Dürst*, Kōiti Hasida+

* Software Laboratory, Aoyama Gakuin University, Fuchinobe 5-10-1, Sagamihara, 229-8558 Japan

http://www.sw.it.aoyma.ac.jp

+ National Institute of Advanced Industrial Science and Technology, 10F 1-18-13 Sotokanda, Chiyoda-ku, Tokyo, 101-0021 Japan

Corresponding author's email address: kaz@it.aoyama.ac.jp

Introduction

Pictograms are simple graphics very well suited for being represented with SVG. However, until now, both our group as well as similar research efforts have used raster data formats exclusively. In this paper, we report on our effort to convert our collection of around 1000 pictograms [2] to SVG, the advantages gained from this conversion, and future benefits we are looking forward to.

Background

A pictogram is an expressive form which can be understood intuitively without a grammar. All pictograms are composed of one or more components, starting with primitives such as circles, rectangles, lines, and curves. There are numerous reports and case studies on the successful use of pictograms in the fields of learning assistance for infants and livelihood assistance for handicapped person.

In Japan, design guidelines for pictograms have been established in 2005 as JIS T 0103 [1]. These guidelines recommend that pictograms be designed by consciously considering combinations of pictograms. As an example, Fig. 3 shows the pictogram for a hospital defined in JIS T 0103. It can be understood as overlaying the pictogram for a cross (Fig. 2) on the pictogram for a building (Fig. 1), with the color of the cross changed to white.

Pictograms and Semantics

Our research is focusing on expressing sentence-level concepts by arranging several pictograms in 2D space, and to study the relationship between 2D arrangements and semanic relationships. This research is greatly aided by SVG. Rather than managing metadata separately from the actual pictograms, metadata about pictogram components and their semantic relationships can be kept as part of the pictogram itself.

Pictograms and Natural Language

The final goal of our research is the automatic conversion from natural language to pictograms and back. For this, we proceed in steps. Natural language sentences (Japanese or English) are analysed and marked up with the GDA (Global Document Annotation, [3]) XML tagset. The structure of a sentence and the graphical structure of the corresponding pictogram, available from SVG, are then compared and matched.

Based on a collection of such data, new pictograms can be created from natural language sentences. A simple example is the replacement of a pictogram that is the object of an action or state by another one. With more and more data as a base, we expect to discover deeper relationships between natural language, semantics, and graphical layout.

Pictogram Authoring

To support the creation of new pictograms and the collection of data, we have implemented a pictogram authoring tool.

Pictograms can be modified in a number of ways: applying layout and form changes such as scaling, mirroring, and rotation, changing color, appling animation, and so on. As a result of experiments, we have established that by applying such modifications, the range of concepts that can be expressed is greatly expanded, reusability is increased, new pictogram vocabulary can be created, issues of color blindness can be addressed, and the primary focus of a pictogram can be made easy to understand.

Currently, our tool allows the application of the above modifications as well as the combination of two or more component pictograms into a new pictogram. Users can search for existing pictograms using English or Japanese, can arrange the pictograms found in different ways, and can associate natural language phrases and sentences with the newly created pictograms. Using SVG has greatly simplified the implementation of this tool. Figures 3 and 4 show some results.

At the moment, users have to supply quite a bit of information, including coordinates, to create a new pictogram, but we are working on a system that will suggest layouts (semi-)automatically based only on natural language input.

Conclusion

We have shown that converting our pictogram collection comprising around 1000 pictograms from raster images to SVG has greatly helped with managing metadata and studying the relationship between natural language structure and pictogram structure. We we are looking forward to further benefits of SVG when moving to more automatic pictogram generation.

Fig.1 Building Fig. 2 Cross Fig. 3 Hospital Fig. 4 A man goes from a school to a hospital.
(This figure is an animation. Please reload if you cannot see a walking man.)

[1] http://www.kyoyohin.org/06_accessible/060100_jis.php (in Japanese)

[2] http://sa.carc.jp/pict/ (in Japanese)

[3] http://i-content.org/GDA/