Combining SVG and models of interaction to build highly interactive user interfaces

The gap between Web design and pure User Interface design is narrowing from day to day, as shown by recent announces around the concept of rich internet application. SVG is at the heart of this evolution because it offers a high-end 2d graphics model, which is familiar to graphical designers, and which fits well within both Web browsers and standalone applications.

IntuiLab designs highly interactive user interfaces, that often feature multimodal capacities: gesture recognition, speech recognition, etc. As it is still difficult to prototype advanced graphical and multimodal applications that fit in a Web browser, we have started to use SVG outside of the browser environment. As regards purely graphical interfaces, our portfolio includes user interfaces with high-end graphics for Air-Traffic control centers, in-car systems and for mobile government services. Using SVG has lead us to improve our usability driven software engineering life cycle and to combine SVG with imperative code written in any programming language and other proprietary markup languages.

SVG fits very well in an iterative design workflow. At the early steps in a project, the graphical structure of the different components that make up a user interface can be expressed as a logical structure. This structure is seen as a set of groups and subgroups by the graphical designer, as a set of components by programmers. It can serve as a contract between the engineers and designers, who can thus start work in parallel: the engineers start programming the components, while the graphical designers experts start designing the graphical aspects of the components. Each time a mockup is required for an intermediate user test, the graphical designer sends the design as a set of SVG files. The binding between the SVG structure and the user interface component is realized through the identifiers of the graphical elements that match the names agreed upon during the definition of the logical structure. With this method, we have been able to integrate the final look and feel of several user interfaces very close to the end of the project, with very low integration costs. Another avantage of this method is that the graphical designs are reproduced with high fidelity in the final user interface,which is not always the case when the programmers have to translate the drawings into programming language instructions. Of course, this requires a rendering engine that conforms to the SVG requirements. For that purpose, we use TkZinc open source graphical toolkit.

We have enhanced this iterative design process based on a separation of graphical elements from the user interface code by introducing interactive components and finite state machines (FSM) as first order objects into our user interface toolkit, IntuiKit. FSMs can be built either declaratively in XML, or with programming instructions. The FSMs are used to control which parts of the SVG trees must be displayed. The link between an FSM and some SVG subtrees is realized in components through a specialized switch-like node that has been introduced into our model. As for FSMs, the switch-like nodes can be built either declaratively in XML or with programmming. Our switch-like node is an extension of the SVG switch node with a condition variable that tests the state of an FSM. SVG trees, FSM and Switches are integrated into the application scene graph that is at the heart of our toolkit. One advantage of this approach is that we do not need complex scripting language code to control the visibility of SVG nodes, as is often the case when programming interactive components with SVG and Javascript for Web browser applications. FSMs provide a simple interface between the code of the application and the SVG that represent the current states of the components.

Now that we have succesfully developped several user interfaces with our toolkit based on this SVG-centric approach (see examples), we are investigating how to specify and reuseanimations and speech grammars for multimodal applications in a similar way. Our hope is that a standardization of finite state machines (or StateCharts) could also in turn be applied to Web based user interface in the future. We also hope to convince everyone that software engineering concerns would be better addressed if DTDs or schemas for describing user-interaction were designed independently of the SVG standard.

About the authors: