SVG Communicating Widgets

Table of Contents

SVG in Widgets and Widget User Agents
Discovering widgets in the environment
Presenting the available widgets to the user
Presenting the active widgets to the user
Enabling the user to move and resize a widget on the screen
Saving and restoring the preferences of a widget
Enabling the user to transfer a widget to another device
Presenting a live icon
Presenting widgets in multiple formats
Widget Communication and SVG
Interface binding
Widget modularity
Our implementations
Related Work

Widgets are lightweight interactive and dynamic multimedia applications. A widget is described using Web technologies and the environment, also called Widget User Agent (WUA), running the widget relies on browser technologies for the display. Many widgets are already available on the Internet to provide the weather information for a particular location, to capture notes, to display news items ... Even though many widget systems exist (Apple Dashboard, Google Desktop Gadgets, Microsoft Gadgets, Opera Widgets, Yahoo! Widgets ...), most widgets are made in a similar way. An XML configuration document provides metadata about the widget such as the name of the widget, the author, a description of the widget... This document indicates also an icon used to represent the widget in docks, pagers ...; and a start file that the browser will use to run and display the Widget. Typically, the start file uses HTML, which uses CSS resources for the styling and JavaScript code for querying the data to be displayed using the XMLHttpRequest API. These common practices are under standardization by the W3C, in the W3C Widgets family of recommendations [1].

The W3C also published in december 2008 the version 1.2 Tiny of the Scalable Vector Graphics language (SVG). SVG Tiny 1.2 being able to describe text, vector graphics, audio, video, animations and interactivity, including using XMLHttpRequest, can be used for creating rich widgets. In this paper, we present how we use SVG within our widget work. This usage is twofolds. First, we use an SVG document as the start file of the widget. A single widget may be displayed on different devices and therefore requires to easily adapt to different screen configurations. Indeed, in this paper, we consider widget mobility, that is the ability to move a widget accross devices. For that reason, SVG and its scalability features are of particular interest. Even though, real adaptation of the widget layout to the device screen configuration requires more than scalability, the use of SVG enables simple adaptation and ensures that the graphics used inside the widget will be displayed with an adequate quality on the different devices. Second, we use SVG documents with some ECMA-Script extensions to implement Widget User Agents.

In existing widgets, the sources of information from which the widget retrieves its textual data (e.g. RSS feeds) or its media (e.g. images or video streams), traditionally web servers, have to be known at authoring time. In our work, we consider the use of widgets in a home environment where widgets are displayed on television sets, mobile phones or other home appliances (e.g. photo frames). In this scenario, widgets may be connected to the Internet to retrieve information or they may not. In that case, widgets are used to display information from one device of the home network onto another one and the address of the source of information cannot be known at authoring time. Additionally, traditional communication means in widgets heavily rely on JavaScript, which can make it inappropriate for constrained devices. In this paper, we present a generic mechanism enabling the communication between widgets and their dynamic environment and we describe the adaptation of this mechanism required for the SVG language.

This section discusses our usage of SVG for widgets and widget user agents.

The W3C is currently producing a family of specifications related to widgets. Among them, we have experimented with the "Packaging and Configuration" [1] and the "APIs and Events" specifications [2]. In [1],the SVG language can be used to represent either the icon or the start file of the widget. This is indicated by the use of links pointing to SVG files inside the XML configuration file of the widget package. There is almost no difference between using an SVG document as a standalone document, outside a widget, and using it inside a widget, except when it is used as an icon. In that case some restrictions apply, for security reasons, and scripts are not executed.

The W3C family of specifications does not standardize the complete behavior of the Widget User Agent (WUA). In particular, how widgets are discovered or proposed to the user, how they can be activated, deactivated, moved ... is not standard. In this work, we have developed two WUAs using the SVG language and ECMA-Script together with some additional APIs. One WUA is used for managing widgets on a mobile phone (see Figure 2) and the other is used on TV sets (see Figure 1).

In both cases, the WUA is implemented by an SVG document which deals with the management of widgets. This SVG document implements the following features.

As explained in the introduction, in this work, we have identified the need for a mechanism enabling communications between widgets and communication entities not known at authoring time. These entities could be either remote servers on the Internet, in the home network or other widgets in the same or in a different WUA. In this section, an overview of our solution is presented. The details of the solution are presented in [3] and [4]. Here, we focus on the usage of this solution with the SVG language.

To enable communications between widget and other entities, we define an abstract communication model. In this model, entities communicate through messages. Messages have zero or more arguments. Sets of related messages are called interfaces.

Each interface is identified by a type and is composed of a set of messages. Each message has a direction (in/out) and is made of several parameters. An incoming message can be notified to the associated widget and the value of the parameters forwarded to the scene. An output message is send when triggered by the scene, its parameters taken from values in the scene.

For all these connections between the scene and the communication description, we primarily use a declarative language. Instants when messages should be sent are described by scene events, e.g. animation starts or mutation events. Similarly, notifications of incoming messages are given using scene events. Values to be used in outgoing messages, or in the other direction to be modified by incoming messages, are described using scene attributes.

For scripted widgets we also introduce the possibility to call script functions upon incoming messages, like a regular event handler; and the possibility to send messages from script using the specific 'invoke' function defined on a dedicated ECMAScript Interface. This interface also offers the 'invokeReply' function to send a reply to an incoming message.

We include the description of the interfaces in the configuration document of the widget package and the WUA interprets it to determine the communication capabilities of a widget. Interfaces start as unbound. WUA are tasked with finding entities with matching interfaces (other widgets, UPnP services, web services, applications...) and binding them as described next.

          <content src="comwidin.svg">
              <cw:interface type="urn:telecomparistech:service:communicationtest:1" serviceProvider="true">
                <cw:messageIn name="switch" outputTrigger="status.boolean">
                  <cw:input name="newTargetValue" setAttribute="set_status.bool"/>
                  <cw:output name="status" attributeModified="status.boolean"/>

In this second example, the interface has a single input message. The use of the serviceProvider attribute declares that the widget UA should expose this interface as a discoverable service to other devices. The messageIn element has one attribute. The outputTrigger attribute is a string which specifies an entity of the scene description. This entity can be an attribute of an element, specified as "nodeID.attributeName", which is the case here: when the attribute is modified, the message is activated in output, i.e. in this case the reply is sent. Or this entity can specify an event, as "nodeID.eventName": when this event is fired, the reply is sent. Or this entity can be a script function name: when this function is called, the reply is sent.

The input element has a setAttribute attribute of type string, specifying an attribute of an element, specified as "nodeID.attributeName". The received message contains a value for argument "newTargetValue": this value is passed onto the scene description entity, i.e. the attribute is set to that value.

When the widget is ready to reply and the outputTrigger has happened, the value found in the scene description entity specified by the attributeModified attribute (an attribute of an element, specified as "nodeID.attributeName") is sent out.

Note: The above is part of the interface of the left widget in Figures 3 and 4.

Our system is implemented in GPAC, an open source multimedia framework for research and academic purposes in different aspects of multimedia, with a focus on presentation technologies (graphics, animation and interactivity), developed at Télécom ParisTech [5]. We have created three widget UAs with wildly different goals:

  1. Our first SVG WUA is dedicated to interactive TV. Its user interface is very limited. It is possible, but not very useful to run multiple widgets concurrently. Position and size of the widgets cannot be changed by the user. It has a "Yahoo! Connected TV" style. [6]

  2. Our second SVG WUA emulates the iPhone UI. Only one widget can be visible at a time, in full screen, alternating with other running widgets and multiple pages of widget icons, à la iPhone home screen. By experience, this is our preferred UA for testing remote widget communication.

  3. Our other WUA is very generic and complex. It allows any number of concurrent widgets, and includes a user interface to load and unload widgets. It has a dock à la Mac OSX. The position, size and state of any widget can be modified by the user, in a Apple Dashboard or Netvibes kind of way.

The WUAs are implemented in a mixture of C++, ECMA-Script and SVG, as an extension to the GPAC browser. Widgets can be implemented in SVG, BIFS or any other media type supported by GPAC, such as Flash, VRML, X3D, LASeR and DIMS, regardless of the widget UA implementation language.

Note: the scripting context of the widget UAs is different from the scripting context of the widgets, which is very restricted and isolated from the widget UA scripting context by a layer of C++. This ensures maximum security from potentially hostile widgets.

Note : we have also implemented other WUA using the BIFS language following the same principles...

[Demo of iPhone-style widget UA]

[Demo of various communicating widgets]

[Demo of widget components]

Here are some of the advantages of our design and implementations:

  1. The system will work with any protocol for discoverable services and devices: our implementation uses UPnP, but we will be extending this in the future.

  2. The actual location of the services is transparent: our widgets are independent from the device they run on; service location independence is provided by the WUAs thanks to the interface type abstraction.

  3. One widget may be connected to multiple services, or one service to multiple widgets, or widgets to widgets.

  4. It is possible to move widgets between devices: we have implemented two mechanisms in this area. The first is a mechanism by which a device serves to other devices the widget which will allow the other devices to drive the serving device. The second is a mechanism by which the user can choose to "move" a widget which already runs on device A to another device B.

  5. It is possible to split widgets into communicating pieces: we have implemented examples of such widgets.

  6. It is possible to have asynchronous connection and disconnection: we have implemented widgets which get connected (resp. disconnected) as soon as the service they are using appears (resp. disappears) on the network.

  7. The system works on very small devices, without a scripting engine: useful scriptless widgets can be designed.

In [7], Sire et al describe an inter-widget communication system. The system is HTML-based and uses scripting APIs for communication. One significant difference with our system is their choice to implement the "widget UA" with a combination of a normal web browser and some PHP code server-side. They define drag-and-drop, which we have not. However, they are confined to communication with widgets in the same browser, whereas our communication is with other entities, widgets and services including those running on other devices. There is also a security issue: a hostile widget has access to widget container information, and from this, to other widgets information.

In this paper, we have presented our work on the usage of SVG for communicating widgets. This work is based on the use of SVG widgets, of an SVG-based Widget User Agent with ECMAScript extensions and on a declarative mechanism for communications between scripted and/or script-less widgets. Our work is to be used in both an interactive TV context and a home-network context. We believe that the combination of W3C widgets, SVG and our communication extension to be uniquely well suited for a global solution integrating widgets from Internet, from the mobile environment, from interactive TV and home network, allowing crossovers.

This work has been partially financed by the European Network of Excellence INTERMEDIA (IST-FP6-38419).