MOBIXPL

A SVG-BASED MOBILE USER INTERFACE FOR SEMANTIC SERVICE DISCOVERY

Keywords: Semantic Web Services , Mobile Devices , Selection Interface , SVG Tiny

Olaf Noppens
Institute of Artificial Intelligence
Ulm University
Ulm
D-89069
Germany
olaf.noppens@uni-ulm.de

Thorsten Liebig
Institute of Artificial Intelligence
Ulm University
Ulm
D-89069
Germany
thorsten.liebig@uni-ulm.de

Patrick Schmidt
Institute of Artificial Intelligence
Ulm University
Ulm
D-89069
Germany
patrick.schmidt@aristaflow.de

Marko Luther
DoCoMo Communications Laboratories Europe GmbH
Future Networking Lab
Munich
D-89069
Germany
luther@docomolab-euro.com

Matthias Wagner
DoCoMo Communications Laboratories Europe GmbH
Future Networking Lab
Munich
D-89069
Germany
wagner@docomolab-euro.com


Abstract


The vision of a mobile Web in which everyday as well as business-related services are accessible from anywhere and anytime will become reality very soon. Mobile operators then have to meet the challenge not to overwhelm users with potentially interesting services but to allow them to easily discover individual services in a flexible and transparent way. This paper describes a novel user interface using a graphical preference specification paradigm which allows preference- based service discovery on cell phone devices. Our implementation adopts recent trends with respect to visualization techniques, like Scalable Vector Graphics for mobile devices (SVG Tiny), as well as semantic service discovery strategies.


Table of Contents


1. Introduction
2. Semantic Service Discovery
     2.1 Semantic Services
     2.2 Discovery Process
     2.3 Discovery Toolbox
3. SVG-based Service Discovery Interface
     3.1 Service Ontology Browsing
     3.2 Preference Expressions Aggregation
     3.3 Implementation and Experiences
4. Summary
Bibliography

1. Introduction

With the incessant improvements of wireless bandwidth, mobile client device capabilities and ubiquitous network access, the number of mobile service offers will increase rapidly. Mobile services provide a broad range of different types of content ranging from music downloads over radio broadcasting, to location- and context-based services which allow the user to effect some actions or changes in the real world. Experience has shown that users tend to maintain control over the service offering in order to select services according to their personal needs in a specific situation or context.

In this paper, we present our work on MobiXpl – a mobile user interface for personalized semantic service discovery. MobiXpl allows users to express their preferences over service characteristics with respect to their personal context. These preferences allow the service discovery process to take only services into account that best match the individual needs. To maximize user-friendliness we propose a zoom-able graph-based preference visualization of service characteristics as well as preference expressions that allows the intuitive creation of personal likes and dislikes. Because of resource limitations of current mobile devices we assume that the discovery process is computed by a component in the mobile network. In our case, MobiXpl is connected via a wireless network connection to the back-end discovery server MobiOnt, which implements the preference-based service matchmaking and discovery process.

The next section shortly discusses the concept of preference- and ontology-based service discovery and introduces a music selection example. Section 3 introduces our approach to preference modeling and visualization for mobile devices that tries to minimize the number of required user interaction. We will end with a summary and an outlook about future extensions.

2. Semantic Service Discovery

Before services can be used, they first have to be found. Service discovery is a mechanism for finding services. Current mechanisms work mostly syntactically by matching service requests and service offerings based on keywords. This often leads to poor answer sets as the ambiguity of keywords may result in mismatches between the user query and service descriptions. In this section we sketch a discovery process and its implementation that leads to more relevant matches by introducing semantic descriptions of both the service offerings and the user requests based on a rich preference formalism.

2.1 Semantic Services

Traditional service discovery mechanisms have limited quality results because these mechanisms lack a common understanding on services descriptions and queries. Achieving good matchmaking results based on a purely syntactic level is hard, if not even impossible. An ontology can provide a semantic specification of a shared conceptualization on a specific domain. By using ontologies service descriptions and queries can be unambiguously defined leading to more relevant discovery results. In the rest of this section we describe our ontology-based semantic service discovery approach by means of a Web radio scenario.

Web radio is one of the highly emerging trends for distributing audio content over the Internet in a legal fashion (e.g., SHOUTcast and radio-locator). We consider a fictitious mobile Web radio scenario which will serve as our running example. This scenario is only one of many possible application for our service discovery toolbox. Just by adding location information the amount of potential application domains is growing, e.g. leisure time activity planer, restaurant finder, shopping guide etc. Web radio stations offer audio content as streamed data in different formats (such as MP3 or Real) and different compression rates to make them accessible to a wide range of mobile media devices with different capabilities. They typically differ from traditional radio stations in providing more specialized audio content. An increasing number of audio streams are further differentiated though content characteristics such as different program types (e.g., streams on SHOUTcast are categorized according to the rubrics music, educational program, news information, etc.).

In our scenario, we assume that Web radio stations can be modeled as Semantic Web Services that are described by their program type, location and audio format on the basis of a service ontology using the Web Ontology Language (OWL). Figure 1 shows an extract of our radio scenario ontology describing program information. Here, jazz programs are, among others, further differentiated into fusion and smooth jazz. Concrete radio streams are classified by type associations to their most specific content description. For example, as a Smooth Jazz stream SkyFM is an instance of the corresponding program characteristic called “Smooth” in Figure 1.

categories.png

Figure 1: Extract of a radio program ontology with sample streams

2.2 Discovery Process

With the increasing number of available services it becomes a more and more demanding problem for the individual user to efficiently find the adequate service in a specific situation. On the other hand, mobile operators are boosting the number of available services. To become a commercial success, mechanisms are needed to efficiently support the user in finding the services he is interested in. In our opinion, making an informed choice of the right services will essential include matching the individual user preferences and dislikes against the services offered in a given situation or context [Balke and & Wagner, 2003a][Balke and & Wagner, 2003b].

User preferences are expressed in terms of preference orderings between service aspects. As proposed for example in preference algebra for data retrieval [Kießling 2002], these partially ordered aspect sets are directly handled without the use of any explicit ranking of service instances or quality values. User preferences are either introduced as selection of a single aspect A or as a special relation with the semantics of considering some aspect A superior to another aspect B. Intuitively, the former expresses a favoritism such as "I like A" whereas an ordering states that a user "prefers A over B". With the help of negation of aspects, one can also model statements for dislikes ("I do not like A"). Complex user preference can be constructed by integrating an existing preference relation into a new one. Kießling et al. [Kießling 2002] suggest Pareto Accumulation to handle equally important user preference according to the Pareto-optimality principle which has been studied intensively for decades in the context of multi-attribute decision problems in the social and economic sciences.

The aim of our preference-based service discovery is to retrieve those services that represent the best matches to a set of given user preferences. The aggregated user preferences define a service request that needs to be mapped to the actual service offerings. The idea of preferences is twofold: on the one hand, preferences are used to prune the set of all available services with respect to the user's wishes. On the other hand, because it is not always possible to fulfill the most specific preference expression, the expression can be gradually relaxed to next-best matches. For this, the matching algorithm starts with the most specific expression which best fits the user's wishes and in the case that no matching can be found, the aggregated preference relation is consulted to determine a relaxation path. In addition to a preference-determined relaxation path the ontology itself can be used for query relaxation. Following the preference principle a service request can be relaxed to more general requests based on the hierarchy of the ontology. Here it is assumed that a near match might still lie within the broader context of the original focus [Balke and & Wagner, 2003a].

2.3 Discovery Toolbox

To evaluate and demonstrate the preference-based service discovery on real mobile devices we have implemented a toolbox which consists of a service provider and discovery server called MobiOnt [Wagner et. al. 2004] that is used by one or more mobile requesters running the MobiXpl client application. MobiOnt serves as a semantic service portal where user requests are evaluated during service discovery utilizing several kinds of relaxation strategies to best fulfill the request. For the task of browsing service ontologies and defining preference in a graphical and intuitive way we implemented MobiXpl as the graphical front-end for the mobile user. As depicted on the left-hand side of Figure 2 MobiOnt provides a standard Web service infrastructure (e.g., accessible via GPRS or UTMS) for interacting and communicating with MobiOnt. For demo purposes we have also implemented a Bluetooth link-up between MobiOnt and MobiXpl.

architecture.png

Figure 2: System architecture

3. SVG-based Service Discovery Interface

MobiXpl is the client application within our service discovery framework. According to the preference driven discovery mechanism this front-end has to support the following main user tasks:

  1. Configuration of available settings (strategy parameter, user presets, etc.)
  2. Browsing through the categories of the service ontology
  3. Selection of one or more characteristics within different categories
  4. Incremental creation of atomic as well as complex preferences over selected characteristics
  5. Start of service discovery process
  6. Selection of a service from the discovery result set

Within a typical work-flow a user sequentially executes these tasks in the given order. However, in order to provide as much flexibility as possible the user can jump from a task to any other task at any time. Communication between MobiOnt and MobiXpl is only required before task 1 and in between task 5 and 6. More precisely, at client start-up MobiOnt has to initially transmit the domain specific service ontology as well as all available domain or user options. Then, when activating the discovery process after preference building MobiXpl has to submit the user specified preferences back to MobiOnt. In return the best matching results are re-transmitted from MobiOnt.

The tasks 1, 5, and 6 are standard in the sense that they do not demand for a special user interface. Instead, users very likely expect to configure and start the server-side service discovery as any other application. Category browsing, aspect selection and preference building (tasks 2-4) require a more elaborated interface. In contrast to standard user interfaces our aim was to avoid expansion lists in order to prevent time consuming scrolling. We therefore developed a graph-based graphical user interface with two-dimensional navigation as well as zooming. In addition, for interoperability reasons, MobiXpl is implemented in a device-independent manner: for handling user interaction it uses the JAVA J2ME platform and for the graphical interface it uses scalable 2D vector graphics. More precisely, we utilize SVG-Tiny as an optional package within the J2ME framework as part of a Java Specification Request (JSR). Almost all mobile phone manufacturers have announced to support this specification. In contrast to an alternative Flash approach, SVG is a platform-independent XML-based format with the advantage of being compact in file size and scalable without quality loss. In addition, within Flash it is difficult to implement a complex application logic due to its limited script-style programming API. MobiXpl has been developed using the official Nokia device emulator for the Nokia S60 (3rd Ed. platform) and S40 (3rd Ed.). The Midlet has been successfully tested on real cell phones from different vendors such as the Nokia 6280, N95 or Sony-Ericsson P990.

The next sections describe the main SVG-based user tasks, namely category browsing, aspect selection, and preference building.

3.1 Service Ontology Browsing

The ontologies of each category are graphical represented in a graph-based structure reflecting the aspect hierarchy. The user can browse the ontology by gradually expanding and de-expanding the aspects. For instance, Figure 3 shows the partially expanded ontology for the program category. Selection and de-selection of an aspect (cf. the highlighted aspect Music within Figure 3) can be established by pressing the middle context-menu "OK" key on the keypad. Users can navigate through the ontology with help of the direction keys, which change the focused aspect similar to the navigation in mobile games. Within Figure 4 the aspect Music is further expanded by one level in comparison to the Figure 3. With respect to the limited display space of a mobile device, focused aspects are automatically re-centered and the whole visualization can be continuously zoomed and paned. Zooming allows the user for fast navigation and for getting an overall picture. In our opinion, such an on-demand expansion paradigm is very suited for mobile devices with a limited screen display because only user-relevant aspects are visible. In addition, this becomes important if the ontology consists of a lot of aspects in different branches. In contrast to contract-list-style visualization of hierarchical structures, a graph-based visualization does not only better uses the available space w.r.t. the x- and y-axis but also is beneficial in understanding the structure when dealing with multiple inheritance. Additional HCI techniques such as thumbnails for not expanded parts of an ontology guides the user (e.g., triangle-style thumbnails in Figure 3 below the aspect Music). This visualization and navigation principles are based on our previous work on visualizing and manipulating ontologies [Liebig & Noppens 2005].

cbrowsing1.jpg

Figure 3: Music aspect within program category

cbrowsing2.jpg

Figure 4: Sub-aspects of Music after gradual explansion

The task of selecting aspects may be cumbersome because of the diversity of categories and huge number of aspects. We therefore also support usage patterns in form of user profiles for a personalized visualization of a service ontology. These patterns allow an adequate pre-selection of aspects with respect to the user's level of expertise and navigation habits. As a result of the user's experience level, MobiXpl also presents a user-specific visualization to the underlying service ontology by creating different ontology views pointing to different aspects in the initial view. This allows a fast and easy way to select aspects of a given domain without presupposing an overall understanding of the whole ontology. In our running example, let us assume teenage users to be typically interested in music. As a result, the ontology view of the program category prunes the ontology by starting with the aspect Music by default. In addition, we extended this behavior by an optional "aspect of interest". The ontology will initially be expanded up to that aspect. The user still has the option to browse up the hierarchy and to explore an alternative branch of course. These kinds of ontology pruning and pre-expansion seem to be a good solution for devices with very limited display capabilities.

3.2 Preference Expressions Aggregation

For each category, the user can separately switch into the preference aggregation mode. Very likely, he will first select a set of aspects and will then aggregate these to preference expressions. However, switching between both modes is possible at any time by pressing the "star" key on the keypad. By default, all previous selected aspects are unbiased (i.e., considered as being equally important). MobiXpl displays them without any relationship between them as shown within Figure 5. Preferences are aggregated to more complex preference expressions by establishing a new preference relationship between two aspects. If the focus lays on the aspect which should be the origin of an aggregated preference, the user can press the middle context-menu "OK" key, and in a second step, he changes the focus (via the direction keys) to the aspect which should be specified as the target of the preference relationship. In our running example depicted through Figure 5 to Figure 7 the user prefers Jazz music over HipHop but he is indifferent between Dance and Jazz music programs. Before MobiXpl will establish a new preference relationship, the user interaction is evaluated in order to avoid redundant or inconsistent preferences. A redundant preference expression is a preference relationship between A and B ("I prefer A over B") where A is a (direct or indirect) generalization of B. Note that it is the nature of an ontology that all individuals which are instances of a sub-aspect are also instances of all (direct an indirect) super-aspects. In this case, the redundant preference can not be used when preferences need to be relaxed because if there are no services available for a super-aspect, there are also no services available for all sub-aspects. Preferences are inconsistent if one specifies a dislike of an aspect A but also likes a (direct or indirect) sub-aspect of A or A itself.

preference-building-1.jpg

Figure 5: Step 1 of preference building: selection of preference origin (Jazz)

preference-building-2.jpg

Figure 6: Step 2 of preference building: selection of preference target (HipHop)

final-preference.jpg

Figure 7: Established preference between Jazz and HipHop

3.3 Implementation and Experiences

Rendering SVG-Tiny on mobile devices has become a standard feature because of the variety of mature SVG players such as TinyLine, BitFlash, etc. In contrast, using the full interactive functionality of SVG-Tiny via Java (JSR-226) is rather novel. Nokia has recently added this package to its Java environment on its S40 3rd Edition phones . Unfortunately it turned out that the Nokia emulator as well as the device implementation has several problems in supporting the full functionality of the Java SVG-Tiny standard. For instance, some devices do not calculate the text bounding box correctly and setting attributes after inserting a SVG element in the DOM tree results in an emulator crash or re-start of the mobile device´s operating system. In addition, official emulators often do not reflect the correct mobile device behavior when dynamically changing the SVG-DOM structure. As depicted in Figure 2 our system architecture consists of a back-end component running the discovery server, and the mobile client application MobiXpl. In the following we only discuss SVG-relevant issues of MobiXpl. After transmitting the XML presentation of a service ontology to MobiXpl a separated graph structure is created for each category. This intermediate graph structure stores data wrt. To a node´s state, such as the expansion status and selection state, as well as layout issues, e.g. coordinates, color etc. For rendering each expanded graph node is mapped to set of SVG elements. If the state of a graph node is changed (e.g. after focusing, selecting, or expanding by the user) the layout is recalculated (if needed) and the graph nodes trigger their corresponding SVG element to reflect the updated node´s state. Removing SVG elements after de-expanding a characteristic turned out to be a computation-expensive operation and therefore the elements of a de-expanded node are not removed but marked as invisible within its SVG DOM structure. This avoids removing and adding them again in case of a re-expansion at a later stage. For performance reasons we have implemented an incremental approach which adds an SVG element to the DOM not before the first time it is needed for the graphical representation, such as an initial appearance due to an expansion.

4. Summary

User-friendliness will be paramount to make mobile and ubiquitous computing a commercial success especially when dealing with potential large numbers of service offerings. In this paper we have presented our work on preference-based service discovery on small mobile phones with a limited screen display. We have presented a navigation and browsing paradigm for defining and aggregating preferences which is based on SVG.

Future work will be spent on extending our toolbox towards enhanced personalization techniques and advanced user modeling such as location-based and situation-aware usage patterns. In addition, usage patterns could also be learnt from observing user interaction to derive typical browsing and preference behavior - also based on a specific situation or location. Furthermore, we will enrich the graphical browsing in order to make it more suitable even for users that do not know the service ontology in advance and to guide the users through the service ontology when they browse the ontology for the first time. As the user is always the key factor for all kinds of interactions and visualization we also plan to evaluate our preference-based service discovery.

Bibliography

[Balke and & Wagner, 2003a]
Cooperative Discovery for User-centered Web Service Provisioning W.-T. Balke and M. Wagner, In Proceedings of the 1st International Conference on Web Services (ICWS'03), Las Vegas, 2003.
[Balke and & Wagner, 2003b]
Towards Personalized Selection of Web Services W.-T. Balke and M. Wagner, In Proceedings of the 12th International World Wide Web Conference (WWW2003), Budapest, Hungary, 2003.
[Kießling 2002]
Foundations of Preferences in Database Systems W. Kießling, In Proceedings of the 29th International Conference on Very Large Databases (VLDB’02), Hong Kong, China, 2002.
[Wagner et. al. 2004]
MobiXpl – Semantic-based Service Discovery on Tiny Mobile Devices M. Wagner and T. Liebig and O. Noppens and S. Balzer and W. Kellerer, In Proc. of Workshop on Semantic Web Technology for Mobile and Ubiquitous Applications (in conj. with ISWC’04), Hiroshima, Japan, 2004.
[Liebig & Noppens 2005]
OntoTrack: A Semantic Approach for Ontology Authoring T. Liebig and O. Noppens, In Journal of Web Semantics, Vol. 3, Issue 2-3, pages 116—131, 2005.

XHTML rendition made possible by SchemaSoft's Document Interpreter™ technology.