Marrying SVG and Java ME Using Advanced Tools

For the SVG Open 2007 conference

Keywords: Java , SVG , NetBeans , Mobility , Mobile Designer

Martin Brehovsky
Sun Microsystems
V Parku 2308/8
14800 Prague
Czech Republic
martin.brehovsky@sun.com

Biography

Martin Brehovsky is a software engineer in Sun's Software Systems Group, where he works on the NetBeans Mobility as a project lead for SVG tools.

Marc Verstaen
Oracle
5450 Great America Parkway
95054 Santa Clara
California
USA
marc.verstaen@oracle.com

Biography

Marc Verstaen has 20 years experience in designing authoring tools for different environments: print, Web, desktop User Interface and Mobile devices. Marc and his team released the first SVG authoring environment for Mobiles 5 years ago.


Abstract


The Java ME application environment is available on virtually every phone sold today, but until recently it was missing advanced 2D graphics capabilities. Developing Java ME applications has typically meant using either high-level LCDUI components, a set of abstract user interface components, which do not provide enough flexibility, or creating custom UIs using low-level graphics, which is flexible but very difficult. With the introduction of the Scalable 2D Vector Graphics API (JSR-226), everything changed - now Java ME applications can display and manipulate rich, animated and interactive 2D SVG graphics (specifically SVG Tiny 1.1) authored by a graphics designer in specialized tools. SVG brings a new style of application development to the Java ME platform, similar to the development process of web applications. A graphic artist can focus on creating a visual presentation for the application, while the application programmer can focus on creating the business logic and can stop spending hours tweaking low-level graphics UI.

Unfortunately, the features and capabilities of JSR-226 haven't been sufficiently reflected by the available tools. while there are very good tools for developing applications for the Java ME platform and for creating standalone SVG graphics, the tools are usually not integrated very well which makes developing SVG-enabled Java ME applications rather difficult. However, this is exactly the area where a collaboration between a software development tool maker and an authoring tool vendor can benefit to both the software developers and the graphic designers.

This paper discusses efforts for integrating NetBeans Mobility as a Java ME development tool and Oracle Mobile Designer as SVG design tools. We will cover various strategies for developing Java ME applications with rich, animated, interactive UIs and discusses how these strategies can be supported by the tools to provide a rich development environment for both the SVG designer and the Java developer where both parties can easily cooperate and work simultaneously on the same project. It also explains a concept of high-level components, utilized by both the authoring and the Java development tool. The high-level components are identified in the SVG file using specific meta-data and are automatically recognized by the Java ME development tool when building the application. Finally, we will discuss all stages of JSR-226-powered application development - from authoring rich SVG content, over utilizing Scalable 2D Vector Graphics API to integrating SVG content with component meta-data and business logic written by the application developer, to the deployment of the created application to real devices available on the market today. Multiple examples and demos are shown for each stage of the development cycle, including demonstration of SVG based Java ME applications running on real devices.


Table of Contents


1. Introduction
2. Enabling SVG in Java - introducing JSR-226
     2.1 JSR-226 Flexibility
          2.1.1 Use of SVG rasterized to bitmaps during development time
          2.1.2 Pre-render SVG to bitmap on device and use bitmaps afterwards
          2.1.3 Use “static” SVG graphics rendered to Canvas
          2.1.4 Use fully dynamic SVG graphics together with events
3. Tools
     3.1 JSR-226 Support in Oracle Mobile Designer
     3.2 SVG Support in NetBeans Mobility
4. Communication between Designer and Developer
     4.1 Naming Conventions
     4.2 Metadata
     4.3 Tools role
5. Future Plans
6. Summary
Bibliography

1. Introduction

In 1999 Sun introduced an environment for mobile phones called Java 2, Micro Edition. It quickly gained a lot of interest and it is estimated Java has been shipped on over 2 billions mobile phones since it's introduction, making it the most popular application platform for mobile devices on the planet. In 2005, Java 2, Micro Edition (J2ME) was renamed to Java Platform, Micro Edition (Java ME).

One of the most important features of Java ME is the fact it is not a single piece of software, nor a single specification, but rather a collection of technologies and specifications that are designed for different parts of the small device market. Because Java ME spans such a variety of devices, it wouldn't make sense to try to create a one-size-fits-all solution.

Because of that, the platform is divided into configurations, profiles,and optional ages. Configurations specify the core language and a base set of APIs that can be used with a certain class of device. Profiles add more specific APIs to make a complete environment for building applications, such as application life cycle or user interface. Optional packages then provide functionality that may not be associated with a specific configuration or profile, such as Bluetooth API or Wireless Messaging API.

As with other Java platforms, all Java ME platform specifications are created using the Java Community Process[JCP]. JCP is a participative process that is used to develop and revise Java specifications, reference implementations and test suites. All specifications are defined collaboratively by the industry and go through a well-defined review process.

The Java ME platform optional packages made it easy to add support for SVG.

2. Enabling SVG in Java - introducing JSR-226

In 2003 Nokia proposed a Java API for viewing and manipulating SVG Tiny 1.1 graphics for mobile devices known as JSR-226 � Scalable 2D Vector Graphics API for Java ME [JSR226]. This API was developed by the Java Community Process and in early 2005 it was approved and released. Since then, many new devices (28+ [JSR226Phones]) started to appear with built-in support for this JSR and many more are expected as this API is a mandatory part of the Mobile Service Architecture Initiative [MSA]. The MSA goal is to define a set of Java ME technologies addressing market needs for high-volume devices supporting Java platform. The MSA specification is lead by Nokia and Vodafone and is supported by many major operators, such as NTT DoCoMo, Orange and Sprint and every major device vendor, such as Motorola, Samsung or Sony-Ericsson, so it is expected most of the devices coming in the near future will be compliant with MSA and therefore will contain also JSR-226.

SVG and Java work together very nicely on devices - SVG is great for creating rich and flashy UIs and Java gives applications access to the underlying mobile platform (e.g. camera, address book, networking, sending messages, etc.). Together they provide a very strong platform for creating modern applications running on mobile devices.

2.1 JSR-226 Flexibility

Using JSR-226 and SVG for developing application does not mean it is “all or nothing”. The API allows several levels of usage for utilizing SVG graphics in Java ME application, making the use of SVG for Java ME development very flexible.

2.1.1 Use of SVG rasterized to bitmaps during development time

In this approach, an SVG image is rasterized to bitmaps during development time based on the resolution of the target device. The application can be developed even without using JSR-226, thus it is also suitable for older devices or platforms without SVG capability. The benefit of this approach is the ability to use SVG resources for application development. This allows sharing the same tools/format for the designers and preparing for devices coming with support for JSR-226. Using adequate tools such as the NetBeans Mobility, it is also possible to generate raster images at different resolutions, based on the information given by the emulators used for developing the Java ME application.

2.1.2 Pre-render SVG to bitmap on device and use bitmaps afterwards

On devices supporting the JSR-226 API but with slow processor and/or small memory it is possible to prerender SVG images into bitmaps during the first start of the application and then store them in the local storage (RMS). This approach benefits from using SVG as a graphical resource through the whole development process, and using fast bitmaps for rendering complex graphics where the performance of the device would not be sufficient. Another important benefit is that since this approach requires a lower processing power, it also saves on battery life, which can be a concern in mobile applications.

2.1.3 Use “static” SVG graphics rendered to Canvas

JSR-226 enables the application to use SVG graphics and display it by “one shot” rendering. This technique is useful when SVG graphics needs to be combined with “legacy” graphics. For example when combining SVG graphics and an interactive game graphics based on GameCanvas. Another example might be a combination of SVG graphics and a video stream rendered through Mobile Media APIs (JSR-135). This approach allows the use of animations and “scripted” animations (i.e., changes done by manipulating the SVG graphics from the Java API), but the application needs to redraw the graphics each time an update is done or when the animation time line is advanced.

2.1.4 Use fully dynamic SVG graphics together with events

The most typical (and recommended) approach to use SVG graphics is to “play” the SVG content using the SVGAnimator class.

This approach uses the application model shown in Figure 1.The Java application loads the SVG document into an SVGImage object, which is then passed to an SVGAnimator object responsible for rendering the image and for sending UI events to the application. The application can then manipulate the Document Object Model (DOM) of the SVG image, and all events or changes done to the DOM are automatically reflected by the SVGAnimator object and rendered onto the screen.

svgopen07-paper-070808-final_img_0.png

Figure 1: JSR-226 Application Model

This approach is by far the most powerful and flexible one as it delivers the SVG experience with all “bells and whistles”.

All four approaches can be combined together even within the same application.

3. Tools

With the increasing application complexity, development tools play an increasingly important role. The SVG and the Java ME worlds both have their own tools for delivering the best user experience for their targeted users.

SVG tools, such as Oracle Mobile Designer, typically target artists or content designers, who are capable of producing visually compelling content. On the other hand, Java ME tools, such as NetBeans with Mobility extension, focus on programmers.

Unfortunately most of the programmers are usually not capable of producing good looking UIs. On the other hand the same can be said also about designers when it comes to programming - designers are usually not capable of producing good and effective code. Therefore it is essential to allow both types of developers to work in their preferred tools and provide an environment which would allow a seamless cooperation between them.

To deliver the optimal experience for both types of developers, we need to have support for viewing/manipulating SVG images in the developer tool and support for delivering SVG-enabled Java components from the designer tool. Currently there are two tools which provide such functionality - Oracle Mobile Designer for designers and NetBeans Mobility for developers.

3.1 JSR-226 Support in Oracle Mobile Designer

Oracle Mobile Designer provides an environment dedicated to SVG content creation. The tools targets designers and as such allows creating interactive animated content without coding. A semantic context has been developed to encapsulate the complexity of SVG when it comes to animation and interactivity. Concepts such as “scene”, “book”, “menu bar” and “button” are build into the product and can be manipulated by designers. In order to increase re-usability of the graphical component and ease project management, Mobile Designer uses the concept of “sharing”. With “sharing”, a button can be defined once as a template and be used (as several instances) in several location in an SVG application without loosing the link to the original shared object. The label of a button can of course be customized, a button can be resized, etc...At any moment, the designer can still modify the original button and see its changes reflected to all the instances of the buttons, while still preserving the geometrical transformation and the current label. This also applies to other components such as “animations” used in Mobile Designer.

The interactive model of SVG is very powerful and more advanced than the one of its competitors. As an example, any graphic property can be animated and the animation can be triggered independently. To enable such a rich model in Mobile Designer, the concept of “segment” has been introduced. For any animation, a default segment is created and used as a reference. The designer can from this reference segment create new segments, with their own animation. These segments can be triggered individually. Triggers, whether they are initiated by a button or a key, can be setup in Mobile Designer, interactively.

svgopen07-paper-070808-final_img_1.png

Figure 2: Oracle Mobile Designer

Of course, all this richness in capabilities would be useless if not accessible from Java through JSR226. We all want to eliminate all the tedious task of coding the connections between events and animations, the hide/show objects, switch scene, connect buttons, launch the correct segment, etc...This is exactly what Mobile Designer does: all this code is generated automatically and placed in the NetBeans project. This frees time for the developer and can allow a designer to create a simple prototype application, with the interactivity and place.

svgopen07-paper-070808-final_img_2.png

Figure 3: Exporting SVG as JSR-226 component for NetBeans Mobility

3.2 SVG Support in NetBeans Mobility

NetBeans Mobility is an extension to NetBeans IDE enabling development of Java ME applications. Both NetBeans Mobility and NetBeans IDE are an open source projects sponsored by Sun Microsystems. NetBeans Mobility has an extensive support for developing JSR-226 applications. It's features include:

svgopen07-paper-070808-final_img_3.png

Figure 4: NetBeans Mobility SVG viewer/composer

svgopen07-paper-070808-final_img_4.png

Figure 5: NetBeans Mobility Visual Designer - flow design

4. Communication between Designer and Developer

During the development of Java ME/SVG applications one of the most important aspects of the development process is the communication between developer and designer. Figure 6 shows a typical workflow for developing applications using rich content, where the content is being developed by a designer and the business logic by a programmer. Since there is a lot of interaction between the designer and the programmer, it is important to make theirs communication as smooth as possible.

svgopen07-paper-070808-final_img_5.png

Figure 6: Typical Workflow for SVG/JSR-226 Application Development

Another important aspect of the collaboration is that designers typically think in terms of low-level graphics elements (e.g. polygons, shapes, curves, etc.) and SVG is perfectly suitable for this level of abstraction. On the other hand programmers tend to think rather in terms of high-level components (e.g. buttons, lists, menus, ...), because this helps them to work on the integration of the UI with the business logic. Furthermore, the designer should ideally be able to change a graphical representation of the high-level component in the SVG document without needing to change anything in the business logic of the application.

Fortunately Java is perfectly suitable for creating high-level component abstractions over low-level graphics. The remaining problem is: how can high-level components be efficiently identified within the low-level graphics presented by the SVG document? There are several strategies for doing that, but the most important ones are use of naming conventions and use of metadata.

4.1 Naming Conventions

To identify high-level components in the SVG document we can use naming conventions and the structure of element identifiers (IDs). These elements are then used by the application as high-level components and they receive events as the user interacts with the application.

In the following example an SVG graphics represent a single menu item for the SVGMenu component coming with NetBeans Mobility tool. SVGMenu is a Java component, which works as a high-level menu. When an SVG image is used for SVGMenu, the tool searches for all elements whose ID starts with the “menuItem_” string and creates necessary calls in the Java code to register those elements.

When the application runs, the SVGMenu then distributes the key events coming from the user to the graphics using DOM focus events. The graphics for each menu item can have an animation attached to the focusin/focusout events. When the user selects a menu item, the graphics is automatically updated using those animations. See code 1 for the menu element's SVG graphics and code 2 for the corresponding Java code declaring the menu element in the runtime. The result of the running application is on Figure 7.


<g id="menuItem_0" transform="translate(261.5,44)">
	<polygon fill="#003856" stroke="#003856" stroke-width="3" 
           stroke-linejoin="round" stroke-linecap="round" 
           points="-323.5,-103.5 -213.5,-103.5 -213.5,-87.5 -219.5,-81.5 -323.5,-81.5 -323.5,-103.5">
		<set attributeName="fill" begin="0; menuItem_0.focusout” to="#003856"/>
		<animate attributeName="fill" begin="menuItem_0.focusin” values="#003856;#fff;#cedfe9" keyTimes="0;0.4;1" dur="0.25s" fill="freeze"/>
	</polygon>
	<text x="-316.49" y="-87.78" font-size="12" font-family="Arial Black" letter-spacing="0.02em" fill="#fff">Music Player
		<set attributeName="fill" begin="0; menuItem_0.focusout" to="#fff"/>
		<animate attributeName="fill" begin="menuItem_0.focusin" from="#fff" to="#003856" dur="0.25s" fill="freeze"/>
	</text>
	<polygon fill="#f00" points="-211.81,-84.12 -216.56,-79.43 -211.81,-79.75 -211.81,-84.12"/>
</g>

                    

Example 1: SVG menu element


// create SVGMenu object
SVGMenu svgMenu1 = new SVGMenu(get_svgImage1(), getDisplay());
// register individual menu elements
svgMenu1.addMenuElement("menuItem_0");
svgMenu1.addMenuElement("menuItem_1");
svgMenu1.addMenuElement("menuItem_2");
svgMenu1.addMenuElement("menuItem_3");

                    

Example 2: SVG menu object declaration

svgopen07-paper-070808-final_img_6.png

Figure 7: Menu screenshot

Another naming convention example uses the ID structure for laying out graphical elements such as the image's background. In many modern devices it is possible to switch the orientation of the display from portrait to landscape mode (and the other way around). Even though one can use preserveAspectRatio attribute for specifying behavior when a viewBox is smaller than the screen where the image is being rendered, this feature does not cover cases when one would like to allow only selected elements to change the aspect ratio (for example background). Fortunately it is possible to provide the necessary logic from the Java side to achieve this functionality and we can use a naming convention to identify elements which should be stretched to fit the display's aspect ratio.

The result of such a functionality can be seen on Figure 8. On the left device, the SVG image, originally designed for 240x320 screens, is shown as is, without any modification. One can easily see the image does not look exactly right on 320x320 screen device. What we can do for Java developers, is to provide a Java component which is able to “fix” the aspect ratio of selected elements and all we need to do is to use a naming convention, so the tool recognizes background elements in the graphics and automatically creates the necessary calls in the Java code.

svgopen07-paper-070808-final_img_7.png

Figure 8: Using "background" element

4.2 Metadata

To help a Java application identify a high-level component we can also use metadata embedded into the SVG image. XML supports namespaces which would allow embedding metadata, but unfortunately, JSR-226 does not require the implementations to support elements in namespaces other that the SVG namespace.

The alternative is to use metadata in the SVG namespace in the form of graphical elements. This is abusing the SVG syntax, but does provide a much needed funtionality. We can use almost any SVG element as a carrier for metadata as long it is made invisible (display=”none”).

In the following example we used an invisible rect element to describe width and height, relative x and y position and number of displayed session items (attribute rx). When the SVG document is loaded, Java code parses this metadata element. When it obtains data from a web service, it updates the SVG image based on the information from this metadata element. Figure 9 shows how one skin is displaying five items per page whereas the other skin displays 6 items per page.

The second metadata element describes a text block, which can display a multi-line text block in our application, even though this feature is not directly supported in SVGT 1.1. The application Java code again parses the metadata element when the SVG document is loaded and when a details screen for the session is shown it uses left margin (attribute x), width of text (attribute width) and height of each line (attribute height) information to populate a text block group (below with id=”textBlock”), with text elements. The text block group also further specifies font family, font size and color. The result can be seen on Figure 10.


<!-- metadata for individual session items -->
<rect id="itemTextMetadata" x="15" y="0" width="220" height="20" rx="5" display="none"/>

<!-- text line width and height and position -->	  
<rect id="textBlockMetadata" x="10" y="0" width="220" height="16" display="none"/>
<!-- now the actual text block -->
<g id="textBlock" fill="#ffffff" transform="translate(0,153)" font-size="16" font-family="SunSansRegular"> 
	<!-- empty now, but will be modified based on the data coming from web -->
</g>

                    

Example 3: Metadata for skin #1


<!-- metadata for individual session items -->
<rect id="itemTextMetadata" x="14" y="0" width="220" height="17" rx="6" display="none" />

<rect id="textBlockMetadata" x="15" y="0" width="250" height="14" display="none"/>
<!-- now the actual text block -->
<g id="textBlock" fill="#35556B" transform="translate(0,153)" font-size="14" font-family="SunSansRegular"> 
	<!-- empty now, but will be modified based on the data coming from web -->
</g>

                    

Example 4: Metadata for skin #2

svgopen07-paper-070808-final_img_8.png

Figure 9: Session elements

svgopen07-paper-070808-final_img_9.png

Figure 10: Multi-line text block

4.3 Tools role

All the information described above can be used even without specialized tools, but with the tools the work of both the designer and developer can be made more productive.

The designer and developer tools allow both parties to agree on naming conventions and metadata structures in such a way that when a developer requires a certain set of components (identified by naming conventions), the designer tool will enforce the designer to supply those elements in the graphics being developed for the application. A similar thing could happen when a designer provides a graphics which uses naming conventions and metadata understood by the developer tool. In this case the developer tool can automatically create objects which correspond to the high-level component represented by the named elements, so the developer can work with the graphics as with the high-level UI components.

5. Future Plans

As time goes forward, there is a lot happening also in both the Java ME and SVG worlds. In the SVG world, a new SVG Tiny 1.2 standard is going to be released soon and it comes with major improvements such as a support for custom DOM events, which could be used to better control the graphics from the application code. In the Java world, a new version of Scalable 2D Vector Graphics API is being developed as JSR-287 and it brings features like support for SVGT1.2 and also full DOM API for parsing the SVG document.

Both initiatives will give both the developer and the designer even more possibilities to communicate. The designer will be able to insert metadata directly to the graphics (as elements in a foreign, non-SVG, namespace) and the developer will be able to read those directly from the application using the full DOM API. Also the developer will be able to send custom DOM events to the SVG document, and the graphics in the document can perform animations triggered by those custom events.

6. Summary

With the introduction of JSR-226, the Java ME platform gained access to SVG as a format capable of delivering rich and animated vector graphics. This API allows the Java ME application developers to easily utilize interactive content created by graphics artists and to deliver visually pleasing applications. The JSR-226 API gives the graphics designer access to the underlying phone platform.

Support for this JSR is being shipped on many devices available on the market today and with the introduction of Mobile Service Architecture (MSA JSR-248), it is expected that most of the devices available in the future will support SVG as a first class image format along with an API (JSR-226) to load, render, manipulate and play SVG content.

Together with advanced support in tools, collaboration between designers and developers can be made very effective as both the designer and developer can focus on their core skillset, knowing they can leverage each other's work and talent.

Bibliography

[JCP]
Java Community Process http://www.jcp.org
[JSR226Phones]
JSR 226 Phones http://svg.org/special/226_phones
[NetBeans]
NetBeans Open Source Project http://www.netbeans.org
[NetBeansMobility]
NetBeans Mobility http://www.netbeans.org/products/mobility/
[NetBeansSVGTutorial]
NetBeans SVG Tutorial http://www.netbeans.org/kb/55/svg-tutorial.html
[JSR226]
JSR-226 http://www.jcp.org/en/jsr/detail?id=226
[JSR287]
JSR-287 http://www.jcp.org/en/jsr/detail?id=287
[MSA]
MSA http://www.jcp.org/en/jsr/detail?id=248
[SVGT1.1]
SVG Tiny 1.1 http://www.w3.org/TR/SVGMobile11/
[SVGT1.2]
SVG Tiny 1.2 http://www.w3.org/TR/SVGMobile12/
[UILabs]
UI Labs http://meapplicationdevelopers.dev.java.net/phoneme_ui_labs.html

XHTML rendition made possible by SchemaSoft's Document Interpreter™ technology.