Developing Mobile AJAX style solutions using SVG

Development pattern and tools explained


Abstract


With the addition of the uDOM in SVG Tiny 1.2, scripting became a possibility in SVG Tiny content. This allows competitive applications to be created in SVG. Especially the inclusion of XMLHttpRequests in the uDOM allows AJAX style of applications to be created. To obtain efficient content creation the new features in SVG Tiny 1.2 requires changes to the tool chains used for creating SVG Tiny content. Not only must the graphics be created but logic in the form of scripting or programming must be possible to develop with support from tools. Although scripting logic or programming always can be done in a basic text editor, development speed can be greatly increased by utilizing advanced environments combining editor, debugger and emulator into one tool known as an Integrated Development Environment, IDE. The increased complexity in the applications increases the savings made using a tool as compared to traditional development and has been a driving force for the current development of IDEs.

The present paper presents tool chains available for developing applications with SVG graphics for two categories of mobile devices: Those supporting the Java interface, JSR 226 to SVG Tiny and those supporting the SVG Tiny natively either in a dedicated SVG player or as part of a browser. The methods used to develop the applications make use of tool chains including an animation tool for adding declarative animations to the content and IDEs for adding advanced logic executing based on events received by the display system. The combined tool, with integrated emulator, makes it possible to do testing with high fidelity early in the development, thus assuring early on that the result corresponds to the actual requirements of the application. In general the amount of errors in the first release can be greatly reduced.

The paper shows that the methodology for developing Java applications and JavaScript applications is very similar. It also demonstrates that the entire development process has tools support both for Java and JavaScript.

For SVG applications using JavaScript the present paper presents a novel IDE focusing on the creation and debugging of SVG Tiny 1.2 applications targeting the mobile domain. The tool has refined the important script development and device emulation parts of the tool chain for SVG Tiny 1.2 content. The tool is based on the eclipse IDE platform and contains, apart from the regular editor, also static syntax checking of scripts. The integrated SVG Tiny 1.2 viewer adds dynamic debugging capabilities to the tool chain and offers testing of user interaction and a preview of the graphics.

Application development for Java and the JSR226 is also discussed. The application creation process is very similar to creating JavaScript applications. Also for this development there exists an IDE with debugging and previewing capabilities. The paper shows that SVG can be easily integrated to Java applications.


Table of Contents

Introduction
Workflow overview
Graphic Design
Declarative animations
Programming
General Workflow
Tools
Create a change – Modifying the DOM
Event handling
Getting/Sending data over the network
Debugging
Deploying
Conclusion

The vision from the mobile phone industry is that the mobile device will mature from phones to generic pocket devices on which new programs and features can be installed and executed. There are today a number of platforms available for deployment of programs on the mobile device. The most used are Java, HTML and native applications on smart phones. As has been shown by recently attractive graphics and efficient HMI is valued by the users and makes the user use more functions. The traditional applications in Java and HTML are somewhat rudimentary in creating graphics. Native applications can create very nice graphics but at the cost of a high development time. SVG provides a means to reduce the development burden and achieve high quality 2D graphics. However to become an effective application and graphics development platform it is important that the tool chains for developing applications are complete, consistent and efficient. While it has always been possible to create static graphics in SVG using tools such as Inkscape and Adobe Illustrator the transition of the static image into an application or animated graphics has been the task of hand coding. With the introduction of the IKIVO Animator graphics designers are able to add animations to the static images without having to hand code XML. Creating animated SVG content can thus be done without having to explicitly understand the XML syntax and semantics of SVG.

As SVG is also winning ground as a basis for applications on mobile devices there is a need to improve the tool chain with tools that can add programmatic parts of the applications. On mobile devices SVG applications belong to one of two paradigms SVG + ECMAScript or Java + SVG. In the former the programmatic behavior of the application is provided by ECMAScript following much along the lines of regular web browser technologies. Creation and testing of this type of application can be greatly enhanced by the use of an IDE as presented in this paper.

In the latter case the programmatic behavior is provided by Java and SVG is used for drawing the graphics. Also for this case there exists an IDE: Netbeans from SUN.

In this paper, the various development tools available will be described. The development process will be exemplified by an interactive SVG application, more specifically a Twitter client, created using some of these tools.

The typical workflow when creating applications with a high degree of graphics contains the pieces:

  • Design/Animation

  • Programming

  • Testing/Debugging

  • Deployment

The relation between the pieces is illustrated in the figure below.


By doing a careful design of the application the three steps in the development can to a large extent be done in parallel. The following sections discuss the different steps in detail.

The first step is graphics design, which is the process of creating the static graphics that is going to be used in the application. The number of graphic images that need to be created depend on the type of application and typically consists of icons, backgounds and figures. The static graphic can be created in any regular tool that can output SVG. There are many tools for creation of SVG graphics; both commercial tools like Adobe Illustrator and Corel Draw, and free alternatives like Inkscape. Very simple, or in situations where full control over all aspects of the application is needed, a simple text editor or using the text editor of an IDE could be a viable alternative.

One of the great strengths of developing applications with an SVG GUI is that the design and logic of the app can be decoupled easily; allowing the designers to create and change graphics independently of the programmers to a large extent.

The figure below shows the graphics elements that are used in the twitter application demonstrated in this paper.


SVG lets you create animations straightforwardly using XML syntax directly in the SVG document. The animation elements are added as child nodes to the targeted elements. The advantage of using declarative animations compared to programmatic changes is that the intrinsic object structure of SVG is used in a more straight forward manner. The animations are declared in time and are defined independently of the frame rate in which the graphics is rendered. This greatly simplifies the process of applying simple animations compared to using ECMAScript or Java to dynamically change the DOM as time progresses.

Like any other SVG element declarative animations can be coded directly in an SVG document using a text editor. However, the process can be streamlined using a tool like Ikivo Animator. For more complex animation the use of a tool is almost required. It lets you create complex animations in a very simple fashion using timelines and drag-n-dropping of SVG elements in a canvas. It also lets you sync together several animations, and prepare compositions for use in more complex applications by creating multiple timelines that can be individually triggered from ECMAScript or JSR-226 DOM-bindings. This is particularly useful in creation of GUI where animations effects are desired on entry and exit of item and to signal that different events have happened.

Adding animations to a static SVG file starts with opening the graphics file in the Animator or import it into a new document. The tool gives a very high level view of the animations and it is now possible to create a number of timelines that can be used to animate different objects or groups of objects. The new timelines are animations that will be triggered from software, i.e. not started automatically, typically in response to events in reaching the application. Creating the animations in the tool is simply a matter of drag and drop. By moving the time marker in the timeline, adding key points and manipulating objects in the stage the animations are created.

The file can then be saved as an SVG Tiny 1.1 or 1.2 file, prepared to be used in mobile applications.

In the sample application three custom timelines have been defined see figure below. The timelines define animations that shall take place when the events selection of a message, deselection of a message and reception of a new message are received respectively.


Even though animations can be added declarative in SVG creating an application also requires programming to be done. Typical feature that are handled in the programming part of the application is communication with servers and handling of user events from the keyboard or similar input devices. The interaction between the program and the declarative part goes via the document object model (DOM). Interfaces or bindings to the DOM exists for different script and programming languages most noticeable are the bindings to ECMAScript and to JAVA via the JSR 226 and JSR 287.

Programming can always be done in a text editor, but efficient development requires the use of an IDE. The better the IDE is the faster is the development of the application. Two IDEs are discussed, Ikivo IDE for development of SVG/ECMAScript applications and Netbeans for development of SVG/JAVA applications.

The general workflow is very similar in the two environments Ikivo IDE and Netbeans IDE's mobile package. Prior to starting the work in the IDE the graphic design and animations must be created. If the work in the IDE’s shall be performed in parallel to the graphics design and creation of animations all identifiers must be defined in advance to allow the program development to start.

The programmer develops the logics and connects it to the SVG model, while testing and debugging in an integrated device emulator. During this process, the programmer may very well realize short-comings in the design, or new features that need to be added. The designer can then, if care is taken not to affect the current DOM, change the graphics and animations in the SVG without affecting the run-ability of the application. This way, developers and designers can work somewhat independently of each other, and reworking of the entire interface of an application can sometimes be done by the designer alone, effectively making many SVG applications easily "skinnable" by nature.

Navigating and modifying the Document Object Model ("DOM") is central in the creation of any interactive SVG application. The changes to the graphics are what the user will experience as execution of the application. The changes to the graphics can be achieved by the two processes as described above: using declarative animations and by programmatically changing the DOM or in the combination of the two programmatically starting and stopping animations.

Depending on the selected programming language different syntax is used to change the DOM. The following examples use the ECMAScript programming language.

  • To get an element with a known id-attribute, simply use the document.getElementById() method.

  • Basic navigation of the DOM can be done by the firstElementChild and nextElementSibling attributes of each node. Using these, any element in the document can easily be looked up. Note that in JSR226, the equivalents are method calls rather than attributes.

  • To access properties, or traits, of an element, there are many different methods depending on the type of trait. E.g. getFloatTrait() can be used to get the floating point value of a trait and setMatrixTrait() to set the transformation matrix of an element. There are also getAttributeNS() and setAttributeNS() for getting and setting of arbitrary raw attributes.

  • To create and insert a new SVG rect element at the end of the root SVG element:

    var newElement = document.createElementNS("http://www.w3.org/2000/svg", "rect");
    document.documentElement.appendChild(newElement);

    The process is similar for every other kind of SVG element.

  • Starting animations from code can be done by locating the desired animation element and call its beginElement method. For example, to start a specific timeline created in Ikivo Animator, one would simply call document.getElementById(timeline_name).beginElement().

One advantage that ECMAScript has over Java is the ability to extend already existing objects in the DOM. One could even extend an entire class of objects by adding properties to the prototype of existing classes/interfaces like SVGElement. This technique is commonly used in Javascript libraries on the web, and can just as well be used in ECMAScript in SVG Tiny 1.2 documents to simplify development and write prettier code. A simple example of this would be adding a "hide" method to all SVG elements like this:

SVGElement.prototype.hide = function(){
	this.setAttributeNS(null, 'display', 'none');
};	

Having declared the above method, the following line would hide the SVG element with the id "myElement":

document.getElementById("myElement").hide();

In the sample application a new messages should appear in stacked message boxes, all with a similar in looks except for the content. To achieve this and allow us to have an arbitrary number of messages in the document, while still being able to edit the design using a graphics tool, a template box is created in the design. The template box is hidden by default. When a new message is received, the template node is cloned by a call to the DOM method cloneNode(true), where the positive boolean paramater specifies that all children and descendants of the node should be copied to the clone as well. The clone is then given message-box-specific convenience methods, like animation triggers, added to it using the extension method mentioned above. It is filled with the message data received from the Twitter servers (as explained in the next section) and is inserted into the document, moved to the correct initial location, and finally made visible.

In ECMAScript, network communication is handled by the getURL and postURL methods. As can be deduced from the method names, they use the HTTP GET and POST methods respectively. Both methods are asynchronous (as in "asynchronous JavaScript and XML", a.k.a. Ajax), taking a callback function as a parameter, so calling them will not block user interactivity. In our Twitter application, getURL is used with a timer to repeatedly poll Twitter's feed of events and update the contents of the SVG document. The received data is in the json format, which conveniently lets us convert the the returned string to an ECMAScript data structure using the eval() function.

 
var twittURL = "http://twitter.com/statuses/public_timeline.json";
var updater = {
	handleEvent: function(evt){
		getURL(twittURL, function(cbData){
			if(cbData.success) {
				var data = eval(cbData.content);
				updateTweets(data);
			}
		});
	}
};

var timer = createTimer(0, UPDATE_INTERVAL*1000);
timer.addEventListener('SVGTimer', updater, false);
timer.start(); //start polling of Twitter server
 				

Using the json format is not the only way to access data smoothly. Sometimes XML is a better alternative (e.g. if an application loads SVG for immediate insertion into the document), and then the parseXML() method will allow you to parse an XML string into a DOM-tree, navigable by the standard DOM attributes and insertable into the main document.

Networking in J2ME is done using a considerably lower-level API. Asynchrony can be achieved by means of threading. The equivalent of the above ECMAScript could look something like this in J2ME:

final String twittURL = "http://twitter.com/statuses/public_timeline.xml";
Thread t = new Thread(){
	public void run(){
		while(true){
			HttpConnection conn = (HttpConnection)Connector.open(twittURL);
			InputStream is = conn.openInputStream();
			StringBuffer str = new StringBuffer();
			int ch;
			while((ch=is.read())!=-1)
				str.append((char)ch);
			is.close();
			conn.close();
			Thread.sleep((int)1000*UPDATE_INTERVAL);
		}
	}
};
t.start();
					

Both the Ikivo and Netbeans IDEs has built-in debuggers that lets you set break-points in code and go through the execution of an application step by step while viewing the state of variables and the visual state of the application in a device emulator. This can be very helpful when eliminating bugs during development and greatly reduces development speed compared to sorting out bugs directly on the device.


One hassle when deploying applications using SVG and ECMAScript is that you often want to separate scripts from the SVG file itself while in development. This leads to problems when distributing the application, as multiple files would have to be downloaded individually by the device/user. One solution would be to merge all scripting into the main SVG file, but this would have to be done manually every time an update was to be released and during testing would be very time consuming.

As a solution, Ikivo IDE gives you the option to deliver multiple file (SVG and external ECMAScript-files) packaged in a 3GP container that can be opened on compatible devices and downloaded/distributed as a single file.

As for deploying JSR-226 applications, it is no different than deploying other midlets. The classes and resources of the application can be neatly packaged in a jar file.

For creating mobile applications with SVG based interfaces there exists two major paradigms: Using SVG/ECMAScript and using Java/SVG. Both of these paradigms have support from tool chains and the applications can be created, tested and deployed using high level tools. The graphics generation and creation of animations can be done without having to go into detail in the XML building up the graphics and the animations. The behavioral part that is controlled programmatically needs to be coded by hand by skilled programmers.