Mixing Dynamic SVG with User Input

Dynamic SVG in a desktop application

Thibaud Arguillère

A former emergency medical doctor, Thibaud (Thibaud [dot] Arguillere [arob] 4D [dot] com) spent many years programming with 4D and C++ for his own small software engineering company.
He then joined the 4D R&D team in 2008, and is now Senior Software Evangelist, traveling around the world to demonstrate solutions developed by 4D SAS.

Vincent de Lachaux

A former medical doctor too, Vincent used to programm with 4D since years. He joined the 4D SAS company since 2003. As soon as 4D included SVG features, Vincent used them and digged into SVG. He is now the internal specialist of SVG in the company.


Abstract


Numerous APIs exist which allow developers to build images based on dynamic data. Thanks to the features SVG offers, these images can be interactive and allow users to drill down into that data. There are also tools which empower users to create their own SVG images.By mixing these two different ideas, a new paradigm can be created wherein the user can actively update dynamic data by interacting with an SVG image.
This presentation will illustrate the needs, challenges and benefits of building upon dynamic SVG with user input, and demonstrate the concept using existing technologies.


Table of Contents

Introduction
Requirements and challenges
Some known tools
Raphaël
Inkscape
Other tools
SVG with 4D v12
The renderer
4D Language for SVG
User interaction
SVG Editor

Introduction

An SVG image is nothing but text, which an application (typically a browser) can render to screen as a picture. The image is described using the APIs, syntax and grammar of the SVG specification. The fact that an SVG file is, at the end, a text file makes it powerful and easy to manipulate. Do you want to change the stroke color of a rectangle? Easy! Just open the file in a text editor, even the most basic one, and then change the stroke attribute, save the file and it is done. Now try to do the same with a gif, a jpeg or a tiff: the basic text editor will fail to open the file, you will need at least a hexadecimal editor.

The same occurs in memory: a programming language can easily modify an SVG image by just modifying the XML. If the language does not have an XML API, the image can still be manipulated: instead modify text inside the SVG. More and more programming languages offer APIs, Frameworks, Plug-ins, etc. to let a programmer to handle SVG dynamically.

And, because SVG is easy to manipulate, it is also easy to mix with dynamic data. This is not only because the image itself can be created from data, but also because these images can be interactive and allow users to drill down into that data. All thanks to the features the SVG specification offers. At the end, dynamic SVG is built with dynamic data, and vice-versa: the user can actively update dynamic data by interacting with an SVG image.

The obvious benefit of dynamic data visualization is that images can truly represent the most up-to-date information with little to no user intervention. By adding interactivity, users can get richer information from an image – embedded links, for example – or add their own annotations or visuals. Finally, this presents an all new way for users to update data: by manipulating the image itself.


Requirements and challenges

Mixing dynamic SVG and dynamic data with user input requires three main elements: an easy API, a drawing tool, and a fast SVG rendering engine.

  • The API. For SVG to be adopted by more developers, its functions need to be easily accessed programmatically. SVG is a text based (XML) specification which is well documented, but developers new to SVG probably find it easier to call a command like “NewSVGLine(coordinates…)” instead of calling low-level APIs that insert a child in the XML tree, add attributes, etc. The API must let the developer easily and intuitively manipulate any SVG: create a new image or modify an existing image.

  • The Drawing Tool. We are talking about letting the end user manipulate the SVG image: obviously we can’t provide an XML editor for drawing lines and updating data! The user expects a drawing tool. While developers can use the API to create their own SVG drawing and editing tool, providing a plug-and-play solution will open the doors to SVG adoption more quickly.

  • The rendering engine and code execution. These must be fast, of course: as everything is dynamically generated, SVG creation and display must happen on the fly. For example, we could decide that when the user moves this or that shape, “something” is calculated, saved to the data and other shapes are modified automatically. This must be quick: detect the change, calculate dependencies, update data, render final image. And all this may occur while the user is still dragging a shape

A question that arises when talking about user-driven, data-manipulating imagery is: how do you limit data control? In other words, one needs to be sure that the user – while being given the power to modify data via an image – cannot destroy or alter sensitive data. On the other hand, data transfer speed needs to be high enough so that user changes are instantly validated on the server side for instantaneous updates. By mixing the three main requirements described earlier, a developer should be able to address those constraints: updates can be checked before being saved on disk in the data file (or sent to the server) and integrity and deletion control can be implemented (for example, the user will not be able to move or delete a specific rectangle).

Recognizing the potential for user-manipulated SVG, there are projects and applications to bring it to greater eminence. A quick search on the internet shows there are several of them, we will discuss two of them: Raphaël and Inkscape.

Raphaël (http://raphaeljs.com/), is a JavaScript library used to create and manipulate SVG across browsers. Creating static SVG is easy (circles, rectangles, path, shapes, …) but Raphaël has two main other interesting behavior: animation and interaction.
Animation: Raphael makes coding animation easy (Note: HTML 5 and css 3 make animation so easy too ;-) but they are not ready yet). Interaction is about allowing developers to attach JavaScript event handlers to DOM-based graphic objects. Most events are supported, which let the developer code whatever he or she wants. Specific information can be displayed when the user moves the mouse over a path, or an action can be triggered when the user clicks or double-clicks an object, etc. Thanks to this interaction, one can easily imagine, for example, updating the displayed SVG by sending an AJAX request to a webserver, which could return some information from a database.

Raphaël It is available under the MIT License (http://www.opensource.org/licenses/mit-license.php) which, to be simple, allows developers use/modify/do whatever you want with it provided the license notice is included in the product. Also notice that Raphaël is nominated for the “Best Open Source Application Of The Year” (2010) on .Net Awards (http://www.thenetawards.com).

Inkscape (http://www.inkscape.org/) is an open-source SVG editor available for Mac OS and Windows. It is a very powerful drawing tool, that includes about all – if not all – a designer expects.

What’s interesting for us – we are talking about user interaction via an SVG image - is that it allows:

  • Creating links out of SVG content, as recommended by the W3C, using the appropriate xlink tag. xlink elements are more powerful than HTML links (“but also more complicated” as Inkscape documentation states!). Of course, the interest here is that Inkscape embeds the settings you define in the Link Property dialog inside an xlink, which enables automatic action when the SVG is displayed in a browser (provided the browser supports SVG…).

  • Adding interaction with JavaScript. The Object Property box lets you to add JavaScript code for several events. Basically, the supported events are all the ones you may need to handle: onclick, onmouseover, etc. Again (as for Raphaël), this JavaScript feature opens the door to a lot of possible interactions: as soon as JavaScript is available, AJAX is around, and getting data from/sending data to a web server a web server is then possible.


Inkscape is available under the GNU General Public License Version 2, June 1991. The website states that “In short, this means you are free to use and distribute Inkscape for any purpose, commercial or non-commercial, without any restrictions. You are also free to modify the program as you wish, but with the only restriction that if you distribute the modified version, you must provide access to the source code of the distributed version”.

Note: “Drawing With Inkscape” is a session given at this current SVG Open 2010

  • SVG Viewer (http://www.adobe.com/svg/viewer/install) is a product from Adobe. It is free and one of its main uses is that it allows display of SVG in browsers that can’t natively handle SVG. For example, Internet Explorer (IE does not natively support SVG. This will change in IE 9). It allows to annotation of SVG graphics, and is also based on JavaScript. Unfortunately, Adobe has dropped support to this plug-in, but it still available for download, with no support for recent OS

  • svg-edit is a web-based, Javascript driven SVG editor (this is the introduction of the main web page, http://code.google.com/p/svg-edit/), hosted by Google. Should we add it is Open Source? It is itself built upon several other Open Source tools or frameworks (jQuery, js-hotkeys, …). Svg-edit does not allow developers to add user interaction: there is not – yet – tools to add links or JavaScript code to a node. But we still mention it in this paper because it is an easy way to create svg images graphically.

    svg-edit is available under the Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)



Is there any solution out there that can do all of the above plus create links between a SVG image and a data store? The answer is yes; 4D v12 (http://www.4d.com) is a solution. 4D is an integrated development platform to rapidly develop and deploy business desktop applications, embedding a database, an SVG rendering engine, a Web server, and many other components into one IDE and deployment platform.

Some parts of the implementation in 4D will not be detailed during the session, because they are not really SVG related and because of the duration of the session. For example, we will not explain how to create the structure (tables, fields, relations…) for the data store in the 4D database engine. Also, how to create the friendly user interface will not be explained (usage of buttons, enable-disable controls, etc.). Instead the implementation of SVG by the 4D language will be the focus. On the SVG side, 4D provides high level tools: native language commands, a component that encapsulates SVG primitive drawing, and another component that displays an SVG drawing area, letting the user modify and create images. But first, we’ll explain how the 4D SVG renderer works.

4D implements its own SVG renderer. An SVG image can be displayed in any form, and user interaction can also be handled.
Internally, the renderer uses native and modern OS APIs. Basically, when time comes to draw the image:

  • The renderer parses the XML DOM and builds a tree of SVG nodes. Each node has a style (fill, stroke, etc.) linked to it, and information useless for drawing is just ignored

  • Then the render pre-calculates font metrics, bounding boxes, and all graphical information that will be used when drawing commands are sent to the OS.

  • Then, the image is rendered in the graphic context using native APIs:

    • On Mac OS, 4D uses Quartz2D, CoreText (was using ATSUI in 4D v11), and CoreImage (Gaussian blur, transparent layers, …)

    • On Windows, current version uses GDI+. But we are currently implementing Direct2D for rendering the SVG. Once done – and for Windows Vista/7 – rendering the SVG will use Direct2D, DirectWrite and D3D (for filters). This will make rendering he SVG on Windows faster than with GDI+


Rendering the image on screen is fast because:

  • Only the first rendering does all those calculation. If the same SVG is displayed a second time, then only the third step is executed, which speeds up the drawing

  • The renderer works with a copy of the original xml tree. This makes drawing changes faster: if an attribute of the SVG is modified, then only this part will be recalculated. Of course, this can lead to some overhead depending on what attribute is modified (for example, changing the attribute of a group embedding several objects), but in any case the first step (parsing original xml) is never done a second time

Low-level access

As we already talked about a bit above, modifying an SVG image can be reduced to manipulating a text or a XML tree. 4D language allows both of those. Using text manipulation commands, the developer can create the SVG from scratch (here, a nice blue rectangle):

// Example of 4D code to create a basic rectangle (string manipulation)
// Create the SVG from text
svgImage := "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>
svgImage:=svgImage+"<svg xmlns=\"http://www.w3.org/2000/svg\" "
svgImage:=svgImage+"xmlns:svg=\"http://www.w3.org/2000/svg\" "
svgImage:=svgImage+"height=\"200\" width=\"200\">"
svgImage:=svgImage+"<rect id=\"rect1\" fill=\"#0000ff\" width=\"100\" height=\"100\" x=\"0\" y=\"0\" />"
// ...etc...other objects...
svgImage:=svgImage+"</svg>"

// Create the XML dom from this text
ref:=DOM Parse XML variable(svgImage)

// Move to an image displayed on screen
SVG EXPORT TO PICTURE(ref;theImage)

		

Using commands such as Replace string, or Position, the developer can modify this pseudo-xml (“pseudo” because there is no tree nor DOM). Another solution is to directly handle the XML. The code to produce the same image would be something like:

// Example of 4D code to create a basic rectangle (DOM manipulation)
ref:=DOM Create XML Ref("svg";"http://www.w3.org/2000/svg")
DOM SET XML DECLARATION(ref;"UTF-8")
DOM SET XML ATTRIBUTE(ref;"height";200;"width";200)

rectNode:=DOM Create XML element(ref;"rect";"x";0;"y";0;"width";100;"height";100;"fill";"#0000ff")

// Move to an image displayed on screen
SVG EXPORT TO PICTURE(ref;theImage)

		

To modify the image, the developer must either parse the text or use the appropriate XML DOM commands. But the fact is: when it is time to dynamically modify the SVG at a low level state, parsing an XML tree is much, much faster than parsing text.

High-level makes it easy

So, developers can build their SVG from a very low level, which requires a good knowledge of the specification (provided one wants to draw more than a basic rectangle!). That is why the 4D SVG Component was created. Basically, it is nothing more than a wrapper for DOM handling. But it makes things easier and more natural when it comes to creating an image. For example, the following code creates the same blue rect than above:

// Example of 4D code to create a basic rectangle (4D SVG component)
$ref:=SVG_New (200;200)
$rect:=SVG_New_rect ($ref;0;0;100;100)
SVG_SET_FILL_BRUSH ($rect;"#0000ff")
theImage:=SVG_Export_to_picture ($ref)

		

In this example, we also could have used all the optional parameters potentially expected by the SVG_New_rect method, that lets developers set the stroke/fill color, the rounding of the corners, …

But in any case, creating a new svg image and drawing a rectangle is done with a more natural language, and so, helps to convince developers that SVG is a great picture format!

Once an SVG image is rendered to screen, in a form for example, it is then possible to let the user interact with this image. In an SVG image, each element has a unique ID (expressed as text). So, basically, the main point is to get the ID of the element the user wants to interact with, and then trigger the appropriate action. Two main examples come to mind when talking about user interaction: doing something when the user clicks (or double-clicks) the element, and doing something when the user mouses over an element.

To handle those events, one needs to get the unique ID of the target element. This means that we need to (1) get the mouse coordinates, (2) check what element is under the mouse, (3) get its unique ID, and (4) do something if this specific object requires to do something (not all objects respond to a double-click, or display information when the mouse is over them).

The 4D engine makes those steps easy. When the user does something with the mouse over a form object (say, a container) that displays an SVG image, the x/y coordinates, relative to the top-left corner of the image, are automatically filled. Once the developer has those coordinates, one single command gives the required information: SVG Find element ID by coordinates. This command returns the ID of an element given an x/y coordinates. It’s that easy. This can be done during a standard clik/double click event, but also during a mouse over (and mouse enter/mouse leave).

There are also commands to find which elements are enclosed in a given rectangle (SVG Find element IDs by rect), or to scroll the image for displaying a specific object (SVG SHOW ELEMENT).