Using Flash to implement SVG and its application to XML Slidy


Table of Contents

Introduction
Rendering SVG in Flash
Editing SVG in Flash
XML Slidy
Conclusion
Figures
References

Microsoft PowerPoint has been the dominant slide presentation tool for many years. This posed a challenge for how to provide a Web-based alternative that wasn't reliant on proprietary formats and which would avoid the need for emailing people slides as large attachments.

The W3C Team generally felt that we should be using Web standards for our presentations, eating our own dog food as it were. The starting point was a way to split a single HTML file into a sequence of linked files that acted as slides. You had to start each slide with an H1 heading element. A Make file was provided to drive the script, but it was fairly cumbersome to use in practice.

I thought that there had to be a better way, and explored the idea of using JavaScript and CSS to dynamically show and hide DIV elements using CSS absolute positioning. This worked quite nicely, and I was able to provide a cross browser script. This is known as HTML Slidy1 and provides a PowerPoint like user interface for navigating between slides.

This has proven to be quite popular, but a substantial number of people aren't comfortable with authoring HTML, and even people who are, find it challenging to develop style sheets for new slide themes. I therefore started looking for ways to support browser based editing of slides that anyone could use without needing to know anything about markup or CSS.

My starting point was to look at the support for rich text editing first introduced into Internet Explorer and later copied by Firefox, Opera and Safari. The "content editable" feature suffers from a lack of interoperability across browsers as well as being very hard to work with when you want to provide the kind of user experience for editing slides that people expect based upon their familiarity with PowerPoint.

I therefore tried to see if I could instead emulate the text caret and use keystroke event handlers to define my own editing mechanism. This turned out to be just about feasible, but ran into problems with the poor state of browser support for keystroke events. It is a shame that browsers have yet to support DOM3 events with its well thought out solution for keystrokes and text input. A work around proved to be just about do-able, with browser specific fixups [Opera was the hardest to deal with].

Around that time, I decided to explore direct use of SVG, given that several browsers now provide native support for it. One big benefit is the nice way SVG is scaled from the document units to the window. This avoids some of the problems when trying to scale HTML to full screen display on different browsers. A quick experiment showed that it easy to use the svg:g element for slides with a small script for slide navigation that hides the current slide and reveals the next. Slides could be created directly in SVG or transformed from a simpler XML format via XSLT. The flexibility of SVG further lends itself to the possibility of implementing a high level slide editor for these formats using SVG and scripting.

Whilst SVG is now implemented natively on browsers like Firefox, Opera and Safari, the implementations vary in which features they support including the details of the scripting API. This makes it hard to write cross browser applications. When it comes to text editing, SVG 1.2 is very promising, but unfortunately the browsers for the most part only support SVG 1.1. A work around is to set event handlers for keystrokes and to emulate a text editing caret (see demos for SVG 1.1 and 1.2), but this runs into the same problems with keystroke event handling as for HTML. A thorough solution is contingent on browser support for DOM3 events and SVG 1.2.

It then occurred to me that maybe the ubiquitous Flash Player might be a better development platform with its much greater penetration than any existing browser. According to Adobe's statistics2, version 9 is now on around 98% of desktop browsers as of June 2008, and this number has been steadily climbing. I quickly decided against using the Adobe developer tools in favor of the little known open source compiler for the "haxe" programming language3, which is like a modernized version of Java, with strong type checking, flexible enums, parameterised types and type inference. Haxe allows you to compile to different versions of Flash (version 7-10) as well as to Neko, a very fast virtual machine that is available as an Apache module (mod_neko). Haxe and necko are open source and were both developed by the amazing Nicolas Cannasse.

The idea was to design a semantically oriented concise XML language for slides (XML Slidy) and to write a compact Flash application that would support viewing and editing slides, and posting them back to the Web server. XML Slidy combines simple markup for slides and slide themes together with SVG for bullet shapes, slide backgrounds and other graphics. I chose to define XML Slidy as a compound document format rather than as a microformat within SVG. This is out of a preference for using XML elements to express the semantics directly, rather than burying them within attribute value conventions.

I started from Helen Triolo's Action Script 3 implementation for rendering a subset of SVG paths in Flash4. With her permission, I ported her code to a haxe library, changing it a lot along the way, and then set about expanding the subset of SVG it supports. Elliptical arcs proved a little tricky, and I think that the implementation notes in the SVG specification could be easily improved upon.

It isn't possible to implement all features of SVG in Flash due to limitations of the Flash drawing API. As an example, SVG requires support for both the non-zero and even-odd fill rules, while Flash9 is limited to just the even-odd fill rule. In practice, this isn't a serious limitation as most graphics don't have complex intersecting paths, and the good news is that the latest version of the Flash Player (v10) now supports the non-zero rule. Flash doesn't support elliptical sections and cubic bezier curves, and both of these are handled by algorithms that draw them using a sequence of quadratic bezier curves.

Flash has a different approach to gradient fills than SVG, and you are required to compute a gradient matrix before drawing the path to which it applies. This means that the path's bounding box is unknown beforehand, which causes problems when the SVG gradient units are relative to the object bounding box. A work around is to draw the path twice, first to determine the bounding box and then again with the corrected gradient matrix. Another challenge is posed by SVG's dashed strokes. Flash doesn't support this, and a work around is to break the line into a series of pieces, which is a little complex for curved paths, as demonstrated by Trevor McCauley5. Another idea is to stroke the line with a repeating gradient, but this won't provide an adequate solution for curved lines.

I also decided not to support the SVG script element as that would have been very difficult to implement in Flash. Again, this isn't a serious limitation at least in the context of use for slide presentations. I have yet to come across anyone who has used scripting within their PowerPoint presentations, although declarative animations for build effects and slide transitions are another matter.

In summary, implementing SVG in Flash has proven to be very practical and avoids the interoperability problems found with native implementations of SVG in current browsers. Flash is also much more widely deployed than even Internet Explorer. Why struggle with existing browsers when you can implement your own XML application that runs on over 98% of browsers including all versions of Internet Explorer! Flash might seem at first to be a competitor to SVG, but it is easy to put Flash to use in SVG-based applications. I am particularly grateful for the SVG test suite as a way of measuring my progress in expanding the subset of SVG supported. I am hoping to be able to get permisssion to open source the haxe SVG library for Flash 9.

My next challenge was to create a Web-based graphics editor in Flash, so that users would be able to create and modify SVG graphics using their browser. Like the SVG renderer, this would be a component library that could be integrated into a range of Web applications. Traditional drawing tools are standalone applications that need to be locally installed. The user interface is often very flexible and can be a little hard to learn. By contrast, most people expect Web applications to be easy to drive and something you can use the first time you come across it. Fulfilling such high expectations isn't going to be easy. My approach has been to try to identify usability problems with existing drawing tools, to come up with alternatives, and to make iterative changes based upon user feedback. This is likely to take a long time to mature, and I am still at an early stage.

Some of the considerations include:

  • The ability to see at a glance where you are when panning and zooming on a large canvas. To support this, I provide a thumbnail view that shows the current view port and allows you to recenter the view with a double click. You can also double click the viewport to recenter the view, and this is why double click was chosen, as a single click is already used for a variety of other purposes, e.g. selection and drawing.

  • The most commonly used controls are placed on a static pane that is always present. These include support for switching between drawing and selection modes, and undo/redo buttons.

  • Other controls are placed on tabbed panes. This saves precious screen space, and offers the user a top level set of choices (shapes, transform, stroke, fill, filters, file and about). Most graphical editors use a toolbar of icons which users have to learn to recognize, I have instead chosen to use words that should be immediately obvious, and through the tabbed pane metaphor, invite users to explore the choices available to them.

  • Each of the panes presents a set of controls which change according to the choices the user has made. For instance on the fill pane, the user can choose between no fill, color, gradient and pattern. Clicking on color, you are offered the means to set the color and its opacity. For color, a pop-up color wheel appears supporting the hue, saturation and brightness model. For opacity, a pop-up slider appears, but you are free to type a value directly if you prefer. This approach allows user to define a sequence of stops for gradient fills.

  • To help people to rapidly get to grips with editing, tooltips are widely used, together with highly contextualized prompt text that indicates a variety of actions that the user could perform. This text will be localized based upon the user's browser preference settings as exposed through HTTP.

  • The (mouse) pointer operates in one of two modes: drawing and selection. One of the considerations was to avoid the need to constantly move the pointer to select a control. You can instead use one of several keystrokes to make changes according to the object the pointer is currently over. People using a trackpad often use one hand to activate the mouse buttons, and the other to control the pointer position. This makes it harder to press a key whilst clicking the mouse button. For that reason, the keystrokes apply their action without the need for a simultaneous click. Small graphical cues next to the pointer indicate the current context, e.g. when you are over the edge of a shape and able to insert a new point or to drag the edge.

  • Grouped graphics can be edited without the need to ungroup them. You can also define properties on groups which is useful for inherited properties such as stroke color. You can select an object by clicking on it and move up and down nested groups with single keystrokes. The context menu accessed by a right click allows you to jump directly to a named group.

Accessibility is an important consideration, and the controls are all designed to be operated via keystrokes when it is impractical to use the pointing device. Flash provides a reasonable set of accessibility features, but the burden is on the developer to use them effectively. One problem is that browsers hang onto the input focus when a web page is loaded. To set the focus to a field in the Flash application, requires a click! A work around is to use a small amount of JavaScript to forward keystrokes from HTML to Flash. A further issue is that on some browsers (e.g. Firefox) tabbing to the next control doesn't work properly across Flash and HTML. If Flash has the focus, the tab sequence is limited to controls within your flash application.

Assistive technology relies on there being an agreed API through which users can drive graphic user interfaces. This is provided by the operating system for built-in classes of user interface controls. Developers are increasingly defining their own controls, e.g. using HTML, CSS and JavaScript, or Action Script in the case of Flash, and this is invisible to assistive technology. W3C's work on WAI-ARIA6 will allow conscientious developers to annotate their Web applications with sufficient information for browsers to support the accessibility API for that platform. The flash.accessibility package provides a means to associate descriptions and shortcuts with objects for use with assistive technology such as screen readers, but lacks the sophistocation of ARIA with its taxonomy of controls, their states and events. Further work is needed allow ARIA to be applied to Flash, and it is my hope that Adobe will cooperate fully with W3C in that regard.

When thinking about recasting Slidy in XML, my first step was to look at the markup created by existing presentation tools, e.g. Open Office, Microsoft PowerPoint and Apple Keynote. All of these are bulky formats, and it seemed to me that it should be practical to design a much more compact format with a clean separation between the slide contents and the definitions for how they should be themed. As a web application, it should be possible for presentations to share a theme by reference rather than embedding copies of the theme in each presentation. Users would be able to change themes, e.g. to use a high contrast theme when needed.

The new slide format should make it easy to create a high level editor in Flash, and should use SVG for graphics, together with a simple means to include video. It should be easy to import content from websites like Flickr and YouTube, and there should be support to import and export slides from and to other formats, e.g. PowerPoint and PDF. The design of the markup language also needs to support ease of use for authors. It would be easy to add all kinds of features into the markup language, but these would bear a cost in terms of how they are exposed to authors.

Here is an example of a presentation with a cover slide followed by a bullet point slide. The root element declares a number of namespaces and defines the view port. The renderer is expected to preserve the aspect ratio when scaling the view port to the current window size. The 'about' element is equivalent to the SVG metadata element, and in this example, provides information in the Dublin Core vocabulary.

<slidy version="1.0"
xmlns="http://example.com/ns/2008/03/slidy"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns:dc="http://purl.org/dc/elements/1.1/"
viewbox="0 0 1024 768">
  <!-- metadata using Dublin Core and other vocabularies -->
  <about>
    <dc:title>Test deck for XML Slidy</dc:title>
    <dc:creator>Dave Raggett</dc:creator>
    <dc:date>2008-03-21T17:39:34</dc:date>
  </about>
  <!-- link to standard theme defining backgrounds etc. -->
  <theme xlink:href="theme.xml"/>
  <!-- one or more slide elements -->
  <slide template="cover">
    <title>Test Slide Deck</title>
  </slide>
  <slide template="normal">
    <title>Second slide</title>
    <points>
      <p level="1">1st level point</p>
      <p level="2">2nd level point</p>
      <p level="1">Another 1st level point</p>
      <p level="2">Another 2nd level point</p>
      <p level="3">Another 3rd level point</p>
    </points>
  </slide>
</slidy>

The 'theme' element uses XLink to reference an external theme, or includes the theme definition as its content. It is followed by a sequence of 'slide' elements. The 'template' attribute identifies the slide's template, allowing different themes to be applied to different kinds of slides. Each slide should start with a title, followed by a sequence of slide components, for example, bullet points, images and tables. SVG graphics may be embedded or referenced.

For bullet points, I initially thought of the HTML 'UL' and 'LI' elements, but experience soon showed that this would significantly complicate the code due to the need to preserve a well formed tree when changing the level (significance) of individual bullet points. It proved to be much easier to represent the level as an attribute rather than as nested elements. The editing user interface follows PowerPoint and Open Office in making it easy to promote or demote the level of a bullet point, and to show or hide the bullet itself.

Here is an example of a theme.

<theme version="1.0"
xmlns="http://example.com/ns/2008/03/slidy"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:svg="http://www.w3.org/2000/svg">
 <style name="normal">
   <svg:defs>
     <svg:circle id="bullet1" r="7" fill="#FF3399"/>
     <svg:circle id="bullet2" r="5" fill="#FF66FF"/>
     <svg:circle id="bullet3" r="4" fill="#9933FF"/>
   </svg:defs>
   <background>
     <svg:rect x="0" y="0" width="400" height="45" fill="#FFEEDD" stroke="none"/>
   </background>
   <style name="title" font-family="serif" font-size="36pt" font-color="red"/>
   <style name="points" font-color="blue" x="0">
     <style name="level1" bullet="#bullet1"  margin-left="20"
      font-color="blue" font-family="Helvetica" font-size="22pt"/>
     <style name="level2" bullet="#bullet2" margin-left="40"
      font-family="Helvetica" font-size="14pt" font-style="italic"/>
     <style name="level3" bullet="#bullet3" margin-left="60"
      font-family="Helvetica" font-size="10pt"/>
   </style>
 </style>
</theme>

The example defines a theme for slides with the template 'normal'. SVG elements can be embedded, and this is used here to define bullets and backgrounds. The 'style' element can be nested arbitrarily, and forms a tree of named styles. The properties are the same as those for SVG, plus additions such as bullet and level, which are interpreted by the code for each kind of slide component, where the nesting of components matches the tree of styles. The svg:defs element can be freely placed to define shared definitions.

The simplicity of the slide and theme formats make them straightforward to transform into other formats, e.g. SVG+script for delivery to SVG viewers, or to PDF for printing. A bigger challenge will be to develop a means to import PowerPoint and Open Office presentations, as both of these lack a clean separation between content and styling, necessitating the use of heuristics to make a plausible guess at the authors intent.

It is probably too early to standardize the markup for slides and themes, and futher experience is needed to evaluate the pros and cons of different design choices. It is my hope that XML Slidy will provide a valuable contribution in the run up to work on standards. For other work on using XML for slides, see e.g. SlideML7 and PowerML8.

The ubiquity of the Adobe Flash Player and the support it provides for XML and rich graphical user interfaces makes it a very attractive platform for XML-based Web applications. The Haxe programming language with its modern features speeds development compared to working in languages like JavaScript. Broadband connections ensure that the download time for the Flash files is very small. As an example, the SVG renderer library component is about 20 KBytes, the size of a smallish image. Larger applications can be split into a number of smaller files and loaded in the background for reduced startup latency. Accessibility remains an area for improvement, and help is sought from Adobe in enabling developers to annotate their applications in a similar way to how WAI-ARIA is used for rich Web applications written in HTML, CSS and JavaScript.

The work described above is still in progress, and is expected to take some time yet to fully complete. In particular, usability and features will depend on continuing user feedback in an iterative development process. I am looking forward to working on support for distributed meetings, e.g. to allow a presenter to drive remote views, together with duplex audio for handling questions from the remote audience. It should also be possible to record presentations for later playback, where the audio is synchronized with slide navigation. Some more details were outlined in my Google Techtalk in early 20069.

Finally, I would like to acknowledge the support of JustSystems10, and my manager Hideki Hiura for enabling me to do this work.

Some screen shots of the SVG editor during its development. The first one shows shape handles, the shape pane, the thumbnail that shows the position of the current view port on the larger canvas, and the context sensitive help and tool tips.


The next screen shot shows the fill properties pane and the pop-up HSV color selection tool.