Using Canvas in SVG


Table of Contents

What is Canvas?
How to use Canvas in SVG?
Why use Canvas in SVG?
Read pixel values
Modify pixel values
Combine vector and raster
Conclusion
References

Canvas was introduced by Apple as an HTML extension for their Dashboard application in July 2004 [hyatt] and short after added to the HTML 5 specification draft [spec]. Citing David Hyatt's original announcement, Canvas is “essentially an image element that supports programmatic drawing. The way it works is that you can invoke a method called getContext on the canvas and then you have access to a whole range of 2d drawing calls” [hyatt]. The following example illustrates this basic concept and draws a transparent green rectangle sized 100 by 75 pixels:

Listing 1: Markup for Canvas in HTML 5

<!DOCTYPE html>
<head><title>markup for Canvas in HTML 5</title></head>
<canvas width="300" height="150">
  alternate content for browsers that do not support Canvas
</canvas>>

<script>
  window.onload = function () {
    var context = document.getElementsByTagName('canvas')[0].getContext('2d');
    context.fillStyle = 'rgba(0,200,0,0.7)';
    context.fillRect(0,0,100,75);
  };
</script>
      

Four main steps are needed to utilize Canvas drawing in HTML 5:

  1. add a canvas element to your document

  2. add a script element to your document

  3. define the Canvas drawing context

  4. use this context to do the actual drawing

As you note, getContext('2d') in step three takes one argument - the "context type". The current Canvas specification defines only one context type named "2d" but also states that “a future version of this specification will probably define a 3d context” [context]. Work on this 3d context has already started at the WebGL™ working group, a collaboration betwen the Khronos Group, AMD, Ericsson, Google, Mozilla, NVIDIA and Opera, which aims to define “a JavaScript binding to OpenGL® ES 2.0 to enable rich 3D graphics within a browser on any platform supporting the OpenGL or OpenGL ES graphics standards.” [webgl].

Once you have obtained the context, you are ready to draw onto the Canvas. You may create rectangles, lines, curves, arcs or texts, stroke and fill objects with color, gradients or patterns and, just as in SVG, scale, rotate, translate or matrix-transform them. In addition, Canvas provides eleven compositing attributes whilst drawing, simple shadows and most important in the context of this paper, allows to draw images, videos or parts of them using the drawImage method and gain direct access to their pixel's rgba-values through getImageData and putImageData. Table 1 presents all available drawing methods and attributes defined by the "2d" context [context2d].


Although methods like fillRect() or lineTo() might indicate some kind of vector usage, Canvas does not preserve its content in vector form. Canvas is all about raster, there is no DOM to access drawn objects as all the drawing calls are translated to pixel values immediately. This is the main difference between Canvas and SVG and one of the reasons, why Canvas is supposed to be considerable faster when it comes to performance. Before we continue exploring how to use Canvas in SVG, there is one more Canvas method worth mentioning - canvas.toDataURL.

This method "returns a data: URL for the image in the canvas. The first argument, if provided, controls the type of the image to be returned (e.g. PNG or JPEG). The default is image/png; that type is also used if the given type isn't supported. The other arguments are specific to the type, and control the way that the image is generated ..." [todataurl]

It's worth noting, that the Canvas specification explicitly mentions image/svg+xml as possible mime-type "if the implementation actually keeps enough information to reliably render an SVG image from the canvas" [ibid]. Unfortunately no browser vendor has implemented image/svg+xml already, so image/png and image/jpeg are the only options available right now. Both return a base64 encoded data: URL which is suitable to be used as xlink:href attribute of an SVG image element.

As it is not yet clear how SVG and HTML5 will play together in the future, the only way to include Canvas in SVG right now is via the foreignObject element. The extra steps required to get Canvas working in SVG are thus as follows:

  1. add the xhtml namespace to your root element

  2. add a foreignObject to your document

  3. add an xhtml:canvas element as childNode of your foreignObject

  4. use getElementsByTagNameNS for getting a reference to your Canvas

Listing 2: Markup for Canvas in SVG

<svg xmlns="http://www.w3.org/2000/svg"
     xmlns:xhtml="http://www.w3.org/1999/xhtml">
<title>markup for Canvas in SVG</title>
<foreignObject width="300" height="150">
  <xhtml:canvas width="300" height="150" >
    alternate content for browsers that do not support Canvas
  </xhtml:canvas>
</foreignObject>

<script type="text/javascript"><![CDATA[
  window.onload = function () {
    var xhtmlNS = "http://www.w3.org/1999/xhtml";
    var context = document.getElementsByTagNameNS(
                  xhtmlNS,'canvas')[0].getContext('2d');
    context.fillStyle = 'rgba(0,200,0,0.7)';
    context.fillRect(0,0,100,75);
  };
]]></script>

</svg>
      

If you want to put an existing raster image onto the Canvas to make use of getImageData and putimageData, there are additional steps needed:

  1. add an xhtml:img element to your foreignObject

  2. get a reference to this image

  3. use drawImage to copy the image to the Canvas

The reason why you cannot us an SVG image element as source for the drawImage method is simple, but painful: the current Canvas specification does not (yet) allow to reference SVGImageElement as source for drawImage and can only cope with HTMLImageElement, HTMLCanvasElement and HTMLVideoelement. This short-coming will hopefully be addressed during the process of defining "SVG in HTML5" behavior and could be extended to allow SVGSVGElement as well. The xhtml:img element in listing 3 uses visibility:hidden as we do not want it to interfere with its visible copy on the Canvas.

Listing 3: Markup for Canvas in SVG using a raster image

<svg xmlns="http://www.w3.org/2000/svg"
     xmlns:xhtml="http://www.w3.org/1999/xhtml">
<title>markup for Canvas in SVG using a raster image</title>
<foreignObject width="300" height="150">
  <xhtml:canvas width="300" height="150">
    alternate content for browsers that do not support Canvas
  </xhtml:canvas>
  <xhtml:img src="http://www.w3.org/Graphics/SVG/logo/tmpSVGlogo.png" style="visibility:hidden;" />
</foreignObject>

<script type="text/javascript"><![CDATA[
  window.onload = function () {
    var xhtmlNS = 'http://www.w3.org/1999/xhtml';
    var context = document.getElementsByTagNameNS(
      xhtmlNS, 'canvas')[0].getContext('2d');
    var image = document.getElementsByTagNameNS(xhtmlNS, 'img')[0];
    context.drawImage(image,0,0);
  };
]]></script>

</svg>
      

Besides the fact that it is always advantageous to mix evolving techniques and thus enhance facilities in creating applications, the ability to control raster values is a big win for SVG. The idea of accessing pixel values in SVG was already part of the first SVG 1.2 Full Draft way back in 2004 where SVGImage.getPixel() was defined as returning an SVGColor-object at point x/y [svg2004]. This chapter has disappeared in SVG 1.2 Tiny and probably will not make it into SVG1.2 Full either. A much better and cleaner way to control raster in SVG can be achieved by Canvas and its drawImage, getImageData and putImageData capabilities instead, so let us explore how this can be done.

When it comes to reading pixel values, context.getImageData(sx,sy,sw,sh) is the way to go. This method "returns an imagedata object containing the image data for the given rectangle of the canvas" [pixelmanip]. Besides width and height attributes, imagedata contains a data object called CanvasPixelArray which “provides ordered, indexed access to the color components of each pixel of the imagedata object. The data must be represented in left-to-right order, row by row, top to bottom, starting with the top left, with each pixel's red, green, blue and alpha components being given in that order for each pixel.” [ibid]

A simple example will shed some light on how this is to be interpreted. Given a Canvas with 2x2 pixels, a black and red pixel in the upper and a green and blue pixel in the lower half, a call to context.getImageData(0,0,2,2).data will result in the following CanvasPixelArray:

CanvasPixelArray = [ 0,0,0,255, 255,0,0,255, 0,255,0,255, 0,0,255,255 ]
      

Accessing pixel values is straight forward: find the offset of the pixels first component and then use the next four values from that point on. For a pixel defined by row and column, the offset may easily be computed with the following formula:

offset = ((row - 1) * canvas.width * 4) + ((col - 1) * 4);
      

Thus the offset for the red pixel in row 1, column 2 of the above example will be 4 and its rgba values 255,0,0,255.

Our first example uses the above formula and displays rgba values for an image according to the users mouse position. This and all successive examples require a small JavaScript library called SVGCanvasElement.js to create canvas elements via foreignObject as shown in listing 2 and import SVG image elements to Canvas as demonstrated in listing 3. Please refer to the the library's source for usage instructions.

As soon as you have access to an images color values, it is easy to compute color distribution statistics. A land cover image with indexed colors in PNG format serves as source for this example. The image, as all other mapping related images presented in the paper, has been created with UMN mapserver [mapserv] for the Tirol Atlas project [atlas]. It covers North Tyrol (Austria) and parts of Southtyrol (Italy) within a 122x122 km wide extent. When you click on the image, a script loops through pixels in the CanvasPixelArray and collects counts for rgb values. Results are displayed as text-nodes containing the color presented as UTF-8 symbol &#x25D9; (◙), the actual rgb() values and their distribution in percent.

Modifying pixels is as easy as reading pixels. Just alter the CanvasPixelArray in place with a for-loop and write it back to the Canvas using putImageData(dx,dy).

This time, we reclassify an 8-bit grayscale elevation image in PNG format according to its altitude values, which are reflected by amounts of gray reaching from 0 to 255. Each subsequent step increments the altitude by 20 meters, thus rgb(0,0,0) represents sea level and rgb(255,255,255) the highest possible elevation computed by 255*20 which results in 5100 meters. Low areas therefore appear in dark and high areas in light gray. When you click on the image, each pixel in the CanvasPixelArray is set to the color defined by the corresponding elevation class. The altered CanvasPixelArray is then written back to the Canvas using putImageData(dx,dy) which in turn results in a reclassified, now colored image. As in example two, distribution for elevation classes and their colors is presented through text nodes as well.

Example four illustrates simple color manipulation by converting color to grayscale on mousemove for a 25x25 pixel wide extent, bottom-right to the current mouse position. The grayscale effect is achieved by picking up the affected region through getImageData(clientX,clientY,25,25) and substituting each pixels r, g and b value by the average of their components. The altered CanvasPixelArray is then written back to the canvas with putImageData(clientX,clientY).

The last five examples do not introduce new Canvas features but try to present a few ideas on how Canvas and SVG could be combined to make use of each others strengths. Let us start with a simple color picker.

Implementing basic color picking in SVG is a task that requires rectangles and a few lines of JavaScript code. The more colors a palette contains, the more <rect /> elements will be needed, each with a handful of attributes to define position and color. This is a perfect match for <canvas/> to step in: use an indexed PNG image instead and pick the colors with drawImage and getImageData. The CSS3 color picker example illustrates how to do this and could easily pick any color from any image by simply substituting the palette with it.

A compact way to prepare icon sets for SVG applications is presented in this example. 82 KDE Oxygen / Status icons sized 22x22 pixels each, have been joined together forming a single PNG image with regular grid layout. When you click on this image, all icons are extracted from the grid and converted to SVG image elements. This task is accomplished by a loop that fills a temporary canvas sized 22x22 pixels with the relevant data extracted from a second canvas holding the original PNG image. It then calls toDataURL('image/png') on the small canvas to obtain a base64 encoded data string which is suitable to be used as xlink:href attribute in the newly created SVG icon. For demonstration purposes, icon positions are set randomly and clicking on them reveals their icon number.

This, and the the next example, is inspired by Andreas Neumann's and Yvonne Isakowski's "Türlersee" application [tuerlersee] presented at the first SVGOpen conference in Zürich 2002. Their paper demonstrated how to create a shaded relief with SVG filter effects applied to an alpha-channel image and how to draw interactive profiles derived from elevation data packed into a JavaScript array.

Instead of reading elevation data from an additional JavaScript array, altitude values are now computed from the alpha channel, which is present already, through Canvas. Mouseover the image to reveal these values and click the image to toggle display of the shaded relief via an SVG feDiffuseLighting filter.

It is also possible to put the alpha channel, representing elevation data, on top of an RGB image as is done in this example. Unfortunately this causes colors to loose saturation, so the only way to reveal original colors again is to apply a feColorMatrix filter that visually deactivates the alpha channel (thanks to David Dailey for pointing the author to the right direction). With elevation data ready in the alpha channel and a fresh land cover/relief base map revealed by the filter, we are now ready to digitize profiles. Each click on the image adds a new profile corner point and as you click the "compute" button, additional points are interpolated at 1px distance along the previously digitized lines. This helps to smoothen the profile which right now is reduced to the minimum. Nevertheless all data is present and could be used to produce interactive SVG-profiles with animation straight away. Just click on the map to start over again.

The last example demonstrates a simple sound map where mousemove events are captured both for raster and vector. Sound clips are played according to raster value read via Canvas or vector type defined by a custom namespaced ta:layer attribute. A JavaScript hash keeps track of which sounds in OGG Vorbis format to play for rgb or attribute values. This examples requires HTML 5 <audio /> and is supposed to work in FF3.5

Mixing Canvas with SVG has huge potential to build stunning interactive applications in the future. Canvas and SVG are not competitors but complement one another as both do what they do so well. Given SVG's capabilities when it comes to user interfaces and Canvas's unique and efficient way of handling raster in the browser, it is time to bring both together and thus make the next step in integrating SVG in HTML5.

Links last accessed: Aug 18th, 2009