Drawing SVG in the Cloud

SVG Tiny for Google Application Engine

Andrew Girow


Abstract


Google Application Engine is a cloud-computing infrastructure for creating and running web applications. Google Application Engine does not allow all Java classes to be used, For example, no graphics classes are allowed. At the same time, there is a need for the server side graphics on Google App Engine. TinyLine fills the gap and brings in the vector graphics to Google App Engine.

We discuss some technical basic behind TinyLine and Google App Engine and show how Google App Engine Java developers could use and benefit from TinyLine.

We go from the simple to more complex examples, starting from the basic 2D drawing capabilities demos and SVG thumbnail images service.

Then we show that thumbnail images are convenient way to present SVG previews to users. The thumbnail images could be used for the SVG listings, SVG directories or for SVG result lists of search engines.

We give an example how TinyLine could help to port other graphical Java applications to Google App Engine platform.

The more advanced example of using TinyLine and Google App Engine brings together full text search of SVG images and visual representation of search results by thumbnail images.


Table of Contents

Introduction
Google App Engine Java
Overview
Java Runtime Environment
Datastore and Services
Sandbox
Server side graphics
TinyLine
TinyLine 2D
TinyLine SVG
TinyLine for GAE
Basic demos
TinyLine 2D Examples
SVG Thumbnail images
Barcode example
SVGArk: Store and Search SVG
Overview
Users
Uploading SVG files
Storing SVG files
Searching SVG files
Limitations
Future development
Bibliography

Introduction

The popularity of Google App Engine Java platform is on the rise. There are lot more applications being deployed on this platform. At the same time, the graphics support on Google App platform is very limited.

There is a need for the richer graphics on Google App Engine. Especially if we are talking about cloud computing when data is being collected and processed in the cloud, there should be an option to visualize this data in the cloud.

Google App Engine Java

Google App Engine Java is cloud computing technology. It allows Java developers to build scalable web applications using standard Java technologies and run them on Google's infrastructure [R1].

Google App Engine Java provides:

  • Java 6 JVM

  • Java Servlets

  • JDO, JPA, JavaMail, and JCache standard interfaces to the App Engine datastore and services

Google App Engine runs Java applications using the Java 6 JVM. Google App Engine uses the Java Servlet standard for web applications.

A web application has a standard WAR directory structure that can have servlet classes, JavaServer Pages (JSPs), static files and data files, the deployment descriptor and other configuration files.

The Java 6 JVM runs web applications in a Sandbox environment. The Sandbox ensures a web application does not interfere with the performance and scalability of other web applications.

A web application, for example, cannot create new threads, write to local file system, make network connections, use JNI or load native libraries.

Google App Engine provides scalable services that web applications can use to store persistent data, access over the network, and perform other tasks.

Google App Engine provides User service API for user authentication. A user that already has a Google account (such as a GMail account) can use that account with an application.

An application can detect if the current user is signed in, and can access the user's email address. Google App Engine also provides the low-level API to generate sign-in and sign-out URLs.

User service API can return the current user's information as a User object. User objects could be saved in the datastore.

The Google App Engine Memcache provides fast, transient distributed storage for caching the results of datastore queries and calculations. It implements JCache API (JSR 107).

The Google App Engine Datastore is a reliable, scalable persistent storage of data. It supports standard Java interfaces: JDO 2.3 and JPA 1.0. These interfaces are implemented using DataNuclueus Access Platform.

The Google App Engine URL Fetch service allows to access resources over the network and to communicate with other hosts using the HTTP and HTTPS protocols.

The Google App Engine Mail service allows sending email messages on behalf of the application user or application administrators. It uses JavaMail API for sending email messages.

The Google App Engine Images service allows transforming and manipulating image data in several formats.

Google App Engine runs Java web application using Java 6 JVM in a Sandbox environment.

When Google App Engine receives a request for a web application, it invokes the servlet that corresponds to the URL. It uses the Java Servlet API to provide the request data to the servlet, and accept the response data.

Google App Engine uses many web servers to run web application. Any request may be routed to any server. It could be not the same server that handled a previous request from the same user.

The Sandbox environment allows Google App Engine to route requests for applications across various web servers, and to prevent one application from interfering with another [R2].

Restriction Alternative
Threads Modify the thread state
Direct network connections URLConnection
Direct file system writes Use memory, memcache, datastore
Java2D Images API, Software rendering
Native code Pure Java libraries

Google App Engine supports client side graphics via Google Web Toolkit (GWT) and Web Browser capabilities (like Canvas, JavaScript or SVG native support).

The server side graphics support in GAE is very limited. It supports some Images API to manipulate image data while Java2D is not allowed.

There is a need for the server side graphics on Google App Engine. Especially if we are talking about cloud computing when data is being collected and processed in the Cloud, there should be an option to visualize this data in the Cloud.

There are a lot of people that feel comfortable with static data visualization. They would like to copy and paste their charts, diagrams or over graphics as still images in PNG or JPEG format into their MS Office Word, Excel or Power Point program.

They would like to place their own caption or memos using simple raster graphics editor, etc.

They would like to see a static preview or capture of a dynamic presentation before they click on it.

Because Java2D is not allowed software rendering should be used for server side graphics in Google App Engine. Software rendering it is exactly what TinyLine does!

TinyLine

TinyLine includes TinyLine 2D and TinyLine SVG [R3].

TinyLine 2D implements a mobile 2D graphics engine for Java platform.

Key features:

  • Small footprint

  • Fast fixed-point numbers mathematics

  • Paths, basic shapes and texts drawings

  • Hit tests for paths and texts

  • Solid color, bitmap, pattern, gradient (radial and linear) paints

  • Fill, stroke and dash

  • Affine transformations

  • Outline fonts

  • Left-to-right, right-to-left and vertical text layouts

  • Antialiasing

  • Opacity

Drawing pipeline

2D drawing pipeline

The TinyLine 2D drawing pipeline has four components: Tiny2D, TinyState, TinyBuffer and TinyProducer.

The TinyLine 2D rendering process is controlled through the Tiny2D object and its state attributes. The state attributes, such as line styles and transformations are applied to graphic objects when they are rendered. The collection of state attributes associated with a Tiny2D is referred to as the graphics state TinyState.

The basic drawing process is the same for any graphics element:

  1. Specify the target surface TinyBuffer.

  2. Specify the appropriate attributes for the graphics by setting the graphics state attributes in the TinyState object.

  3. Define the shape or text you want to draw.

  4. Use the Tiny2D object to render the shape, paths or text, by calling one of the Tiny2D rendering methods.

During the rasterization, the Tiny2D object reads the graphics state attributes TinyState and applies them to the drawing process of the graphics primitives. The rasterization target is the TinyBuffer object (also named as "canvas") where all is drawn. When the drawing process has completed the TinyProducer is notified that new pixels on the TinyBuffer "canvas" are ready to be send onto a device screen.

The painters model

TinyLine 2D uses the painter's model for its imaging. In the painter's model, each successive drawing operation applies a layer of "paint" to an output "canvas" - TinyBuffer object. The paint on the pixels buffer (TinyBuffer object) can be modified by overlaying more paint through additional drawing operations. This model allows you to construct sophisticated images from a small number of primitives.

When the paint is not completely opaque the result on the pixels buffer is defined by the (mathematical) rules of the simple alpha blending.

Coordinate spaces

TinyLine 2D supports three coordinate spaces: character space, user space and device space. Transformations between coordinate spaces are defined by transformation matrices, which can specify any linear mapping of two-dimensional coordinates.

Graphics elements

TinyLine 2D supports two basic types of graphics elements that can be rendered onto the TinyBuffer object:

  • Shapes, which represent some combination of straight line and curves

  • Text, which represents some combination of character glyphs

As about raster images, a raster image is presented via the TinyColor class as array of values that specify the paint color and opacity (alpha).

Shapes and text can be filled and stroked. Each fill and stroke operation has its own opacity settings; thus, you can fill and/or stroke a shape with a semi-transparently drawn solid color, with different opacity values for the fill and stroke operations.

Text and fonts

In TinyLine 2D texts are rendered just like shapes. Therefore, coordinate system transformations, painting - all TinyState graphics state attributes apply to text in the same way as they apply to shapes or paths.

Additional text attributes include such things like the writing direction (text layout), font specification (TinyFont) and painting attributes which describe how exactly to render the characters.

The TinyFont class includes the information necessary to map characters to glyphs, to determine the size of glyph areas and to position the glyph area.

Colors

TinyLine 2D supports the following built-in types of paint that can be used in fill and stroke operations using the ARGB color space:

  • Single color

  • Gradients (linear and radial)

  • Patterns (raster images)

TinyLine SVG uses (includes) TinyLine 2D rendering capabilities and implements SVG Tiny 1.1+ engine for Java platform.

Key features:

  • SVG Tiny 1.1+ engine

  • Supports SVG fonts, raster image and text elements, paths

  • Supports SMIL animations and events

  • Allows both textual and gzipped SVG streams

  • Compact code

  • Easy to use API

SVG pipeline

SVG drawing pipeline

During the parsing process SVGParser callbacks build internal OM (SVGNode tree) from the input SVG stream. The SVGDocument has the SVGNode tree root as its member.

The SVGRaster translates SVGNode objects into graphics primitives and draws them (rasterization process) using Tiny2D onto the TinyBuffer object.

The TinyProducer interface gets a notification that new pixels of the TinyBuffer object are ready to be send onto physical screen.

The SVGRaster class rasterizes the tree of SVGNodes onto the TinyBuffer. Each SVGNode "paint" operation draws over some area of the pixel buffer. When the area overlaps a previously painted area the new paint partially or completely obscures the old. When the paint is not completely opaque, the result is defined by the simple alpha blending rules.

Nodes in the SVG tree have an implicit z drawing order. Subsequent nodes are painted on top of previously painted nodes.

When a graphic object is rendered, the geometry, image, and attribute information are combined to calculate which pixel values must be changed.

TinyLine is extremely portable across Java flavors because it uses a small set of standard Java SDK classes

  • java.lang.Exception

  • java.io.InputStream

  • java.lang.Object

  • java.lang.System

  • java.lang.Throwable

It made TinyLine run on Google App Engine platform without any efforts.
Because the Google App Engine Sandbox environment does not allow AWT Image class instead of drawing TinyBuffer on an AWT Image, we should serialize TinyBuffer object into PNG image data.
For SVG Tiny support, we should be able to load (read) PNG image data into TinyBuffer object.

Raster images

Neither Google App Engine nor TinyLine provide reading and writing raster images API.

SVG Tiny should support at least one of the following formats PNG or JPEG. It seems that PNG support is a lot easier than JPEG.

The PNG reading and writing classes are in com.tinyline.appengine.image package:

    com.tinyline.appengine.ImageWriter
    com.tinyline.appengine.AppImageLoader
			

Here ImageWriter converts a TinyBuffer object into a stream of bytes in PNG image format. AppImageLoader reads an input stream in PNG image format and converts it to a TinyBuffer object.

2D and SVG Canvases

TinyLine for Google App Engine classes includes

    com.tinyline.appengine.AppTiny2DProducer
    com.tinyline.appengine.AppTiny2DCanvas
    com.tinyline.appengine.AppViewerCanvas
			

AppTiny2DProducer is a Google App Engine implementation of the TinyProducer interface. TinyProducer provides an interface for objects which can produce the image data from the TinyBuffer object (pixels buffer)

    /**
     * Determine if there is a consumer that is currently
     * interested in data from this TinyProducer.
     */
    public boolean  hasConsumer();

    /**
     * Send a rectangular region of the buffer of pixels to any
     * consumer that is interested in the data from
     * this TinyProducer.
     */
    public void     sendPixels();

    /**
     * The imageComplete method is called when the TinyProducer is
     * finished delivering all of the pixels to a consumer.
     */
    public void     imageComplete();
	

AppTiny2DCanvas is the Google App Engine implementation of the Tiny2D Canvas.

AppViewerCanvas is the Google App Engine implementation of the SVG Tiny Canvas. It implements ImageLoader interfaces

    /**
     * Returns a TinyBuffer for the given image URL or path.
     */
    public TinyBuffer createTinyBuffer(TinyString imgRef);
    /**
     * Creates a TinyBuffer which decodes the image stored in the specified
     * byte array, and at the specified offset and length.
     */
    public TinyBuffer createTinyBuffer(byte[] imageData, int imageOffset, int imageLength);
	

Basic demos

TinyLine for Google App Engine comes with a list of 2D examples in com.tinyline.appengine.demos.tinylinegae.tiny2d package.

They show how Java developers could use TinyLine and Google App Engine for the server side 2D graphics. See [R4]. Here are just some of them.

The Colors example shows how to draw with a single ARGB color.

TinyLine 2D examples

The Patterns example shows how to draw with patterns (bitmaps) colors. Also, it shows how to draw on different targets and store and restore the graphics state object.

The Gradients example shows how to draw with gradients colors.

TinyLine brings SVG server side rendering to Google App Engine platform.

This example demonstrates how to build a very basic SVG Thumbnail service using TinyLine and Google App Engine. See [R4].

Thumbnails are used to help users in recognizing and organizing images. Most up to date operating systems, desktop environments, image-organizing programs and search engines use thumbnails.

Thumbnail images are convenient way to show SVG files previews to users before they decide to download or play real SVG files.

Thumbnail images could be used for SVG listings, SVG file directories or for SVG Result Lists of search engines.

SVG drawing service fetches an SVG Tiny document from the Internet and exports a PNG image of the given size - a Thumbnail image.

SVG Thumbnail service

Function generateImage () in com.tinyline.appengine.demos.tinylinegae.TinyLineServlet does the job:

    /**
     * Generates an image that corresponds to the request.
     */
    private byte[] generateImage(HttpServletRequest request)
    		throws IOException, ServletException
    . . .

    /* Initialize SVG canvas */
    viewerCanvas = new AppViewerCanvas(w, h);
    /* Load the default SVG font if needed */
    if (SVGDocument.defaultFont == null)
    {
    	SVGDocument doc = viewerCanvas
    			.loadSVG("http://tinyline.com/svgt/download/guide/example/images/helvetica.svg");
    	SVGFontElem font = SVGDocument.getFont(doc,
    			SVG.VAL_DEFAULT_FONTFAMILY);
    	SVGDocument.defaultFont = font;
    }
    /* Load the SVGT image */
    viewerCanvas.goURL(urlStr);
    /* Return the corresponding PNG image */
    return imageWriter.generateImage(viewerCanvas.t2d.getTarget());
    

Here are the sample Thumbnail images in generated from the same SVG file:

Sample thumbnail images

TinyLine for Google App Engine helps to port other graphical Java applications to the Google App Engine platform.

It is particularly easy to port Java applications that are already using SVG. As example, we selected Barcode4J library [R6].
Barcode4J is has a wide range of barcodes it supports, it is written in Java and it has a modular design. Therefore, we expected to spend fewer hours to port it.

Here are some basic facts about barcodes. A barcode is an optical representation of data. Barcodes that present data in lines and spaces between them are called 1D barcodes. There are also 2D barcodes that represent data as squares, dots and other patterns.

Barcode example classes:

    com.tinyline.appengine.demos.barcode.BarcodeServlet
    com.tinyline.appengine.demos.barcode.SVGTinyCanvasProvider
    

BarcodeServlet is the main entry of the Barcode example application. It reads parameters from HTTP requests, creates all Barcode4J classes and sends HTTP responses.

SVGTinyCanvasProvider class implements SVG Tiny producer for Barcode4J library.

SVGTinyCanvasProvider supports the SVG Tiny requirements:

  • SVG Tiny does not support styling with CSS

  • SVG Tiny supports presentation attributes

  • SVG Tiny supports user units only, except for the 'width' and 'height' attributes on the outermost 'svg' element where percentage units are also supported.

For example, deviceFillRect () function that draws 'bars' uses presentation attributes and converts CSS units to user units by calling toTiny() function:

    public void deviceFillRect(double x, double y, double w, double h) 
    {
        Element el = createElement("rect");
        el.setAttribute("x", toTiny(x));
        el.setAttribute("y", toTiny(y));
        el.setAttribute("width", toTiny(w));
        el.setAttribute("height", toTiny(h));
        detailGroup.appendChild(el);
    }
    
Barcode example

Here are the sample barcodes generated by the Barcode example.
(Code128 barcode, PDF417 barcode and UPCA barcode)

Samples barcodes

SVGArk: Store and Search SVG

The more advanced example of using TinyLine and Google App Engine brings together full text search of SVG images and visual representation of search results by thumbnail images.

SVGArk [R5]. is an example application that shows how to:

  • Store and search SVG files;

  • Use thumbnail images for organizing SVG listings, SVG directories and SVG search result lists.

Here is the screen shot of SVG Ark home page that displays the recently added images.

SVGArk home

SVGArk demonstrates the technologies that could be used in search engines, image-organizing services and other cloud based applications.

Google App Engine provides several useful services based on Google infrastructure. Users service allows SVGArk to integrate with Google user accounts.

Users can use their Google accounts (Gmail) to sign in to SVGArk application. SVGArk can detect whether the current user has signed in, and can redirect the user to a sign-in page. While a user is signed in to the application, SVGArk can detect whether the current user is an administrator or not.

For example in the SVG file upload code (UploadServlet) we check if the current user is allowed to upload files.

    UserService userService = UserServiceFactory.getUserService();
    if (userService.isUserLoggedIn())
    {
        isAllowedUser = true;
        user = userService.getCurrentUser();
    }
    

The same way we check if the current user can edit information, etc… After the user logged in, he could see the latest added SVG images List.

A registered user can upload an SVG file via the webpage's Upload form.

SVGArk upload

The file upload servlet com.tinyline.appengine.demos.svgark.FileUploadServlet obtains the uploaded file data from a multipart form post using classes from the Apache Commons FileUpload Streaming API [R7], [R8] and stores it into the datastore.

The recently uploaded file becomes visible in My Images view:

SVGArk My Images

Currently, Google App Engine imposes a limit of 10 MB on file upload (and response size) as well as 1 MB limits on entity you can store in the datastore.

It should be fine for the purposes on SVGArk.

Google App Engine includes support for two standards Java API for the datastore: JDO and JPA. DataNuclueus Access Platform provides these interfaces, with an adapter for the Google App Engine datastore.

For the SVG Ark, we use the JDO API to store and retrieve SVG files from the datastore. JDO allows you to store Java objects in any datastore with a JDO-compliant adapter. We use Java annotations to tell JDO how to store Java objects in the datastore, and how to retrieve them from the datastore.

The following classes provide functions to store and retrieve SVG files from the datastore:

    com.tinyline.appengine.demos.svgark.SVGFile
    com.tinyline.appengine.demos.svgark.PMF
    com.tinyline.appengine.demos.svgark.SVGArk
    

SVGFile

This simple class defines basic properties for an SVG file: file owner, SVG raw data, date, sharing setting and etc... We annotate these fields with @Persistent to tell DataNucleus to store them as properties of objects in the Google App Engine datastore.

SVGFile class defines getters and setters for the properties. Using setters is the simplest way of ensuring that the JDO recognizes the object fields updates.

The class also defines a field named id, a String object annotated as @Persistent and @PrimaryKey. It pays a role of the entity key that is unique over all entities in Google App Engine datastore.

For the optimization purpose there is a field for the thumbnail image of the given SVG generated during the file upload.

    @PersistenceCapable(identityType = IdentityType.APPLICATION)
    public class SVGFile
    {
        @PrimaryKey
        private String id;

        /* File owner email address. */
        @Persistent
        private String fileOwner;

        /* File owner nickname how it appears on the screen */
        @Persistent
        private String fileOwnerNickname;

        @Persistent
        private String fileName; 
    

PMF

Each request that uses the datastore creates a new instance of the PersistenceManager class. It takes time to initialize PersistenceManagerFactory instance. Because we need only one instance, we can store it a static variable and use it for multiple requests.

    public final class PMF
    {
        private static final PersistenceManagerFactory pmfInstance =  
        JDOHelper.getPersistenceManagerFactory("transactions-optional");

        private PMF()
        {
        }

        public static PersistenceManagerFactory get()
        {
            return pmfInstance;
        }
    }
    

SVGArk

Now we can store SVGFile objects in the datastore

    public static int insertSVGFile(User user, String fileName,
        String contentType, String sharing, String caption,
        InputStream fileStream) throws IOException
    {
        svg = new SVGFile(fileid, fileOwner, fileOwnerNickname,
        fileName, fileSize, contentType, sharing, caption, data);

        PersistenceManager pm = PMF.get().getPersistenceManager();
        Transaction tx = pm.currentTransaction();
        try
        {
            tx.begin();
            pm.makePersistent(svg);
            tx.commit();
        }
        finally
        {
            if(tx.isActive())
            {
               tx.rollback();
            }
            pm.close();
        }
        return RC_OK;
    }
    

SVG files are search engines friendly.

Because SVG places text as XML character data, XML-compatible search engines could find text strings in SVG content similar to other XML documents [R9]. In SVGArk an SVG image caption plays the same role of context as a HTML portion surrounding a link to an image. The next component is hyperlinks. We just look for all xlink:href attributes values in a DOM.

There is a still lack of full text search support in the Google App Engine datastore. Google marked this issue as "Started", but there is no a release date yet. Therefore, we should check the alternative approaches.

One of the options is Apache Lucune project [R10] that provides Java-based indexing and search technology. Yet, there is problem with the index file that you could not save to file system. The workarounds like virtual file system are too complex for SVGArk.

We based SVGArk search on the example project [R11] that uses self-merge joins to do some basic full text search.

The approach is based on the two main points:

  • An Google App Engine datastore entity could be queried efficiently based on the list of properties via self merge-joint. See [R12] for details;

  • The (stemming) reduction of words towards their basic form should be done to allow inexact search.

There are some limitations of the SVGArk approach.

Each SVG file cannot be bigger than 1MB SVG. There is a workaround to use list of entities for one big SVG file or to use the Blobstore API, which allows apps to serve data objects that can be up to 2 gigabytes in size.

The number of search terms is limited (~ 5). An unlimited number of query terms could result in potentially costly self merge-join.

Bibliography

[R1] Google App Engine Java Overview http://code.google.com/appengine/docs/java/overview.html

[R2] Google Developer Days Brazil 2009 - Java Appengine http://www.slideshare.net/chanezon/google-developer-days-brazil-2009-java-appengine

[R3] TinyLine http://www.tinyline.com

[R4] TinyLine GAE: Basic Demos http://tinylinegae.appspot.com/

[R5] SVGArk: Store and Search SVG http://svgark.appspot.com/

[R6] Barcode4J library http://barcode4j.sourceforge.net

[R7] Apache Commons FileUpload Streaming API http://commons.apache.org/fileupload/streaming.html

[R8] Google File Service project http://code.google.com/p/google-file-service/

[R9] SVG Tiny 1.2 Specification, W3C Candiate Recommendation http://www.w3.org/TR/SVGMobile12/

[R10] Apache Lucune project http://lucene.apache.org/

[R11] Guestbook example with full text search http://code.google.com/p/guestbook-example-appengine-full-text-search

[R12] Google I/O 2009 - Building Scalable, Complex Apps on App Engine http://code.google.com/events/io/2009/sessions/BuildingScalableComplexApps.html