Using SVG for graphically rich 2D content in mobile 3D games

Keywords: 3D, mobile gaming, texture generation, SVG

Kiia Kallio
Software Engineer
Fathammer Oy
Helsinki
Finland
kkkallio@fathammer.com
http://www.fathammer.com

Biography

Kiia Kallio works as a software engineer in Fathammer, a company creating 3D game technology for mobile devices. He has long experience in the computer game industry, both as a programmer and a graphic designer. Prior to his career at Fathammer, he worked at Remedy Entertainment in a team that made "Max Payne", an award-winning 3D action game.


Abstract


Traditionally, textures and other 2D content in 3D games have been based on bitmaps. In a mobile 3D gaming environment the storage space limitations pose new challenges for the game content. Although processing power and runtime memory limitations allow visual quality comparable to Sony Playstation, the storage space is reduced to a fraction - instead of a CD-ROM the game should fit into a few megabyte memory cartridge.

The benefits of vector graphics, independency of resolution and small storage space requirements, have not been generally considered significant from the point of view of traditional 3D game developers. In mobile 3D gaming, these make vector graphics an attractive alternative.

The biggest problem with vector formats is the limited artistic expression. Vector graphics tend to produce images that are not suitable for rendering natural materials - things that are essential to rich 3D environments. However, SVG has a mechanism that can be used to get around these limitations: filter effects. SVG filters are powerful in the hands of a programmer who knows image processing and SVG specification inside out. The lack of good tools for artists is a problem though. SVG Filter Editor, included with the X-Forge™ 3D game engine, is a tool that enables artists to construct the filters visually without writing SVG code, but still allowing access to all the filtering features of SVG specification.

The SVG file format is not optimal for tight storage limitations in one aspect: as a text based format it takes more space than necessary. Also the XML parser adds up to the size of the executable, and DOM tree has quite large runtime memory requirements. This is solved in X-Forge™ by moving the parsing process from the load time to the content creation tool chain by using a pre-parsed binary format.


Table of Contents


Problem background
     Mobile 3D gaming environment
     Mobile vs. console and desktop games
     Experiences from mobile 3D games
Solution
     Requirements
     Evaluation of alternative technologies
     SVG and the requirements
Components of the solution
     Considering an external toolkit
     XML parser and DOM handling
     Feature subset
     Content creation
     Summary
Conclusions
     Images
     Visual quality
     File sizes
     Problem areas
Future developments
Bibliography

Problem background

Mobile 3D gaming environment

Today's high-end mobile devices, notably PDA's running Pocket PC, Pocket Linux or Palm 5 operating systems, and cell phones running Microsoft Smartphone or Symbian operating system (for example Nokia Series 60 phones such as N-Gage or Sony Ericsson UIQ phones such as P800), are becoming a new platform for 3D games.

These devices typically run on some variant of ARM processor, with clock speed of 100-400 MHz. The amount of memory usually varies between 4 MB and 32 MB, but is restricted by the fact that the same memory is used also for storage, not just as run-time memory. The devices usually have colour displays, with resolutions ranging from 176x208 to 480x320. The display hardware is typically very straightforward, without any display acceleration components.

Within these limitations it is still possible to create 3D games that compete with the quality of console and PC games of the 1990's. Development in tight mobile environment requires however different approach from the methods used in desktop or console systems.

Mobile vs. console and desktop games

The form factors of mobile devices along with the varying usage situations give some new challenges to the game design. On devices primarily meant to be used as phones, the controls for playing games may be far from optimal. Also the size of the display and its poor resolution can be limitations. From the technical point of view however, the lack of memory and storage space is the biggest problem.

In the beginning of 1990's, CD-ROM entered the gaming market and changed a lot in console and desktop gaming. Prior to CD-ROM era, games were distributed on diskettes or game cartridges, with sizes ranging from few hundred kilobytes to few megabytes. Although some games were distributed as a pile of diskettes, arrival of CD-ROM media allowed much more massive games when the size of the content was no more a limitation: one CD could store a massive amount of 700 megabytes of data. One of the most successful systems having a CD-ROM was Sony Playstation, released in 1994. On desktop PC's, CD-ROM quickly became the standard media for distributing games in mid-1990's as well.

In respect to 3D rendering quality, today's mobile devices can rival with Playstation. In the amount of content however, the situation is even worse than it was in the times before CD-ROM. Most mobile devices for instance don't have any changeable media whatsoever. In the best case there are some changeable cartridges of a few megabytes, but for anything as massive as CD-ROM we still have to wait a few years. Typically the games are crammed to the device memory along with operating system, all the other applications and data, and there should be some memory left for running the applications as well. For devices with as little as 4 megabytes of memory this is clearly a problem. For comparison: Playstation has total of 3.5 MB of runtime memory (2 MB main memory, 1 MB video memory, 0.5 MB audio memory) devoted for one single game at a time, and the whole CD-ROM for storage. [PSX] Clearly mobile games require some clever strategies to overcome the situation.

Experiences from mobile 3D games

The experience gathered from 3D mobile games this far suggests that biggest portion of the content in a 3D game is still 2D content. This consists of textures for the 3D scenes and bitmap graphics for the user interfaces.

The following figures are from "Stunt Run", a game that was made at Fathammer and bundled with Sony Ericsson P800 phone. They demonstrate the content sizes in a 3D mobile game:

Content sizes of "Stunt Run", a 3D car racing game:
uncompressed zlib compressed compression ratio
Total size of content 1 140 137 bytes 577 999 bytes 50.70 %
3D content 340 318 bytes 161 116 bytes 47.34 %
2D content 573 820 bytes 291 653 bytes 50.83 %
Audio content 188 358 bytes 115 693 bytes 61.42 %
Misc content 38 916 bytes 10 819 bytes 27.80 %

Table 1

In the final game, all content was packed to a single zlib-compressed package file. For testing purposes, each content category was packed to similar file. Table 1 shows the compression results.

Solution

The natural solution for the problem with storage space was to seek methods for defining 2D graphics more efficiently.

Bitmap formats have several benefits for this kind of use, as they are the standard way of doing things. Artist are familiar with content creation tools and techniques, and with enough resolution bitmaps allow unlimited artistic expression, thus being suitable for use both as textures and user interface components. Bitmap handling is fast and straightforward also on the programming side, as bitmaps are thoroughly supported by the graphics engine in any case. Bitmaps can be used for UI, textures, fonts etc. with no extra code bloat.

However, to fit the tight storage requirements, bitmap graphics should be scaled to low resolution and compressed heavily. As visual quality is one of the main concerns of a gaming application, this is unacceptable.

The mobile space is also populated by various devices with different screen sizes and resolutions. Bitmap graphics have to be prepared in suitable resolution for each individual device.

Requirements

A better solution than bitmap graphics was clearly needed. The requirements for the solution can be summarized as following:

Evaluation of alternative technologies

After evaluating various solutions, SVG was chosen to be the technology to use. The competing technologies were other vector formats, such as Macromedia Flash, and various algorithmic texture generation methods. However, texture generation methods usually provide rather monotonic textures with no possibility to add features defined by the artist: for instance natural phenomena such as stone or wood grain is easy to model, but for instance a stone wall with a door and windows is not. Vector formats on the other hand don't have the functionality required for texture creation.

When used as textures, vector images still have to be rendered into bitmaps, but for UI components it is possible to render the vector image directly to the screen, thus saving the memory required for a bitmap.

SVG and the requirements

Although SVG doesn't completely fit all the requirements, it gets so close that it is possible to fill the missing gaps.

As a text based format, SVG consumes more storage space than necessary. Although compression can solve this partially, the results are not as good as they could be. SVG is not really a lightweight technology either: the SVG specification is huge [SVG] , and an XML parser and DOM add to the size of the implementation. Although lighter profiles such as SVG Tiny [SVGMobile] do exist, they don't provide enough functionality when considering the visual quality. Rendering vector graphics directly to the display surface takes typically only a little amount of memory, but some operations, such as filtering or complex nesting, require more. Also DOM implementations are typically hungry for memory. The tools for SVG content creation also have some space for improvement, especially on the filter creation side.

However, there are also extra benefits of choosing SVG: content is scalable, which allows using the same content on a variety of devices with different screen resolutions. SVG can be used also for animation, which provides great size optimisations when compared to video formats.

Components of the solution

The X-Forge™ game engine is a C++ based multi-platform mobile 3D game engine. The engine isolates the developers from the peculiarities of the operating system of the device, and allows them to concentrate on the most important: the game itself. X-Forge™ is not just 3D rendering engine, but also provides other components such as audio and networking, and high-level functionality such as collision detection and physics [X-Forge] .

Considering an external toolkit

On some platforms, the engine is distributed as a part of the operating system, but in most cases the engine is linked to the game application. The binary size of the engine, depending on which features the developer has chosen to include, ranges typically from 200 to 500 kilobytes. The smallest readily available mobile SVG implementations are in the class of 200-700 kilobytes. Adding one of those would have doubled the binary size. They are not necessarily multiplatform products, and usually lack some features - such as filters - that are vital for good visual quality.

Integration of an external SVG library to the engine would also have meant duplicate code: the engine already has a solid application framework, a math library, code for bitmap image manipulation, an efficient antialiased polygon rendering algorithm etc. In order to avoid code bloat, external SVG library would have required heavy modifications to be able to use as much code from the X-Forge™ framework as possible.

Therefore, a decision to develop an internal SVG rendering engine was made.

XML parser and DOM handling

As an XML based format SVG also requires an XML parser and DOM tree. XML has several benefits as a file format, but for defining content of a mass-market product, a 3D game, these are not exactly required. Game developers typically place more effort in order to protect the game content from tampering than allow open access to it. XML files tend to grow rather large as well - an effect we have tried to avoid in the first place.

To reduce the size of the implementation and run time memory requirements, and also to boost the file parsing performance, we decided to move the XML parsing and DOM handling offline, and use a binary format for the graphics in the engine. This has been done for XML before, for instance WAP uses WBXML (WAP Binary XML) [WBXML] , and Plazmic uses a binary SVG format in their media engine [Plazmic] . Another benefit from using a custom binary format is increased protection of intellectual property, as unauthorized use of the game content is not as easy as with plain SVG.

However, to also allow direct SVG use in the future applications, the binary format was designed in a way that it follows SVG specification closely. It can be described as a compiled format of a SVG DOM that can be executed directly by a virtual machine. Although the compiler is currently an offline tool written in Java, integrating a C++ version along with an XML parser to X-Forge™ game engine is all that is required for direct support of SVG in the future.

Feature subset

SVG - when using the full profile - provides all the functionality for the artists to allow unlimited artistic expression. The biggest improvement in this respect when comparing SVG to other vector formats is the filter support. In addition to the usual drop shadow effects, filters can be used for creating a wide variety of effects for texture images. As figures 2-6 demonstrate, SVG images can have the full visual quality of bitmap images.

SVG profiles for mobile use - SVG Tiny and SVG Basic [SVGMobile] - are not designed according to the requirements for visual quality. Features such as filtering are considered secondary, and more attention is placed on multimedia and interactivity features. Therefore, the subset of SVG supported by X-Forge™ 3D Game Engine is not based on current mobile profiles, although features required for SVG Tiny are included in the subset definition.

Because SVG is used as a replacement for bitmap graphics, features like scripting, interactivity, text editing capabilities etc. can be provided through the game application in a more efficient manner. Inclusion of these features was not seen as a necessity. X-Forge™ is focused on 3D games, and thus doesn't require the functionality required by web or cartography applications.

Content creation

A variety of tools already exist for SVG creation. However, all the tools we have evaluated have neglected the power of SVG filters, and support for filters is mediocre at best. Existing tools either require writing filter descriptions by hand, or allow only slight modifications of existing filter descriptions. Artists need tools where filters can be edited visually, without thorough understanding of SVG source code.

To overcome this limitation, the toolset for creating SVG content to be used in mobile 3D games required an addition, a SVG filter editor. As X-Forge™ is targeted for mobile platforms, not desktop environment, it was not possible to use X-Forge™ as the platform for tool development. Instead, the tool was written in Java using Batik SVG toolkit.

SVG filter editor is a stand-alone application that can be used along with a vector graphics package that has SVG support. At least Adobe Illustrator is suitable for this, as importing SVG filters from external files is easy and artists are usually familiar with the software.

The emphasis in the tool development was placed on two main principles: the tool should allow unlimited access to SVG filter functionality, while being a tool for artists, not programmers. As the typical user of the software is a professional game artist, the idea of the user interface was not to hide functionality from incompetent users, but to provide a visual representation of the SVG filter that is can be understood and manipulated without writing a single line of SVG code.

As one SVG document can contain several filter definitions, the tool allows previews of these either one by one or all combined in the finished image. There can be several previews of different filters visible at the same time, but only one filter is the current active filter, the one that is being edited. A filter is represented in a layer graph, where the filter effects are stacked on top of each other and connected with pipes. The pipes can be dragged around in order to rearrange the connections between the filter effects. Also the order of the filter effects can be changed by dragging them.

Screenshot.png

Figure 1: Screenshot of SVG Filter Editor

Summary

In conclusion, the full SVG solution of X-Forge™ consists of following components:

Components of SVG solution of X-Forge™:
Offline tool chain
Vector editing package
  • Any vector editing package with SVG support
SVG Filter Editor
  • Tool for editing SVG filters
SVG compiler
  • Two versions: a command line compiler and an exporter in SVG Filter Editor
  • The implementation can be modified so that this is hosted on the mobile device
XML Parser
  • Only a minimal parser could be hosted on the mobile device.
DOM Translator
  • Translates the SVG DOM to binarized format.
Mobile implementation
Vector Graphics Processor
  • SVG byte code virtual machine
Primitive Translator
  • Translation of SVG drawing commands to vector primitives
Primitive Draw
  • Vector drawing
Surface Toolkit
  • Bitmap handling

Table 2

Conclusions

The SVG solution of X-Forge™ is approaching completion, and some test data is already available. Following images demonstrate the results that can be achieved by using SVG in mobile games.

Images

Following images were created using SVG Filter Editor and a vector-editing package. Although SVG support of X-Forge™ game engine is not yet in production use, most of the images are based on real production material.

Marble.png

Figure 2: Marble

Marble is a procedural texture created by using only SVG filters. It contains no drawn geometry, except the rectangle to which the filter is applied. The texture is looping in both x and y direction. This is achieved by using the stitchTiles-attribute of feTurbulence filter primitive, and feTile filter primitive.

Wall.png

Figure 3: Wall

A wall from "Unfinished Business", a 3D shooter game example. The texture is looping in x direction.

Wall_with_hole.png

Figure 4: Wall with a hole

A more complex wall texture from "Unfinished Business". The texture is looping in x direction, and can be aligned seamlessly to the wall texture of image 3.

Ceiling.png

Figure 5: Ceiling

A ceiling texture from "Unfinished Business". The texture is looping in x and y directions.

Menu.png

Figure 6: Menu background

Background of a menu from "Stuntcar Extreme" car racing game. The image is mostly based on vector art and uses only one filter for shading.

Visual quality

As the images above demonstrate, SVG can be used for creating images that are indistinguishable from bitmap images.

Figure 2 , "Marble" demonstrates results that are similar to procedural texture generation software. Figure 3 , Figure 4 and Figure 5 however demonstrate the real power of SVG in texture creation: combination of vector art and filter effects can produce results that are not possible by using only vector images or procedural texture generation.

Figure 6 demonstrates that SVG is suitable also for menu graphics and other 2D content in addition to textures.

File sizes

The savings in storage space are even more impressive. An example SVG file in 256x256 pixel size takes 10 kilobytes of disk space. The same image, converted to JPEG format takes 30 kilobytes when saved with quality where compression artefacts are not disturbingly visible. When the SVG file is converted to binary format used in X-Forge™, the file size drops even lower, to modest 3 kilobytes. Sizes this small can be achieved in JPEG format only by compressing the image beyond recognition with the highest compression settings. Even after this, the file can be compressed with zlib, typically bringing results of 50% compression.

Table 3 demonstrates the file sizes of example images:

Image: Size as SVG Size as binary Size as compressed binary Size as PNG Size as JPEG
Marble 2 802 bytes 935 bytes 487 bytes 63 887 bytes 14 701 bytes
Wall 4 993 bytes 1 930 bytes 873 bytes 98 807 bytes 24 679 bytes
Wall with a hole 10 027 bytes 3 963 bytes 1 623 bytes 97 651 bytes 31 415 bytes
Ceiling 8 624 bytes 3 355 bytes 950 bytes 80 762 bytes 18 894 bytes
Menu background 19 480 bytes 11 310 bytes 6 507 bytes 63 601 bytes 23 304 bytes

Table 3

JPEG compression of the example images was made with lowest possible quality where visual artefacts were not disturbing. In the application used for compression, quality was selected in the range of 0 to 10; typically the values were between 6 and 8.

When using SVG images, another significant issue with file sizes has to be kept in mind: SVG files can reference other image files. For instance in image 6, Menu background, the single most complex vector component is the Stuntcar Extreme logo. This can be saved to a separate file, and be referenced from all the images that contain the logo, thus reducing the total amount of data considerably. In practice this may cause some problems though, as content creation tools don't necessarily support this approach very well.

SVG Filter Editor, although still having room for improvement, has been proven to be a solid tool for creating SVG filters without going into the complexities of writing SVG code. As with all tools in computer graphics, efficient usage of the tool still requires good understanding of the underlying principles, SVG filter model and document structure.

Problem areas

The speed of image construction is a concern. Heavy use of SVG filters easily creates images that take long to render. Therefore, artists need to be aware of methods for reducing the execution times: some filters - such as lighting effects - are heavier than others, some effects can be performed in a lower resolution by adjusting the filter kernel size etc. Complex filter effects can also require surprising amounts of runtime memory for intermediate bitmaps. Currently the tool chain doesn't provide means for artist to measure these properties, although rendering times in the SVG Filter Editor provide some hints. To remedy the situation, a tool that can load binary SVG files and measure the rendering speed and memory consumption using the actual X-Forge™ game engine is required.

Future developments

The problems with storage space of mobile gaming devices are likely to become less severe as the storage capacities grow. However, as the processing power, display resolutions and device memory sizes grow as well, the games require even more textures and other 2D content. Storage solutions that can handle that data in plain bitmap formats are not likely to appear on the smallest devices: optical disks or hard drives require mechanical parts and consume space and battery power, and thus will appear only on the higher-end devices.

Games played on mobile phones - or devices that are combinations of a game console and a phone - can also have connectivity features. These can be used for instance for downloading game content, but as the connection speeds are quite modest and the users pay for the amount of transferred data, it is good service to provide the data in as compact form as possible, even if the storage is not a problem. If the transfer times can be reduced to minimum with small content sizes, a solution where the data is stored only on a server and fed to the clients at request is possible to implement as well.

In wider scope, the textures created in SVG prove that SVG can be used to achieve effects that have been associated only with bitmap graphics. Since SVG is scalable and has small file size, there is a wealth of other applications in addition to mobile 3D gaming that benefit from switching from bitmaps to SVG.

Bibliography

[PSX]
Sony PlayStation/PSOne technical specifications, Available at http://www.playstation.com.
[SVG]
Scalable Vector Graphics (SVG) 1.0 Specification, J. Ferraiolo, editor, W3C Recommendation, 4 September 2001. Available at http://www.w3.org/TR/SVG/.
[SVGMobile]
Mobile SVG Profiles: SVG Tiny and SVG Basic, T. Capin, editor, W3C Recommendation, 14 January 2003. Available at http://www.w3.org/TR/SVGMobile/.
[X-Forge]
X-Forge product brochure, Available at http://www.fathammer.com.
[WBXML]
WAP Binary XML Content Format, B. Martin, B. Jano, editors, W3C Note, 24 June 1999. Available at http://www.w3.org/TR/wbxml/.
[Plazmic]
The Suitability of SVG for Deploying Wireless Applications, J. Hayman, Conference proceedings of SVG Open 2002. Available at http://www.svgopen.org/papers/2002/hayman__suitability_of_svg_for_wireless_applications/.

XHTML rendition created by gcapaper Web Publisher v2.0, © 2001-3 Schema Software Inc.