Generating SVG weather maps and meteorological graphs using Magics++

Using SVG for meteorological maps

Table of Contents

Advantages of using SVG
SVG as a possible format for operational webplots

The European Centre for Medium-Range Weather Forecasts (ECMWF) is an international organisation supported by 31 Member States and Co-operating States. ECMWF produces operational medium-range forecasts up to 15 days and extended range forecasts for up to six months in support of Member States' forecasting activities. A subset of the products is also made available to the wider meteorological community, i.e. WMO members, and to the commercial sector via the catalogue of products. As part of its commitment, ECMWF provides software to read their data and visualise it. The Magics++ graphics library, freely available under the Apache license, is developed by ECMWF to display numerical weather analysis and forecast fields and observational data on geographical maps. Various forms of non-geographical maps, e.g. box plots and bar charts, are also supported.

For over 25 years ECMWF has provided and maintained the MAGICS graphics library to plot meteorological data and statistics. Until recently this library was mainly written in Fortran and its implementation streamlined for printing with its main output format, PostScript. Over the last four years a new library, Magics++, has been developed to replace MAGICS. Using C++ and having a flexible object-oriented architecture makes Magics++ easier to maintain and extend for future requirements. New requirements have arisen mainly from a shift of focus from print-oriented to web-oriented graphics production.

As the black box diagram below shows, Magics++ understands many data formats common in meteorology, such as Grib, Bufr and NetCDF. Magics++ offers various forms of displays to visualise data: for example geographical plots with contours and shading at various projections or complex (statistical) graph plots, such as meteograms and climateograms. Magics++ has a flexible mechanism for laying out plots on pages. Magics++ with its flexible design allows output in different formats through so-called output drivers. SVG is one of these new drivers.

The MAGICS library only offered a Fortran interface which was not easy to maintain on web servers. Therefore it was decided to add an XML based format, called MagML to describe meteorological plots. These MagML files are processed by Magics++ to generate the desired plot. At ECMWF and other sites web services are developed which use MagML to describe plots requested by the users.

Magics++ is freely available under the Apache license from [1]. Magics++ is also the plotting engine behind the next release of Metview. Metview is a meteorological workstation developed at ECMWF.

As shown in Figure 1, Magics++ can handle many meteorological data file formats. These are some of the features users can use to visualise meteorological data:

  • High quality contouring

  • Automatic and user specified titles

  • Automatic legends

  • WMO observation plotting

  • Display of satellite data

  • Change between geographical projections

  • Allows users to control layout of plots on pages

With the redesign of Magics++ the list of graphics output formats was reviewed. SVG looked like an ideal candidate. The possibility of integrating SVG output directly into web pages and also to use as a possible print format were ideal for our purpose. An added bonus was the ability to add interactivity to the displays through JavaScript; the format's openness is also an advantage.

SVG as an XML based format is directly editable, scriptable and is widely supported by professional graphics tools. SVG is supported by many open source and commercial editing software packages (Adobe Illustrator, Inkscape). SVG allows the definition of Meta data as loose text description and/or as RDF to be used in semantic web applications. This and the fact that SVG files contain searchable strings make it possible to search and catalogue the graphics. SVG documents can also be analysed by a hosting web page through JavaScript to use its contents, such as the Meta data, for generating titles and menus.

Moving away from a paper oriented format also means that the aspect ratio of A4 paper need not to be used which can mean savings in empty space around the graphics on web pages and improves the layout of the page.

Magics++ design is modular and an output driver can be added without changes to the meteorological part of the library. Two drivers were added to investigate SVG support. The first one is based on the Cairo graphics library [2] and supports beside SVG also PostScript, PDF and PNG. The second driver is our own effort to write SVG code directly.

The Cairo library is used in many open source projects, such as Firefox and Inkscape, and with this its continued maintenance should be safe. The coding was fast and easy and quality of the output was good. However the generated SVG is static without animation and interactivity. There is no option to make the SVG plot scalable by excluding the size definition in the main SVG tag. The Cairo based output driver will remain part of Magics++ to make use of the good quality output in PostScript and PDF. For SVG output however, it was decided to develop our own output routine to make the output more flexible.

Implementation of our second own designed SVG output driver was also straight forward, with the exception of supporting fonts and raster satellite data which can be of large volume. To avoid the generation of large SVG files, satellite and other raster based data were written into PNGs through the libgd graphics library [3] and then imported as external files. The use of external files is unfortunate for the handling of plots. JavaScript code was embedded to enable the toggling between different layers of information and to enable a magnifier glass facility.

From our tests with various tools and browsers, we decided to avoid some parts of the SVG specification which were not widely implemented or which show different results on various tools. SMIL based animations, SVG fonts settings and patterns were some of the features we decided not to use. Colour gradients were not used, but this was more because technical plots make no use of this feature.

The speed of implementation was increased through the fact that new SVG code could be tested first in a plot before being coded in the output driver. Also was it helpful to be able to study other SVG code to learn from.

An important duty of ECMWF is to make forecast plots available to its customers through its web page. The plots displayed on the web are generated by Magics. Currently weather maps are available in three graphical file formats on the web. A GIF of the plot is displayed on the page while the same plot is also offered for download as a PostScript or PDF from a side menu. The reason for this is that users require the map in a vector format in order to zoom in and for high quality print-outs. PostScript is the format in which all maps are generated in the first place and the GIF and PDF are generated by using the convert command of the ImageMagick package. This setup has its drawbacks. Often the time to convert the PostScript into GIF and PDF takes longer than generating the map in the first place. Also extra storage space is needed to store all three formats.

We looked at various aspects of how we use graphics and came to following conclusions about SVG:

  • Printers: Even though a draft specification is available, SVGPrint has not been successfully implemented yet. Short tests with CUPS were undertaken to support SVG but failed. Without direct support for printers, conversions to other formats (discussed below) become more important. On a more positive note, Firefox 3 and Opera support the printing of SVG, even if embedded inside HTML.

  • Browsers: The SVG support in Firefox, Opera and Safari has improved immensely in the last years. As expected the first time load was noticeable. However the delay was shorter than what was anticipated. Resizing and interaction through scripts however were fast. The SVG graphics adjusted to the size available on the page and zoom functions such as the Ctrl-'-' and Ctrl-'+' from the browsers took effect on the SVG graphics. Unfortunately Internet Explorer supports SVG only through plug-ins with limited functionality.

  • Documents: Many professional layouting tools such as Adobe Illustrator and Scribus support SVG directly. Docbook supports inclusion of SVG, while Latex seems not to be able to handle SVG.

  • Screens: Only limited tests were done on screens but SVG showed its full potential as a scalable format. While PNGs at different resolutions need to be maintained to support different screen sizes, a single SVG was only needed to support standard and HD screens. In future we might be interested to take advantage of SVG's wide support by many Mobile phones.

  • File size: In comparison to PostScript, SVG's are smaller, but not by much. However SVG supports compression which makes an improvement of about 80%.

  • Conversions to other formats: There are many tools to convert images and convert from ImageMagick does a good job if a raster output is required. Other rasterizers are Batik (part of Apache) which is Java based, but was found to be slow. Smaller tools such as rsvg are faster with the same effect. Conversions of SVG's into other vector formats, such as PostScript (for printing) and PDF, was not as straight forward as we had hoped. The only tool we found useful was the editor Inkscape, used in command line mode with the option --export-ps. There might be the possibility of a XSLT based solution, but there was no time to follow up this idea.

From the beginning it became clear that a major weakness was the support of viewers and in web browsers. Even though over the last two years we have seen huge improvements in the support of SVG in many web browsers and software, Internet Explorer from Microsoft is an exception to these developments. This is important since our web statistics shows that 80% of our users use the Internet Explorer.

Also not all content can be handled in a single SVG file. Raster data is stored in external files to have better performance and faster speed downloading SVG content. The main products of the centre being contour maps and statistics plots this should not be a main concern, but Magics++ as a meteorology plotting library will support this output.

Another concern is the file size. In general slightly smaller than PostScript it is still much larger than PNG. For most SVG plots the majority of the data stored is for the coastlines. These would not be needed if SVGs would only be used as overlays as done by Google Maps.

The implementation of an SVG output driver within Magics++ has shown the potential the format has for a richer user experience. The quality of maps is very good and all graphical features of Magics++ could be supported.

Regarding the web plot production, there are no plans yet to replace the current output formats with SVG. This is due to the lack of SVG support by printers and web browsers, notably Internet Explorer.

SVG output might grow in importance with more dynamic web sites using AJAX and OGC web services.

Thanks goes to other members of the Graphics Section working on Magics++, Sylvie Lamy-Thepaut, Iain Russell, Sandor Kertesz and Fernando Ii. Thanks also goes to the web group at ECMWF supporting these studies.