Open-Source Airport and Instrument Procedure Diagrams in SVG

Andy Clark
Sun Microsystems, Inc.
USA

Biography

Andy Clark was a member of the development team for IBM's XML4J parser implementation which was renamed "Xerces" and donated to the Apache Software Foundation to form the basis of the Apache XML Project. He was the chief architect of the Xerces Native Interface (XNI) and initial implementor of the Xerces2 parser. Although Andy no longer works on the Xerces project code directly, he remains active in the project in the areas of design and usage. He now works for Sun Microsystems, Inc. as a developer in the N1 Grid group.

K. Ari Krupnikov
Sun Microsystems, Inc.
USA

Biography

Originally a database developer, Ari Krupnikov has worked with XML technologies since 1999. With Sun since 2003, Ari was a research associate with the University of Edinburgh in Scotland between 2001 and 2003 where he worked on XML Schema databinding. He worked for a succession of startups prior to that. Ari learned to program on an IBM 370.

In addition to Private Pilot certificate with instrument rating, Ari holds an MSc form the University of Edinburgh.


Abstract


In our experience, SVG offers the easiest path to generating two-dimensional graphics. SVG does not live in a vacuum, though -- it usually augments web pages or printed material which are often generated from the same source within a set of workflow processes. A typical workflow includes data aggregation, XSL transformations, and specialized processing code. Tools that can combine existing and custom XML processing components to manipulate data and generate disparate outputs greatly simplify the life of application developers. In this paper we present STnG (Streaming Transformations and Glue) , a tool we developed for this purpose. As an example, we show how STnG simplifies generation of airport diagrams.


Table of Contents


Problem Domain and State of the Art
     Our Objective
Components of a Transformation
     STnG Overview
     Transformation Framework
Workflow
     Source Format Description
     Parsing
     XSL Transformations
     Coordinates
     Web Services Integration
Future Development
Conclusion
Footnotes
Bibliography

Problem Domain and State of the Art

Terminal Procedure Publications (TPPs) are designed to facilitate air traffic safety and channel the flow of air traffic around airports ( Figure 1 ). These procedures include predefined arrival and departure routes, weather minima and terrain clearance information for individual airports and aid pilots both in flight planning and during flight. TPPs make heavy use of graphics to help pilots maintain situational awareness. Aeronautical information can change quickly, so the International Civil Aeronautics Organization mandates a 28-day update cycle for TPPs. In addition, terminal procedure publications have traditionally been distributed on paper which are voluminous (e.g. ~10,000 pages for the continental US) and wasteful because of the frequent updates.

TPPs.png

Figure 1: Sample Airport diagram from FAA TPPs converted from PDF (left) and scanned paper diagram from Jeppesen (right)

In recent years, electronic TPPs became available as standalone publications or for use in in-flight navigation systems such as moving-map GPS (Global Positioning System) . In the United States, the FAA (Federal Aviation Administration) began publishing PDF versions of its TPPs ( Paragraph 48 ). The PDFs are in the public domain and cover the entire US and its possessions. As such, PDF files can only be viewed on screen or printed on paper; it is not possible to make any other use of them, such as to couple the graphics to a moving-map display driven from a GPS receiver, or incorporate them into a search system that would select only those procedures that a given aircraft is capable of following. A number of commercial packages allow broader use of the data, notably from Jeppesen ( Paragraph 48 ), but they utilize closed, proprietary formats that make for high barriers to entry for developers who wish to use them.

Raw aeronautical data are available from two US government sources in the form of relational database dumps. The US Geospatial Intelligence Agency compiles and makes available the DAFIF (Digital Aeronautical Flight Information File) which includes detailed information on close to 10,000 airports worldwide, as well as airways, radionavigation facilities, etc. The FAA distributes a similar dataset with more detail than DAFIF but it only covers the US. Produced by US federal agencies and paid for by US taxpayers, both are in the public domain, but their formats are not straightforward to use.

Our Objective

We wanted to produce TPPs in a re-usable electronic form that is open, searchable, indexable, printable and useful for flight planning. Potentially, this format could become part of an inflight navigation system. We chose DAFIF because of its global coverage and postulated XML as our data format, using SVG for graphics because it simplifies the generation of two-dimensional graphics from complex data. With these decisions in place, our task boiled down to transforming a set of text files into structured XML from which all output, including SVG diagrams, would be generated.

Components of a Transformation

A general outline of our pipeline is shown in Figure 2 . The first stage is a parser that reads DAFIF text files, finds subordinate records, and joins them into a single, hierarchical structure. The parser is very simple: it reports records as XML elements; record fields as element attributes; and subordinate records as child elements. An XSLT filter then reorganizes the data into a richer XML structure that is easier to process in subsequent stages. Next, a Java class performs trigonometric calculations to project geodetic coordinates in DAFIF to Cartesian coordinates which we use to generate SVG. A SOAP exchange then augments DAFIF data with aerial photography images. Finally, XSLT stylesheets generate HTML documents and SVG representations of the data which are transcoded into PNG bitmap equivalents using Apache Batik.

XDafif-overview.png

Figure 2: Flow diagram

Aside from parsing DAFIF data files and calculating geodetic projections, which are implemented as custom Java code, the pipeline uses existing components. However, most of these components were never designed to work together. Therefore, the technical challenge shifts to writing the "glue" that connects the components together. We've used this project as a testbed for our Streaming Transformations and Glue tool.

STnG Overview

STnG is a Java tool for transforming XML documents. It describes processing pipelines, expressed as XML documents, that make it easy to tie standard and custom components together to implement applications. Even though we generate SVG diagrams in this project, it should be noted that STnG was not specifically designed to produce SVG. It is a general tool for processing XML that helps break complex transformations into manageable chunks and, as such, is applicable to SVG generation.

A STnG pipeline consists of a reader followed by one or more filters which may fork the input into separate processing paths. Each path can be terminated with a serialization component to save the processing results. By default, STnG uses standard Java mechanisms for XML parsing and serialization, but the user can override the defaults and select specific parser or serializer classes. We use this feature in this project to read DAFIF data files. Also, a STnG pipeline doesn't have to be linear: a switch may direct different parts of the input document to different processors or output destinations. Switches use standard XPath patterns to address parts of the source document as another way to segment the source document for specific processing.

STnG documents are converted to Java source code which can be compiled amd used as a standalone application or as a component in a larger Java application. The STnG architecture is described in detail in Paragraph 48 .

Transformation Framework

The skeleton STnG transformation description is shown in Figure 3 .

<st:transform
     xmlns:st="http://ns.lib.aero/STnG"
     xmlns:java="http://ns.lib.aero/foreign/java.sun.com/"
     xmlns:batik="http://ns.lib.aero/foreign/xml.apache.org/batik">
  <st:target>
     <java:class package="aero.lib.dafif">XDafif</java:class>
  </st:target>

  <st:reader java:class="aero.lib.dafif.parsing.DafifReader"/>

  <st:switch>
    <st:case xpath="ARPT" recursive="true">

      <!-- break the stream into a series of independent documents -
           one for each airport  -->
      <st:chunker/>

      <!-- =============================================== -->
      <!-- Not shown: cleanup, projection and SOAP filters -->
      <!-- =============================================== -->

      <!-- generate SVG and PNG -->
      <st:tee>
	<st:xslt>
	  <st:src href="svg/arpt.svg.xslt"/>
	</st:xslt>
	<st:tee>
	  <batik:rasterizer media-type="image/png"
                            href="arpt/{@ARPT_IDENT}-diagram.png"/>
	</st:tee>
	<st:serializer href="arpt/{@ARPT_IDENT}-diagram.svg">
	  <st:output standalone="yes" indent="yes"/>
	</st:serializer>
      </st:tee>
      <!-- produce an HTML document -->
      <st:xslt>
	<st:src href="html/arpt-html.xslt"/>
      </st:xslt>
      <st:serializer href="arpt/{@ARPT_IDENT}.html">
	<st:output method="html" indent="yes"/>
      </st:serializer>
    </st:case>
  </st:switch>
</st:transform>
    

Figure 3: A skeleton pipeline

This pipeline starts with a reader (aero.lib.dafif.parsing.DafifReader) which reports the entire DAFIF dataset as a continuous XML document. This document is quite large: the set of tab-delimited DAFIF data consumes about 130Mb; the raw XML reported by the parser, about 500Mb. The size of the resulting document prevents us from using traditional XML methods to process the data. However, the individual airports do not depend on each other so we can use STnG's switch component to split the stream into smaller documents -- one per airport. For brevity, we've omitted most of the processing from Figure 3 and skip directly to output.

NOTE: We find it a common requirement to split large input documents into smaller chunks that can fit in memory for XSLT processing. This construct simplifies such splitting and was one of the original use cases for STnG.

For every airport, we wanted to generate multiple files: an XML document that we could reuse later, an HTML webpage, two SVG diagrams (icon and detailed view), and PNG equivalents of the SVGs. With the STnG pipeline description, tee elements duplicate the input stream so that different filters may be applied to different branches and results serialized to different files. Note different XSLTs applied to HTML and SVG branches in Figure 3 . By the time the document reaches these filters it has already been converted to a form that is easy to process into SVG (e.g. all geographic anchors show <x,y> coordinates relative to the airport's center) -- all that remains is to map elements like runway to svg:rect. Note also the conversion to PNG: the batik:rasterizer is a wrapper around the Apache Batik transcoder ( Paragraph 48 ) which takes SAX events that represent an SVG document and transcodes them to a raster image.

complete-screenshot.png

Figure 4: Finished HTML page

Workflow

Source Format Description

DAFIF consists of 74 semi-normalized tables, each of which ships as a tab-delimited text file. 37 of these are directly relevant to airports. The relationships between the most important ones are summarized in Figure 5 whereas Figure 6 shows records from two of the tables (airport and runway) that pertain to an airport.

relations.png

Figure 5: DAFIF file relationships relating to airports

/ARPT/ARPT.TXT:
db1.png
DAFIFT/ARPT/RWY.TXT:
db2.png

Figure 6: Source data fragments

We had to address three main issues before we could generate diagrams from the data files:

  1. Different records that describe the same airport reside in different files and must be joined.
  2. DAFIF records are only partially normalized. Some records contain fields that refer to distinct objects as a way to minimize table count; others split fields between separate records. For instance, due to physical field length limitations, textual remarks that are too long to fit in a field may span several records. XML has no such limitations, and introducing a richer, hierarchical structure makes the data easier to work with.
  3. DAFIF specifies coordinates in geodetic coordinates, <latitude,longitude>. To present a flat two-dimensional map, we need to project these to Cartesian coordinates, <x,y>. DAFIF provides coordinates in fractional degrees (e.g. <40.081944,-75.010558>) as well as in degrees, minutes and seconds (e.g. <N40045500,W075003801>). We use fractional degrees for computation.

The parser and subsequent filters perform these transformations.

Parsing

The parser produces output that closely matches the source tables' structure but in XML form. A sample result for an airport is shown in Figure 7 .

<ARPT ARPT_IDENT="US73343" NAME="NORTHEAST PHILADELPHIA" STATE_PROV="42" ...>
   <!-- ... -->
   <RWY ARPT_IDENT="US73343" HIGH_IDENT="24" LOW_IDENT="06" HIGH_HDG="241.0" .../>
   <RWY ARPT_IDENT="US73343" HIGH_IDENT="33" LOW_IDENT="15" HIGH_HDG="333.0" .../>
   <!-- ... -->
</ARPT>

Figure 7: parser output

NOTE: None of the intermediate results are ever serialized. The parser's output, as well as that of filters we describe below, only exists in the form of SAX callbacks. For convenience, however, we refer to "XML documents" as if real document content were flowing in our pipeline.

XSL Transformations

Even though STnG allows custom code to be used within the pipeline description, many tasks can be accomplished with standard XML utilities such as XSL Transformations (XSLT). As such, STnG provides a component to perform transformation using stylesheets.

The <st:xslt> element inserts an XSL transformer component into the pipeline. The component transforms SAX events flowing into it and transmits the transformation result down the pipeline as output. The stylesheet is specified using the <st:src> element as a child of the XSLT component as shown in Figure 8 . Alternatively, STnG allows authors to include XSLT stylesheets verbatim as children of <st:xslt> which is convenient for small stylesheets and encourages modularization.

  <st:xslt>
    <st:src href="xslt/ARPT-cleanup.xslt"/>
  </st:xslt>

Figure 8: STnG XSLT filter

Our application pipeline uses the XSLT component to clean the airport information generated by the DAFIF reader, as shown in Figure 9 .

<RWY IDENT="06/24" LENGTH="07000" RWY_WIDTH="00150" SURFACE="ASP" CYCLE_DATE="200011">
   <LE IDENT="06" HDG="61.0" SLOPE="0.0" TDZE="116.0" LAND_DIS="07000" TAKEOFF="07000">
      <coords>
         <WGS_LAT WGS_DLAT="40.077508">N40043903</WGS_LAT>
         <WGS_LONG WGS_DLONG="-75.020264">W075011295</WGS_LONG>
         <ELEV>111.0</ELEV>
      </coords>

Figure 9: Cleaned-up document - output from XSLT filter

Coordinates

We wrote a Java class to perform the trigonometry calculations required by our application. Since our diagrams cover small areas, this class uses a simple flat Earth projection centered on the ARP (Airport Reporting Point) . A switch initializes ARP coordinates for every airport that passes through it, and calculates distances from it to all other coords.

XDafif-detail.png

Figure 10: STnG switch flow diagram

<java:class-variable type="double" name="arpLat"/>
<java:class-variable type="double" name="arpLon"/>
<java:class-variable type="FlatEarthApproximation" name="fea">
  new FlatEarthApproximation (FlatEarthApproximation.UNITS_FEET)
</java:class-variable>

<st:switch>
  <st:case xpath="ARPT/coords" recursive="false">
    <st:content-filter>
      <sax:endElement>
        arpLat = Double.parseDouble (<st:value-of select="WGS_LAT/@WGS_DLAT"/>);
        arpLon = Double.parseDouble (<st:value-of select="WGS_LONG/@WGS_DLONG"/>);
        super.endElement (uri, localName, qName);
      </sax:endElement>
    </st:content-filter>
  </st:case>
  <st:case xpath="coords" recursive="false">
    <st:content-filter>
      <sax:endElement>
        double thisLat = Double.parseDouble (<st:value-of select="WGS_LAT/@WGS_DLAT"/>);
        double thisLon = Double.parseDouble (<st:value-of select="WGS_LONG/@WGS_DLONG"/>);
        AttributesImpl fromArp = new AttributesImpl ();
        fromArp.addAttribute ("", "north", "north", "CDATA", Double.toString(fea.distanceNorth(arpLat, thisLat)));
        fromArp.addAttribute ("", "east",  "east",  "CDATA", Double.toString(fea.distanceEast(arpLat, arpLon, thisLon)));
        super.startElement ("", "from-arp", "from-arp", fromArp);
        super.endElement ("", "from-arp", "from-arp");
        super.endElement (uri, localName, qName);
      </sax:endElement>
    </st:content-filter>
  </st:case>
  <st:default>
    <st:passthrough/>
  </st:default>
</st:switch>

Figure 11: STnG switch with embedded Java code

Note how Java code is embedded directly into the pipeline: STnG allows authors to specify event handlers for SAX callbacks inline. The callbacks have access to document state and can reference values in it through the <value-of> element. The XPath expression is evaluated at runtime and returns a string. Note also that that both cases override the endElement callback. In STnG's streaming environment, only nodes that already have passed through a filter are visible. Any legal XPath expression can be used in a value-of, but ones that address nodes that have not been seen yet will evaluate to empty strings, even if the nodes will eventually pass through the filter. By the time an element's end has been reported, its child elements have already registered with the switch.

This filter augments <coords> elements with an extra child, <from-arp>, as shown in Figure 12 .

  <coords>
     <WGS_LAT WGS_DLAT="40.077508">N40043903</WGS_LAT>
     <WGS_LONG WGS_DLONG="-75.020264">W075011295</WGS_LONG>
     <ELEV>111.0</ELEV>
     <from-arp north="-1615.999403681382" east="-2716.014381311095"/>
  </coords>

Web Services Integration

Microsoft TerraServer hosts high resolution aerial photographs USGS (United States Geological Survey) that cover the entire US. Images are stored as 200x200 pixel JPEG tiles. TerraServer exposes a SOAP API that provides, inter alia, a method to retrieve tile identifiers based on geodetic coordinates ( Paragraph 48 ). We use this method to augment our diagrams with areal photography.

SOAP is a messaging framework that is normally used through an API that maps underlying communication to method calls or function invocations on a remote server. However, underneath the programming interfaces, SOAP is simply a protocol for exchanging XML documents, typically over HTTP POST. The SOAP request document is relatively straightforward to construct and the XML result can be processed easily with XSLT, without resorting to specialized libraries. STnG provides the <st:http-post> component to send POST requests over HTTP.

  <st:xslt>
    <st:src href="xslt/terraserversoap/coords-GetAreaFromRect.xslt"/>
  </st:xslt>
  <st:http-post href="http://www.terraserver-usa.com/TerraService2.asmx">
    <st:header name="Content-Type">text/xml</st:header>
    <st:header name="SOAPAction">http://terraserver-usa.com/GetAreaFromRect</st:header>
  </st:http-post>
  <st:xslt>
    <st:src href="xslt/terraserversoap/GetAreaFromRect-svg.xslt"/>
  </st:xslt>

Figure 13: HTTP POST Pipeline Stage

An XSLT extracts coordinates from the stream and constructs a SOAP request, complete with a SOAP envelope and feeds it to the HTTP POST filter. The filter (configured with HTTP headers required for SOAP) sends it to the remote server, parses the server's response , and sends it down the pipeline. Another XSLT extracts tile information from this response and constructs an SVG that puts separate tiles into a single image. As part of the larger pipeline, this is performed for every airport in the source document.

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" 
               xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soap:Body>
    <ts:GetAreaFromRect xmlns:ts="http://terraserver-usa.com/">
      <ts:upperLeft>
        <ts:Lon>-75.020264</ts:Lon>
        <ts:Lat>40.090261</ts:Lat>
      </ts:upperLeft>
      <ts:lowerRight>
        <ts:Lon>-75.00155</ts:Lon>
        <ts:Lat>40.073903</ts:Lat>
      </ts:lowerRight>
      <ts:theme>1</ts:theme>
      <ts:scale>Scale8m</ts:scale>
    </ts:GetAreaFromRect>
  </soap:Body>
</soap:Envelope>

Figure 14: SOAP request with envelope

TerraServer SOAP messages are too verbose to include here, but TerraServer's excellent documentation pages contain complete examples ( Paragraph 48 ).

<svg:svg width="400" height="400"
         xmlns:svg="http://www.w3.org/2000/svg" 
         xmlns:xlink="http://www.w3.org/1999/xlink">
  <svg:image x="0" y="0" width="200" height="200" 
             xlink:href="http://terraserver-usa.net/tile.ashx?t=1&amp;s=13&amp;x=311&amp;y=2773&amp;z=18"/>
  <svg:image x="200" y="0" width="200" height="200" 
             xlink:href="http://terraserver-usa.net/tile.ashx?t=1&amp;s=13&amp;x=312&amp;y=2773&amp;z=18"/>
  <svg:image x="0" y="200" width="200" height="200" 
             xlink:href="http://terraserver-usa.net/tile.ashx?t=1&amp;s=13&amp;x=311&amp;y=2772&amp;z=18"/>
  <svg:image x="200" y="200" width="200" height="200" 
             xlink:href="http://terraserver-usa.net/tile.ashx?t=1&amp;s=13&amp;x=312&amp;y=2772&amp;z=18"/>
</svg:svg>

Figure 15: SVG generated from TerraServer SOAP response

The result is incorporated into the HTML webpage shown in Figure 4 .

Future Development

We continue developing both our terminal procedure publications and the STnG tool. Here are some of the directions we want to expand these projects in the future:

Conclusion

Sean McGrath, CTO of Propylon, wrote in 2002 ( Paragraph 48 ):

We think this is excellent advice.

In this project, neither our input nor our primary output was XML. However, using XML allowed us to take advantage of existing tools, reducing the amount of custom code in our application. The only code we had to write was reading DAFIF data and calculating geodetic projections, code that would have been required regardless of the implementation. An alternative would have been to write a Java or Python application that read the DAFIF data into objects, performed the necessary calculations, and programmatically generated HTML, SVG, and PNG output. This approach, when applied to transformations, is difficult to implement correctly and time-consuming to debug. This is because Object-Oriented decomposition favors objects with complex behavior and long lifecycles. In contrast, our pipeline is made up of small filters that are not bound to a particular object model beyond generic XML. This lead to an adaptive, data-driven design and made it easy to change formatting or create additional output targets easily.

STnG made this application quick to implement and easy to change. It provided the impedance matching between various XML components, partitioning the problem into smaller, more manageable chunks. This allowed us to concentrate our efforts on application workflow and avoid getting mired in debugging cycles. We believe that future enhancements to our application will also benefit from the STnG model.

Footnotes

  1. DAFIF represents an SQL database, and of course an RDBMS could easily perform the joins. However, constructing a database, populating it and using it only once for a transformation proved more of a hassle than benefit. The amount of code required exceeded what was needed for a custom parser.

  2. You can specify a reader for <st:http-post>; by default, STnG uses an XML reader.

Bibliography

[STnG]
http://www.mulberrytech.com/Extreme/Proceedings/html/2003/Krupnikov01/EML2003Krupnikov01-toc.html
[Batik]
http://xml.apache.org/batik/
[Jeppesen]
http://www.jeppesen.com/
[FAA TPP]
FAA terminal procedures in PDF format http://www.naco.faa.gov/index.asp?xml=naco/online/d_tpp
[GML]
Geography Markup Language from Open GIS Consortium http://www.opengis.org/
[TerraServer API]
API documentation for Terraserver SOAP API http://terraserver-usa.com/About/AboutTerraServiceAPI.htm#GetAreaFromRect. Cf. sample SOAP messages for this API at http://terraserver-usa.com/TerraService2.asmx?op=GetAreaFromRect
[XPipe]
http://xpipe.sourceforge.net/BinaryStuff/xpipeny.ppt

XHTML rendition created by gcapaper Web Publisher v2.0, © 2001-3 Schema Software Inc.