Real Time Flood Warning Decision Support Systems Utilizing Client-Side XML/XSLT/SVG

For the SVG Open 2005 conference

Keywords: Flood, SVG, XSLT

Bruce Rindahl
Project Engineer
Leonard Rice Engineers, Inc.
2000 Clay St. Suite 300
Denver
CO
USA
rindahl@lrcwe.com

Biography

Mr. Rindahl is a Colorado professional engineer with 25 years experience in water resources. He holds a B.S. in Civil Engineering and a M.S. in Appliced Mathematics. In the last four years he has been developing real-time flood warning decision aids using XML/XSLT/SVG technology. These decision aids include rainfall totals, streamflow values, radar images, weather readings, advanced meterological models, and aerial photographs. The displays integrate data from several sources and include syncronized animation of all data sources. In addition, he hes developed programs and interfaces to convert GIS data to XML format for SVG displays.


Abstract


Real-time flood warning Decision Support Systems (DSS) have been developed in the past several years to aid emergency managers to respond quickly to rainfall and flooding events. In the past four years, the Urban Drainage and Flood District (UDFCD) in Denver Colorado have developed several web based DSS interfaces using XML/XSLT/SVG technology. These displays allow real-time analysis of radar images, thunderstorms, rainfall, river stage, inundation areas, and future rainfall potential. This paper will discuss the specific techniques developed for these displays.

The UDFCD has been operating a real-time flood warning system since the early 1980's using radio technology to send rainfall and stage information to a computer base station. The commercial software operating the base station has evolved over the years to include detailed displays of hydrographs, maps, and summary tables of the flood-warning system. Nevertheless, the user had to be directly connected to the base station either directly at the machine or via a dial-in connection. In 1994 the first web based displays were created by delivering pre-formatted tables or GIF images via an early web server program. Since that time the demand for web-based information from the UDFCD flood-warning system has increased dramatically along with the types of displays and information available. The price for this demand has been an ever increasing load on both the processor and bandwidth of the UDFCD base station.

Starting in 2002, the UDFCD requested we investigate potential new technology available to display real-time decision aids in a web display while reducing the load on the base station. Several promising applications were developed using client-side XML/XSLT to create a full graphical interface using SVG technology. The displays allowed the integration of GIS information for spatial reference along with the real-time rainfall and stage information. Programs were developed on the base station to deliver any flood-warning system data in XML format via the existing web server software. XSLT stylesheets were then created to transform the XML data from the base station into SVG using exclusive client-side processing. This greatly reduced the bandwidth and processing load on the base station. In addition, additional data sources were converted to XML allowing an integrated DSS for emergency managers. The additional data sources included radar images, aerial photographs, projected thunderstorm movement, hydro-meteorological data, and post-storm rainfall observer readings. GIS data is also integrated in the display for additional reference for the user. The use of SVG allowed not only the common graphical display of this information, but the ability to animate the information in a synchronized fashion. The XML data stream also allowed us to develop prediction models using spreadsheets with an automatic update for current data.


Table of Contents


1. Background
     1.1 Images
2. Approach
     2.1 Images
3. Other Displays
     3.1 Images
4. Compatibility with other Platforms
Acknowledgements

1. Background

The Urban Drainage and Flood Control District (UDFCD) in Denver, Colorado has been operating a real-time flood warning system since the early 1980's using a protocol called ALERT to collect hydrological information to a computer base station. ALERT is an acronym for Automated Local Evaluation in Real Time, which is a method of using remote sensors in the field to transmit environmental data to a central computer in real time. This standard was developed in the 1970's by the National Weather Service and has been used by the National Weather Service, U.S. Geological Survey, Army Corps of Engineers, Bureau of Reclamation, numerous state and local agencies, and international organizations.

The ALERT standard is event driven. This means the only data transmission is when a raingage bucket trips or a predefined stage change is detected by a stage sensor. Thus most of the time the base station is in low demand recording small amounts of ALERT data. However, when significant storm events occur, hundreds of sensors can be transmitting almost at once and causing CPU load on the base station. During these significant events is also exactly when users are demanding the most information on the rainfall amounts and potential flooding. The real time nature of this system is quite remarkable - the time between a remote raingage bucket tip and an update to the appropriate display is approximately one second. A commercial software package called NovaStar operating the base station has evolved over the years to include detailed displays of hydrographs, maps, and summary tables of the flood-warning system. However, the user had to be directly connected to the base station either directly at the machine or via a dial-in connection. In 1994 the first web based displays were created by delivering pre-formatted tables or GIF images via an early web server program. Since that time the demand for web-based information from the UDFCD flood-warning system has increased dramatically along with the types of displays and information available. The price for this demand has been an ever increasing load on both the processor and bandwidth of the UDFCD base station. Figure-1 shows a plot of the total ALERT readings received by the UDFCD base station in readings per hour compared with the actual data displays served (not web hits) for the month of June, 2004. Notice the significant rainfall events of June 8 and June 27 along with the high data requests during that time.

Beginning in 2002, the UDFCD was interested in investigating new methods to display and analyze ALERT data via the standard web interfaces available. Several graphical technologies were investigated including custom Java applications, Flash displays and SVG. After review of the cost and requirements of the project, SVG was chosen for further development. It was felt that custom Java applications would be too costly and project specific while Flash could not create constantly changing displays on the fly with no intermediate user interaction. SVG had the advantages of an open standard, could be created on the fly with XML/XSLT transformations from the actual real time data, and had an existing free plug-in available for a standard browser.

1.1 Images

RealTime-fig1.svg

Comparision of the hourly http data requests in Kilobytes per hour with the actual real-time ALERT readings logged each hour

Figure 1: ALERT Server Usage (June 2004)

2. Approach

In order to utilize the capability of SVG for display of real-time flood warning information, the ALERT data was first converted to XML format. This was a relatively easy task as the existing NovaStar software package controlling the ALERT base station had several utilities to control the output format of data requests. In this manner, the rainfall, weather, and flood stage values could be served via http as xml data requests. Other agencies including the National Weather Service (NWS) and the National Oceanic and Atmospheric Administration's (NOAA) Forecast Systems Laboratory (FSL) have provided some of their data in XML format. Finally, other web-based data sources are converted via shell scripts from a delimited format to XML via the sed program.

Once the data was available in a standard XML format the next task was to develop XSLT stylesheets to convert the data into an SVG display. Several specific stylesheets were developed for individual SVG displays that reflected the actual data received from the ALERT base station. The format of these stylesheets was kept as generic as possible to allow them to be re-used for different applications. For example the dates and times module from EXSLT.org was utilized in several stylesheets to compute both the time of data transformation and data synchronization. In addition, since many of the data sources reported the location of the data readings in latitude/longitude pairs, a module was developed to convert the latitude and longitude values into either UTM coordinates or state plane values as the need arose.

Finally, a technique was developed to create and display the real-time data in a SVG enabled web browser. Since the client was utilizing the Microsoft Internet Explorer (IE) web browser, the first priority was to create displays for that platform. IE cannot directly display inline SVG created by XSLT stylesheets so a different approach was utilized. This involved creating a "blank" SVG file with placeholders (g tags) for the actual data. The XSLT stylesheets create a HTML file with the the blank SVG files referenced using embed tags. In addition, a javascript function was written that directly manipulates the SVG DOM and adds the appropriate display required. This function was called via the onload parameter of the blank SVG document.

To give a detailed description of this technique, a portion of the real-time rainfall display will be described. The "blank" SVG file, rainfall.svg has the following basic structure:

<svg id="map" width="800" height="600" viewBox="2930000 -1900000 420000 360000" onload="init(evt)" onzoom="rezoom(evt)">
	.
	.
	.
<g id="raingages" font-weight="bold" font-size="8000" text-anchor="middle" visibility="hidden">
	<g id="rain_5min" >
	</g>
	<g id="rain_10min" >
	</g>
	<g id="rain_30min" >
	</g>
	<g id="rain_1hour" visibility="visible" >
	</g>
	<g id="rain_6hour" >
	</g>
	<g id="rain_24hour" >
	</g>
</g>
	.
	.
</svg>

Example 1: SVG file structure

The first thing to note is the viewbox values for the SVG file. All of the displays are in actual GIS values (the y values are negated). This permits both the coordination of different SVG files to be aligned and the addition of new elements to be precisely located. The rainfall display includes a shaded relief background image, SVG vector files for streams, roads, lakes, and counties. Next the various "g" tags with unique id values are placed where the rainfall readings are required. All the five minute rainfall readings are added to the "rain_5min" tag element and the other readings are added in their respective locations. This allows all the readings to be switched on and off as desired.

The init function called by the onload event is created by the XSLT stylesheets and the XML data.

For example a portion of the XML rainfall data file reads:

<gage_rain id="1010" min_5="0.04" min_10="0.08" min_15="0.16" min_30="0.75" hour_1="2.99"
hour_3="2.99" hour_6="2.99" hour_24="2.99" shef_id="DVWC2" name="Denver West" lat="039:43:57"
long="105:09:07" />

Example 2: XML data file structure

This is the real-time reading of gage 1010 during a storm event showing the actual rainfall amounts at the time in inches.

In the HTML file created by the XSLT stylesheet, a javascript function is created called "_new_gage":

function _new_gage(id,x,y,color,val) {
 var root = document.embeds[0].GetSVGDocument();
 var gage = root.createElement('g');
 var t = root.createElement('text');
 t.setAttribute( 'x', x );
 t.setAttribute( 'y', y );
 t.setAttribute( 'fill', color );
 t.appendChild(root.createTextNode(val));
 gage.appendChild(t);
 return gage;
}

Example 3: JavaScript Function

This function takes as values the gage id, the x and y coordinates, the color, and value of the text reading. It first creates a new "g" element and a new text element. The text element then has the x, y, and fill attributes set to the appropriate values. Finally, a TextNode element is created with the actual reading and appended to the text element. The text element is appended to the g element and returned.

At this point, all the data contained in the XML data file is converted into javascript function calls and placed in a function called init. This funcition is called by the SVG onload event.

function init(evt) {
 var SVGDoc = document.embeds[0].GetSVGDocument();

 var rain_10m = SVGDoc.getElementById("rain_10min");
 var rain_30m = SVGDoc.getElementById("rain_30min");
 var rain_1hour = SVGDoc.getElementById("rain_1hour");
 var rain_6hour = SVGDoc.getElementById("rain_6hour");
 var rain_24hour = SVGDoc.getElementById("rain_24hour");
	.
	.
	.
 rain_10m.appendChild(_new_gage('1010','3097897.452546865','-1691883.7047196834','black','0.08' ));
 rain_30m.appendChild(_new_gage('1010',3097897.452546865,-1691883.7047196834,'magenta','0.75' ));
 rain_1hour.appendChild(_new_gage('1010',3097897.452546865,-1691883.7047196834,'red','2.99' ));
 rain_6hour.appendChild(_new_gage('1010',3097897.452546865,-1691883.7047196834,'magenta','2.99' ));
 rain_24hour.appendChild(_new_gage('1010',3097897.452546865,-1691883.7047196834,'black','2.99' ));
	.
	.
	.
}

Example 4: JavaScript Init Function

This adds the text readings to the SVG display at the proper location grouped by the time step so all the readings for a single time step can be displayed at once. Additional SVG attributes can also be added in this manner including full animation.

A comparison of the load on the server can be made at this point. The XML file giving current rainfall totals for 150 raingages with 6 time steps is 40 Kb. A comparable GIF image showing one rainfall time step and a medium complexity background image is 47Kb. The SVG display can show all 6 time steps at once showing a 6 to 1 advantage in bandwidth. This does not account for the process time on the base station to create the detailed background GIF image or the highly detailed SVG image now available on the client machine.

Figure 2 shows the actual display of this interface.

2.1 Images

RealTime-fig2.jpg

Real-Time display of ALERT rainfall data. Note the color coding of the text values and the station name displayed as a mouse over function. Also note the extensive GIS background information in SVG format allowing pan and zoom functionality. The legend allows various layers to be turned on and off

Figure 2: SVG Real-Time Rainfall Display

3. Other Displays

Using the techniques described in the previous section, several real-time displays have been created for displaying ALERT data and other sources. One of the first displays created was to show current weather conditions. By monitoring air temperature, dew point and wind conditions, it is possible to estimate when conditions are likely of promote severe thunderstorms. The FSL maintains a near real-time database of weather data from a variety of data providers throughout the United States. During the course of this project, the FSL graciously agreed to provide the data in XML format on-line. An SVG display is created by an XSLT stylesheet of current air temperature, dew point, wind speed and direction was created from this data source. By hovering over the wind speed symbol, a text summary appears giving the latest reading from the particular sensor. Figure 3 shows a snapshot of this display.

One of the most complex displays created is the thunderstorm tracking display. The University Center for Atmospheric Research (UCAR) has developed a program called TITAN that identifies thunderstorms from radar reflectivity. The thunderstorms are generalized as ellipses and are defined by location, rotation angle, major and minor axis length, and maximum reflectivity. Storm can merge or split and the direction and speed of the storms are estimated by TITAN. The ease of defining an ellipse in SVG and the animation functions make this an ideal candidate for display in an SVG environment. UCAR runs the TITAN model in real-time on the latest radar images and writes the storm parameters in XML format. Storm movement is tracked for the last hour. An XSLT stylesheet transforms the XML data into a series of SVG ellipses that are fully animated for the entire period of storm tracking. All parameters of the ellipse including location, size, rotation, and predicted movement are displayed. In addition, the corresponding radar images from the National Weather Service are loaded into the display and synchronized with the TITAN storm tracks. Finally, the ALERT raingage data readings for the last hour are also animated in time with the storms. Figure 4 shows a track of the storm that struck the Denver area on June 8, 2004.

River stage and discharge is also tracked in real-time using SVG. Stream flow values in XML format is converted into SVG via another XSLT stylesheet. Flow data from three different sources are converted to XML and transformed into a hydrograph (time vs. discharge) plot. The use of SVG allows not only zooming of the plot but permits the user to hover over a particular spot on the graph to display the actual flow reading and time. Multiple stream gages can be plotted on the same graph and are aligned with the correct time scale. Figure 5 show a typical hydrograph of several stream gages on the South Platte River through Denver.

Finally, predefined floodplains can be shown overlain on aerial photographs using SVG. The aerial photographs are obtained on the fly via the Microsoft TerraServer web site. By using SVG with the actual GIS coordinate system the floodplain outline can be shown exactly with the rectified aerial photographs. The pan and zoom functions of SVG are combined with scripts that reload successively higher resolution photographs from the TerraServer site. Figure 6 shows a detailed display of the floodplain display.

3.1 Images

RealTime-fig3.jpg

SVG display of real-time weather conditions. The wind barb symbols are created based on wind speed and the rotation angle is determined by the wind direction. SVG text of the stations air temperature, dew point, wind gust and time of last reading are displayed via the mouseover attribute.

Figure 3: SVG Weather Display

RealTime-fig4.svg

SVG display of thunderstorm tracking. The display is an animation of the previous hours' radar images and thunderstorms. Storm direction and speed is indicated by the black arrows at the center of each storm. Storm intensity is color coded and the display is synchronized with the radar images.

Figure 4: SVG Thunderstorm Tracking Display

RealTime-fig5.svg

SVG display of real-time streamflows. A time versus discharge plot of several stream gages is created from different data sources. SVG text of the flow readings and times are displayed via the mouseover attribute.

Figure 5: SVG Hydrograph Display

RealTime-fig6.jpg

SVG display of floodplain delineation over aerial photography. The floodplain outline are SVG path elements while the photographs are image elements linked to the Microsoft TerraServer site. By utilizing pan and zoom functions, a higher resolution photo is updated via user interaction.

Figure 6: SVG Floodplain Display

4. Compatibility with other Platforms

All of the original development work for this project was done for Internet Explorer 6 (IE6) using the Adobe SVGViewer plug-in. The target audience of emergency managers and UDFCD staff were using this browser. During the course of the development, investigations were made as to the portability of the XSLT and SVG files to other platforms.

At this time, the only known platform to be compatible with the programs described in this paper is Windows with IE6. However, it appears that the Safari browser on the MAC OS may also work with these programs. Safari has announced it will implement XSLT using the libxslt package and supports the Adobe SVGViewer. Based on this news, all the XSLT stylesheets were checked for compatibility with libxslt. Numerous changes were made and all stylesheets are now fully supported by libxslt. A major benefit is that the EXSLT extensions from EXSLT.org are natively implemented in libxslt. IE6 did not have support for EXSLT but was easily added via JavaScript extension functions. The author did not have access to the latest Safari browser so this could not be tested. Other platforms using an SVG enabled browser and libxslt may also display these applications.

Extensive research was made into the ability of the Mozilla browser to display these images. Unfortunately, at the present time this browser cannot display these applications. There are several general issues preventing proper display on Mozilla. The first is the current lack of zoom, pan and animation support in the Mozilla SVG implementation. The thunderstorm tracking application animates several data sources to properly show the movement and effects of storm movement. The other displays without animation can work in Mozilla but they utilize pan and zoom functions to provide more detailed information to the user. In addition, the XSLT implementation in Mozilla does not support the EXSLT extensions. A few of these extensions, notably dates, math and node-set are critical to identify the time the transformation takes place and the synchronization of the display of the various data sources. Finally, all of the described applications perform client-side transformations with the static information (i.e. GIS data) stored locally while the actual real-time data is requested on the fly by the XSLT stylesheets via the document() function. This real-time data is located on various servers such as the UDFCD base station and is at a different domain from the local machine. The Mozilla browser does not allow this behavior as a potential security risk and cannot be altered in the current configuration. IE6 using SP2 identifies this potential risk and either allows the cross-domain data via a pop-up confirmation or through configuration settings. It is hoped that these issues can be resolved and the applications can be utilized on several platforms.

Acknowledgements

The Urban Drainage District and Flood Control District and Kevin Stewart funded the majority of this work. The University Center for Atmospheric Research and David Yates funded the TITAN interface. Thanks to the the Forecast Systems Lab and the National Weather Service for embracing the XML data standards. HydroLynx Systems, Inc. is the manufacturer and distributor of the NovaStar real-time data collection, alarm monitoring and control system software. EXSLT.org provided ideas, code and a tremendous resource for expanding on the power of XSLT. Numerous people provided example code and ideas for XSLT and SVG to point me in the right direction.

XHTML rendition made possible by SchemaSoft's Document Interpreter™ technology.