Introduction

The "GDAL Provider stability and performance within MapGuide" initiative will address long standing stability and performance issues with the GDAL FDO Provider running under MapGuide Open Source.

 

Phase 1 - Initial Investigation (complete)

The initial investigation into performance and stability on Windows is complete.  A C++ dll mismatch on Windows between the MapGuide Server, GDAL, and the GDAL plugins for ECW and MrSID were causing instability.  Prebuilt GDAL binaries corrected the issue.  See MapGuide 2.2 GDAL Raster Provider for more details. The GDAL provider exhibits reasonable stability under light to moderate user loads.  Scalability is limited to approximately 2 cpus.

 

Phase 2 - Implementation of Resampling Methods

The GDAL Provider currently processes all raster data using nearest neighbour resampling.  This provides good performance and may not be suitable for all raster data types.  For example, text labels on cartographic imagery can become unreadable when nearest neighbour resampling is applied.  The second phase of the GDAL Provider Initiative will introduce configurable resampling methods to MapGuide and FDO. 

 

Phase 3 - Improve Scalability and Performance on Windows and Linux

GDAL Provider processing is currently is limited to a single thread to alleviate thread safety issues between MapGuide/FDO/GDAL.  This gives MapGuide stable albeit slower performance on Windows.  To accommodate heavy user loads, multi-threaded processing should be enabled.  The third phase of the GDAL Provider Initiative will correct the thread safety issues yielding a stable and performant provider on both Windows and Linux. 

Donors

Pacific Alliance Technologies

Explore Australia

Ennoble Consultancy

Spatial Integrators

Status of Provider and Recommendations

The GDAL Provider in MapGuide is usable if a Visual Studio 2008 build of the GDAL plugins and dependant libraries are used.  Please see MapGuide 2.2 GDAL Raster Provider for implementation details.  Testing was performed against the 64 bit MapGuide 2.2 RC2 build on Windows Server 2008 Web Edition.  A summary of the file types supported under various configurations is provided below.  Other raster formats may work:

Configuration	Multi-thread Stable	Single-thread Stable	Not usable
Windows XP 32 bit, MapGuide 2.2 32 bit	TIFF, SID, ECW, JPEG2000 *note 1	TIFF, SID, ECW, JPEG2000	none
Windows Server 2008 64 bit, MapGuide 2.2 32 bit	TIFF, SID, ECW, JPEG2000 *note 1	TIFF, SID, ECW, JPEG2000 *note 1	none
Windows Server 2008 64 bit, MapGuide 2.2 64 bit	TIFF, SID, ECW, JPEG2000	TIFF, SID, ECW, JPEG2000	none
CentOS 5 or Ubuntu 9 32 bit, MapGuide 2.2 32 bit	not tested	not tested	not tested

Notes:

1. This configuration was not specifically tested by OTX Systems.  It is expected to work based on test results from other configurations.

An eight hour load test with 8 concurrent users completed successfully using the 64 bit MapGuide 2.2 RC2 build.  During the test, a 200MB increase in memory footprint was noted.  This may be due to memory leaks.  For MapGuide sites handling heavy user loads, a nightly restart of the MapGuide Server service should be implemented to reduce the impact of memory footprint increase.

Future Work

Based on testing performed to date, the GDAL Raster provider is relatively stable and exhibits usable performance.  Future work will include increasing scalability to more than two cores and adding support for additional resampling methods.  The currently used nearest neighbour resampling is problematic for raster images containing text. 

Funding and Work Performed to Date

This initiative has received partial funding from the community and 22 hours of work have been performed. Future funding will be used to support Phase 2 and Phase 3 of the initiative.

Work Performed - Phase 1	Hours	Date
Create a development/test environment on a multi-CPU machine	3	Jan 12, 2010
Create GDAL Initiative page	1	Jan 21, 2010
Debug PNG issues in Beta build	.5	Aug 24,2010
Perform single user interactive testing of GDAL with 32 bit MG on 32 bit Win XP.  ECW, MrSID, tiled tiff seems to work.  Normal tiff loses colorization.  JPEG2000 works if ECW or MrSID plugins are installed.	2.5	Sept 9, 2010
Email and phone ERDAS re: ECW SDK licensing.  The standard license does not permit use in a server environment.  Users of MapGuide should contact ERDAS to discuss licensing requirements.  Updated the GDAL in MG 2.1 page in Trac to make this clear to the user community.	2	Sept 30, 2010
Build MapGuide and Fdo from source on test box to debug GDAL Provider issues	2	Nov 19, 2010
Investigate Ticket 1476.  MrSID plugin throws an invalid heap error w/ 32 bit MG on 64 bit OS.  The plugin uses Visual Studio .Net 2003 dlls.  They may not be compatible with WOW64.	2	Nov 21, 2010
Check Visual Studio .Net 2003 (VS .Net) compatibility with newer operating systems.  Support for VS .Net was dropped in Vista.  Plugin recompile with Visual Studio 2008 may be required.  Check raster format usability on Windows Server 2008 64 bit for TIFF, MrSID, ECW, and JPEG2000.  Install http://vbkto.dyndns.org 32 bit VS2008 GDAL binaries.	4.5	Nov 22, 2010
Create test map and Grinder test script for MGOS 2.2 RC2 64 bit raster test.  Test runs against a 25 image sid, ecw, tiff, jpeg2000 catalog.  All formats work with 64 bit MGOS and VS2008 gdal plugins	3	Dec 29, 2010
Run 8 user test for 8 hours.  No errors.  Memory footprint grew by 200MB indicating some memory leakage.  Performance did not scale (max 2 cpu out of 8)	1	Dec 30, 2010
Document GDAL install configuration	1	Dec 31, 2010

 

Related Defects and Tickets

Ticket #1196 MapGuide unresponsive using GDAL provider

Ticket #1121 MrSID images fail to display in MapGuide 2.1

Ticket #1476 mgserver lock up using GDAL Plugins (mrsid)