Performance Testing A New Geographic Information System

Background

A Major UK Utilities Company decided to develop a new Geographic Information System (GIS) in order to provide its contractors with instantly available Safety Certificates (maps) showing the precise location of power cables.  The provision of such accurate information helps to reduce accidental damage to cables and resulting customer inconvenience, as well as improving safety for engineers in the field.  As the new GIS would be safety critical it was absolutely crucial that it was tested thoroughly to ensure that it would perform well before it was launched.  Sopra Group was commissioned by this Major Utilities Company to Performance Test its new GIS. 

Performance Testing aims to verify the performance of an application and its architecture prior to launch; it is a crucial part of the Testing lifecycle because the later a performance defect is detected the more expensive it is to correct.  Sopra Group ran Performance Tests upon the Client's new GIS application in order to identify any under-performing parts of the system so that appropriate corrections could be implemented and then re-tested prior to it 'going live.' 

Solution

Together with the Client Sopra Group determined the system's peak load profile - an estimate of the highest predicted number of concurrent users of an application performing a specific number of transactions within a set period of time.  In this instance calculations indicated that the application needed to service 320 users generating 240 map image requests per minute. 

Testing a system at peak load allows the measurement of response times of all the business-critical transactions, and close monitoring of the application environment (web/application/database servers) in this way can highlight potential system failures. 

Sopra Group also worked closely with the Utilities Company to understand its key business priorities along with their associated level of risk before defining the Performance Test scripts.  Detailed business analysis ensured that the right elements of the system were tested and that time and resources weren't wasted on testing business-as-usual functions. 

Benchmarking Exercises

After several Performance Test runs the system was made stable, but system response times were still below the required level.  Benchmarking exercises were then undertaken to determine why the system couldn't perform under the required load:

• Horizontal Scaling – this showed that the system could be scaled out to include additional application servers and that future system performance would not be affected if additional servers needed to be added. 
• Virtualisation vs Physical Hardware – the Client was intending to host the new system on virtualised software, but testing showed that the application couldn't perform at the required level on a virtual server.  In light of this the Client decided to run the GIS on physical servers, which enabled system performance to exceed the requirements.

Benefits

Sopra Group's rigorous approach to Performance Testing helped to ensure a successful launch of the new GIS and afforded the Client a number of benefits including:

• Launch of a stable system – ensuring the system was reliable and stable was absolutely vital as the application is safety critical.
• Flexible system – it can be expanded to include different application servers allowing it to respond to the growing needs of the business.
• Cost-effective solution – the test scripts were clearly focused on relevant business priorities which helped keep testing costs to a minimum. 
• Clear documentation of processes – Sopra Group's best practice approach ensured that the behaviour of the system under load was thoroughly documented; this will enable the Client to manage capacity and maintenance planning in the future without third-party input.