Outsourcing Simulator Development: pitfall or panacea?

by Roger Jerram

Outsourcing the development of simulator software is a strategic decision for the Eurocontrol Experimental Centre (EEC). The ESCAPE real-time ATC simulator was developed and has evolved over the years using in-house staff, so why change?

For Darren Smith, Technical Manager of the ERS Research Enabler and responsible for simulator development, one answer lies in cost control. “Two years ago, most of the software development headcount was made up of external contractors and there was little sign of the trend being reversed,” he said. “We wanted better cost-effectiveness from large software development contracts encouraging economies of scale and from common products where costs can be shared with suppliers”.

In short, ERS wanted more return on its investments in external staff. There was another concern: quality. “We felt that with outsourcing we could get better quality from the use of industrial solutions where the software base is robust and well-tested, and from a more professional working relationship with suppliers” said Smith.

Cost control, economies of scale and access to specialized resources are among the most common reasons cited by organizations for outsourcing software development. But there are ‘hidden’ costs and substantial risks.

A 2003 survey by People3, a Gartner Group company, found that while just over 20 percent of the polled companies that outsourced IT work reported a cost savings greater than a fifth, another 20 percent did not achieve any cost reductions at all. Almost ten percent experienced an increase in costs. People3 concluded that many companies often neglect to factor in all costs associated with managing the outsourcing contracts.

The highly configurable and scalable ESCAPE simulator is used in different environments for various purposes in the EEC at Brétigny and elsewhere: Maastricht UAC, IANS in Luxemburg and CRDS in Budapest being the main “external” users. “Outsourcing by ERS will work only if the needs of all customers are taken into account,” said Hervé Parvais, head of the Simulator Facilities Management Centre of Expertise and responsible for deployment of the simulator. “All users will need effective support from suppliers during component integration and verification”.

For Colin Meckiff, head of the Sector Safety and Productivity Research Area and one of the simulator’s major research clients, the quality of the contracts and the attention to risk management will be crucial to success. “We are in an R&D environment and need a degree of flexibility from the software suppliers,” he argued. “If the contracts are not sufficiently thought-out in this respect then our experimental simulations could fall short of our stakeholders’ expectations”.

Starting February 2004, the simulator software has been divided for outsourcing into large groups of components chosen to be functionally coherent. Contracts for the first two of these groups were let last December following competitive calls for tenders. Offers for a third group were received this February. By the end of 2005 all of the five groups are targeted for outsourcing.

Mindful of the risks posed by poor definition of contracts and processes, ERS and the Project Support Office of the EEC joined forces with Burkhart von Erlach, then of the Procurement and Special Agreements Section of the Finance Directorate in Brussels, to produce a model contract. Burkhart is now heading the Finance Team at the EEC. “I will be closely following the results of our work on the contracts,” he said, being now in a good position to do so. “We will review their effectiveness and feed back our experience for other outsourcing projects”.

All outsourced component groups share the same ‘Generic’ contract part while each has its own ‘Component Annex’. The Generic contract goes hand-in-hand with the ERS Requirements Management Plan that describes in detail the business process used to manage the contracts. From contract initiation to termination, from high-level analysis of user requirements to final acceptance of software, the internal and external interfaces have been studied, discussed and described.

All the documents are subject to configuration management and agreement with the suppliers. “We need to maintain with our suppliers a common set of plans and processes for Requirements, Software Configuration and Integration Testing,” said Régis Berteaud, manager of the contract for the AIR component group. “This will help ensure a common understanding between us and the suppliers, and between the different ERS Contract Managers.”

Whatever the scope and quality of contract documentation, cost escalation and a purely commercial view taken by suppliers remain key risk issues. “Budgeting and cost control are critical factors in the equation for success,” points out Hervé Parvais. “There is a risk of unexpected costs due to bad interpretation or definition of requirements.”

A payment model was developed for the Generic Contract to help ensure continuity of support and on-time delivery of tested software within a stable financial framework. The model includes one-off payments for successful Initialisation and Completion, regular fixed payments for project and contract management and operational support subject to agreed metrics, and ‘tasking’ payments for new software development. The tasking payments are phased with varying percentages due upon delivery of Design Documents and Test Plans, Component delivery, Integration (site) acceptance and Validation (final) acceptance. Suppliers that ‘play the game’ better can be rewarded by more front-loading of the percentage payments (and vice-versa of course).

The first two contracts to be let, AIR and HMI, each use a different model for client-supplier interfacing. For the AIR group, STERIA will at first install an in-house team under their dedicated Project Manager. They will progressively move to more off-site working while retaining on-site those supplier personnel necessary for operational support. This contrasts with the approach taken for the HMI group by Barco Orthogon AG who will from the start work almost completely offsite in their company premises in Bremen, Germany. We will be interested to see how these two approaches work in practice.

We should not forget that outsourcing ERS demands changes from its staff. According to Patrick Peeters, head of the Development and Integration Centre of Expertise: “Moving from home-made development to outsourcing implies drastic changes to working practices and skills. In fact it is a new type of job.” The new emphasis is on systems architecture, requirements management, acceptance and contract management. Three years ago ERS had more than forty contractors onsite, today we have less than ten. The EEC has invested in a programme of training and specialist consultancy to help staff make the change. “It concerns mainly the change in the project management from "maître d'oeuvre" to "maîtrise d'ouvrage", the production of the technical specifications and the acceptance of the deliverables,” said Peeters. “The staffing policy of the Centre is to allocate people to activities closer to our core business in ATM. The clear and common positioning of the Business Manager and the Line Managers has greatly helped to pass the message.”

So will the strategy lead to pitfall or panacea? It’s too early to say. We have set our objectives: better cost-effectiveness measured by services obtained per unit cost, and better quality measured by newly-defined metrics. We have put in place processes and contract structures designed to support these objectives. One thing is certain: there can be no going back to the ‘old’ ways. Apart from concerns over cost and quality the management of in-house contractors risked contravention of the local labour regulations. Time will tell if we have succeeded with outsourcing and we will review the performance of the contracts early next year.