In the Case Studies section:
Design-Reality Gap Case Study No.5
An Integrated Information System for Defence Force Management in the Middle East
Case Study Author
Anonymous
Organisation
The Republican Defence Force (RDF), under control of the Department of Defence (DoD), consists of the nation's army, air force and navy.
Application Description
The application was the planned introduction of an integrated information system (IIS) that would cover all branches of the Republican Defence Force. This would not only computerise but also integrate a variety of management information systems, including human resource IS, budget IS, accounting IS, and procurement/payment IS.
Application Drivers
Until the mid-1990s, management of the RDF was a hierarchical activity that relied mainly on manual information systems. As the RDF grew during the 1990s, this approach became increasingly inefficient and ineffective. A decision was therefore made to upgrade the management information systems, with a view to improving management processes, and improving the image of the RDF. This also coincided with a more general move to try to improve the functioning of the whole apparatus of government, and with a more general spread of IT, which some senior and middle-rank officers mis-perceived as an overnight solution to a variety of managerial and administrative problems.
Stakeholders
The main stakeholders are those in senior positions (Secretary for Defence and other DoD senior officials; Head of the RDF, Heads of the individual Defence Services, and other senior officers); key advisers or close colleagues of those senior officials/officers; middle-ranking officers who will be involved in the IIS introduction process, including the head of the IT Unit, and others who will be using the outputs of the system; and ordinary service men and women, who will be recorded on the system and may also be operators or users. Local IT firms also had some stake. They can be represented on the following matrix of power and interest. The most important stakeholders will be those combining highest power and highest interest (senior officers) and there is a continuum down to the least important who have lowest power and lowest interest (ordinary service personnel).
|
High |
|
Secretary for Defence / Other senior DoD officials |
Head of RDF / Heads of Defence Services / Other senior officers |
| Level of Power | Medium |
Head of IT Unit |
Advisers / Close colleagues of senior officials/officers |
|
|
Low |
Majority of service men / women |
|
Middle-ranking officers Local IT firms |
|
Low |
Medium |
High |
|
Level of Interest |
||||
Design-Reality Gap Analysis
Design-reality gap analysis compares the assumptions/requirements within the application design with the reality pertaining just before that design was implemented along seven 'ITPOSMO' dimensions:
- Information : the design made relatively few changes to the content of formal information that was in use within the Republican Defence Force just prior to IIS implementation. It did implicitly assume that informal information would not be used in the RDF - whereas in reality that had been important. The design explicitly required that information be presented in different ways to those use pre-computers, and that information flow in different ways. In particular, the design required a sharing of data within and between the different arms of the RDF. That was quite different to the pre-existing reality by which officers jealously guarded their data both for reasons of personal power and concerns about national security. This created a large/medium design-reality gap on this dimension. Gap score: 8.
- Technology : the design assumed the use of a broad range of new software and hardware, including a series of networked information systems spread across the whole of the RDF. The initial reality was almost entirely manual management information systems, with some PCs in use for word processing. There was also a lack of nation-wide telecommunications infrastructure - even a lack of electricity in some RDF locations. This created a large design-reality gap on this dimension. Gap score: 8.5.
- Processes : the design assumed significant changes to pre-existing RDF management and information-related processes. Although the basics of many management decisions were similar in design and initial reality, those decisions were designed to be made more on the basis of formal information, with a removal of pre-existing duplications and some bureaucratic checks, and with an integrated approach to decision-making, particularly financial decision-making, that was not present in initial reality. This created a medium/large design-reality gap on this dimensions. Gap score: 7.
- Objectives and values : the design assumed that the objectives of the project (automation of processes, integration of processes) were shared by all stakeholders. In reality, pre-IIS, some senior and middle-rank officers shared these objectives, but others did not and DoD officials felt reluctant about the project. The design also assumed a rational, objective culture within the Republican Defence Forces. In reality, the culture was heavily personalised and tribalised, with management processes strongly affected by conflicts between senior officers, some of whom belonged to different tribes. This reality also jarred with a design assumption that cross-functional team-working would be valued. Overall, there was a medium/large design-reality gap on this dimension. Gap score: 7.
- Staffing and skills : the design assumed the presence of a broad range of staff competencies. These included: good contract design and management skills; strong technical skills from a local IT contractor; sound project management skills; IT skills among all relevant service personnel, including officers; and IT knowledge and awareness among all senior officers. In reality, pre-IIS, virtually none of these competencies were present. For example, the local contractor turned out in practice to be inexperienced and lacking both managerial and technical skills. To take another example, most service personnel lacked IT skills. A few had word processing skills, but those fell below the operational skill requirements assumed in the IIS design. Overall, there was a large design-reality gap on this dimension. Gap score: 8.5.
- Management systems and structures : the design incorporated a notion of decentralised, flexible but integrated decision-making systems. This clashed with the pre-existing reality of autocratic, hierarchical, centralised but disconnected management systems and structures. This meant a large/medium design-reality gap on this dimension. Gap score: 8.
- Other resources : the design requirement was for budgetary expenditure of some US$5m to be spent on the IT, plus further expenditure for general staff training and IT staff recruitment, to be spent over an 18-month implementation period. The RDF did have such resources in reality. There was also a requirement for commitment of a considerable amount of time by senior and middle-rank officers in the planning of system design and installation. In reality, staff had some other calls on their time: devoting the required time to IIS would necessitate the sidelining of some other duties. This meant an overall medium/small design-reality gap on this dimension. Gap score: 4.
- Overall : there was an overall large/medium design-reality gap in this case. Total gap score: 51.
Design-Reality Gap Changes During Implementation
It emerged that the system implemented was a product initially designed for the private sector. Although the RDF design removed inappropriate private sector components, in practice those components were not actually removed by the initial contractor. In other words, the IIS was never customised to specific RDF processes and systems (nor, implicitly, to its objectives and values). The design-reality gap on these dimensions therefore actually increased after initial implementation. The initial contractor was then removed, and a new one introduced, which did attempt some of the designed customisations, bringing the design-reality gap back down to initial levels.
The staffing and skills gap was narrowed somewhat by IT training provided to service personnel. However, this left the gap in other competencies untouched. That gap was affected by the change in contractors - reducing as a more competent contractor was introduced, but growing again when some key project personnel left to work in the private sector. Overall, the gap reduced to a score of c.7.5.
The technology gap narrowed as new IT was brought in (thus changing the reality of the RDF to make it more like the design and bringing the score at the time of case analysis to around 5.5), but other dimensions did not alter because the IIS failed to make an impact on the reality of organisational functioning.
Finally, after a few years, and despite the obvious problems with the system, the design was expanded to incorporate two additional computerised modules, thus creating some modest increase in most design-reality gaps.
In all, after some years of implementation, the overall design-reality gap has not significantly reduced from the initial score.
Evaluation: Failure or Success?
Introduction of the integrated information system has been largely unsuccessful, bordering on a total failure, as might well be expected with a large-medium/51-score design-reality gap.
The system was implemented, but most modules have never worked satisfactorily and never been used, or were just run in parallel with existing practices but not made part of those practices. At the time of analysis - some six years after project initiation - of the five initially envisaged modules, only one is in operational use. In cost-benefit terms, the project has been little short of disastrous, with hundreds of meetings, training sessions, and other uses of staff time plus the direct financial costs all being invested with very little to show on the benefit side of the balance sheet.
Learning: Identifying Causes of Near-Total Failure
The dimensional gaps are arranged in descending order in the following table, which focuses on the pre-implementation situation:
Dimension |
Rating |
Technology |
8.5 |
Staffing & Skills |
8.5 |
Information |
8 |
Management Systems & Structures |
8 |
Processes |
7 |
Objectives & Values |
7 |
Other Resources |
4 |
With such high gap ratings for so many dimensions, it can be argued that all except 'other resources' were a contributory cause of the largely unsuccessful outcome. As noted above, the greatly-lengthened timescale for the project and the gradual introduction of IT means that 'technology' in practice should not be seen as a prime cause of the failure. The remaining dimensions can be grouped into three related areas as causes:
- Trying to change the way of working too much at once: Information/Management systems & structures/Processes dimensions . By changing all these dimensions - looking for a fundamentally different set of working practices, the project tried to do too much at once.
- Mismatch with organisational culture and personal interests: Objectives & values dimension . Like all armed forces, the RDF is a very traditional organisation but also one with some strong individual personalities. The way the IIS was designed conflicted directly with both the traditional culture and the self-interests of at least some senior figures.
- Absence of key competencies: Staffing & skills dimension . Such an ambitious project required a strong set of complex competencies that simply were not available.
Any one of these can be problematic - together they created a strong likelihood of failure.
Recommendations: Reducing Design-Reality Gaps
To reduce the risk of failure, e-government projects must reduce problematic design-reality gaps. This means either making the design more like current reality and/or making reality more like the design. Hindsight recommendations for improvements on this project fall into two main camps:
i. Generic gap closure recommendations
These are approaches to e-government projects that can help generally to reduce gaps on a number of dimensions:
- Scope limitation : a very simplified e-government analogy is to say that technology and working practices are like a person's two legs. If you try to change just one, that's like asking a person to take one leg off the ground: the result is that they are a little unsteady but can still balance and stand up. Just so, with e-government, if you just change one of the two elements - i.e. changing the technology but keeping working practices the same, or changing working practices but keeping technology the same - then the organisation may be a little unsteady but it can still keep going. But if you try to change both at once, that's like asking a person to take both legs off the ground at once: the result is that they will fall down. Just so, with e-government, if you change both technology and working practices simultaneously, then the project will often fall down. The initial project should have been much more limited in its scope and ambition. It should initially have aimed just to introduce computers and networks to the RDF, but to leave current working practices untouched. It could have done this in one of two ways. Most obviously, by automating some of the current administrative/management working practices, but leaving those practices as close as possible to the original form - i.e. retaining the same information, the same processes, the same management systems and structures, the same personnel - as much as possible. Alternatively, by focusing on roll-out of basic network applications - email and Web access - to a broad range of locations and personnel. This would not change any of the main work of the RDF, just automating some communication and data-gathering practices. In either case, this would mean a significantly smaller overall design-reality gap.
- Incremental/step-by-step approach : this is another way of looking at avoiding the perils of over-ambition. Instead of computerising and integrating all modules at once across the whole RDF, the project design would have been better advised to take an incremental approach. This could have been incremental in terms of modules: computerising the HR function first, then moving on budgeting once HR was successfully completed. If that still appeared too ambitious, the project could also have been incremental in terms of services: focusing just on the navy, say, and then rolling out a module to the other services once the navy module was proven to work well. The incremental approach breaks down the overall design-reality gap into smaller individual elements.
- Participation : the IIS project was conceived by a small clique of senior officers. It would potentially have been better if a broader range of stakeholders had been involved in the process. In that way, a richer sense of reality would likely have been incorporated into the project's design, resulting in a smaller overall design-reality gap. However, the design of this technique may itself be mismatched to RDF realities, which neither involve nor value participative approaches to any great degree.
ii. Specific gap closure recommendations
These are actions taken on e-government projects that can help specifically to reduce gaps on a particular dimension:
- Objectives and values : there is relatively little that could be done specifically about this dimension. The project design incorporated a very top-down approach to project implementation, that virtually imposed the system on middle and lower ranks. The reality of their objectives and values could perhaps have been moved a little towards the design through clear and interactive communication that explained the value and role of the information system, and that listened to concerns. However, this - in itself - is an approach that is out of synch with the autocratic and centralised way of working in RDF. The only true way to address the objectives and values gap would have been to design a quite different e-government system that did not conflict with existing motivations, interests and culture. Email might be one such system, since everyone likes and needs to communicate with others and since email is flexible enough to support a range of different objectives and values.
- Staffing and skills : a broad range of awareness-raising and training activities should have been initiated in order to address some of the competency gaps identified above. The problem of poor/absent contract, change and project management skills was less tractable since many were not available in the country. Training could have gone some way to help, but many of these can only be developed in reality through experience. So, should the RDF have looked for help outside the country? There are two problems here. First, the fact that external consultants can quite often be poorly in touch with national and organisational realities, so their designs mismatch those realities - you reduce the staffing and skills gap at the expense of increasing other dimensional gaps. Second, e-government applications in security and defence are highly sensitive: few governments would welcome foreigners. The only solution is to reduce the scope and ambition of the design so it requires fewer/lower competencies, and so comes closer to reality.
Action Recommendations
All of the above represent retrospective recommendations: ways in which the project could have been better implemented if it had recognised design-reality gaps earlier. Some of these are not of much use to the existing project, since they essentially argue that this was the wrong project, and that RDF should have started with a completely different e-government application. Some, however - incrementalism, better communication, training - could be used to try to rescue IIS.
Case Details
Author Data Sources/Role : Application User/Participant Role
Outcome : Largely Unsuccessful. Reform : eAdministration (managing process performance).
Sector : Security Services (Defence).
Region : Middle East. Start Date : 1996. Submission Date : January 2003