In the Techniques section:
Techniques
eGov Risk Assessment: Simple Factor Rating
"Is my e-government project likely to fail?"
This page offers one technique for answering this question by assessing project risk. Follow this link for other techniques.
Analysis has shown that there is a general set of success and failure factors that affect e-government projects. The risk assessment technique presented here asks you to rate the presence/absence of these factors. Follow this link for a detailed explanation of these factors (and some related case examples).
Assessing the Success/Failure Factors
Assessment consists of questions relating to a series of factors, with attached rating numbers.
Notes:
- Work through the factor questions one by one.
- You give a rating number for your answer. The rating for each answer can be anywhere on a scale from zero to ten: the higher the number, the less risky the project. Examples are only given for zero, five and ten, but all numbers in the range are possible.
- It may be necessary to conduct further data gathering (interviews, document analysis, observations, questionnaires, etc.) in order to adequately answer the questions.
- The term 'public agency' is used to mean the department, Ministry, parastatal, or other public sector organisation in which the e-government project is being undertaken.
- There is a worked example at the end of the page.
1. Drivers
Factor Question 1a . How strong is the drive for change from outside the public agency (e.g. from central government, or from an aid donor, or from citizens, etc.)?
Answer Rating : from 0 for 'non-existent' through 5 for 'moderate' to 10 for 'intense'.
Factor Question 1b . How strong is the drive for achievement of e-government goals from key agency officials?
Answer Rating : from 0 for 'non-existent' through 5 for 'moderate' to 10 for 'intense'.
2. Strategy
Factor Question 2 . Is there a clear, long-term strategy for e-government that sees IT as a means to achieving broader reform objectives?
Answer Rating : from 0 for 'no strategy at all' through 5 for 'partial strategy' and/or 'only partly clear' and/or 'somewhat unstable' and/or 'sees IT more as an end than a means' to 10 for 'strategy that meets all the stated criteria'.
3. Management
Factor Question 3a . How good is project management for the project?
Your assessment should cover at least the following six sub-points as examples of good practice: the presence of clear project responsibilities; consideration of risk; good monitoring and control; good organisation of resources (including staff); good management of partnerships (with private suppliers and with other public agencies); and effective procurement.
Answer Rating : from 0 for 'very poor' through 5 for 'moderate' to 10 for 'very good'.
Factor Question 3b . How good is change management for the project?
Your assessment should cover at least the following four sub-points as examples of good practice: strong leadership from a project champion; support of senior officials and other powerful stakeholders; use of incentives to create commitment and ownership among stakeholders (including operational staff); and strong stakeholder involvement that builds support.
Answer Rating : from 0 for 'very poor' through 5 for 'moderate' to 10 for 'very good'.
Factor Question 3c . How much are key players just focusing on personal self-interest and playing politics?
Signs of this that could form a checklist include infighting; resistance where loss of power is feared; "me too" copying of e-government solutions for image purposes; obsession with electoral impacts and short-term kudos; and corruption.
Answer Rating : from 0 for 'very much' through 5 for 'somewhat' to 10 for 'very little'.
4. Design
Factor Question 4 . How effective and realistic is the design of the e-government project?
Your assessment should cover at least the following four sub-points as examples of good practice: an incremental/piloting approach; quick and feasible objectives; strong stakeholder involvement ensuring design meets real needs; and an understanding in the design of the 'human factor', including local culture and values.
Answer Rating : from 0 for 'very ineffective and unrealistic' through 5 for 'moderately effective and realistic' to 10 for 'very effective and realistic'.
5. Competencies
Factor Question 5 . Are the required competencies present?
Assessment of competencies should at least approximate to a three-dimensional matrix that covers: a) the different categories of competency (skills, knowledge and attitudes); b) the different stakeholder groups (system developers, system managers, organisational managers, system operators, system users, etc.); and c) the different areas of competency (strategic, change/project management, information systems development and management, hands-on, interpersonal, 'intelligent customer' (contracts, suppliers, procurement), etc.).
Answer Rating : from 0 for 'completely absent' through 5 for 'some presence' to 10 for 'all present'.
6. Technology
Factor Question 6 . Is the technological infrastructure adequate?
Assessment would cover the presence and resilience of data systems, software, hardware, and telecommunications. Where appropriate, this can include assessing whether systems (data, software, hardware) supposed to 'talk to each other' are actually compatible.
Answer Rating : from 0 for 'wholly inadequate' through 5 for 'moderate' to 10 for 'completely adequate'.
7. Other
Factor Question 7 . Are there other factors likely to cause the e-government project to fail?
These might relate to money or timescales as well as other issues.
Answer Rating : from 0 for 'yes' through 5 for 'perhaps' to 10 for 'no'.
Presenting, Analysing and Using the Results
Overall Rating Analysis
The simplest thing you can do is add up all the rating scores and interpret them according to the following table.
Overall Rating |
Likely Outcome |
|---|---|
0 - 20 |
Your e-government project will almost certainly fail unless action is taken. |
21 - 40 |
Your e-government project may well fail unless action is taken. |
41 - 60 |
Your e-government might fail totally, or might well be a partial failure unless action is taken. |
61 - 80 |
Your e-government project might be a partial failure unless action is taken. |
81 - 100 |
Your e-government project may well succeed. |
Force-Field Diagram
An alternative presentation would use a force-field diagram, which lists drivers/enablers on one side and constraints on the other. To represent these, a rating score of 5 on any factor would be taken as neutrality (no overall impact, so not included in the diagram); a score above 5 means the factor is an enabler; a score below 5 means the factor is a constraint. The size of difference from 5 represents the strength of enabler or constraint. The strength is represented by the length of the arrow in the force-field diagram. This is illustrated in the worked example below.
When identifying factors as enablers, wording can be changed to the positive aspect of the factor (e.g. changing 'project management' to 'good project management'), and vice versa when factors are identified as constraints (e.g. 'poor project management').
A slight variation is to use the weighting system described below, with the width of the arrow line related to the weight.
Individual Factor Analysis and Action
The rating scores for each individual factor can be presented using a table or force-field diagram arranged to show the scores in order from largest to smallest. The factors identified as the largest constraints are those that should be prioritised for action if risks of failure need to be addressed. Follow this link for further details about actions to take to reduce the risk of failure based on the factor model.
Variations on the Basic Technique
The basic technique makes a questionable assumption - that all factors are equally important to the success and failure of the project. A more complex variation would involve two rounds. In the first round, the risk assessment team would assign a weight to each of the identified factors - external pressure, internal political desire, overall vision/strategy, project management, change management, politics/self-interest, design, competencies, technological infrastructure, other. An 'ordinary' factor might be given a weight of 1; a factor that was considered 'important' in the particular e-government project could be given a weight of 2; and a factor that was considered 'very important' in the particular project could be given a weight of 3. The weighting score would be multiplied by the rating to give an overall set of weighted ratings. For example, if external pressure was felt to be very important for this specific project, it could be given a weight of 3. If that pressure was found to be only moderate, it could be given a rating of 5. The overall weighted rating for that factor would be 3 x 5 = 15.
A more complex variation again would involve three rounds. In the first round, the risk assessment team would assess which factors are relevant to the success and failure of the particular e-government project. It would consider not only those factors listed just above, but also other factors as well. The second round would weight these factors. The third round would give a rating score to each factor.
Pros and Cons of this Technique
This technique is simple and quick to understand and put into practice. On the downside, at least in its simple form, it assumes that "one size fits all" e-government projects, which we know is not really true. Also, the questions, being relatively few in number, sometimes try to cover quite a range of different issues in a single question.
Worked Example
A new Web-based procurement system is being implemented by the Ministry of Transportation in Gedactia. Introduction of the system is being promoted and partly-funded by an external donor, which has put in place many of the formal skills and technology required, but the project has relatively little internal support. Is this e-government project likely to succeed or fail? An assessment and answer are given below.
Questions, Answers & Ratings
Factor Question 1a : How strong is the drive for change from outside the public agency (e.g. from central government, or from an aid donor, or from citizens)?
Answer : The donor is pretty keen on seeing significant changes in the way the Ministry operates.
Rating : 8
Factor Question 1b : How strong is the drive for achievement of e-government goals from key agency officials?
Answer : Officials are relatively happy with the way things work at present, and are generally fearful of change, particularly change involving IT.
Rating : 2
Factor Question 2 : Is there a clear, long-term strategy for e-government that sees IT as a means to achieving broader reform objectives?
Answer : It is questionable whether there is really any commitment within government to stated reform objectives, and officials are only just now starting to identify policies and strategies for IT.
Rating : 2
Factor Question 3a : How good is project management for the project?
Answer : The donor has insisted on bringing foreign consultants in to help manage the e-procurement project. They bring with them a strong set of rational, textbook skills in project management and experience of projects in industrialised countries. On the downside, they have little awareness of the specific issues facing the Gedactian public sector.
Rating : 7
Factor Question 3b : How good is change management for the project?
Answer : There is little or no internal support or championing of the project, and little internal ownership. The consultants have made some half-hearted attempts to get stakeholders involved, but they seem to see the donor as their main client.
Rating : 2.5
Factor Question 3c : How much are key players just focusing on personal self-interest and playing politics?
Answer : There is a fair amount of this going on; even, arguably, within the consulting firm which seems to care more about its reports and its appearance than about a successful system. Many internal staff are resistant to the project, partly for personal reasons, but also because they don't feel e-procurement is an appropriate priority for the Ministry.
Rating : 2
Factor Question 4 : How effective and realistic is the design of the e-government project?
Answer : The consultants have created a very professional-looking design that incorporates an incremental/step-by-step approach, and some quick wins. However, it feels more like an 'off-the-shelf' solution than one that is matched to local needs and conditions.
Rating : 5
Factor Question 5 : Are the required competencies present?
Answer : On the technical side, things are looking good - the consultants know what they are doing, and they are well advanced in providing hands-on training for local staff. On the other hand, many of the human and change-related skills and knowledge are lacking, especially within the Ministry.
Rating : 5
Factor Question 6 : Is the technological infrastructure adequate?
Answer : The consulting firm seems likely to be able to introduce a sound technological infrastructure for the project. There are some compatibility issues, particularly the lack of IT access among smaller suppliers.
Rating : 7
Factor Question 7 . Are there other factors likely to cause the e-government project to fail?
Answer : Not that can be identified at present. Sufficient funds are being by the donor for the first two years of the project, though a question hangs over the ongoing financial sustainability of the project.
Rating : 7.5
Table of Results
In the following example, the first column lists the factors, the second column lists the rating score, and the third column lists the difference from 5 that is then drawn in the force-field diagram below.
Factor |
Rating Score |
Enabler/Constraint Strength |
|---|---|---|
External pressure |
8 |
Enabler 3 |
Internal political desire |
2 |
Constraint 3 |
Overall vision/strategy |
2 |
Constraint 3 |
Project management |
7 |
Enabler 2 |
Change management |
2.5 |
Constraint 2.5 |
Politics/self-interest |
2 |
Constraint 3 |
Design |
5 |
Neutral |
Competencies |
5 |
Neutral |
Technological infrastructure |
7 |
Enabler 2 |
Other |
7.5 |
Enabler 2.5 |
Overall Rating |
48 |
- |
Force-Field Diagram
Conclusions and Action
Given the overall rating of 48, the project is at some risk of failure unless action is taken. One main set of actions would be to try to overcome or reduce the main identified constraints (i.e. starting at the bottom right of the force-field diagram and working upwards, trying to identify ways to ameliorate each constraint in turn). Follow this link for further details about actions to take to reduce the risk of failure based on the factor model.