9 Steps for Better Evaluation & Decision Making
1. Define the task
In any decision process, the critical first step is making sure that you have written down a clear and concise definition of the decision “task”. The best decision making model or assessment system may fail if you have not defined the end objective of the decision making process. Read more...
While not complex, the definition should go beyond something like “select some good projects” to a statement which is clearly linked to your goals – such as “select the top 20 projects which maximize the potential benefits to our stakeholders”. You can provide more detail as to the “benefits” you want, who the “stakeholders” are, etc. Keep it clear and simple and you will start your decision process on the right foot.
2. Selecting criteria
It often seems easy to create selection criteria for a decision process. In fact, five minutes of thought will often come up with some selection criteria for just about any assessment or evaluation process. Creating the right selection criteria, however, is the most critical step. Read more…
Structured brainstorming with key stakeholders and members of the decision making team can be used to create a set of selection criteria which are clear, which focus applicants on putting the best proposals forward and which enable the team to make decisions which have the maximum chance of achieving your goals. Processes for organizing and articulating each of the selection criteria make the odds even better.
3. Rating and ranking
“Please rate on a scale on 1 to 10”. How many times have you been asked to rate or rank something this way? How many times have you wondered what the real difference was between a “7” and an “8” – and – is a “6” twice as good as a “3”? Read more…
Did you ever stop to think if your view of a “7” would be the same as everyone else’s? Rating or ranking is the heart of evaluating anything. Using “traditional” ways of ranking, however, leave the entire decision process open to subjectivity and personal bias. Using clear language to describe the “steps” in a rating or ranking “scale” ensures that everyone from the applicant to the reviewer to the decision makers fully understand what each step means in the context of the goals to be achieved. Only then can the rating or ranking component of a decision making process be fully effective.
4. Creating application forms
The input to every evaluation process is an application form. Creating application forms, however, is often difficult and frequently starts the decision making process off on the wrong foot. To do its job most effectively, the application form should follow a few basic rules. Read more…
- Keep it clear, concise and simple
- Ask for exactly the information that will be used in the decision process and make it clear to the applicant that there will be no other information considered
- Ask for the information in a structured, highly visible way which is clearly linked to the goals and objectives of your program/organization – and – which matches exactly the decision making framework
- Tell the applicant that they will get feedback in exactly the same order
5. Applicant self assessments
When completing an application, everyone tries to put their best case forward. Often, however, that case is presented through “rose coloured glasses” and when subjected to peer review during a rigorous decision process is found to be optimistic. Read more…
If the applicant is presented with the set of decision criteria and the full evaluation framework to be used by the reviewers and is asked to do a “self assessment” using these, the benefits to both the applicant and your program can be huge. Applicants present information which is more realistic and better focused on the decision process giving them a better chance. Your program will receive a better grade of application with more potential to achieve your goals.
6. The review process
The peer review step in a decision making process presents significant time and logistical challenges. One secret for making it easier is to make sure that the information presented to the reviewers and the responses requested from them is clearly organized and focused on the decision criteria – and only the decision criteria. Read more…
If all the information related to “The Project Plan” is always in the same place in every proposal and if the framework for how the reviewer is to assess the information is laid out in clear, language- related steps linked to the goals of your program, the review process becomes very logical. Giving the reviewers the opportunity to comment on their rating/ranking/assessment as they complete each criteria provides valuable information for the assessment process. Reviewers find that doing each review, and multiple reviews, takes up to half the time and gives them a feeling of having done a “better job”.
7. Compiling the results
Putting the results of a review (scores from ranking and rating) into a spreadsheet is often the most time-consuming part of a decision making process. If the evaluation or peer review team have also been asked to provide expert opinion and comments, the task of gathering and organizing this input can become overwhelming. Read more…
Using a database (there are many platforms available) makes the task of gathering the information much easier – and – it also makes the jobs of organizing, analyzing and presenting the results not only easier, but in some cases feasible. One key is to have the database and its analyses directly linked to the decision criteria and to build the outputs so that they clearly show how each application – and all the applications combined – will help achieve the goals and objectives of your program and organization.
8. Presenting the results
Presenting long lists of numbers or big spreadsheets as the primary data source for a decision making process is almost sure to result in a group of tired and bleary-eyed decision makers – and could significantly downgrade the quality of the decisions made. Read more…
If the decision information is gathered in a way such that it can be compiled and presented in a few consistent, high visualization impact charts and graphs, decision makers can quickly grasp the strengths and weaknesses of an application. They can then focus discussion on problems or issues and reach consensus. A 2×2 Grid and a bar chart profile showing the strengths and weaknesses of each application for each decision criteria, accompanied by focused comments from the peer review team, give the decision makers everything they need to know.
9. Making a desicion
The application forms have gone out, the applications have come in, have been compiled and presented to decision makers. How do you ensure that the “best” decisions will be made? Read more…
The key is to allow decision makers the time to discuss the things they really must discuss– the problems, discrepancies, lack of agreement, etc. – while ensuring that they spend minimum time on endless cycles around the table of “I agree..”, “I really like this one too..”. Presentation of information should clearly show which applications are very strong (and likely to require only minimal discussion) those which are weak (and which should be discussed to make sure that nothing has been missed) and the group “in the middle” where significant discussion is likely to be needed. Arrange the agenda to make sure that each group gets sufficient time – and – that there is time at the end to stand back, look at everything and make sure that everyone feels comfortable with the end result.