Careful what you measure and compensate on, as that intevitably influences behaviors. Time and budget performance is but one measure of ability to plan and execute projects, and has little to do with whether a project meets business objectives. How many points does a project manager get that's brave and aware enough to recognize that a halfway through a project it isn't going to add business value, and raises that issue so it can be addressed?

A good project manager is tactically competent within the discipline; a GREAT project manager is also strategically astute, and never loses sight of the business outcome they are trying to effect.

If that means sometimes lobbying to kill a bad one, or extending a project a bit to ensure an effective transition to a production state rather than simply throwing a complex deliverable over the fence in the interest of personal performance metrics, these are the project managers you want to reward and hold up as examples.

Comment is spot on. I worked in an organisation where metrics were worshipped and an attempt was made to use cold, hard metrics to create objective evaluations of project managers. Measurement drives behaviour. Many PM's focused on getting metrics right - often to the detriment of project delivery or what was in the company's best interest. Even the best metrics can be manipulated. I don't believe it is possible to get a truly objective evaluation. Chasing metrics will eventually cause true delivery to go down the drain.

This article points out that you need to get your metrics right and a balanced approach. Metrics are fine so long as they're correct! I have found that subjective evaluation alone can be just as bad - it deteriates to the eyes of the beholder. I believe that any position requires strong leadership with frequent communication - period. That's the only way any method can be effective.

Objective measures, while important, are only part of the PM performance focus. Delivering the requirements, on time, within budget, and with high quality, for a project that doesn't fit where the organization is headed anymore is of limited value. I'm assuming the writer's objective scoring accommodates project changes (due to risk/issues, prioritization decisions, response to business needs, etc.) rather than point in time measures from the early stages of the effort. Also, the PM that continues to leave trails of burned bridges, pissed off performers, etc. isn't the type that I'd give an "A" to. Need to focus on the holistic side of Project Management to really guage the merit of the PM's performance.

The time and budget criteria might need to be dialed up or down on the impact scale depending upon the role the PM had in authoring the scope and deliverables of the project ... coming in later and being handed a no win situation should be taking into account ...

To me this is just 40% of a Project Manager's evaluation. There is a lot more to it. If you promote this culture - aren't you driving a culture of not allowing them to experiment or be creative, as there is an element of risk of failure there.
Moreover, Project could fail due to many reasons beyond the control. Its how they developed and definitely the quality. You could deliver a project the Way Users Ask Vs the Way Users Need. Where do we recognise this?
The set of developers resign as they were exhausted after 1 project - as they were streatched beyond their limits. But the project was successful! Where do we account this?
To summarize its a mix of results and the journey that a Project Manager should be evaluated on.

This is a good example of how function points can be used successfully. E.g. if the design calls for building and/or modifying 1000 fp for $1M, The Software Unit Cost is $1000/fp. Along the way, when the customer says "Did I ever mention we need...," the size will increase. But if the extra functionality can be delivered at $500/fp, it might be a good Business decison to do it now.
Actually we often see projects that get done on time, with satisfied customers but the budget gets exceeded by 20%. Using FP, PMs can demonstrate how the project delivered 50% more functionality for the 20% overage. This is a very positive Productivity statement.
Nevertheless, I agree that metrics are not the only consideration. Teams and PMs must still use Managerial Judgement and make Business decisions for Business reasons.

I agree with the comments, the very essence of project management is its dynamic nature, which demands a more fluid method to evaluate project managers. While metrics will play a role, other elements elements like - whether the business purpose was met, the complexity of the project need to be factored in.

Man, I thought this philosophy had died two years ago.

Back in the early '70s (yeah, before many of CIO's readers were born) a group from Harvard published the results of a study to determine why bonus plans fail. They determined that there was one and only one rule for a successful incentive (bonus) plan ... the bonus has to be directly tied to a behaviour which is both under the direct control of the person receiving the incentive and desired by the organization.

In short ... measuring ANY management on a success or failure basis is a total, absolute waste of energy, will inevitably result in a negative result and violates the basic rule of successful incentive-based management. Sorry, it ain't that easy. Get over it.

Yes, success or failure is important and should be incorporated into an evaluation. But who ever said that a project manager had any control on the success or failure of a project? In fact, the construction industry has a tool which is 80% accurate in its ability to predict success or failure of a project. The LAST item on the list from a timing point of view is the selection of a capable project manager! Which implies that a project manager can only influence success or failure in less than 20% of the cases! (Remember not all failures can be controlled -- that's why we have risk management).

My suggestion to you is that the best you can hope for is that a good project manager will tell you when the project should be killed, that when the project goes off the rails he/she will be able to bring it back and that as far as can be humanly done he/she will control the scope creep.

(I also suggest that you read some of the work done by Barry Boehm in EDS itself ... one of the cute things he discovered is that there is a 500% variation in the time required by different people to complete a specific, known task -- which means that no estimate or budget is worth anything until the team has been assigned.)

Glen Ford
http://www.TrainingNow.ca

It's always tough to measure good PMs and I (like most others) usually rely on past successes versus failures. These metrics are a little scary. I can't stop thinking about the following scenarios:

The CFO just called and said there's a cash flow problem and we need to delay phase 2 of the project....oops I just lost 14 points. Or...The CEO changed his/her mind on a requirement and we need to include another $2M in hardware...oops another 20 points down the drain.