Reviving CMMI – a Conclusion?

June 3, 2013

Third item on this topic, I know. Some people believe that  I spend too much time complaining without proposing a solution, so here is my proposal: measurement and analysis should be expected from the start.

Many times, when I have asked for evidence of the implementation of measurement and analysis, I have been provided with evidence of project monitoring and control. Measuring that your project tasks are progressing, that you are respecting delays and budgets is not part of MA, it is part of WMC/PMC. It should not be difficult to include in either the model or the appraisal methodology a better set of examples of how MA should be applied, and place this as a requirement in the appraisal process.

Currently, the appraisal method focuses on the practices; during the training, we say that the goals are what is important and are required, the practices are only “expected”. However, during the appraisal, we focus on measuring the practices rather than the goals. We assume that if all the practices are in place, then the goal must be satisfied, if one practice is missing, then the goal is not satisfied. And even if we did, I cannot help but notice that the goals (and the purpose) statement all focus on activities, on tasks, on practices, and not on results.

Let’s require business measurements and demonstrations of results for each goal. CMMI is (supposedly) there to help achieve business productivity results and not just to do a series of tasks to get a certificate we can hang up behind the reception desk.

Why do you do “Requirements Management”? Show me results. Show me that the number of issues related to unidentified change requests has diminished; show me the reduction in unexpected requirements appearing during the V&V stage; show me data that show how the time to find a deviation from customer requirements in your project plans and other work products has gone down, because you have implemented a successful “two way traceability”. Show me measurements that demonstrate the impact on the quality of your products and services, on customer satisfaction. If you cannot show me that, you have not implemented a useful approach to requirements management. Of course, people will argue that you cannot measure improvements at maturity level 2, you can only really have useful measurements at maturity level 4. But that is not true. You do not need control charts or five years of history to show a trend. Because if you did, no one would manage to satisfy the expectation for a quality trends in the PPQA process area at maturity level 2 – or maybe you have just skipped over that passage and focused on having a static checklist? If you implement a practice, whether from the CMMI or elsewhere, it is because you expect to see a change one that practice is performed; if you expect to see a change, there is something which can be measured.

This is not a complicated addition to the model, it is only a clarification of the real purpose behind each one of the goals. A change in the appraisal process of this magnitude would ensure that people understand that CMMI actually has a benefit, and it would allow us (finally) to have some decent metrics as to the value of the approach.

I don’t know if anyone will read this or pay attention, but I am glad I got it off my chest. Hopefully, my next post will be about something else. Maybe I will develop a business based appraisal system of my own, but if I am not supported by the community at large, no one will use it and I will waste my time: it is so much easier to do what the model says without thinking.

and you might get a certificate to hang up, a logo to put on your website…



  1. The problem with numbers in general, is that human beings will find a way to gamble the system. That’s why I find measurements to be like nuclear fission, you can use it to do tremendous amounts of good of really bad things like the atom bomb.

    I think that when CMMI fails, it is because it was never meant to protect it’s adopters against misuse and it can’t do that now.

    Armed with numbers, any new (and clueless) project manager or team leader or empowered member, can start to interpret those (wrongly) and start to create havoc in a pressing situation by making bad decision based on partial number analysis. “IE: 1) In order to increase quality, bonuses will be based on numbers of bugs fixed” Then obviously the developers start checking in buggy code and fixing dozens of bugs that shouldn’t be there in the first place” 2) All projects except utilitarian projects must have an above 80% code coverage. Then developers start making everything they can get away with an utility to avoid the painful threshold.

    (I’ve personally seen allot more wrong interpretations of numbers than good ones)

    It may not be the fault of measurements by themselves, that they can be sometimes misused. However making sense of numbers if very very very hard in practice, otherwise we wouldn’t have statisticians and researchers. Most people will simply not get it, no matter what.
    CMMI puts these powerful tools in the hands of almost anyone without any regard of how they might be used.

    For me this is the point of failure.

    My 2c

    • Good point, BG. However, it is not because people continue to drive fast and use radar detectors that we should remove speed limits and controls. What I would like to see is a demonstration of business benefits. We probably need to train people better in the concept of measurement and analysis (and not just high maturity statistics). A CLA should be able to recognize an understanding of key improvement topics like MA and QA, but they still have to pass tests in which they show they remeber what is GP2.6 and 2.7. I reiterate my main point: appraisers need to be able to see real business benefits, not just activities.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: