Working on a CMM Assessment Team

I just completed an extended assignment working on our lab's CMM assessment team. In case you don't know what I'm talking about, CMM stands for the Capability Maturity Model, which is a set of software development best practices created by the Software Engineering Institute (SEI) at Carnegie Melon University.

We concentrated our research on the six Key Practice Areas (KPAs) for CMM Level 2, which focus on how repeatable your organization's business practices and procedures are. The KPAs were Requirements Management, Software Project Planning, Software Project Tracking and Oversight, Software Quality Assurance, Software Configuration Management, and Software Subcontractor Management. If you would like to learn more about these key practices, you can download the CMM documentation from the SEI here.

After a week of training in the CMM practices and assessment procedures from a SEI-certified assessor, our team took three weeks to review an extensive set of our lab's plans, schedules, documents, and meeting notes for three projects. From this research, we came up with initial observations about our lab's practices. We followed this up with a week of interviewing our management team and individual engineers, QA, and technical writers to confirm that they knew that documented practices existed, where to access them, and how to use them. We took the notes from these interviews (six 1.5 hour interviews over three days) and used these to confirm our initial work (and make new observations). When we were done, we had made approximately 150 observations about our lab practices, which were presented to the entire engineering group on the last day of the assessment.

For those of you who may be considering conducting a CMM assessment for your organization, I would like to share three of the most important things we learned during our assessment:

• Attempt to map your organizational language to the language used in the CMM. I can not stress the importance of this enough! As soon as our team was able to do this, making specific observations for our lab became a lot easier.
• Try to map CMM job titles to specific people in your organization. A lot of the CMM practices refer to specific job titles, like "Software Engineering Manager." You should become familiar with the CMM glossary definitions of all of these titles and attempt to define who specific people in your organization fit these job descriptions. This will also help you make observations easier, since you can put a specific face with a CMM practice.
• Schedule your interviews so you don't interview similar groups on the same day. We scheduled all of our interviews with engineering managers and product management for the opening day of the assessment, and we didn't have time to absorb what the first group said before going into our second interview of the day. Why is this a problem? You end up asking management more strategic, business-oriented questions than you do individual contributors. If we had to do this again, we decided that we should schedule a manager interview and an individual contributor interview on the same day, which would give us more of an ability to absorb what each group said prior to the next interview.

Personally, I found this to be a satisfying assignment. I was removed from my normal area of functionality and was put in a position to look at the big picture of how my lab works. I was able to look at reports and documents I do not normally review, and it was comforting to see how much deliberate thought is put into many of the decisions made during the life of a project. The assessment team did have some recommendations on how specific tasks could be done better, but as a whole I was glad to see that a majority of our procedures lined up with CMM best practices.

If you're asked to work on an assessment team, give it some thought -- it's a lot of work, but it's also pretty rewarding in the end.