Culture of Discipline

Title stolen from a chapter in Jim Collins' book "Good To Great".

I had arranged for a demo of Agile development to our team and I thought maybe it was something that we could employ. However, to my dismay, one of our software practitioners (in fact he is one of the development managers) didn't like the process; he called it micro-management and claimed that his engineers would kill him if he tried something like that with them [in jest of course].

The process that was demonstrated was called SCRUM and it employs meetings every day asking developers what they did yesterday and what they're doing today. Moreover, this particular process asks for developers to enter their time spent doing development on a daily basis. It definitely involves more careful monitoring of people's time. However, this is an effective scheme for determining if software schedules will be met and for averting disaster. The bi-weekly or weekly status meetings don't cut it anymore. I can understand his point of view. I used to be a developer myself and probably would have hated to determine how long a task would take or be constantly asked how much time I spent on something everyday. [A little bit of "Don't tell me what to do"]

This needs to change if we're going to ever get good at software management. This is a culture change in the way of treating software engineers and asking them to change their behavior, i.e., to become more disciplined. As a software engineer, I was arrogant and cavalier. I prided myself on being an expert at C or C++ (or whatever it was I was doing) and any company should be glad to have me and my expertise. If I need to take as long as I took to get something done, then that's how much time it should have taken. Or if something wasn't done that should have been done, I was busy doing something else that was just as (if not even more) important. My time was my own; I didn't have to tell anyone what I was doing; and for the most part management didn't care. If I got things done a day early, no one had to know about it and I would surf the web and take it easy the next day. But that kind of attitude isn't how you build great software nor a great organization.

Micro-management is a dirty word. No manager probably wants to be called that. It's a death knell to be referred to as one. However, the truth of the matter is if you were to form new habits, you need to act as one. If you're losing weight, until you form the habit of watching what you're putting in your mouth, you will not lose weight. You need to micro-manage what you eat. If you're learning a new skill, you need to pay attention to the minutiae of each motion (e.g., swinging a tennis racket). Similarly, the same goes for management. If you're going to introduce new habits (or processes) within your team, you need to buckle down and enforce them until they become a habit. You need to micro-manage. If you're going to kick off a new process and expect your team to follow it, you're going to have to monitor and micro-manage them and quickly correct errors. This is a fact of life. Don't let the fear of being called a micro-manager deter you!

In fact, one of my most memorable experiences being 'micro-managed' was at a start-up company. A new senior executive had joined at a time when our product had fallen behind. This new exec quickly set up daily early morning meetings with key staff and a list of the issues. It was almost like a SCRUM meeting. He wanted updates on each issue and what progress was made since the previous day. At first I considered the meetings a pain. [I don't want to be at the office by 8 !!] It was intense, being in those meetings. You had to own up! And you knew what was expected of you at tomorrow's meeting. But after a while, as the issues were closed, one after another, and our software became mature, we really became cohesive as a team/company and felt proud that we were pulling together to put out a quality product. We eventually had a great product ready for the market. This guy did what his predecessor didn't do. His predecessor was hacking away at the weeds whereas this guy knew what needed to be done and did it. No surprise that he is now a Senior VP at IBM.

In fact, to really build an effective organization at all levels, that "micro-management" ethos needs to be practiced by every individual in the organization. Each individual needs to be disciplined enough to carry out his/her duties to the fullest. Jim Collins states: "People in the good-to-great companies became somewhat extreme in the fulfillment of their responsibilities, bordering in some cases on fanaticism". I liken micro-management to being present in each moment. If you're present in each moment of your life and fully engaged with what's going on, then you will fulfill your responsibilities. Or you will speak up and address something when it doesn't make sense and so on.

Effective Software Managers

Here's a link to a little blurb on being an effective software manager:
Effective Software Manager Blurb

and here's a link to the article the blurb also references:

What Makes an Effective Software Manager?

The author in the first link writes about his manager getting rid of ineffective people. This is really interesting because it's exactly the first thing Jim Collins writes about in "Good to Great": First, get the right people on the bus!

I also feel "Making the Call" [referenced in the two articles above] is incredibly important. I've found 'making the call' and 'taking ownership' go hand-in-hand. I'm seeing real-life examples where people who don't do so lead their employees in circles, running only to stand still and never without any sense of completion.

An example:

There is a service provided to us by an outside party and the contract was never signed. The service was executed nevertheless and the first quarter invoice for 2007 is still yet to be paid. Myriad attempts to get this invoice paid has led nowhere. I'm not sure who was responsible for dropping the ball. I do know there were many attempts by IT to pay the invoice. In addition, the contract was never signed for this service and this has been outstanding for almost a year. An email that I had sent out regarding the matter was never responded to. It would be amazing for a change to hear someone say, "OK, great. Let me look at that contract, sign it and hand it right back to you" and for them to actually do it. Instead, unreturned emails and phone calls are the norm. If my business user would 'make the call' or 'take ownership' this would make mine and many employees' lives easier. Instead, the issue gets buried until the next go-round.

I think in an environment like the bank where IT serves its business users and IT has traditionally acted subservient to the business, making the call is especially important. Many IT people I've seen don't want to challenge their business users and state their terms. Instead, they're led astray by business users who have no IT knowledge. I've actually found that when I grab the reigns and state unequivocally what the options are and what the challenges are of each option, then usually the business will comply. After all, we serve their interests. Why shouldn't they trust us? In fact, IT can be in a position where we can actually LEAD the business. However, when we are too eager to please or afraid to be clear about our intent, communication breaks down and we (IT) can be sent off on wild goose chases; goose chases, we might be resentful about, because we weren't clear from the get-go and because we were afraid to be straight with our users.

Metrics

We use MQC (Mercury Quality Center) to store our defects and enhancements. We have about a year's worth of data in there and no one has thought to look and see what it tells us. I recently undertook such an effort. I export ALL defects/enhancements from MQC into an Excel file which is then parsed by an Access Database. I then wrote various queries to examine the data.

Here are some metrics that I want to examine after each release of our software to see how we are doing:

1) # of Outstanding Defects as of completion of this release ; broken down by severity and what version they were found in [to see how long they've been lying around]

2) # of Outstanding Enhancements to be implemented; broken down by priority and when they were requested [to see how long they've been lying around]

3) Breakdown of Defects/Enhancements for this latest release by Component to see which component of the software was touched the most.

4) Average [Mean, Median, & mode] Time to Fix a Defect [Critical, High, Medium, Low] until now

5) Average [Mean, Median & mode] Time to Implement a New Request [Critical, High, Medium, Low] until now

NOTE: If we apply Metrics 4 & 5 after each release, we can see if the average decreases over time to see if we're getting faster at delivering fixes and enhancements.

These metrics are pretty simple but will give us terrific information on where we are and what we need to work on to improve our process.

Support from the Top

If Bosses, Line Managers & PMs made it clear they want to see:

1) Burndown charts where they can see the progress to date vs. what was originally planned. And that they expect to see progress every day. This means that even if the manager doesn't have time for a status meeting today, if they wanted to see one s/he should be able to. It should be updated daily in accordance with what was accomplished. Management needs to catch errors early and to see if a project is going off-course in order to be pro-active.

As with daily scrum meetings, why not have a scrum-like managerial meeting where a burndown chart is presented and progress to-date is discussed, progress to-be-made is discussed and obstacles are presented?

2) Working software after each iteration! This is the only true measure of progress! Managers should ask that the software be demo'ed after each iteration.

more projects would be delivered on-time and on-budget! More users would be happy.

Some Agile Practices

Agile Practices that I'm thinking of implementing and as adapted to the environment I work in.

1) Have daily meeting for 10 minutes; Being agile requires open communication; everyone should know what's going on all the time.
Each person states:
a. What s/he accomplished yesterday
b. What s/he's going to do today
c. Any pain points/issues/obstacles

2) Have a process improvement meeting every month or even bi-weekly in the beginning
a. Figure out what's working/not working in the process and make adjustments

3) The BA should be available as development proceeds in each iteration to answer questions that come up and to test. This happens naturally in practice anyway. Much as we prefer a complete spec up-front (which few people read anyway), we should be honest about what the actual process is. Do we want to spend time developing the best spec ever up-front or proceed with Just Good enough requirements and start coding so we get working software?

These questions can be asked in a one-off basis (developer calls BA) or during the short, daily status meeting.

4) Use Burndown charts rather than project plans to track progress. Burndown charts are effective and especially look good to management because you can compare your original estimate to how you're actually doing. You can absorb all kinds of info including your 'velocity', where you hit bumps in the road, etc. Traditional project plans do not offer this because you just keep overwriting the same plan over and over with the latest dates. The n-th plan, after it is updated, looks like it was supposed to happen by design when in reality, you made it conform to the current status of the project. There is no way to compare the n-th plan to the 1st plan to show you how close you were to your estimates. And if you can't do this, how are you going to improve your process? Process Improvement depends on measurement and re-measurement! [I know people will object and say MS Project allows you to do such and such; and this might be true but to date, I haven't found anyone who uses this feature.]

5) Iterative Development and scheduling by feature delivery rather than Initiation, Definition, Design, etc.

6) Get estimates based on time to implement features rather than Initiation, Definition, Design, etc. These estimates would likely be more accurate and something developers can give and live up to.

7) Deploy prototypes and let developers have a go at them to give feedback so you can make it even more bulletproof and to also advertise to them what's coming. Prototypes help developers get familiar with upcoming changes and new features and will enable them to give better estimates on implementation time after playing with a prototype. Prototypes also act as requirements specifications in and of themselves. Rather than have a developer wade through a huge specification document, s/he can fire up the prototype and see the actual behavior demonstrated! A prototype is worth a thousand pages [of requirements documents]!

8) After incorporating feedback from the Dev team, I would go show my users. The prototype needn't do everything. It's a good idea to storyboard a typical scenario and walk the user through it and paint a picture. Picture a child enraptured by a parent telling a story. You want to fully engage the user with a prototype and have s/he contribute meaningful feedback.