Legacy Software

Working at an investment bank is challenging. There's constant pressure to deliver, not screw up, reduce operational risk (which is the new buzzword that represents the same concepts that we've always been wanting less of), reduce TCO (another buzzword), etc. and of course to produce new software that helps the business do deals or reduce their manual workload.

Here are a few of the challenges:

• Maintaining Legacy Software and/or trying to Decomm Legacy software
• Satisfying audit & internal regulatory requirements that have little impact
• Hiring button pushers rather than smart people who can actually debug issues

I will address the first two points in this post & leave the last for another day.

We have tons of old, legacy software whose original developers have moved on. I'm talking about software that just runs and that is mostly a black box. We feed them (i.e., keep the lights on, reboot the servers on a routine basis, monitor them) so they keep running but occasionally they fail. That means I need to pull a developer who is working on a new project to resuscitate the old software. This is painful because the developer is not very familiar with this software and it takes time to figure things out. Users may be impacted and thus screaming and escalating to senior managers, so this adds to the pressure. Note that most if not all of this software was not constructed with the latest agile practices like TDD so there are no unit or integration tests to speak of, and the documents (if they even exist) are yellow & crusty.

Of course, making changes to this software is risky, because the developer could make a change that impacts another part of the software and break something else. And without a good suite of tests, how will you ever know? Given the time pressure to get this working again in production, we can't do adequate testing. So we release into production and hope for the best. And we document what we learned in a wiki or FAQ, so we can at least build our knowledge base (hopefully). Then we all context-switch back to our new software development.

This is an oft-repeated pattern in my group and I would not be surprised if this is universal. (or I would be surprised if this was NOT universal!) Additionally, morale slips because we lose time fixing old software, we disappoint both the users of the legacy software and the new users who are expecting their new software on time, etc. Developers become disappointed because they can't seem to get traction on the new software as well.

So as a general rule of thumb, we should try to minimize modifications to the legacy code-base. If you need to do it, then do it. But let's look at one 'need' in more detail; this brings me to the 2nd bullet point above: Satisfying audit & internal regulatory requirements that have little impact

I'm currently on a project to satisfy an internal audit point (not a government regulatory item) that requires me to modify a legacy piece of software (which by the way we are trying to decommission). This change, I'm finding is incredibly risky. One, it touches at least a dozen files across the board and the software is responsible for getting all kinds of data intra-day and End of day. Two, the developer is in Singapore whereas the rest of the team is in NY and so we can't even talk real-time about the changes. Three, there are other, new components that the legacy software will need to make use of and so may require a 3rd party to be available on stand-by. Taken all together, it would have been wiser to get an exemption for this.

Unfortunately, this comes back to the way I presented the problem to my manager. I hadn't thought through all the ramifications of the change. I only knew I didn't think it was in our best interests. But I went to my manager and all I said was "Look, we have this audit point but we want to decomm this software so do we really want to do this?" and he said, "Yes, we need to". So off, I marched.

My point is that I didn't have a strong enough view on the situation. At that moment in time, I only had a 'leaning' but not a clear enough picture of what the whole thing would entail. If I did, I could have possibly persuaded him that this wasn't in our best interests to do.

So what's the lesson here? When modifying legacy software, (1) get the scope of the change (how long it will take, impact assessment, how many files/modifications need to be made, etc), (2) assess overall risk, (3) if too risky, push back and make the case as persuasively as possible.

Audit points, regulatory risk, operational risk are big buzzwords that make people jump and immediately act but wait! not so fast...  (1) there could be abuse of those terms and they can be inappropriately applied, especially if there are other agendas involved like politics & (2) they should not all be treated equally. Some audit points just aren't as important as others and we need to really understand what's important and what's not. It's similar to taking something in the spirit of the law and not just going by verbiage. So if the risk to satisfy this point is riskier than not satisfying it, well, that should be taken into account. We shouldn't just blindly do it because that's what the audit point says. Let's have a conversation about it and negotiate.

So in writing this point and thinking further about this, what I will do is: re-examine the exact scope of the change, and if warranted, re-have the same conversation with my manager asking for an exemption. I'll fill him in on the exact risks and have him sign off on email if he still wants to proceed. That way, if there are issues, he can't say he didn't know about them. And I'll have done my due diligence.

The Start-Up of You

The title is a reference to Thomas Friedman's Op-Ed column in the NYTimes, dated 12-July-2011. Click here for the article: http://www.nytimes.com/2011/07/13/opinion/13friedman.html?_r=1&scp=2&sq=thomas%20friedman&st=cse

 It caught my eye because it sounded similar to "The Brand of You" by Management Consulting guru, Tom Peters.

In any case Friedman is dead-on. So dead-on that I came out of semi-retirement to even write this post! :)

I won't rehash what he writes but I will raise examples in my own workplace that illustrates what he states.

E.g., We've been having cuts at the investment bank where I work, and it's just one in a parade of many. People have to constantly look for new jobs or are worried that they'll lose their existing one. Constant re-invention is a must. Workers must always look for ways to add value (which is one of the things I stated in my earlier posts). You can't be idle in this day & age. Look around for what else needs to be done. Kaizen! Continuous Improvement! People who are bored at work simply aren't looking. If you claim to be bored, look at it is a tremendous opportunity to use your imagination to poke into something/get involved and make suggestions.

Another eg. I've seen is what happens after the layoff. In my own group, we were at a height of 100+ people at one point. Now we're down to under 50. People got let go & others left on their own. But the company never replenished. We still have to maintain our ongoing operations however. So what does that mean? Many people are filling multiple roles and are busier than ever. We've gotten pretty damn lean and efficient. And furthermore, we're not just going to hire anyone. We did have a few openings for contractors. But it took several months to fill them. Why? Why when there's a glut of IT people out there did it take us so long? Because as Tom Friedman put it, "we are increasingly picky". We want "people who not only have the critical thinking skills to do the value-adding jobs that technology can’t, but also people who can invent, adapt and reinvent their jobs every day, in a market that changes faster than ever."


I also look at how I can make the job more efficient. I think in terms of the franchise model. The key to franchise success is how to replicate your business easily and cheaply. McDonalds hires burger flippers. These burger flippers don't stick around all that long, I'd gather. It's an easy skill to train them in, and if they leave, you train the next one. And so on. They have it down to a science. Well, even in IT, a similar thing can happen. I am working with a system that has traditionally caused a lot of noise. It takes someone of high intelligence to figure out what's going on. Since I've come into the group, I've whittled the noise down slowly but surely. I've gotten rid of the false positives. Now the system is beginning to hum. Any errors that actually do pop up are being documented in a HOW-TO guide. When I am done, suddenly we don't need that person of high intelligence anymore. That person can be re-assigned to work on more interesting tasks. And in that person's place? You got it: a burger flipper. Someone of mid-level intelligence who can follow instructions and handle basic tasks and attend to those errors. Not the greatest job, but hey, it's a foot in the door and people can move on once they get bored. Even for this job, though, I wouldn't just hire anyone. I'd want someone who could master the task easily and then say "What else? What else can I do?" 

Pure PMs vs. BAs

I've been noticing lately that pure Project Managers on Wall Street are losing popularity. Of course I have only a limited perspective but the sentiment I'm getting is that pure PMs aren't adding as much value. In fact I see a trend towards hiring more BAs with PM experience, or Senior Dev Leads/Project Managers. So some specialty role + PM experience is a more popular choice with employers.

A pure PM might fit into a PMO role better, but an actual project manager doing projects is better off diving into the details.

Collaborating & Engagement with Colleagues

I am on a new project with new colleagues. I had been working with the same personalities for so long now that I forgot what it was like to go through the Forming/Storming/Norming phases again!

It's interesting to see how some people prefer to work alone even when the work requires collaboration. I actually think this is an easy trap to fall into. In this era of the knowledge worker, each worker has such specialized skills and knowledge of his/her area that it requires serious effort for someone else to intervene and pick up a new task or role. It is just easier to wait for the person who specializes in that area to pick up the task. This can result in long wait times for things to get done, but that's a topic for another post.

A side effect of this type of specialization however is that people start to work in their own silos. Because one person can do so much in just their one area, a type of rigidity sets in and people only do what they know. If something is outside their bailiwick, they will assume someone else will pick it up; or they'll punt the task over to someone else and wash their hands of it and rush back to their own specialty business. In fact, that other person may be new and has no clue and is waiting for direction from you!

It's becoming harder to recognize that picking up the phone and talking to your colleague might be a good idea. A few face to face meetings and phone calls are the equivalent of 1000 emails (or IM chats) just as a picture is worth a thousand words. I moved the project many fold in 1 day than it had moved in several months.

There is nothing like the feeling of several of your colleagues hunched over a screen trying to work out a problem together. Research shows group problem solving is way more productive and efficient than solo solving. In fact, in a project management course I took a few months back I was witness to this exact phenomenon. We were told to do an exercise by ourselves. After that, we had to do the exercise as a group. This provided for a lot of debate because each person had their own views on what the correct answer was. We had to come up with a consensus and convince each other. Eventually we arrived at a solution. Then we graded our own individual solutions and then the group solution. What do you think the result was? The group solution score was higher than any of the individual solutions! I was astounded.

I can see myself get lethargic at times as well and just want to stay at my desk and manage from there. I have gotten facile however at recognizing when this happens and am able to override it and keep myself moving. I will make that extra phone call to push a dev ticket ahead or will sit with my colleague at his cubicle for an hour discussing an issue and how to address it. These are the little things that when added up move projects and eventually implement a company's strategy.