January 7, 2009

The missing link between management information and agile methods

It is now pretty much accepted, and to some degree starting to be confirmed by actual empirical data, that agile software development can lead to better bang-for-the-buck than the traditional requirements gathering -> analyze -> design -> develop -> test iterations with comprehensive documentation in each step of each iteration. The concept is rather simple: Do not spend time on details you are unsure will have value, and always make incremental improvements to a working product that you get up and running as quickly as you can.

Also, by involving the product owner directly in prioritizing product features, adapting to changing customer requirements will be built into the process. Another element of many of the agile methods is cross-functional and often self-managed teams that respond to a changing environment, rather than following a detailed plan. When you think of it, the conceptual leap from lean manufacturing as perfected by Toyota to agile software development is maybe not that large – at least not in underlying principles.

But, even if the customer’s senior management accepts the notion of increased value and efficiency through flexibility, they are often not willing to accept the notion of just developing in a broad direction to create as much valued functionality as possible until the money runs out. Some more structured planning and reporting is expected. So how do you handle this as a project manager? This is the topic of this post and I will not go any more into the details of agile methods as such here.

Often the solution chosen, if you cannot get the customer to choose an all agile approach (which I believe in practice often means a time and material type contract), will be a hybrid. Pragmatic as I am, this is an approach I have used in some of my projects. The problem is that this often results in a missing link between the software development process and the management information. This leads to the classical situation so often captured in the Dilbert cartoon, where engineers and management live in two distinct worlds with little communication coming across with the intended information still present…

This presents you as a project manager with two challenges:

  • How to translate the legal obligations and high level plans into meaningful guiding for the day-to-day work for your developers, without falling back to a detailed pyramid of specifications and the team spending a too large part of its time producing documentation that will not be used
  • How to translate the actual work done and progress made into a world with more structured plans with fixed milestones

These challenges are of course not new, but what makes them different than just a work breakdown type of task is that the details of what happens in the execution step is not planned upfront. It thus take a lot of skill to be able to agree the right high level milestones, so that they can be reached using agile methods for the actual execution. It really is a challenge to be as specific as necessary for the customer to be confident the value will be delivered as agreed, but not so specific that it kills the whole concept of agility upfront. In my view, this is maybe more an art than a science – so I am afraid I can provide no simple checklist to achieve that goal, but it helps to know that it is a challenge and think about it upfront.

And there is no reason to paint it all black. It is indeed possible to provide those missing links. For instance the burndown charts and list of completed issues of Scrum in the agile world provide excellent input for a Highlight Report in the structured planning world of PRINCE2. Similarly breaking down a PRINCE2 Stage Plan into a product (and later sprint) backlog of Scrum or a similar method can work just fine. However, there is of course tension between the different basic philosophies that you should discuss with your client. But as long as your client buys in to the idea that you actually deliver more and more relevant products, even if she might not be willing to let go of more traditional planning, I think that this hybrid model can work better than not trying to achieve any of the benefits of agile software development at all.

Agility as a more general term is certainly a key treat for a project manager to be successful. However, my take is that the toolkit of the agile software development movement often fall way short of covering what the customer needs to feel comfortable with her project. To avoid the situation with the missing link, the project manager must be comfortable operating in both worlds and tie them together in a coherent whole, while enjoying the trust of both the developers and the customer. Then the situation will hopefully not be exactly as Scott Adams sees it below:

I have heard some argue that the role of the project manager is obsolete when moving to agile/lightweight software development methods. I’d like to argue that it is more important than ever, but more demanding – as you have to be comfortable in a broader span of processes and skills than before. I will probably return to the role of the project manager and the mapping to roles in Scrum in a later post (should she be the scrummaster or the product owner do you think?).

Remember what Keynes once wrote: It is better to be roughly right, than precisely wrong. Agile methods are a way of achieving just that, but the direction you go must be managed along the way and senior level management may not want to adapt 100% to this new way of working. As the project manager, you thus have to be the translator and understand both worlds – in other words be, or at least provide, the missing link proposed by the title of this blog post.

Let me know what you think by leaving any comments you have below. As always, I hope to learn a lot from them.

6 comments:

  1. Hi buddy! A couple of thoughts:

    Getting teams to go "all agile" is hard enough; trying to "mix and match" agile and traditional approaches is going to be an even bigger challenge. It's a delicate balancing act between specifying too much or too little (or even the right vs. the wrong stuff).

    With the imminent economic meltdown, more projects will be fixed-price. This places more of the financial risk on the consultancies and probably means a less fluid change management process (ie. the client has to pay).

    The next year is going to be interesting to watch.

    ReplyDelete
  2. Cheers mate! Thanks for the comment.

    The starting point for my post is that it is sometimes not at all possible to go "all agile" for the whole project from contract management to actually writing the code, even if you want to. I think this also plays back a little bit to my previous post about alignment of dimensions.

    Sometimes the team can move pretty long towards "all agile" if they want to, but without customer collaboration and contracts aligned – this basically leaves the PM with two options[1]: 1) "No thanks, no agile", or 2) Try to mitigate the stretch. I think that hybrid approaches are very risky, but they can also work rather well. When it does, I think I would say in spite of contracts and because of the collaborative environment with the client...

    The point on right vs. wrong things (and not just level of detail) is a good one, if not always easy to categorize right - let alone do. I discussed something related with a colleague of mine, Ole Kroknes – you might have seen a brief interview about roughly the same in the Norwegian Computerworld publication, on Friday in the context of outsourcing. It is actually a little bit similar: If the wrong things are specified (or they are specified in too much detail) from the customer, synergies and scale effects will be replaced by customization and the price of tailoring...

    PS. I agree about the downturn and rigid change mgmt in fixed price agreements, but don't you think that being able to have a short "time to value" and make fast incremental improvements may be a driver for adoption of more agile methods in this environment as well?

    [1] - Sometimes the PM is not given this choice – he is just handed the project to deliver (thank you very much). The contract may very well be established and agile vs. traditional methods might not even be a topic in the client's mind.

    ReplyDelete
  3. This comment has been removed by a blog administrator.

    ReplyDelete
  4. This comment has been removed by a blog administrator.

    ReplyDelete
  5. You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. class reunion websites

    ReplyDelete
  6. excellent piece of information, I had come to know about your website,i have read some posts of yours by now, and let me tell you, your site gives the best and the most interesting information. This is just the kind of information that i had been looking for, i would regularly watch out for the new posts.
    Agile Project Management

    ReplyDelete