The missing link between management information and agile methods

It is now pretty much accepted, and to some degree starting to be confirmed by actual empirical data, that agile software development can lead to better bang-for-the-buck than the traditional requirements gathering -> analyze -> design -> develop -> test iterations with comprehensive documentation in each step of each iteration. The concept is rather simple: Do not spend time on details you are unsure will have value, and always make incremental improvements to a working product that you get up and running as quickly as you can.

Also, by involving the product owner directly in prioritizing product features, adapting to changing customer requirements will be built into the process. Another element of many of the agile methods is cross-functional and often self-managed teams that respond to a changing environment, rather than following a detailed plan. When you think of it, the conceptual leap from lean manufacturing as perfected by Toyota to agile software development is maybe not that large – at least not in underlying principles.

But, even if the customer’s senior management accepts the notion of increased value and efficiency through flexibility, they are often not willing to accept the notion of just developing in a broad direction to create as much valued functionality as possible until the money runs out. Some more structured planning and reporting is expected. So how do you handle this as a project manager? This is the topic of this post and I will not go any more into the details of agile methods as such here.

Often the solution chosen, if you cannot get the customer to choose an all agile approach (which I believe in practice often means a time and material type contract), will be a hybrid. Pragmatic as I am, this is an approach I have used in some of my projects. The problem is that this often results in a missing link between the software development process and the management information. This leads to the classical situation so often captured in the Dilbert cartoon, where engineers and management live in two distinct worlds with little communication coming across with the intended information still present…

This presents you as a project manager with two challenges:

• How to translate the legal obligations and high level plans into meaningful guiding for the day-to-day work for your developers, without falling back to a detailed pyramid of specifications and the team spending a too large part of its time producing documentation that will not be used
  
• How to translate the actual work done and progress made into a world with more structured plans with fixed milestones

These challenges are of course not new, but what makes them different than just a work breakdown type of task is that the details of what happens in the execution step is not planned upfront. It thus take a lot of skill to be able to agree the right high level milestones, so that they can be reached using agile methods for the actual execution. It really is a challenge to be as specific as necessary for the customer to be confident the value will be delivered as agreed, but not so specific that it kills the whole concept of agility upfront. In my view, this is maybe more an art than a science – so I am afraid I can provide no simple checklist to achieve that goal, but it helps to know that it is a challenge and think about it upfront.

And there is no reason to paint it all black. It is indeed possible to provide those missing links. For instance the burndown charts and list of completed issues of Scrum in the agile world provide excellent input for a Highlight Report in the structured planning world of PRINCE2. Similarly breaking down a PRINCE2 Stage Plan into a product (and later sprint) backlog of Scrum or a similar method can work just fine. However, there is of course tension between the different basic philosophies that you should discuss with your client. But as long as your client buys in to the idea that you actually deliver more and more relevant products, even if she might not be willing to let go of more traditional planning, I think that this hybrid model can work better than not trying to achieve any of the benefits of agile software development at all.

Agility as a more general term is certainly a key treat for a project manager to be successful. However, my take is that the toolkit of the agile software development movement often fall way short of covering what the customer needs to feel comfortable with her project. To avoid the situation with the missing link, the project manager must be comfortable operating in both worlds and tie them together in a coherent whole, while enjoying the trust of both the developers and the customer. Then the situation will hopefully not be exactly as Scott Adams sees it below:

I have heard some argue that the role of the project manager is obsolete when moving to agile/lightweight software development methods. I’d like to argue that it is more important than ever, but more demanding – as you have to be comfortable in a broader span of processes and skills than before. I will probably return to the role of the project manager and the mapping to roles in Scrum in a later post (should she be the scrummaster or the product owner do you think?).

Remember what Keynes once wrote: It is better to be roughly right, than precisely wrong. Agile methods are a way of achieving just that, but the direction you go must be managed along the way and senior level management may not want to adapt 100% to this new way of working. As the project manager, you thus have to be the translator and understand both worlds – in other words be, or at least provide, the missing link proposed by the title of this blog post.

Let me know what you think by leaving any comments you have below. As always, I hope to learn a lot from them.