Why we always miss the date

– “In the next version I want your team to deliver this amount of functionality. I want you to estimate the effort and get back with a committed date you are going to deliver on”.

Anyone who has led development effort in our industry for a while had heard these exact or similar words many times. They also usually trigger an emotional response – “Committed date? How can I commit to a date if there are so many factors that affect the delivery which I do not fully control?”. After enough such experiences, in which our arms get twisted to provide a committed date, this natural response gets suppressed and we find this idea quite acceptable.
The fact is that the nature of most software projects is that these are extremely complex endeavors in both technical and social aspects. Moreover, they are constantly changing in those areas, which makes a proper management even more challenging. This nature of software projects results in realization that we cannot predict the delivery date with perfect accuracy and precision, as we are expected to.  So if we come up with a date it would be a lie. A real software estimation is a range of dates rather than a single date.

Let me step aside for a moment to clarify the difference between accuracy and precision. Accuracy is when your estimation date range matches the final outcome. Precision is when your estimation date range is narrow. For example, if you estimated your delivery date to be between Aug 3 and Aug 15 and you eventaully delivered on Aug 10 you were being accurate. If you were to deliver on Aug 25 or Jul 20 you would be inaccurate. Now, if you estimated your delivery date to be between Aug 3 and Aug 5 you were being very precise but not at all accurate since your estimation range did not contain the actual delivery date (which happened on Aug 10). So our goal in the estimation game is to be always accurate while increasing precision as we get closer to delivery. This is often referred to as the software projects’ cone of uncertainty. The further we are from the delivery the wider the estimation range, due to numerous uncertainty factors.

Still, more often than not we get our arm twisted by the customer to provide a concrete date. So what should we do?

Once we come up, internally, with a date range (meaning that the second date reflects a pretty high probability of delivering on or before that date, say 90%) we could communicate that date as a commitment, to be on the safe side. Even if the customer accepts this (usually unacceptable by him) date, the chances are still pretty low that the team will make it. And the reason for that is a so-called “student effect”. This effect is named after a human’s natural tendency to procrastinate unpleasant tasks to the latest time possible (just as a student postpones an exam preparation until it is too late). Two factors influence the degree of the student effect: how unpleasant a task is and how stressed we are (or how big the sense of urgency is – it is usually proportional to how close we are to the due date with the remaining work). Therefore, we need to minimize the effect of these two factors.

The first factor, the “unpleasantness” of a task, can be partially handled by matching the type of tasks to people. When people like the kind of tasks they are assigned to they usually focused on them a big part of the day.

The second factor, the sense of urgency, can be achieved by two different techniques. But first let’s understand why sense of urgency is so effective. What happens when we feel that the due date is far away? Well, all kinds of unimportant things. For example, we go to that all-hands meeting (2 hours!) just because we were invited; we answer phone calls from all kind of people that need our attention; we read mails and whatsapp messages that pop up frequently, we go for an 1.5 hour lunch etc., etc. Would we do all those things if we had a major release in two days while our tasks are gating it? Obviously not. We would shun ourselves from those interruptions. In other words, the sense of urgency “squeezes” the unimportant staff out of our daily work. In order to achieve the sense of urgency we have two techniques. The first is to track daily process using, for example, Scrum-like daily coordination meetings. Each day every team members “commits” himself to the task for the next 24 hours and reports back the next day the outcome of that commitment. In this case we break down the big pressure towards the end of the release into short, small daily pressures. The second is to separate due/target date from committed date. In this technique we communicate to the team a different date than what we commit to customers. Actually this is a good old buffer with a disturbing attribute that it is concealed from the team.

The first technique is generally far more preferable since it promotes transparency (while the other is actually a manipulation) as well as allows for constant monitoring of progress. Still, there are cases where the second is also an acceptable option.

Notice, that all the above assumed no change in delivery scope. Things get much more complicated when that happens.

To summarize, in order to improve our delivery predictability we need to work with date ranges while trying to keep the team engaged and moderately pressured on a daily basis before delivery.

Leave a Reply

Your email address will not be published. Required fields are marked *