Not long ago, I participated in a discussion about effort estimation in Scrum. The basic point is simple: In order to plan a Sprint, we need to have reasonable estimates for how much work the team can do in the Sprint (its “velocity”), and how much work is needed to implement each Story under consideration.
The discussion revealed a major split between two approaches for estimating effort and velocity. While both defined a sizing unit named “point,” the definitions were quite different.
Points as a Measure of Effort
“Effort” refers to an amount of work. The units of effort are time-based, such as “person-hours” or “person-days,” and describe the time spent on a specific task by a person.
Effort differs from duration in that a single person-day of effort may be spread across two or more calendar days, if the person only devotes part of each work day to the task. The ratio between the effort someone spends on a task, and the duration of the task, is his availability. Even someone who is officially dedicated to a particular task full time will have some of his day taken up by overhead, such as meetings, phone calls, and email. (A reasonable rule of thumb for the availability of someone dedicated full-time to a task is 75%, meaning he is likely to spend about six of his eight work-day hours on the task.)
When points are taken as a measure of effort, the usual definition is that one point equals eight person-hours, or one perfect person-day. Team members then estimate a Story’s size based on how much effort they believe the Team will expend to implement the Story.
The Team’s velocity is the number of points of effort available for the Sprint, based on the members’ availability and the number of workdays in the Sprint. Given the velocity and point estimates for Stories, it is easy to determine how many of the top candidate Stories will fit into the Sprint.
Points as a Measure of Complexity
It seems obvious that the complexity of a Story affects the effort required to implement the Story, so complexity makes sense as a sizing metric. Unfortunately, there is no standard means to define complexity for a Story. (Function-Point and related techniques might provide such a standard, but are not commonly used in Scrum projects.) Instead, various estimation techniques are used to create a relative numerical scale, such that a Story with a larger point estimate is more complex than one with a lower point estimate.
Estimation techniques often analogize Story complexity to common physical tasks, such as “moving piles of dirt” or “painting the house,” or to commonplace scales such as “T-shirt sizes.” Historical information is very important, so that a Team can say, “This Story is about as big as that three-point Story X from last Sprint, so we should estimate it at three points.”
Team velocity is then not computed a priori from team size, but based on historical data regarding how many points the Team implemented in the last Sprint. Again, once Story sizing and Team velocity are known, it is easy to determine how many of the top candidate Stories will fit into the Sprint.
Which is Better?
We can’t answer the question unless we know what “better” means. Possible measures of “better” include
- Reliability of predictions about the Team’s ability to complete a planned set of Stories per Sprint
- Ease with which Team members understand and internalize the scale
- Ability to measure increases in productivity over time
- Support for Scalability, by ensuring uniformity of definitions across Teams
- Transparency and comprehensibility to external stakeholders
- Opacity and insulation from interference by external stakeholders
The first measure should be the most important, since it is the reason for having an estimation process. The other measures are secondary (and 5 and 6 are diametrically opposed).
So which approach is better, given these measures?
I believe that Effort estimates do better for measures 2, 4, and 5, while Complexity estimates do better for measures 3 and 6. I have seen no compelling evidence that either approach is superior for the key measure, #1. In other words,
It doesn’t matter whether you choose Effort or Complexity estimates for Story sizing and Sprint planning. They both work.
Heated debate between comparable alternatives usually reflects strongly-held values. If one were clearly superior, there would be no need for debate. If neither were superior, and strongly-held values were not an issue, no one would care enough to engage in debate. The fact that such a debate exists tells me that such values have come to the fore, and I find this interesting.
What are these values? I can’t speak for everyone, but here are my guesses:
Effort estimation appeals to those who prefer metrics that can be measured against an objective standard, such as time. These people
- Value scalability, and so prefer metrics that are uniform across teams
- Have a good relationship with external stakeholders, and want to provide the latter with useful status information that is easy to understand
- Like to create mathematical models, and find them useful
Complexity estimation appeals to those who prefer metrics that cannot be measured against an objective standard. These people
- Do not want anything that smells of the ‘bad old days’ of waterfall projects, such as time-based metrics, because the latter open the door to criticism when performance differs from estimates
- Do not trust external stakeholders, and wish to keep status information private within the Team, and unintelligible to outsiders, because outsiders may try to meddle if they know what is happening within the Team
- Dislike mathematical models, and do not trust them
I don’t believe that the above reasons provide the whole story for why some people prefer one sizing metric over the other. I’m sure that personal feelings about what feels most natural play a role, and I’m also sure that those who started with a particular approach and found it successful see no reason to change.
However, I’ve seen enough insularity (‘Scrum islands’) and distrust between development teams and business stakeholders to find these plausible influences for shaping metrics. I’ve also seen strong knee-jerk reactions among Scrum experts against anything that they associate with failed waterfall-style projects. I think the latter is a mistake, and leads to throwing out the baby with the bath-water, but it is definitely an influence.
So at least in some cases, I suspect that a preference for subjective measures of Story sizing and velocity is driven partly by distrust and an ‘us versus them’ attitude. If so, I think it would be wise to build bridges and improve trust, rather than accept the status quo.