I’m moving on…

Posted December 15, 2009 by Kevin Thompson, Ph.D.
Categories: Uncategorized

I’ve joined cPrime as an agile/Scrum instructor and consultant. I’d been talking with the good folks at cPrime for several months, and am very happy to be joining them.

Part of my work for cPrime will be to write for the cPrime blog, which means that Deep Scrum will go dormant. Some of my old from Deep Scrum material will appear on the cPrime blog, and so will all of my new material. I expect to produce a similar mix of instructional articles, musings about how and why Scrum works (and occasionally, doesn’t work), and opinion pieces.

So come over to https://www.cprime.com/blog/ and say hi! And if you have been following my writing on LinkedIn (where my profile is www.linkedin.com/in/kevinthompsonphd), be sure follow my cPrime profile in the future (www.linkedin.com/company/cprime-inc.).

A Brief Defense of Time: Estimating Sizes for Scrum Projects

Posted November 12, 2009 by Kevin Thompson, Ph.D.
Categories: Uncategorized

Not long ago, I participated in a discussion about effort estimation in Scrum. The basic point is simple: In order to plan a Sprint, we need to have reasonable estimates for how much work the team can do in the Sprint (its “velocity”), and how much work is needed to implement each Story under consideration.

The discussion revealed a major split between two approaches for estimating effort and velocity. While both defined a sizing unit named “point,” the definitions were quite different.

Points as a Measure of Effort

“Effort” refers to an amount of work. The units of effort are time-based, such as “person-hours” or “person-days,” and describe the time spent on a specific task by a person.

Effort differs from duration in that a single person-day of effort may be spread across two or more calendar days, if the person only devotes part of each work day to the task. The ratio between the effort someone spends on a task, and the duration of the task, is his availability. Even someone who is officially dedicated to a particular task full time will have some of his day taken up by overhead, such as meetings, phone calls, and email. (A reasonable rule of thumb for the availability of someone dedicated full-time to a task is 75%, meaning he is likely to spend about six of his eight work-day hours on the task.)

When points are taken as a measure of effort, the usual definition is that one point equals eight person-hours, or one perfect person-day. Team members then estimate a Story’s size based on how much effort they believe the Team will expend to implement the Story.

The Team’s velocity is the number of points of effort available for the Sprint, based on the members’ availability and the number of workdays in the Sprint. Given the velocity and point estimates for Stories, it is easy to determine how many of the top candidate Stories will fit into the Sprint.

Points as a Measure of Complexity

It seems obvious that the complexity of a Story affects the effort required to implement the Story, so complexity makes sense as a sizing metric. Unfortunately, there is no standard means to define complexity for a Story. (Function-Point and related techniques might provide such a standard, but are not commonly used in Scrum projects.) Instead, various estimation techniques are used to create a relative numerical scale, such that a Story with a larger point estimate is more complex than one with a lower point estimate.

Estimation techniques often analogize Story complexity to common physical tasks, such as “moving piles of dirt” or “painting the house,” or to commonplace scales such as “T-shirt sizes.” Historical information is very important, so that a Team can say, “This Story is about as big as that three-point Story X from last Sprint, so we should estimate it at three points.”

Team velocity is then not computed a priori from team size, but based on historical data regarding how many points the Team implemented in the last Sprint. Again, once Story sizing and Team velocity are known, it is easy to determine how many of the top candidate Stories will fit into the Sprint.

Which is Better?

We can’t answer the question unless we know what “better” means. Possible measures of “better” include

  1. Reliability of predictions about the Team’s ability to complete a planned set of Stories per Sprint
  2. Ease with which Team members understand and internalize the scale
  3. Ability to measure increases in productivity over time
  4. Support for Scalability, by ensuring uniformity of definitions across Teams
  5. Transparency and comprehensibility to external stakeholders
  6. Opacity and insulation from interference by external stakeholders

The first measure should be the most important, since it is the reason for having an estimation process. The other measures are secondary (and 5 and 6 are diametrically opposed).

So which approach is better, given these measures?

I believe that Effort estimates do better for measures 2, 4, and 5, while Complexity estimates do better for measures 3 and 6. I have seen no compelling evidence that either approach is superior for the key measure, #1. In other words,

It doesn’t matter whether you choose Effort or Complexity estimates for Story sizing and Sprint planning. They both work.

Heated debate between comparable alternatives usually reflects strongly-held values. If one were clearly superior, there would be no need for debate. If neither were superior, and strongly-held values were not an issue, no one would care enough to engage in debate. The fact that such a debate exists tells me that such values have come to the fore, and I find this interesting.

What are these values? I can’t speak for everyone, but here are my guesses:

Effort estimation appeals to those who prefer metrics that can be measured against an objective standard, such as time. These people

  • Value scalability, and so prefer metrics that are uniform across teams
  • Have a good relationship with external stakeholders, and want to provide the latter with useful status information that is easy to understand
  • Like to create mathematical models, and find them useful

Complexity estimation appeals to those who prefer metrics that cannot be measured against an objective standard. These people

  • Do not want anything that smells of the ‘bad old days’ of waterfall projects, such as time-based metrics, because the latter open the door to criticism when performance differs from estimates
  • Do not trust external stakeholders, and wish to keep status information private within the Team, and unintelligible to outsiders, because outsiders may try to meddle if they know what is happening within the Team
  • Dislike mathematical models, and do not trust them

Conclusions

I don’t believe that the above reasons provide the whole story for why some people prefer one sizing metric over the other. I’m sure that personal feelings about what feels most natural play a role, and I’m also sure that those who started with a particular approach and found it successful see no reason to change.

However, I’ve seen enough insularity (‘Scrum islands’) and distrust between development teams and business stakeholders to find these plausible influences for shaping metrics. I’ve also seen strong knee-jerk reactions among Scrum experts against anything that they associate with failed waterfall-style projects. I think the latter is a mistake, and leads to throwing out the baby with the bath-water, but it is definitely an influence.

So at least in some cases, I suspect that a preference for subjective measures of Story sizing and velocity is driven partly by distrust and an ‘us versus them’ attitude. If so, I think it would be wise to build bridges and improve trust, rather than accept the status quo.

When Scrum is too Much

Posted November 2, 2009 by Kevin Thompson, Ph.D.
Categories: Uncategorized

Scrum was designed as a lightweight process framework for software development, with the intent to impose a minimal amount of overhead and structure. So it is interesting to consider cases when Scrum is itself too “heavyweight” for a project, and something lighter is more appropriate.

The “Sweet Spot” for Scrum

Scrum is optimized for projects that have high uncertainty (due to the inability to estimate effort accurately), a high likelihood of frequent changes of direction (due to rapidly-changing requirements), and the need to respond quickly to customer requests. For these reasons, it mandates short development cycles of uniform length, to minimize risk and maximize responsiveness to rapidly-changing customer needs. The emphasis on short cycles enables a predictable schedule of releases, each of which is the result of one or more development cycles.

The “Sour Spot” for Scrum

Let’s think about some examples of projects for which Scrum is not particularly useful.

I don’t use Scrum to make a cheese sandwich. I use a waterfall process, following a template devised long ago: I get out two pieces of bread, slice enough cheese for the filling, and assemble the sandwich.

I also don’t use Scrum to build a set piece for a play. Instead, I collaborate with as many other volunteers as we can reasonably use, and work on the piece until we have to quit for the day, or until it is done. After one or more work sessions, we finish the piece and move on to the next. This process is neither Scrum nor waterfall in nature, but bears more of a (rather vague) resemblance to Kanban.

So what do these examples tell us? Basically, they say that Scrum is much less useful when the work is thoroughly understood in advance, or when the delivery date is not specified, and not cyclic.

A Real-World Example: LifeHints

I’ve been helping out some friends at LifeHints over the last year, working with them to set up development guidelines. My first thought, of course, was that the LifeHints folks should use a Scrum process, but I soon changed my mind.

Why not use Scrum? Well, before their first release, they had no established customers, and so no customer requests, for features or release schedules. Thus the cyclic nature of Scrum provided no particular benefits, and responsiveness to customers was not an issue.

What was in issue was the need to build a system incrementally, starting with the simplest scenario (logging in and showing a home page), and adding functionality bit by bit over time until the application had enough to be useful. This process definitely required agility, as the requirements evolved in unpredictable ways over time. It also required daily, or near-daily, builds that worked, so that everyone could see how the new features were shaking out.

In a nutshell, the LifeHints team, which contained both Product Owner and Team member roles, worked in short but variable-length iterations, refining requirements and refactoring code as needed, until the evolving vision and the evolving product converged to a useful reality.

This process was very effective, and I suspect it occurs frequently in pre-release startup companies. I also suspect that, as the company and its customer-base grow, something more formal will eventually be required, and Scrum will become an attractive option.

Conclusions

Everyone knows the saying, “When your only tool is a hammer, every problem looks like a nail.” Scrum is a tool in the toolbox of project management, and, like any tool, is better-suited for some situations than others. For the people at LifeHints, Scrum was too structured and prescriptive for the company’s stage of development. Given the focus on Scrum as a lightweight framework, I find the irony gently amusing.

(For the curious: LifeHints provides guidance on what things you need to do to live a life that reflects your personal preferences. You can check it out at www.lifehints.com.)

Welcome to Scrum Island! Now go home.

Posted October 27, 2009 by Kevin Thompson, Ph.D.
Categories: Uncategorized

I’ve noticed a curious insularity in discussions among Scrum experts. These discussions focus mainly on how to get a Scrum team to run as smoothly as possible. Less commonly, there are discussions about how to introduce Scrum into a company, which focus on the benefits of Scrum for efficiency of development.

In short, the talk is about the view from “Scrum Island,” where the Scrum team lives. Either the islanders are discussing how to make the island work better, or they are talking about how to get foreigners to fund the island. One could be excused for thinking that the islanders have no concern for those who live elsewhere. In fact, I’ve heard of (but not personally experienced) cases where Scrum teams are actively hostile to the notion that outsiders have any right to know what is happening on the island.

The insular view that the team should be protected by an “ocean” (a procedural firewall) that makes it a black box from the outside is understandable, but unfortunate. It is true that the team’s internal workings should be protected from meddling by outsiders, but outsiders have a real need to know what the team has done, is doing, and plans to do.

The Foreigners

The software development process, and the teams that make it happen, do not exist in a vacuum. They exist in the context of a business that has customers, and which contains important roles outside the engineering department. Many of these outsiders are stakeholders for the process. Either they provide inputs to the process, or they are impacted by the process. These stakeholders (“foreigners”) have legitimate reasons to know what is happening on Scrum Island.

Who are these Stakeholders, and what do they want?

  1. Customer Support personnel need to enter problem reports into the bug-tracking system, complete with some assessment of impact or priority. They also need to know when bugs are scheduled to be fixed, or have been fixed, in order to respond to questions from customers.
  2. Professional Services personnel are in a similar position to Customer Support. They interact with customer who have purchased a service offering, and who expect that their bug reports and feature requests will receive attention.
  3. Not only do Salespeople get requests for features from current or potential customers, but they also must communicate expectations about the schedule for feature development. As a result, they place a high priority on getting this information, because it has a direct impact on quarterly sales.
  4. Directors, Vice Presidents, and C-Level executives need to plan out business strategy that is aligned with development schedules, which means they need to know (and influence) development plans.

Bridging the Ocean

Bridging the space between the “Islanders” and the “Foreigners” requires three things: tools, procedures, and attitude. Of these, the last is the most important, because it enables the first two to work smoothly.

“We’re all in this together” goes a lot further towards success than “us versus them.” If the Scrum team and external stakeholders can work together easily, the result can be very good. However, if the Scrum team refuses to share information, or external stakeholders try to interfere with the current Sprint’s work, the results will be dismal.

As for tools, the classic “sticky notes on whiteboard” solution for displaying Sprint content and progress does not meet the needs of stakeholders well except in very small companies. When teams and stakeholders are no longer on the same floor, the visibility suffers; when they are in different buildings, it vanishes. A more effective approach is to use an agile project-management tool, which provides a “single source of truth” for requirements, schedule, and progress for everyone, including the teams and external stakeholders. Many such tools exist (e.g., Rally, VersionOne, ScrumWorks), and they can be very effective at providing the information stakeholders need. (As an added benefit to both side, the Scrum Master is freed from the need to write reports for stakeholders.)

“Procedures” refers to the practice of entering requirements and plans into the project-management tool, and keeping the status information current. As Scrum teams have to do these things anyway, in order to function, there is no additional cost to supplying the information to external stakeholders.

Moving to the Continent

We don’t really want Scrum Islands. They arise as accidents, or in response to perverse incentives. What we really want is to have everyone, on Scrum teams and not, working together smoothly. A Scrum process provides the opportunity for tremendous visibility into work done and planned, and this information is valuable throughout the company. Making the information available is not difficult, and brings real rewards to everyone involved with supporting, selling, planning, and building products.

So let’s say good-bye to Scrum Island, and move to the Land of Agile Business. We’ll all be happier.

Scrum as Project Management

Posted October 17, 2009 by Kevin Thompson, Ph.D.
Categories: Uncategorized

Experienced project managers, trained in the formal practice of project management, can be forgiven for feeling that they have entered the Twilight Zone on first encountering Scrum. While Scrum is a “process framework,” optimized for managing software projects, almost nothing about it seems familiar to someone with a PMP certification. And yet, if Scrum is about project management, its concepts should make sense to “classic” project managers.

How can we resolve this paradox?

Scrum Revealed

The apparent discrepancy between Scrum concepts and standard project management concepts is due partly to unfamiliar terminology, and partly to unfamiliar tradeoffs. Let’s look first at the terminology.

Terminology

The terminology of Scrum reflects the practice of working in short, fixed-length cycles called Sprints, a set of which produces a Release of the product. With this understanding, we can translate Scrum terms into language more familiar to project managers:

Project Management Term Scrum Term
Schedule Sprint (or Release)
Scope Sprint Backlog
Work Breakdown Structure Task Breakdown
Productivity Velocity
Estimate to Complete Burndown Chart

The correspondence is straightforward. The burndown chart, for example, is just the graph of remaining planned work versus time, which should trend down to zero on the last day of the Sprint. The Sprint Backlog is the set of requirements (“Stories,” in Scrum) planned for implementation in a Sprint.

Tradeoffs

The key to understanding Scrum is to understand what success means for a software project, since the definition of success drives the process.

Project managers are familiar with the “iron triangle” of scope, schedule, and cost. For any project, changes to any of these affect the others. For example, if the scope (or effort needed to achieve it) was underestimated, cost and schedule may have to increase to achieve the scope.

The traditional definition of success requires implementing the planned scope, on schedule, and on budget. When this isn’t possible, the next best thing is to trade off, perhaps extending schedule, adding resources (cost), or reducing scope. In most cases, however, achieving the specified scope, or something close to it, is the most significant part of the definition of success for the project.

Software is different. Unlike houses, software products evolve incrementally, and modest increments of functionality can provide significant new value to customers. Moreover, customer needs can change rapidly, as some of today’s expectations turn out to be less important in six months than other needs that materialize three months out.

The definition of success for most software projects is not to deliver a fixed scope in six months, but to provide desired features quickly in response to urgent (and changing) customer needs. Responsiveness trumps scope as the most significant element of success.

The need to optimize responsiveness drives us to an agile concept of project management, characterized in Scrum by

  • Short cycles (Sprints)
    • Allows frequent course corrections in response to changing customer needs
    • Enables quick delivery of urgent customer needs
  • Fixed schedules (uniform length for Sprints)
    • Guarantees reliable scheduling of release-quality code
  • Completion of features in priority order within each Sprint
    • Guarantees top-priority features will be completed even if actual effort for planned features exceeds estimates significantly

Scrum trades off between scope and schedule by freezing schedule and adjusting scope as necessary. The reason we do not fix scope is because effort estimates for new-feature development have consistently proven unreliable in the software industry, and rather than fight the losing battle for more accuracy, we optimize for what we can predict (schedule, which helps in planning delivery dates), rather than what we cannot (the delivered scope).

Conclusion

The apparent departure of Scrum projects from standard project-management concepts turns out to be an illusion. In fact, Scrum processes are tightly-choreographed and involve careful planning, as any successful project does. The illusion of otherness arises from the unfamiliar terminology, and an unfamiliar tradeoff of scope versus schedule. In the end, an effective Scrum project is indeed following sound project-management practices.

New Collaboration Tool for Distributed Scrum

Posted October 11, 2009 by Kevin Thompson, Ph.D.
Categories: Uncategorized

I’ve recently had the opportunity to meet the good folks at Sococo, and review their interesting new Web-based collaboration tool. Team Space addresses a long-standing problem for every distributed Scrum project I’ve seen, namely, the absence of instant, on-demand collaboration that approaches the effectiveness of co-location.

I’ve used multiple flavors of instant messaging and Web-based presentation tools (like Skype and WebEx), but these solutions have significant limitations. They are very effective at addressing the needs for which they were created, but they were not created as general-purpose collaboration tools. Conducting distributed stand-up or sprint-planning meetings with these tools is possible, but laborious. The effort required to set up connections can make using these tools for ad-hoc collaboration impractical.

Sococo’s Team Space takes a very different approach. It provides a virtual office (complete with cubicles and conference rooms), which shows the availability and activities of everyone who is logged in to the application. Users can start online voice or chat discussions, share application views, and pull in more people as needed. The always-on nature of the product provides an on-demand experience, which differs from the scheduled nature of standard teleconference solutions in a manner reminiscent of how DSL connections differ from dial-up modems.

Team Space is in beta-test phase right now. The company is seeking beta testers, and is offering free accounts to people who want to try Team Space for their projects.

If the difficulty of collaborating with distributed team members has been a thorn in your side, I suggest heading over to Sococo and giving Team Space a try.

What should a Scrum Master ask an Interviewer?

Posted October 8, 2009 by Kevin Thompson, Ph.D.
Categories: Uncategorized

David Bland (at Scrumology) has a great list of 10 questions a Scrum Master should ask an interviewer, after arriving for a job interview. One thing I look for when interviewing candidates is the quality of their questions, and this list would impress me. I intend to use it the next time I want to impress my interviewers! Check it out.