Archive for October 2009

Welcome to Scrum Island! Now go home.

October 27, 2009

I’ve noticed a curious insularity in discussions among Scrum experts. These discussions focus mainly on how to get a Scrum team to run as smoothly as possible. Less commonly, there are discussions about how to introduce Scrum into a company, which focus on the benefits of Scrum for efficiency of development.

In short, the talk is about the view from “Scrum Island,” where the Scrum team lives. Either the islanders are discussing how to make the island work better, or they are talking about how to get foreigners to fund the island. One could be excused for thinking that the islanders have no concern for those who live elsewhere. In fact, I’ve heard of (but not personally experienced) cases where Scrum teams are actively hostile to the notion that outsiders have any right to know what is happening on the island.

The insular view that the team should be protected by an “ocean” (a procedural firewall) that makes it a black box from the outside is understandable, but unfortunate. It is true that the team’s internal workings should be protected from meddling by outsiders, but outsiders have a real need to know what the team has done, is doing, and plans to do.

The Foreigners

The software development process, and the teams that make it happen, do not exist in a vacuum. They exist in the context of a business that has customers, and which contains important roles outside the engineering department. Many of these outsiders are stakeholders for the process. Either they provide inputs to the process, or they are impacted by the process. These stakeholders (“foreigners”) have legitimate reasons to know what is happening on Scrum Island.

Who are these Stakeholders, and what do they want?

  1. Customer Support personnel need to enter problem reports into the bug-tracking system, complete with some assessment of impact or priority. They also need to know when bugs are scheduled to be fixed, or have been fixed, in order to respond to questions from customers.
  2. Professional Services personnel are in a similar position to Customer Support. They interact with customer who have purchased a service offering, and who expect that their bug reports and feature requests will receive attention.
  3. Not only do Salespeople get requests for features from current or potential customers, but they also must communicate expectations about the schedule for feature development. As a result, they place a high priority on getting this information, because it has a direct impact on quarterly sales.
  4. Directors, Vice Presidents, and C-Level executives need to plan out business strategy that is aligned with development schedules, which means they need to know (and influence) development plans.

Bridging the Ocean

Bridging the space between the “Islanders” and the “Foreigners” requires three things: tools, procedures, and attitude. Of these, the last is the most important, because it enables the first two to work smoothly.

“We’re all in this together” goes a lot further towards success than “us versus them.” If the Scrum team and external stakeholders can work together easily, the result can be very good. However, if the Scrum team refuses to share information, or external stakeholders try to interfere with the current Sprint’s work, the results will be dismal.

As for tools, the classic “sticky notes on whiteboard” solution for displaying Sprint content and progress does not meet the needs of stakeholders well except in very small companies. When teams and stakeholders are no longer on the same floor, the visibility suffers; when they are in different buildings, it vanishes. A more effective approach is to use an agile project-management tool, which provides a “single source of truth” for requirements, schedule, and progress for everyone, including the teams and external stakeholders. Many such tools exist (e.g., Rally, VersionOne, ScrumWorks), and they can be very effective at providing the information stakeholders need. (As an added benefit to both side, the Scrum Master is freed from the need to write reports for stakeholders.)

“Procedures” refers to the practice of entering requirements and plans into the project-management tool, and keeping the status information current. As Scrum teams have to do these things anyway, in order to function, there is no additional cost to supplying the information to external stakeholders.

Moving to the Continent

We don’t really want Scrum Islands. They arise as accidents, or in response to perverse incentives. What we really want is to have everyone, on Scrum teams and not, working together smoothly. A Scrum process provides the opportunity for tremendous visibility into work done and planned, and this information is valuable throughout the company. Making the information available is not difficult, and brings real rewards to everyone involved with supporting, selling, planning, and building products.

So let’s say good-bye to Scrum Island, and move to the Land of Agile Business. We’ll all be happier.

Scrum as Project Management

October 17, 2009

Experienced project managers, trained in the formal practice of project management, can be forgiven for feeling that they have entered the Twilight Zone on first encountering Scrum. While Scrum is a “process framework,” optimized for managing software projects, almost nothing about it seems familiar to someone with a PMP certification. And yet, if Scrum is about project management, its concepts should make sense to “classic” project managers.

How can we resolve this paradox?

Scrum Revealed

The apparent discrepancy between Scrum concepts and standard project management concepts is due partly to unfamiliar terminology, and partly to unfamiliar tradeoffs. Let’s look first at the terminology.

Terminology

The terminology of Scrum reflects the practice of working in short, fixed-length cycles called Sprints, a set of which produces a Release of the product. With this understanding, we can translate Scrum terms into language more familiar to project managers:

Project Management Term Scrum Term
Schedule Sprint (or Release)
Scope Sprint Backlog
Work Breakdown Structure Task Breakdown
Productivity Velocity
Estimate to Complete Burndown Chart

The correspondence is straightforward. The burndown chart, for example, is just the graph of remaining planned work versus time, which should trend down to zero on the last day of the Sprint. The Sprint Backlog is the set of requirements (“Stories,” in Scrum) planned for implementation in a Sprint.

Tradeoffs

The key to understanding Scrum is to understand what success means for a software project, since the definition of success drives the process.

Project managers are familiar with the “iron triangle” of scope, schedule, and cost. For any project, changes to any of these affect the others. For example, if the scope (or effort needed to achieve it) was underestimated, cost and schedule may have to increase to achieve the scope.

The traditional definition of success requires implementing the planned scope, on schedule, and on budget. When this isn’t possible, the next best thing is to trade off, perhaps extending schedule, adding resources (cost), or reducing scope. In most cases, however, achieving the specified scope, or something close to it, is the most significant part of the definition of success for the project.

Software is different. Unlike houses, software products evolve incrementally, and modest increments of functionality can provide significant new value to customers. Moreover, customer needs can change rapidly, as some of today’s expectations turn out to be less important in six months than other needs that materialize three months out.

The definition of success for most software projects is not to deliver a fixed scope in six months, but to provide desired features quickly in response to urgent (and changing) customer needs. Responsiveness trumps scope as the most significant element of success.

The need to optimize responsiveness drives us to an agile concept of project management, characterized in Scrum by

  • Short cycles (Sprints)
    • Allows frequent course corrections in response to changing customer needs
    • Enables quick delivery of urgent customer needs
  • Fixed schedules (uniform length for Sprints)
    • Guarantees reliable scheduling of release-quality code
  • Completion of features in priority order within each Sprint
    • Guarantees top-priority features will be completed even if actual effort for planned features exceeds estimates significantly

Scrum trades off between scope and schedule by freezing schedule and adjusting scope as necessary. The reason we do not fix scope is because effort estimates for new-feature development have consistently proven unreliable in the software industry, and rather than fight the losing battle for more accuracy, we optimize for what we can predict (schedule, which helps in planning delivery dates), rather than what we cannot (the delivered scope).

Conclusion

The apparent departure of Scrum projects from standard project-management concepts turns out to be an illusion. In fact, Scrum processes are tightly-choreographed and involve careful planning, as any successful project does. The illusion of otherness arises from the unfamiliar terminology, and an unfamiliar tradeoff of scope versus schedule. In the end, an effective Scrum project is indeed following sound project-management practices.

New Collaboration Tool for Distributed Scrum

October 11, 2009

I’ve recently had the opportunity to meet the good folks at Sococo, and review their interesting new Web-based collaboration tool. Team Space addresses a long-standing problem for every distributed Scrum project I’ve seen, namely, the absence of instant, on-demand collaboration that approaches the effectiveness of co-location.

I’ve used multiple flavors of instant messaging and Web-based presentation tools (like Skype and WebEx), but these solutions have significant limitations. They are very effective at addressing the needs for which they were created, but they were not created as general-purpose collaboration tools. Conducting distributed stand-up or sprint-planning meetings with these tools is possible, but laborious. The effort required to set up connections can make using these tools for ad-hoc collaboration impractical.

Sococo’s Team Space takes a very different approach. It provides a virtual office (complete with cubicles and conference rooms), which shows the availability and activities of everyone who is logged in to the application. Users can start online voice or chat discussions, share application views, and pull in more people as needed. The always-on nature of the product provides an on-demand experience, which differs from the scheduled nature of standard teleconference solutions in a manner reminiscent of how DSL connections differ from dial-up modems.

Team Space is in beta-test phase right now. The company is seeking beta testers, and is offering free accounts to people who want to try Team Space for their projects.

If the difficulty of collaborating with distributed team members has been a thorn in your side, I suggest heading over to Sococo and giving Team Space a try.

What should a Scrum Master ask an Interviewer?

October 8, 2009

David Bland (at Scrumology) has a great list of 10 questions a Scrum Master should ask an interviewer, after arriving for a job interview. One thing I look for when interviewing candidates is the quality of their questions, and this list would impress me. I intend to use it the next time I want to impress my interviewers! Check it out.

Introduction to Scrum: Benefits and Practices

October 5, 2009

Scrum is a lightweight agile process framework used primarily for managing software development. Scrum is

  • lightweight because it has few prescribed elements
    • Three roles: Team, Scrum Master (often a Project Manager), Product Owner (often a Product Manager)
    • Three meetings: Sprint Planning, Daily Scrum, Retrospective
    • Three artifacts: Product Backlog, Sprint Backlog, Burndown chart
  • agile because it maximizes responsiveness to changing customer needs
  • a process framework because it is not a process, but a collection of practices and concepts around which a process can be built

For those who are not already “doing Scrum,” the key question is not, “How does it work?” but, “What are the benefits?” This question does not have a unique answer, because it depends on who is asking. Benefits to developers, project managers, and salespeople are different.

This article identifies key benefits of Scrum, and the Scrum practices that produce them.

The Benefits of Scrum

Different stakeholders want different things from a software development process.

  • Developers want to write code, not documents.
  • Quality Assurance engineers want to create test plans that ensure product quality, and have high-quality code to test.
  • Project Managers want a process that is easy to plan, execute, and track.
  • Product Managers want features implemented quickly, with no bugs.
  • Services and Support personnel want to know exactly what is in all product releases, and have a reliable means to satisfy customer requests for bug fixes and enhancements.
  • Sales personnel want to know what is “in the pipeline” for future releases.
  • Customers want all of their feature requests and bug-fixes done quickly.
  • Executives, Program Managers, and PMO personnel want to know exactly what is happening, and what is planned to happen.
  • Everyone wants happy customers.

The list seems long, but the key points are few:

  • Team satisfaction and productivity are maximized when effort spent on non-deliverable items (e.g., internal documentation) is kept to a minimum.
  • Maximizing quality at each stage minimizes re-work at following stages, and maximizes product quality seen by customers.
  • Responsiveness is best achieved by fulfilling customer requests quickly.
  • Everyone who cares should be able to see all relevant information about project plans, status, and history.

Thus the best real-world development process devotes as little effort as possible to deliverables the customer doesn’t value, provides relatively bug-free code at the start of testing, delivers all relevant information to everyone who needs it, and fulfills customer requests quickly.

It is no coincidence that Scrum was designed to satisfy these points.

How Scrum Provides its Benefits

The following sections describe how Scrum practices produce the desired benefits.

Team Satisfaction and Productivity

The “team” consists of the development and Quality Assurance engineers who do the hands-on work of creating a high-quality product. Team members generally find their greatest satisfaction when they can do work that is rewarding.

  • For developers, this means designing and writing computer software.
  • For QA engineers, this means defining the exact criteria for success through the test cases they develop.
  • For all team members, this means producing something they are proud of.

Productivity goes hand-in-hand with eliminating unnecessary work. Scrum addresses team satisfaction and productivity by emphasizing work that is valuable (as a deliverable) and rewarding (to the team), and de-emphasizing what is not (non-deliverable artifacts).

In practice, “non-deliverable artifacts” usually consist of internal documentation about product requirements and design, which customers do not see or value. Scrum projects do require some written documentation, but minimize it by relying as much as possible on real-time communication between people. Thus a Product Manager will write brief requirement descriptions (called “Stories”), and elaborate on the details as needed in discussions with team members.

The requirement for effective real-time communication means that one of the following must be true for all team members, Product Managers, and Project Managers (in order of decreasing desirability):

  1. All are in the same building
  2. All are in the same city
  3. All are in time zones that overlap at least four hours per day
  4. All are willing to spend hours per day outside normal working times (e.g., transoceanic teams).

The last three cases can only be made to work if real-time teleconference and Web-conference capabilities are available on demand.

Maximizing Quality

Teams implement Stories to the requirements, in a very literal sense: An implementation is not complete (a story is not “done”) unless it satisfies the requirements, as defined in the test cases. While test-driven development is not required for Scrum, test cases do define whether the requirements have been met, and no story is complete unless it passes all of its test cases. If bugs arise, developers fix them until the tests succeed.

This practice ensures that each Story implementation is bug-free, with respect to the requirements, at the time of its completion. It does not prevent regression bugs, so additional testing is necessary after all development is frozen. However, the quality of the product going into regression testing is higher than is the case for products going into the final test period for waterfall projects, and high quality ripples through all stages of the process.

Maximizing Responsiveness to Customers

Responsiveness means providing turnaround to customer requests in a manner that is consistent with customer priorities. Since instant turnaround is not possible, the next best thing is to respond quickly to high priorities, and less quickly to low priorities.

The only way to deliver any new feature or bug-fix quickly is to work in short development cycles, which is why the basic unit of Scrum development, the “Sprint,” is typically 2-4 weeks in length. Longer cycles, composed of two or more Sprints, are also common and often referred to as “Releases” (which is not a Scrum term).

Productivity and job satisfaction both require that people are productively employed, not sitting idle, which means that parallel work for team members is the norm. The two strategies for parallelizing work on a set of Stories are

  • Parallel work on serial Stories. The whole team collaborates on one Story, until completion, then begins work on the next.
  • Parallel work on parallel Stories. Each team member works on a different Story, until completion, then starts on another one.

Since Sprint lengths are “time boxed” (have rigidly-enforced durations), and unexpected problems can occur, it is often not possible to complete all work planned for a Sprint. For this reason, it is critically important that Story development be serialized as much as possible. This allows us to deliver, say, eight of ten planned Stories when only 80% of the expected work can be completed. In contrast, the parallel-Story strategy might produce no completed Stories at all in this case, and deliver zero value to customers.

The need to serialize Story development implies another important Scrum concept: Ranking. The set of Stories planned for a Sprint is called the Sprint Backlog, within which Stories are ranked (sequenced) for implementation. The Product Manager (say) is responsible for ranking the Stories, so that the most important ones are done first. (The Sprint Backlog is a subset of the larger Product Backlog, which contains all un-implemented requirements.)

The combination of short development cycles and ranking of requirements maximizes responsiveness to customer needs.

Providing Transparency

“Transparency” means that all steps, inputs, and outputs of the development process are visible-but to whom?

In the narrow sense, as typically described in books on Scrum, transparency applies to the internal membership of the team, the Scrum Master, and the Product Owner, as they need to know the status of the project every day. In this case, and for co-located teams, transparency may be provided by posting index cards or sticky notes with the current Story and task status, along with the current burndown chart, in a public location. The Scrum framework essentially guarantees this level of transparency.

(A “burndown chart” is a bar or line chart showing, each day, the amount of this Sprint’s planned work that remains to be done. The ideal progress is indicated by a diagonal line, trending down to zero on the last day, against which the actual state is compared.)

Transparency in the wider sense means that every stakeholder who has a need for project status information has immediate access it. “Status information” includes not only the status of the current Sprint, but the content of past Sprints or Releases, and the Product Backlog. The Scrum framework does not provide a standard practice to meet this need, but it provides excellent an excellent foundation for meeting it.

Transparency for stakeholders and distributed teams can be achieved via agile project-management applications (e.g., Rally or ScrumWorks), to which all team members and stakeholders are given access. These applications store all requirements and task definitions, track work status, and provide sophisticated reports. They enable distributed teams to collaborate, and allow stakeholders to query for the information they need, without adding a burden on the team or Scrum Master.

Conclusion

Scrum is designed to optimize team satisfaction and productivity, product quality, responsiveness to customers, and transparency for stakeholders. The key practices that enable these benefits include de-emphasizing work on non-deliverable items, implementing and finishing each Story in a Sprint Backlog in rank order, working in short Sprints of 2-4 weeks, and making past, present, and future project information available to all stakeholders.