James Shore

An Approximate Measure of Technical Debt

19 Nov 2008

James Shore/Blog

Technical debt sucks, and it's a particularly common problem for the teams I work with. Technical debt affects everything they do. It disrupts plans, kills productivity, and creates defects. There is much wailing and gnashing of teeth, but strangely, these teams put very little effort into paying off the debt. (That's probably why they got into this mess in the first place.)

Part of the problem is that technical debt is, well, technical, so only programmers can see it. Managers and stakeholders can see the effects of the debt (the aforementioned wailing and gnashing of teeth), but they can't see the cause. So when programmers complain about the quality of the code, they often ignore them. "We're behind schedule already!" they cry, brandishing their whips. "Row faster!" And the poor programmers work as hard they can, taking more shortcuts, incurring more debt, and making things even worse.

Not that programmers are perfect. There are plenty of programmers who haven't a clue about good design, and they merrily rack up technical debt without even realizing they're doing it.

Measuring Technical Debt

I've been searching for a measure of technical debt (or its inverse, design quality) to help address this problem. Now, no measure is going to turn bad programmers into good programmers. Design quality is a human problem, and compilers eat spaghetti code just as easily as they eat clean code. But I think it would help if people could at least tell when they were making things worse. Unfortunately, traditional measures such as McCabe's cyclomatic complexity don't do a great job of exposing the large-scale technical debt I see most often.

I think I've finally found the right metric. It's a good one. Years of research has shown that this metric correlates closely with cost, effort, and bugs. I'm surprised nobody thought of it before now.

Ready?

It's lines of code.

The Case For SLOC

I'm not kidding. SLOC isn't a perfect metric, but I think (source) lines of code approximates technical debt. (And in case anyone is confused, I mean that more lines of code represent more technical debt. More lines of code, more technical debt, worse code quality. SLOC = bad, got it?)

Think about it. If we define technical debt as high cost of change, then SLOC fits the bill perfectly. Estimation gurus have long known that lines of code are correlated to effort and defects. In fact, many estimation tools work by taking a size estimate (either in lines of code or its language-neutral equivalent, function points), then running it through an algorithm that estimates project length, effort, and cost.

If that isn't enough, what's the most common form of technical debt? Duplication! Duplication! Duplication! Either the blatant duplication of cut-and-paste coding (leading to duplicated lines of code) or the subtle duplication of poor abstractions (leading to repeated concepts and algorithms). Cut duplication and you improve code quality. As Martin Fowler said in 2001, "I have an increasing sense that pig-headed determination to remove all repetition can lead you a long way toward a good design."

And then there's Carter's Compass, coined by John Carter: "I know I'm on the right track when by deleting code I'm adding functionality." Who hasn't had the wonderful experience of adding features while reducing the overall size of the codebase? It's a great feeling. You just know you're doing good work.

Nothing's Perfect

SLOC is an imperfect metric, and one objection is that adding features increases lines of code. Although you can offset that increase by reducing technical debt elsewhere, a perfect programs will have no technical debt to reduce.

I don't think this is a real-world problem, and perfect people won't need this metric. (Mostly because they don't exist.) Real, non-trivial systems have plenty of opportunities to reduce size even as features are added. Even if nearly all of those are worked out--and I'll believe it when I see it--then you can use the metric to check that the rate of increase says low.

Another failure case of SLOC as a metric of technical debt is that it could encourage other types of bad code: people creating monstrous one-liners, for example, or refusing to make small methods because of a language's boilerplate.

We can work around this problem by counting executable statements rather than actual lines of code. For example, the following Java code counts as one statement:

public class HelloWorld {
  public static void main() {
    System.out.println("Hello, World!");
  }
}

The Spag

So here's my proposal. The unit of technical debt is the "Spag (Sg)," from "spaghetti." You can pronounce it "Spag" (rhymes with "gag") or "Spug" (rhymes with "bug") depending on your preference. (I almost went with the "Foote," in homage to Brian Foote's and Joseph Yoder's awesome Big Ball of Mud design pattern, but I wasn't sure if Brian would appreciate having a measure of code crap named after him. So Spag it is.)

To keep the numbers sensible, and to prevent obsessing over micro-optimizations, one spag equals 1,000 statements, and spags are never calculated to more than one decimal place. So the Java "Hello World" program above has 0.0 spags of technical debt, and a program with a million statements has a kilospag.

(A kilospag is spagload of technical debt, and if you've ever worked on one of those programs, you know just how bad it is.)

I think the best way to use the spag is to track trends in your codebase. It's particularly appropriate when you know you have problems and you want to see your progress, or if you need a visual way to show managers and stakeholders that you've been accumulating technical debt. ("Since you told us to 'meet the deadline or else,' our rate of spag accumulation has quadrupled. We need to start paying more attention to design quality and paying off this debt, or we're going to have serious productivity and quality problems.")

It's a fun idea with a silly name, but I also think it's got real potential. Try it out on your projects and let me know how it works for you.

Comments

The Decline and Fall of Agile

14 Nov 2008

James Shore/Blog

It's odd to talk about the decline and fall of the agile movement, especially now that it's so popular, but I actually think the agile movement has been in decline for several years now.

The State of the Art

I've seen a shift in my business over the last few years. In the beginning, people would call me to help them introduce Agile, and I would sell them a complete package that included agile planning, cross-functional teams, and agile engineering practices.

Now many people who call me already have Agile in place (they say), but they're struggling. They're having trouble meeting their iteration commitments, they're experiencing a lot of technical debt, and testing takes too long. So they hire me to help them with one of these things. When I go visit, I see a team that is nominally agile, but is suffering huge numbers of problems and is anything but the joyful, settled, smooth-running workplace I expect from an agile organization.

Other consultants I've talked to report the same experience. "Rescuing Scrum teams keeps me in business," joked one colleague. It's funny because it's true.

The Role of Scrum

Scrum is undeniably the winner of the agile method wars. Thanks to the Scrum Alliance's vast (and lucrative) network of Certified Scrum Trainers and Certified ScrumMaster courses, when people say "Agile," they usually mean Scrum. So when "Agile" fails, it's generally Scrum that's failing. And Scrum is incomplete, purposefully so. "Scrum is like your mother-in-law," Ken Schwaber said at the recent Agile Vancouver conference. "It's constantly pointing out your shortcomings." The trick is that you're supposed to learn from that feedback and fix your problems.

But because Scrum works in short cycles and doesn't include any engineering practices, it's very easy for teams using Scrum to throw out design. Up-front design doesn't work when you're using short cycles, and Scrum doesn't provide a replacement. Without continuous, incremental design, Scrum teams quickly dig themselves a gigantic hole of technical debt. Two or three years later, I get a call--or one of my colleagues does. "Changes take too long and cost too much!" I hear. "Teach us about test-driven development, or pairing, or acceptance testing!" By that time, fixing the real problems requires paying back a lot of technical debt, and could take years.

What frustrates me the most is that this situation is entirely avoidable. In a green-field environment, the solid agile engineering practices included in Extreme Programming pay for themselves within the first few months. Without XP's agile engineering practices, code quality and productivity asymptotically decreases over time. With them, productivity starts lower, but then it asymptotically increases. I can't prove it, but my sense is that the two curves cross at about the eight-week mark. Maybe sooner. Agile engineering practices aren't only important--they pay for themselves! Doing anything else is pure negligence... if you understand your options. Scrum is silent on the matter.

Scrum, Misapplied

It's not entirely Scrum's fault. Agile development isn't just about good engineering practices. It's also about acknowledging the importance of people in software development, forming cross-functional teams, obtaining high-bandwidth communication, constantly reflecting and improving, delivering value, and changing plans to take advantage of opportunities.

Scrum includes these other points. It says that the team should be cross-functional and recommends collocating the team in a shared workspace. It says the team should deliver a valuable, shippable product at the end of every Sprint, and that the team should self-organize, discover impediments, and remove them.

Oh, and it also has a few mechanical things about a monthly Sprint and daily Scrum. Trivial stuff compared to the rest. But guess which part people adopt? That's right--Sprints and Scrums. Rapid cycles, but none of the good stuff that makes rapid cycles sustainable.

You're Doing It Wrong

There are a lot of teams right now failing with Agile. These teams are working in short cycles. The increased planning frequency has given them more control over their work and they're discovering and fixing some problems. They feel good, and they really are seeing more success than they were before.

But they aren't working in shared workspaces or emphasizing high-bandwidth communication. They're don't have on-site customers or work in cross-functional teams. They don't even finish all of their stories by the end of each Sprint, let alone deliver releasable software, and they certainly don't use good engineering practices.

These teams say they're Agile, but they're just planning (and replanning) frequently. Short cycles and the ability to re-plan are the benefit that Agile gives you. It's the reward, not the method. These psuedo-Agile teams are having dessert every night and skipping their vegetables. By leaving out all the other stuff--the stuff that's really Agile--they're setting themselves up for rotten teeth, an oversized waistline, and ultimate failure. They feel good now, but it won't last.

Failures in the Midst

It's human nature to only do the stuff that's familiar and fun, and that's what has happened with Agile. People look at agile methods as a chinese menu of practices, choose the few that look cool, and ditch the rest. Unfortunately, the parts they leave out are the parts that make Agile work. Scrum makes it worse by ignoring important (but hard) agile engineering practices, and the Scrum Alliance makes it worse still with their armies of trainers--some good, some not--issuing dubious "ScrumMaster" certificates to people who demonstrated competence in connecting butt to chair for two days.

So, unfortunately, a lot of self-described Agile projects are going to fail. They're failing right now. And eventually Agile will take the blame, and it will pass, as all fads eventually do.

It's too bad. The good Agile--the real Agile--it really works. I've seen it. My colleagues have seen it. It's been repeated hundreds of times, and some of those projects have succeeded for years. But those hundreds of successes will be drowned out by the thousands of failures.

Can we prevent this trend? I don't know. Perhaps we should start by giving up on the "Agile" brand name. It's never been clearly defined, which has allowed a lot of dysfunctional projects to call themselves "Agile."

Or maybe we need to stop selling Agile. Maybe we need to say, "Agile is hard, and you can't master it by sitting through a two-day course." Maybe we need to be firm and say, "Sorry, if you don't use agile engineering practices, if you don't have high-bandwidth communication, and if you don't include a strong customer voice, you're not going to succeed. Try something else instead." Scrum is popular because it's easy--and that's part of the problem.

Whatever we do, we need to do it soon. Agile is failing all around us, and I'd hate for the failure of the fad to take down the truly useful ideas that we started with.

Comments

The Art of Agile Development: Iteration Planning

15 Oct 2008

James Shore/Agile-Book

in 99 words

Iterations are timeboxed to one week and follow a strict schedule:

Plan iteration

Commit to delivering stories

Develop stories

Prepare release

Demonstrate release

Hold retrospective

To plan, measure the velocity of the previous iteration (total the estimates of "done done" stories). Select stories from the release plan that match the velocity. It shouldn't take long.

Assuming programmers are your constraint, they brainstorm and estimate engineering tasks. Ask the on-site customer about detailed requirements when necessary. Compare the task estimates to last iteration's to confirm the plan's feasability.

Post the stories and tasks prominently and mark them when complete.

New Information

Kanban Systems

Inside the Book

Iteration Planning
The Iteration Timebox
The Iteration Schedule
How to Plan an Iteration
The Commitment Ceremony
After the Planning Session
Dealing with Long Planning Sessions
Tracking the Iteration
When Things Go Wrong
Partially Done Work
Emergency Requests
The Batman
Sidebar: Daily Iterations
Questions
- How should we schedule time for fixing bugs?
- If we don't estimate stories during iteration planning, when do we estimate stories?
- All the available tasks depend on tasks that other pairs are working on right now. What should I work on?
- What should the batman do when there are no outstanding support requests?
Results
Contraindications
Alternatives
- Iteration Length
Further Reading

Comments

Kanban Systems

15 Oct 2008

James Shore/Blog

Commentary on The Art of Agile Development's Iteration Planning practice.

In the last few years, Kanban Systems have become an increasingly hot topic in the Agile community, thanks in part to the Poppendiecks' wonderful introduction of Lean Manufacturing principles. Three people who have been working on Kanban systems in the Agile space are David Anderson, Arlo Belshee (video), and Kenji Hiranabe.

Despite the interest, I haven't seen any good overviews of Kanban systems. In this essay, I'll try to fill the gap. I'll describe what Kanban systems are, why they work, and when you should use them.

Kanban Described

Kanban systems are an approach to scheduling work. Typical Agile projects use time-boxed iterations. At the beginning of the iteration, the team meets and chooses stories from their backlog that can be done by the end of the iteration. Over the course of the iteration, they develop those stories, and at the end of the iteration, they ship them.

Kanban systems are different. Instead of using time-boxed iterations, Kanban systems focus on a continuous flow of work. The team takes a story from the backlog, develops it, and ships it as soon as it is done. Then they take the next story from the backlog, develop it, and ship that story. Work is shipped as soon as it's ready, and the team only works on one story at a time.

That's the ideal, anyway. Approaches differ. My favorite approach is based on Arlo Belshee's, and that's what I'll describe here. (Arlo, with his usual flair for names, calls it "Naked Planning.")

Sketch of a kanban planning board

MMFs and the Backlog Queue

The primary work unit for the team is the "minimum marketable feature," or MMF. An MMF is the smallest feature that has value to the market. As Arlo puts it, MMFs are "the smallest features we can sell."

The team maintains a backlog of MMFs in a queue. This queue is strictly limited in size. Arlo allows seven MMFs in his backlog; I usually prefer two or three. A small backlog is good because it limits the amount of time an MMF can wait, which prevents them from getting stale.

The team only works on the top MMF in the queue. Stakeholders can add, remove, and move the other MMFs in the queue as much as they want, so long as they don't exceed the queue's size limit.

Work in Progress

The team works on the top MMF in the backlog queue until it's done. They use simultaneous phases, so there's only three classifications: waiting, in progress, and done. MMFs are either in production ("done"), at the top of the backlog queue ("in progress"), or waiting.

The use of simultaneous phases is probably the biggest difference between Arlo's approach and David Anderson's. David Anderson doesn't use simultaneous phases, so his MMF-equivalents have to travel through multiple stages: Engineering Ready Queue, Systems Analysis, Development, Build, Test, Release Ready, and In Production. This means that his team works on multiple MMFs at a time, which leads to a need for complex configuration management, and I'm sure it lengthens his cycle time.

The team brainstorms and tracks the tasks required to complete the current MMF only. As tasks are completed, they're removed from the board; as new tasks are discovered, they're added. When all the tasks are done, either the MMF is done or the team has forgotten something.

Once the MMF is done and the on-site customers have approved it, the software is released. The MMF is taken off the queue and development starts on the next MMF.

Tracking and Estimating

There's no estimating of tasks or MMFs in this approach. Instead, you use what Arlo calls "Disney Line Planning": "Your wait from this point will be XXX days." In other words, you measure the typical time it takes to complete an MMF, from start to finish, and extrapolate to various points in the queue. Since MMFs can vary in size, this is a rough projection, not a commitment.

MMFs and tasks are tracked on index cards stuck to a large whiteboard. You could use an electronic tool if you wished, but for a colocated team, I recommend the low-tech approach. A big whiteboard is constantly visible and encourages discussion and collaboration in a way a computer can't.

Urgent Work

If there's an emergency, a support request, or some other urgent need, there's an empty slot on the board marked "Urgent." The team can put an MMF in that slot at any time without having to go through the regular backlog queue. The team strives to finish urgent items quickly, and they try to keep the slot empty and available at all times.

If the "Urgent" slot is full and another urgent item appears, it has to be added to the backlog queue.

Bugs

As with other Agile approaches, the team is expected to fix bugs before the MMF is done. If bugs are found afterwards, the fixes are scheduled with a new MMF. If a bug needs to be fixed immediately, the team uses the "Urgent" slot.

Understanding Kanban

The name "Kanban" is really a misnomer. "Kanban" means "sign" in Japanese. In Toyota's Kanban system, when a station on the manufacturing line ran out of inventory, they would place a sign in an empty box and send it up the line to indicate that they needed more inventory.

But despite the name, the sign is not the most important part of the Kanban system. What Toyota was really trying to accomplish was "one-piece flow." One-piece flow means manufacturing items one at a time rather than in batches. It creates flexibility, improves productivity, eliminates waste, increases throughput, cuts time to market, reduces the impact of error, and reduces the cost of inventory. It's a Good Thing.

Kanban is a hack around the limitations of the physical world.

In manufacturing, one-piece flow is hard to achieve. Sometimes efficency requires that you build up inventory, like when you truck items from one location to another. Kanban is a hack around this limitation. It's a way of controlling the amount of inventory in the system. But the ideal is still one-piece flow: working on just one item at a time, with no inventory waiting. Lean coaches will actually remove Kanban cards to shake things up and spark further improvement to the system. As Jeffrey Liker says in The Toyota Way (emphasis his):

...TPS experts get very impatient and even irritated when they hear people rave and focus on kanban as if it is the Toyota Production System. Kanban is a fascinating concept and it is fun to watch... When is the kanban triggered? How are the quantities calculated? What do you do if a kanban gets lost? But that is not the point... The challenge is to develop a learning organization that will find ways to reduce the number of kanban and thereby reduce and finally eliminate the inventory buffer... So kanban is something you strive to get rid of, not to be proud of.

So when you hear "Kanban System," don't think of little cards pulling work from upstream stations. The ideal of the Kanban system is to eliminate inventory and achieve one-piece flow. And in software, we can. We aren't subject to the same physical limitations that a manufacturing plant is.

That's why I prefer Arlo's approach to David Anderson's. By using the Extreme Programming approach of simultaneous phases, everyone on the team works on one MMF and gets it to done, then they work on the next. Work-in-progress inventory is reduced to the absolute, bare minimum: just one MMF. (Beat that, Toyota!) By doing this, we create flexibility, improve productivity, eliminate waste, increase throughput, and do all of the other wonderful things that one-piece flow gets you.

When to Use a Kanban System

I like the idea of Kanban systems, but I'm not sure they're appropriate for everyone. This might just be unfamiliarity: I have a lot more experience with the old-fashioned iteration-based approach.

The good thing about Kanban systems is that they really do a great job of eliminating waste. They eliminate iteration planning, they react extremely well to changing needs, and they work well in chaotic environments. The theory behind them is sound, and if you are a fan of Lean ideas, then using a Kanban system will seem like a no-brainer.

On the flip-side, one of the biggest problems I see with agile teams is "Agile thrashing." In Agile thrashing, the team flails about, making progress on a lot of disparate ideas, but never really creating anything significant. I worry that Kanban systems, with their limited backlog, will exacerbate this problem.

I'm also concerned that Kanban will be too advanced for teams new to Agile planning. One of the great things about iterations is that they're timeboxed. At the end of the week (or month), time's up! You're done. Many teams struggle with this. They have trouble getting to "done done" and the timebox helps them recognize and eventually fix the problem. Timeboxes also prevent scope creep and they also force important trade-off decisions to be made. Overall, they're an excellent tool for improving the team's self-discipline, and Kanban systems don't seem like they'll provide those benefits.

In the field, I've seen Kanban work best in chaotic environments where upcoming features don't have much in common. I don't think it's a coincidence that the initial examples of Kanban come from those sorts of environments. David Anderson's team was responding to change requests. Arlo Belshee's team worked in a start-up, where frequent, entrepreneurial shifts in direction are commonplace. One team I worked with used a Kanban system to respond to post-release change requests, but plans to switch back to iterations once they start planning their next release.

So, should your team use iterations or a Kanban system? Here's my criteria:

If your team is new to Agile planning, use iterations. Be super-disciplined about getting everything "done done" each iteration, using slack, and achieving a stable velocity so you can consistently make and meet iteration commitments. This will force your team to learn important lessons about how Agile works.
If you're producing a new product or a significant new release of an existing product, use iterations. Create a vision, do release planning, get a good product manager, and make sure he's heavily involved. Your goal is to create a compelling product with significant value, and you'll benefit from a gestalt view of the product and the predictability iterations provide.
If you're in an environment where the feature backlog is constantly in flux or hard to predict, particularly if you can release the software on a whim, try the Kanban system. Entrepreneurial environments, maintenance of mature products, and recent releases are all examples of environments with unpredictable backlogs. The Kanban system will convert that chaos into a smooth, constant flow.

And if none of these categories seem to fit, go ahead and give the Kanban approach a try. You'll learn something new and hopefully have fun doing it. There's a lot of potential here, and I don't think we've explored all of the options. Try it! Let us know how it works out.

Comments

The Art of Agile Development: Risk Management

10 Oct 2008

James Shore/Agile-Book

in 99 words

Risk management allows you to make and meet commitments.

Manage common risks with risk multipliers. Manage project-specific risks by brainstorming disasters and their causes. Create mitigation and contingency plans for serious risks and define unambiguous triggers for taking action.

Combine risk multipliers with project-specific risks to project your chances of meeting commitments, and report those probabilties as "commitments" and "stretch goals." Lower your risk and improve projections by including slack in every iteration and getting stories "done done".

Remember that success is more than meeting commitments. Sometimes it's better to delay delivery and create a better product.

Examples

Use Risk Management to Make Solid Commitments

Estimate Inflation: A Cautionary Tale

Inside the Book

Risk Management
A Generic Risk-Management Plan
Project-Specific Risks
Sidebar: Monitoring Risks
How to Make a Release Commitment
Sidebar: Predicting Release Dates
Success over Schedule
When Your Commitment Isn't Good Enough
Questions
- What should we tell our stakeholders about risks?
- Your risk multipliers are too high. Can I use a lower multiplier?
- We're using a scopeboxed plan, and our stakeholders want a single delivery date rather than a risk-based range. What should we do?
Results
Contraindications
Alternatives
Further Reading

Comments

Estimate Inflation: A Cautionary Tale

09 Oct 2008

James Shore/Blog

Commentary on The Art of Agile Development's Risk Management practice.

I liked my example of using risk management to make commitments, so I made a longer example that was pulled directly from many projects I've seen.

Unfortunately, I made it a little too real. There's scope creep, pushy stakeholders, and poor decisions. (The poor decisions are the killer.) It's a lousy example of risk management, but it's a great cautionary tale. I present: What Not To Do.

(If you haven't read my other essay yet, read it first.)

Prior to Iteration 1

The team has just been formed in order to work on an important new initiative for the company. They're experienced Agile practitioners, but they've never worked together before, so they don't know what their velocity is going to be. They start by playing the planning game, coming up with stories, and grouping them into minimum marketable features.

Feature	Cost	Total
A	34	34
B	7	41
C	13	54
D	2	56
E	6	62
F	26	88
G	17	105
H	9	114
I	11	125
J	5	130

They also settle on a release date. It's 12 weeks from now. They don't know their velocity, but they take a guess by dividing their man-days by pi (8 devs × 5 days per iteration ÷ π). They get a velocity of 13.

Because their velocity is a wild-ass guess, they use the "risky" column in the risk table.

10%: 156 (13 velocity × 12 iterations ÷ 1 risk)
50%: 78 (13 velocity × 12 iterations ÷ 2 risk)
90%: 39 (13 velocity × 12 iterations ÷ 4 risk)

"We can commit to delivering feature A and we'll work on features B, C, D, and E as stretch goals."

The on-site customers are quite unhappy with this result--they can't go to market unless features A, B, C, and D are done. They pressure the team to increase the estimate. Dividing by pi is ridiculous, they say; why should it take the team more than three days to do one day's worth of work? They say that the velocity should be 32 (8 devs × 5 days per iteration × 80% effectiveness). The team eventually splits the difference and predicts that the velocity will be 23.

10%: 276 (23 velocity × 12 iterations ÷ 1 risk)
50%: 138 (23 velocity × 12 iterations ÷ 2 risk)
90%: 69 (23 velocity × 12 iterations ÷ 4 risk)

"We're not comfortable with the velocity estimate, but if it's accurate we can commit to delivering features A, B, C, D, and E. The remaining features will be our stretch goal."

The best thing the team could have done was to delay making estimates; they have zero data and nothing to make commitments with. But customers demand estimates. It's a tricky situation, but I would have tried to get the customers to wait.

Iteration 1

The team makes makes a commitment to deliver 23 story points in the first iteration, and they do. Everyone is impressed, although a few people are concerned that things didn't really get done.

10%: 276 (23 finished + 23 velocity × 11 iterations ÷ 1 risk)
50%: 150 (23 finished + 23 velocity × 11 iterations ÷ 2 risk)
90%: 86 (23 finished + 23 velocity × 11 iterations ÷ 4 risk)

Iteration 2

The next iteration is a disaster. There's a bit of work to finish up from iteration 1, and those take longer than expected. They keep finding bugs. They don't even start half of the stories that they planned on for this iteration, and they only finish one. That one story gives them a velocity of two.

10%: 45 (25 finished + 2 velocity × 10 iterations ÷ 1 risk)
50%: 35 (25 finished + 2 velocity × 10 iterations ÷ 2 risk)
90%: 30 (25 finished + 2 velocity × 10 iterations ÷ 4 risk)

A few people argue that two is a ridiculous velocity. They should at least count the stories that were partially done. After some arguing about how much of those stories were done, and how much work they did on the stories from iteration 1, they decide on a velocity of 15.

10%: 188 (38 finished + 15 velocity × 10 iterations ÷ 1 risk)
50%: 113 (38 finished + 15 velocity × 10 iterations ÷ 2 risk)
90%: 76 (38 finished + 15 velocity × 10 iterations ÷ 4 risk)

The team is falling into a classic trap here: they don't like the projections, so they're making up numbers that they do like. This is leading them to overcommit and do shoddy work that has to be revisited. It's gonna bite 'em in the ass... it always does.

Iteration 3

The problems continue. The team signs up for 15 points of new stories, but they have to work on the partially-finished stories from iteration 2 as well. They only finish one new story (worth three points), but they decide that their velocity was 17 because they finished all the stories from iteration 2 and partially finished all of the stories from iteration 3.

10%: 208 (55 finished + 17 velocity × 9 iterations ÷ 1 risk)
50%: 132 (55 finished + 17 velocity × 9 iterations ÷ 2 risk)
90%: 93 (55 finished + 17 velocity × 9 iterations ÷ 4 risk)

Iteration 4

The problems have finally become too significant for the team to ignore. They don't get any new stories completely done this iteration because they spent all of their time finishing stories from previous iterations. If they've really done as much as they say they have, they should be halfway done by now... but they haven't even finished feature A.

The team goes back to the original stories and estimates and discovers that they've really only finished 28 points. They've been double-counting stories because they've been spreading the work out over multiple iterations. If they only count stories they start and completely finish in a single iteration, their velocity over the last three iterations has been 2, 3, and 0.

It's an "oh shit" moment for everyone. They decide to quit gaming the system. They reset their velocity to 2 and make a renewed commitment to only signing up for what they can get "done done" every iteration.

10%: 44 (28 finished + 2 velocity × 8 iterations ÷ 1 risk)
50%: 36 (28 finished + 2 velocity × 8 iterations ÷ 2 risk)
90%: 32 (28 finished + 2 velocity × 8 iterations ÷ 4 risk)

The numbers look terrible, and it tests the team's newly-found resolve. They decide to warn stakeholders that something is going wrong, but not share the new numbers until they've had a few more iterations. "We've discovered a problem with our estimating approach and we probably won't be able to complete as many features as we thought," they say. "We'll have more information for you in two weeks, at the end of iteration 6."

Iteration 5

The team holds true to their promise not to do anything that they can't get to "done done". They make sure all the existing stories are done and do some new stories, too. They finish 8 points of new stories.

10%: 92 (36 finished + 8 velocity × 7 iterations ÷ 1 risk)
50%: 64 (36 finished + 8 velocity × 7 iterations ÷ 2 risk)
90%: 50 (36 finished + 8 velocity × 7 iterations ÷ 4 risk)

Feature	Cost	Total
A	45	45
B	7	52
C	13	65
D	2	67
E	6	73
F	26	99
G	17	116
H	9	125
I	11	136
J	5	141

The customers are busy, too. They take a hard look at what's been done and realize that feature A needs some more polish. They add 11 more points of stories to feature A.

They look at the results and say, "With the data we have right now, we can commit to delivering feature A and we can work on features B and C as a stretch goal." The on-site customers are worried--they need features A, B, C, and D to go to market. They reset their plan to include just features A, B, C, and D and start thinking about what to tell management next iteration.

Iteration 6

Great news! The team finished feature A and has started on feature B. The customers thought about adding a few more stories to feature A, but decided that there's too much risk of not going to market. They're constraining scope until they know more.

The other good news is that the team has finally cleared out the partially-done work. They were able to spend the entire iteration focused on this iteration's stories, and as a consequence, their velocity went up! Their new velocity is 10.

10%: 106 (46 finished + 10 velocity × 6 iterations ÷ 1 risk)
50%: 76 (46 finished + 10 velocity × 6 iterations ÷ 2 risk)
90%: 61 (46 finished + 10 velocity × 6 iterations ÷ 4 risk)

Feature	Cost	Total
A	45	45	Done!
B	9	54
C	17	71
D	4	75
E	6	81
F	26	107
G	17	124
H	9	133
I	11	144
J	5	149

It's not all good news, though. The customers look at features B, C, and D and realize that they're going to have to make some changes. They do their best to keep the changes small, but the scope still increases.

The team has an uncomfortable conversation with their stakeholders. "We made some mistakes in the beginning--we were too aggressive in our estimates, and that caused us to overlook some serious problems. We've corrected the situation, but we're still recovering. The good news is that we've finished feature A and we're committed to delivering feature B. We're working on features C and D as a stretch goal for the end of iteration 12. Because we can't go to market without C and D, we might have to ship later than we wanted. We'll keep you informed, and we won't make the same mistake again."

Iteration 7

The team continues to make good progress, completing 10 more points of stories, including feature B! They're also cleaning up some of the design flaws that the first few iterations introduced into the codebase. The programmers are starting to feel like things are under control.

10%: 106 (56 finished + 10 velocity × 5 iterations ÷ 1 risk)
50%: 81 (56 finished + 10 velocity × 5 iterations ÷ 2 risk)
90%: 69 (56 finished + 10 velocity × 5 iterations ÷ 4 risk)

"We've finished feature B and we're on track to delivering features C and D. If next week goes well, we may be able to eliminate the risk of shipping late."

Iteration 8

Another great iteration. The team's velocity actually goes up, to 12. They've had four solid iterations and the programmers are feeling confident in their ability to get things "done done". The team decides that they're ready to move to the "rigorous" column of the risk table. This does wonderful things for their projections.

10%: 116 (68 finished + 12 velocity × 4 iterations ÷ 1 risk)
50%: 102 (68 finished + 12 velocity × 4 iterations ÷ 1.4 risk)
90%: 95 (68 finished + 12 velocity × 4 iterations ÷ 1.8 risk)

"We're happy to report that we can now commit to delivering features C and D on time. We're committing to shipping as scheduled. We're also ready to commit to shipping feature E and we'll work on feature F as a stretch goal."

Iteration 9

Feature	Cost	Total
A	45	45	Done!
B	9	54	Done!
C	17	71	Done!
D	4	75	Done!
E	6	81	Done!
K	13	94	(new)
Polish	15	109	(new)
F	26	135
G	17	152
H	9	161
I	11	172
J	5	177

The team's velocity continues to go up, to 14. They finish features C, D, and E. The on-site customers have been demonstrating the product to users, though, and they discover that they're missing some features that end users need. They also identify some further polish to features A, B, C, D, and E that's necessary to wow the market.

10%: 124 (82 finished + 14 velocity × 3 iterations ÷ 1 risk)
50%: 112 (82 finished + 14 velocity × 3 iterations ÷ 1.4 risk)
90%: 105 (82 finished + 14 velocity × 3 iterations ÷ 1.8 risk)

"We've finished features C, D, and E. They'll be in the release next month. Our market research has also revealed a new feature, feature K, that's important for this release, so we're committing to putting that in. We've also identified some additional polish and tweaks that will make our users happier, so we're working on those as a stretch goal."

Iteration 10

The team is on fire! Their velocity goes up again, to 16. They finish feature K and start working on the polish. The customers go out on a limb and add feature F back into the schedule, although it's not supported by the projections... but they mitigate the risk by looking at how they can cut scope if they need to.

10%: 130 (98 finished + 16 velocity × 2 iterations ÷ 1 risk)
50%: 121 (98 finished + 16 velocity × 2 iterations ÷ 1.4 risk)
90%: 116 (98 finished + 16 velocity × 2 iterations ÷ 1.8 risk)

"We've finished feature K and we're committed to finishing the additional release polish. It looks like we'll have some extra time, so we're going to put feature F in as a stretch goal--we might have to cut parts of it, but we know what to cut if we do."

Iteration 11

The team's velocity is 16 again and they finish the release polish. They're doing a great job of keeping the code clean, including slack in each iteration, and getting everything to "done done," so they can sustain that velocity indefinitely. The customers are a little disappointed that the velocity didn't keep going up, but they cut scope on feature F as planned.

10%: 130 (114 finished + 16 velocity × 2 iterations ÷ 1 risk)
50%: 125 (114 finished + 16 velocity × 2 iterations ÷ 1.4 risk)
90%: 123 (114 finished + 16 velocity × 2 iterations ÷ 1.8 risk)

"Everything's on track. We're going to have a great release next week."

Iteration 12

Another good iteration. The team finishes 16 points of work and the customers adjust scope to match. They perform their release (using their automated script, of course) and spend the following week documenting their work, conducting a retrospective, learning from their experience, and having a well-earned party.

Analysis

This was a made-up example, but it's based on the real experiences of teams I've worked with. I often see teams that are struggling because they overcommit each iteration. As a result, they have trouble meeting their commitments and--ironically--have much lower performance than they should.

In the end, this team delivered 130 points of work, which was (purely coincidentally) the same amount they originally planned. But they only did that well because they backed off of their aggressive, made-up estimates and let the team do their work properly. You can see in iterations 2-4 how badly the team's real velocity dropped off. Once the poor iteration planning was fixed, the team's velocity improved, and steadily rose to its optimum value of 16^*... a value they could have reached much earlier if they hadn't wasted so much effort on trying to meet unrealistic commitments.

^*Optimum for this team, that is. Every team's optimum velocity is different.

When I work with teams like this one, the first thing I do is help them learn how to deliver on time every iteration. This almost always involves reducing the team's velocity to what they can actually accomplish. This takes the pressure off the team and allows them to do the important clean-up work that they've been postponing. The result? Their velocity steadily improves as they clear out the damage.

The team in this example was lucky in that they discovered and fixed the problem themselves. Most teams have trouble taking the drastic action they need. As a result, their velocity never stabilizes. They struggle to make accurate predictions and the constant pressure causes the team to make dumb mistakes. They don't deliver on their commitments.

You can avoid this mistake by only committing to what you actually deliver. Only count stories that are totally, 100%, "done done". You'll be happier, you'll get more done, and you'll be able to make meaningful projections and commitments.

Comments

Use Risk Management to Make Solid Commitments

08 Oct 2008

James Shore/Blog

Commentary on The Art of Agile Development's Risk Management practice.

Making and meeting commitments. Over and over, when I ask executives what they want from their teams, I've found this to be one of the top priorities.

That's why the Risk Management practice in The Art of Agile Development is one of my favorites. It's all about being able to make and meet commitments, and we have three great tools in there: risk multipliers, risk-adjusted burn-up charts, and project-specific risks. Today, I'll show you how to use the first two to make solid commitments to your executives.

Risk Multipliers

Risk multipliers account for common risks, such as turnover, changing requirements, work disruption, and so forth. These risk multipliers allow you to set a date, estimate how many story points of work you'll get done, and be right. It's a simpler version of the risk curves you'll see in good books on estimating and project management.

	Risk Multiplier
Chance	Rigorous Process	Risky Process	Description
10%	1	1	Ignore--almost impossible
50%	1.4	2	Stretch goal--50/50 chance
90%	1.8	4	Commitment--virtually certain

(As we explain in the book, these multipliers are estimates gleaned from DeMarco & Lister's RISKOLOGY simulator and Todd Little's detailed analysis of hundreds of projects. I'm guessing somewhat at how accurately they apply to XP. The most accurate approach is to calculate your own risk multipliers from past project history, but most companies don't track that data.)

The way this works is that you take your iteration velocity and extrapolate out to your delivery date. That's the best estimate of how many story points you'll finish under ideal circumstances. So if your velocity is 14 and you have ten iterations left, you predict that you'll finish 140 more story points before you release.

Then you apply the risk multipliers according to the type of process you're using. If you have a solid velocity and get everything "done done" every iteration, you get to use the "rigorous" column. Otherwise, you use the "risky" column.

To continue the example, let's say your team has a rigorous approach. To make a commitment, you apply the multipliers, like this:

Chance	Story Points Finished	Description
10%	140 (140 ÷ 1)	Ignore--almost impossible
50%	100 (140 ÷ 1.4)	Stretch goal--50/50 chance
90%	78 (140 ÷ 1.8)	Commitment--virtually certain

This allows you to make commitments to stakeholders and executives. "We are virtually certain to finish 78 more points by our release date, so we're committing to delivering features A, B, and C. We have a 50-50 chance of completing 100 points, so we're planning on features X, Y, and Z as a stretch goal."

The Risk-Adjusted Burn-Up Chart

The risk-adjusted burn-up chart allows you to track your commitments alongside your progress. It's based on Phil Goodwin's burn-up chart, which is a variant of Scrum's burn-down chart. Here's how it works.

Start by drawing a chart with iterations on the bottom and story points on the left. Mark the release date with a dashed line, as shown.

At the start of every iteration (typically, every week), plot two points on the chart. At the bottom of the chart, plot the number of story points that are "done done". At the top of the chart, plot the total story points--"done done" plus the number remaining in the plan. This gives you a standard burn-up chart.

Now add the risk numbers. Calculate your probability of completing various numbers of story points using the risk multiplier approach described in the previous section, and then plot those numbers on a vertical line. You can either draw the vertical line on the release date and change it every week (as shown in the figure above), or you can draw it on the current iteration (as I'll show in a moment).

The result is an information-rich chart that shows you three trends:

Completion progress;
Changes to the plan; and
What you can commit to.

Example

Here's an extended example of these ideas in practice. Imagine a team of six programmers. They've been working together for a while now and have a solid velocity of 12. They start a new project that's due in six weeks. After playing the planning game, they have a prioritized list of minimum marketable features that looks like this:

Feature	Cost	Total
A	34	34
B	7	41
C	13	54
D	2	56
E	6	62
F	26	88

Based on their past velocity of 12 and their history of getting iterations "done done," the team calculates the following estimates:

10%: 72 (12 velocity × 6 iterations ÷ 1 risk)
50%: 51 (12 velocity × 6 iterations ÷ 1.4 risk)
90%: 40 (12 velocity × 6 iterations ÷ 1.8 risk)

They report this to their stakeholders by saying, "We commit to delivering features A and B, and we'll work on feature C as a stretch goal. Features D and E are unlikely to make it, and we almost certainly won't have time to do feature F."

(Sorry about the image quality. Also, the 10% and 90% should be reversed.)

Iteration 1

The team signs up for 12 points and the iteration goes smoothly. They deliver 12 points. The customers leave the plan unchanged in hopes that the team's predictions are wrong. The team recalculates their risk-based projections:

10%: 72 (12 finished + 12 velocity × 5 iterations ÷ 1 risk)
50%: 55 (12 finished + 12 velocity × 5 iterations ÷ 1.4 risk)
90%: 45 (12 finished + 12 velocity × 5 iterations ÷ 1.8 risk)

(The 10% and 90% should be reversed in this picture.)

Iteration 2

It's a national holiday and most of the team is out for a few days this week. The team uses all of their slack but is unable to get everything done. They deliver 7 points, but they keep their projected velocity at 12, knowing that the holiday was an unusual event.

10%: 67 (19 finished + 12 velocity × 4 iterations ÷ 1 risk)
50%: 53 (19 finished + 12 velocity × 4 iterations ÷ 1.4 risk)
90%: 46 (19 finished + 12 velocity × 4 iterations ÷ 1.8 risk)

They report the news: "We are still committed to delivering features A and B, and feature C remains a reasonable stretch goal. Features E and F will definitely not be done and we're removing them from the plan."

(The 10% and 90% are correct in this picture and all the remaining ones.)

Iteration 3 & 4

Iterations three and four proceed without incident and the team delivers all 12 points each time.

10%: 67 (43 finished + 12 velocity × 2 iterations ÷ 1 risk)
50%: 60 (43 finished + 12 velocity × 2 iterations ÷ 1.4 risk)
90%: 56 (43 finished + 12 velocity × 2 iterations ÷ 1.8 risk)

The team reports, "Great news! The work's been proceeding smoothly and we've finished features A and B. We're now working on features C and D, and we're adding feature E back in as a further stretch goal."

Iteration 5

Disaster strikes! A critical database crashes and everyone is pulled to work on it. The team only delivers 6 points this iteration.

10%: 61 (49 finished + 12 velocity × 1 iteration ÷ 1 risk)
50%: 56 (49 finished + 12 velocity × 1 iteration ÷ 1.4 risk)
90%: 55 (49 finished + 12 velocity × 1 iteration ÷ 1.8 risk)

The team's report: "Due to the unexpected problems this week, we're not going to get to feature E after all, and any problems on the scale of this week's will mean canceling feature D, too. However, we're committed to delivering feature C and we're going to do our best to get feature D done for you."

Iteration 6

No problems arise and the team gets all 12 points done, for a grand total of 61 points completed. They finish features C and D and spend the extra points adding polish to existing features and documenting their work. Rather than being upset that the team didn't get the whole plan done, stakeholders are happy that their expectations were exceeded. Everyone gets a big raise and they all live happily ever after.

The end.

PS: One of the cool things about this technique is that it automatically accounts for Steve McConnell's famous cone of uncertainty. Look at the risk bars over time and you'll see how they become smaller and more certain as time progresses, just as McConnell predicts.

The Art of Agile Development: The Planning Game

01 Oct 2008

James Shore/Agile-Book

in 99 words

Customers have the most information about value: what best serves the organization. Programmers have the most information about cost: what it takes to implement and maintain those features. And remember, every decision to do something is a decision to not do something else.

The planning game brings together customers and programmers so that they may maximize value while minimizing costs. Anybody may create stories. Programmers estimate the stories, and customers prioritize them. Programmers and customers may question each others' decisions, but each group has final say over its area of expertise.

The end result is a single prioritized list.

Commentary

Coulda, Shoulda, Woulda

Inside the Book

The Planning Game
How to Play
- Overcoming disagreements
Sidebar: On Disappointment
How to Win
Questions
- Won't programmers pad their estimates or slack off if they have this much control over the plan?
- Won't customers neglect important technical issues if they have this much control over the plan?
- Our product manager doesn't want to prioritize. He says everything is important. What can we do?
Results
Contraindications
Alternatives

Comments

Coulda, Shoulda, Woulda

01 Oct 2008

James Shore/Blog

Commentary on The Art of Agile Development's The Planning Game practice.

Prior to The Planning Game, the most popular approach to prioritization went by the name "MoSCoW"--for "Must Have," "Should Have," "Could Have," and "Won't Have."

Actually, that's not quite true. The most popular approach, then and now, was to say, "Everything's important. Do it all." Which obviously doesn't yield great results.

And that's problem with MoSCoW, too. People fear that MoSCoW actually stands for "maybe," "next time," "never," and "never ever." So the savvy stakeholders mark their stuff as "must have." We end up with the "do it all" approach, except that now we get to spend our time arguing over how many items can go in each category and whether something is really a "must" or not. Bleh. No thanks.

That's part of the genius of the planning game. In the planning game, you don't argue about the priority of any individual story. Everything is assumed to be important. There's just one overriding question:

How does this story compare in importance to the other stories we have?

It's a surprisingly useful distinction. You see, if you use something like MoSCoW, in order to prioritize your story ahead of Fred's story, you have to say, "Well, Fred's story is nice, but it's clearly not a must like mine is." And Fred, being human, gets all defensive and has to protect his sacred honor by explaining how his story really is important or he wouldn't have bothered wasting everyone's time with it.

In the planning game, there's only one list of stories, and the stuff at the top of the list gets done first. No ties, no "do them both at the same time"--one list. And so you end up with conversations like this: "Fred's story is clearly very important, and much more important than these stories here. But I think mine should be done first for reasons X, Y, and Z." And now you're talking about how each story compares against the others rather than dealing with abstract categories.

It's amazing how well this works. People have conversations and solve problems together rather than arguing about poorly-defined categories. It achieves results surprisingly quickly.

There are some preconditions to making this work. When I've seen the planning game fail, it's been for two reasons:

Failure #1: No decision-making authority. In other words, the people prioritizing are not the real decision makers. This often happens when the team is unwilling or unable to get stakeholders in the room. It kills prioritization discussions because nobody has the authority to compromise. Pissing matches result.
Failure #2: Relying on electronic planning tools. Electronic planning tools have their place--they're useful for coordinating distributed teams, for example--but planning sessions, especially initial planning sessions, require disagreements to be exposed and worked through. Face-to-face discussions are essential for this to go well. (A big table makes it much easier to see the whole plan, too.)

Ultimately, prioritization is about coming up with priorities that are going to make the most people happy. Or, to be slightly more realistic, the priorities that will be least disappointing to your major stakeholders. Since everybody has a different opinion about what's important, this requires getting together, talking about what you find important, working through disagreements with your peers, and finding common ground. It's a very human process, and the planning game excels at making that process work.

Comments

Software Development Experts James Shore and Diana Larsen to Teach Grant-Funded Training Courses on Agile Development Techniques

25 Sep 2008

James Shore/Press-Releases

Portland, Oregon, September 29, 2008 - James Shore and Diana Larsen will teach two grant-funded courses on Agile development techniques in Portland in October. The courses are offered through the Software Association of Oregon (SAO) and Worksystems, Inc (WSI). This training is funded, in part, with Employer Workforce Training Funds, a Federal grant, administered by the Oregon Department of Community Colleges and Workforce Development.

Read More...

Comments

James Shore

An Approximate Measure of Technical Debt

Measuring Technical Debt

The Case For SLOC

Nothing's Perfect

The Spag

The Decline and Fall of Agile

The State of the Art

The Role of Scrum

Scrum, Misapplied

You're Doing It Wrong

Failures in the Midst

The Art of Agile Development: Iteration Planning

in 99 words

New Information

Inside the Book

Kanban Systems

Kanban Described

MMFs and the Backlog Queue

Work in Progress

Tracking and Estimating

Urgent Work

Bugs

Understanding Kanban

When to Use a Kanban System

The Art of Agile Development: Risk Management

in 99 words

Examples

Inside the Book

Estimate Inflation: A Cautionary Tale

Prior to Iteration 1

Iteration 1

Iteration 2

Iteration 3

Iteration 4

Iteration 5

Iteration 6

Iteration 7

Iteration 8

Iteration 9

Iteration 10

Iteration 11

Iteration 12

Analysis

Use Risk Management to Make Solid Commitments

Risk Multipliers

The Risk-Adjusted Burn-Up Chart

Example

Iteration 1

Iteration 2

Iteration 3 & 4

Iteration 5

Iteration 6

Further Reading

The Art of Agile Development: The Planning Game

in 99 words

Commentary

Inside the Book

Coulda, Shoulda, Woulda

Software Development Experts James Shore and Diana Larsen to Teach Grant-Funded Training Courses on Agile Development Techniques