It’s hard escape the topics of Scrum and Agile Development these days (well, unless you don’t know any tech people). And it seems like the conventional thinking has congealed around a single oft-repeated conclusion: if you're doing custom software, you should use Scrum. As often happens with popular ideas, though, the prominence of Agile methodologies has engendered its own backlash. You won’t have to look far to find lengthy blogposts by disgruntled developers eager to tell you everything that’s wrong with Scrum. And the thing is, much of what they say is pretty spot on.
A lot of the complaining is a perfectly reasonable response to exaggerations by evangelizers, many of whom talk about Agile as if it were a panacea to cure all your software development ails. But what makes Scrum an effective methodology is that it takes into account that development projects are big, complicated beasts. There will never be an ideal methodology for every situation, but the whole point of Scrum is that it encourages developers to adjust in the direction of the most ideal method for any given set of circumstances.
Below we’re going to look into four of the main areas where developers run into problems as they try to apply the methodology to projects in the real world. If you’re unfamiliar with the principles of Scrum, you can read up on them and how they can save you time and money—at least in theory—here.
How Well Do Scrummasters and Product Owners Really Get along?
The Scrummaster is the guy or gal on the developer side who coordinates the various teams who are each tackling their own component of the application. The Product Owner is the one representing the business unit the software is being built for. The development team may work for the same business as the people who will eventually use the application, or they may be from an outside firm. Either way, the Scrum process requires a high degree of engagement from people on both sides, primarily between the Scrummaster and the Product Owner.
In the literature on Scrum, this relationship is often described as a sort of utopia where everyone’s goals are aligned and no conflicts of interest ever arise. But, especially when the developers are from an outside firm, the typical dynamic eventually kicks in where the client wants the developers to do more for less and the developers want to do less for more. Ideally, the developers will demo some component at the end of each sprint and the Product Owner will provide feedback. But that’s assuming the Product Owner is willing to put in the time and effort for one thing, and be reasonable in his or her requests for changes for another.
It takes a lot of commitment on the part of the Product Owner if Scrum is going to work as it should. But the frustrations and failures aren’t always on the business unit side of the project. There needs to be a lot of openness to plan revisions on both sides, which can be a real problem for managers accustomed to having top-down control. And this brings up another critical dimension: for the bottom-up organization of development teams to work, they have to have clear goals.
Is the Value of User Stories Entirely Fictional?
The Product Backlog is basically the overall description of how the application should work, and it’s made up of User Stories, hypothetical scenarios describing how people in real work situations will use the software. If you’re not using Scrum, you’ll probably have some kind of Requirements Document instead which describes the application’s functionality more directly. The advantages you get with a Product Backlog are that you can—you’re even expected to—go back and change it based on feedback from Sprint Review Meetings, and that you’re getting closer to a real-world understanding of how the software should function than you would with any list of if-then rules.
Unfortunately, there are times when creating User Stories is going to feel like adding an unnecessary step, and an unnecessarily complicated one at that. Going from a good old-fashioned Requirements Document to some working code is a much more straightforward process than trying to turn stories into software. And the flexibility that’s supposed to be the main strength of the Product Backlog can also be a weakness. It can lead to developers feeling like they’re trying to find their way from point A to point B with a constantly shifting map. So there needs to be a way for all the teams to maintain a sense of direction toward the project goals. That’s where the meetings come in, right?
When Do All Those Meetings Become Mere Hoops for Developers to Jump through?
The Scrum methodology calls for daily meetings where the developers report their progress and discuss any obstacles they’re facing. Then there are bi-weekly meetings where they discuss progress and obstacles with the Scrummaster. And of course there are also the Sprint Retrospectives for the devs and the Sprint Reviews for the business unit at the end of every two-week sprint. That’s a lot of meetings. The danger is that after sitting through too many of these gatherings the developers may fall into the habit of treating them like mere formalities. They’ll contribute little, and what they do contribute will be of little substance. Instead, they’re basically performing for the team leader or Scrummaster—and they’ll get increasingly annoyed because they feel they’re being made to jump through stupid hoops.
All these troubles you run into when implementing Scrum actually arise from not really doing Scrum. For whatever reason, one or several of the people involved in process aren’t, well, involved in the process. The result is that managers end up taking what are supposed to be agile methods and applying them rigidly. One good test of agility which would be well in-keeping with Scrum principles is whether the team could potentially abandon an Agile approach mid-project and try using a more traditional methodology until the next retrospective. But there’s no such thing as perfect agility, and if too many team members are disengaged there’s not much you can do.
The Big Question: Does Scrum Work?
By suggesting that Scrum runs into problems when it ceases to be Scrum, many will charge that I’ve succumbed to the “No true Scotsman” fallacy (and you hear about this fallacy all the time in discussions of Scrum). If projects based on this methodology fail more often than not because one or more of the principles are prohibitively difficult to apply, then we can’t rightly claim that it’s an effective methodology. But what many in the new crop of Scrum Haters fail to consider is that we need to assess the strengths of each methodology relatively, not absolutely. Pointing to parts of Scrum that frequently go wrong is well and good, but the real question is whether Scrum goes wrong more or less frequently than other methodologies.
I can imagine a large survey of software development projects that compared the success rates of various methodologies. And that would ultimately be what it takes to settle the question of relative effectiveness. As far as I know, however, no such study has been conducted (so the debate will likely carry on indefinitely). In lieu of a scientific survey, though, we have to rely on our own anecdotal evidence, which points to the greater effectiveness of Scrum. Eric Potter describes what happened when his development team switched to Scrum in the middle of a project they’d begun with the Waterfall methodology.
The members of Potter’s team all agreed that Scrum was better. But the surprising part was that even the client noticed the improvements. Potter writes:
“One executive told me that while he didn't know fully what Scrum was, he could ‘feel the difference.’ He also mentioned that once we switched, he felt that the stakeholders had more meaningful interactions with the developers. We were finally implementing features that worked the way the business unit wanted them to work.”
The software development practice at Aptera quickly adopted Scrum for nearly all of its projects. Since then, the team has encountered all the difficulties discussed above, but there is still a strong consensus among all the members that the new methodology works far better.
Also check out: