When (not why) “Agile” and especially Scrum are terrible

This article is a response to “Why ‘Agile’ and especially Scrum are terrible” as posted on Michael O. Church’s blog. He is not so much attacking the agile manifesto, as he he is attacking the reality of “agile”, the way it portraits itself and the excesses.

If you read the comments on Reddit then you find that there are two camps: scrum advocates and scrum haters. The haters complain how scrum leads to bad behavior and the advocates argue that this behavior is caused by doing scrum wrong. In this post I will focus on this behavior that, both parties agree, should be avoided.

1) Being judged personally

You should not be judged on hour by hour productivity as a software developer, because constant daily competition is not healthy. What particular tasks, technical debt or research tasks each developer worked on in a sprint is irrelevant. Peer pressure is the most effective way to keep people productive and looking at individual performance kills peer pressure. If there is micro-management going on, then that has nothing to do with being “Agile”.

Although there may be a high level of transparency, that does not mean that management should be evaluating people’s performance at the end of each sprint. The story points are designed to prevent management from doing this. The retrospective allows for criticizing the performance and effectiveness of people in the team by the team.

Especially web-based “Agile” registration tools must be used with care (or avoided). They are not part of “scrum” as white-boards and stickies are preferred. Also they may add lots of bureaucracy and may be misused to draw conclusions. Conclusion that may destroy the valuable trust relationship between employer and employee as they are more likely to be wrong than right.

If you see “agile” and “scrum” as tools to achieve “accountability” or “predictability” you are missing the point. It is merely a method to talk about work done instead of work to be done.

2) Developers being considered the lowest rank

The “product owners” and “scrum masters” do NOT outrank “team members”. Although they prioritize the user stories, they should not not choose which ones get executed. Also they should not assign tickets. If they feel superior or they are treated (or paid) as such, then something is wrong and not in the last place because good team members are all-round technical experts and these are extremely hard to find.

Team members are stakeholders as well. They can not only choose which tickets to work on (from the top of the stack), but also add stories (often technical debt) and argue why they are important.

One of the goals of agile is to eliminate blame and fear in the workplace. In a good team people do not fear to say “I was stuck with my code all day and got nothing done” during a stand-up. People should understand their pain and instead of blaming them offer some help.

3) Short term focus

Under Agile, technical debt can pile up not being addressed, because non-technical business people decide on priorities. They will not see a problem until it’s far too late or, at least, too expensive to fix it.

This pileup may be the result of  a straightforward application of “scrum/agile” principles. The focus may easily shift to only things that bring business value or “visible features” as I like to call them. These principles might work when imposed/implemented by an engineering team onto itself, but they break apart instantly the moment the management tries to impose them from outside.

There’s no reason why major refactoring can’t be tackled as a story. And having metrics to measure this (e.g. steadily increasing time to deliver features) should make it easier, not harder, to make the business case.

Conclusion

Short term work under high scrutiny is not fun nor agile, avoid it.

About Frank, our fantastic handyman

johnny-automatic-electric-drillAt home, we have a handyman. His name is Frank. He is very experienced and all-round and was able to take on every construction or repair job we gave him in the past years. Frank is easy going, never stressed out and often takes a break to smoke a cigarette and enjoy the weather. Although we pay him by the hour, we do not complain about his smoking as he asks a reasonable fee and works quite efficiently. Also, my wife really likes the guy, as she feels he can be trusted and always understands her really well.

For instance, last year he redid our kitchen, which turned out really nice. And although we wanted to go for a four ring gas hob he convinced us to take a five-ring hob with a central wok burner. He argued this would hardly take any extra space and would be very convenient in case we needed to use a bigger pan. Every time we have visitors over, my wife refers to Frank and how happy she is about the choice we made.

This year we were getting the bathroom done, but since we only have one shower in our apartment I asked Frank: “When will the bathroom be done?”. He answered “that depends”, “why are you asking?”. I told him about my concern of using the shower and he smiled. “I can make sure the shower can be used every time I leave as long as you don’t mind the mess.” he replied. Well… that solved one of my worries, but I was not completely satisfied, so I kept asking.

“But, when will the bathroom be done? I mean.. how much money is it going to cost?”. “Those are two different questions” Frank answered, “The first one depends on how often and how long I will be here and the second thing depends on what the bathroom should look like and what other more important jobs you have around the house”. “Hmmm… I understand”, I mumbled, but had the unpleasant feeling that he was dodging the question.

My wife has a very busy and responsible job, but she is also the one that decides on the interior related things in our house. Not that I do not care, but we both know that she has “feeling” for these things and I (an IT guy) do not. I told my wife that I could let Frank in and pour him his coffee, but that I did not see how we would organize this bathroom rebuild. She had an easy solution: She asked Frank to promise to work 3 days a week for at least 2 months. He worked on Monday, Tuesday and Thursday, since he had another job going on Wednesday and Friday. Every Thursday evening he would leave late, so he could meet my wife to show what he did and discuss with her what he was going to work on next week.

I would not have taken that route and would have probably asked an interior designer to draw the bathroom in 3D. Then I would have asked several building contractors to quote me with a price and a delivery date. I’m sure my wife would not have been as pleased as she is now with our new bathroom. The bathroom turned out exactly as we wanted, she even feels she has built it herself. Another good thing is that we only paid Frank for the work he actually did and the materials he needed. Of course not everything went flawlessly: Frank had to reroute a sewer pipe because we wanted the toilet in a different position. That was a lot of hassle, but my wife really wanted it like that, so it was our own choice in the end.

I’m glad I’m in IT doing agile projects, because construction work is not my cup-of-tea. 😉

Hey government! Agile software tenders please..

This is a follow-up on my Dutch article on Agile government software development tenders. I believe that the current tender strategy on software builds is causing a waste of (tax) money by our (Dutch) government.

Apparently governments think that bigger tenders lead to lower costs. They probably think that higher volume allows for lower margins. This is an economic rule and this means that it should be true for all industries, right? Not for software development. Big tenders lead to BDUF (Big Design Up Front), which means: less control, steering, validation and thus higher risk of failure.

If you calculate the risk of failure as part of the costs, then the simple economic rule “higher volume allows for lower margins” is no longer true (for software development).

So for the above misconception software development projects are made as big as possible. The government will try to describe what should be build in this huge contract. In the software industries these descriptions are called “specs” (or specifications). There can be hundreds of specs. The contractor has no other option than to “build to spec”, which means that even if the specification is wrong. he has to build it wrong, because the spec is part of the contract and this is agreed.

Specs fail, Agile does not

Even if the spec was written correctly in 2011 and the specs were finished by 2012 and then the project finished in 2014, it may be that the specs are no longer valid. This is what we call in software development a “moving target”. So now there are 3 options:

  1. Ignore that we are building something that is not needed anymore.
  2. Adjust the specs, but this means we have to renegotiate price and delivery time.
  3. Move the work to “extra work” category in which it will be charged an hourly fee.

Every deviation or unclarity in the original spec can be used to earn extra money or make the deadline fluid. Writing a spec is very hard. So hard that it is, in my opinion, impossible to write an unambiguous and good spec in a reasonable amount of time.

So as a government you have to choose between two evils. Either you hold the contractor to build something that is not useful or you change the spec. In the first case the project fails and in the second case you have to pay extra and/or accept that the deadlines shifts.

To make matters worse the government rules clearly state that the cheapest offer should be chosen. I believe that contractors understand the rules of the game and that is why they are underbidding. This leads to an even more stubborn contractor as he needs to “earn back” the money he “lost” to win the tender.

Not lean at all! Reduce waste now

This is a very ugly game that is played with our tax money. And in my opinion the government should apply what we have learned years ago in the product software business: work Agile. In Agile we only commit on effort and not on spec. We let a group of experts work for 2 weeks on a project, look at the result, steer the direction and continue.

Governments should make a few domain experts available for the project. These people should be seen as the “customer on-site”. The domain experts and the contractor winning the bid should use a SCRUM way of working to guarantee short feedback cycles.

Then one last thing: Software building should no longer be seen as projects, but should be considered products. If people work on software for a limited time, they are less committed to its quality and reduced complexity. Also the software that the government offers should be constantly changing as does our world, the government regulations and all other software around us.

Further reading:

  1. http://www.computerworld.co.nz/article/492220/agile_approach_may_clash_govt_tender_policy/
  2. http://www.computerweekly.com/news/2240187680/Government-tenders-for-agile-digital-developers
  3. http://diginomica.com/2014/04/22/uk-government-spending-millions-agile-development-coaching-training/

 

Agile ontwikkelen bij ICT aanbestedingen overheid?

Hieronder een (zo goed als) lopend verhaal over de ICT problematiek bij aanbestedingen door de Nederlandse overheid. Het bestaat uit een viertal quotes over hetzelfde onderwerp, maar uit verschillende bronnen. Ze geven een overzicht van de pijnpunten en pleiten tevens voor het gebruik van de “Agile” methode bij ICT aanbestedingen.

Bart van Oosterhout in Intermediair (2007):

Een op de vijf grote ict-projecten bij de overheid komt nooit van de grond, en dat kostte tot nu toe miljarden. De Tweede Kamer wil nu een parlementair onderzoek. Maar in de ict-sector weet men allang waarom het zo vaak misgaat. Over megalomanie, fantasy- deadlines en de wederzijdse wurggreep van de aanbesteding.

Sander Hulsman quote Marc Maas (QNH Consulting) in Computable op 05-09-2014:

Net als bij infrastructuur besteden overheden te grote percelen aan. Dit heeft verschillende effecten. Zo zorgt een grote dealwaarde voor grote belangen bij bedrijven om de deal binnen te halen. Natuurlijk zorgt dat voor de gewenste prijsdruk, maar zorgt ook voor neveneffecten. Zo wordt de deal te scherp aangeboden waarbij bedrijven genoodzaakt zijn om te sturen op meerwerk. Dit kunnen ze omdat de klant de aanbesteding te generiek uitvraagt, waardoor veel interpretatieruimte ontstaat. Deze wordt gebruikt door de leverancier. Dit zorgt voor misverstanden en frustratie.

Paul Weghorst schrijft in 2012 op scrum.nl:

Binnen het uitbesteden is een traditie gegroeid van uitgebreide specificaties opstellen, een aanbieding aan de markt vragen in een proces met minimale interactie, de goedkoopste laten winnen en vervolgens een contract opstellen met hoge boeteclausules. Of dit allemaal echt wordt voorgeschreven door de wetgever is nog maar de vraag, maar het is wel de standaard van vandaag. We weten ook dat dit zelden verloopt zoals het bedoeld is: gemiddeld 35% van de specificaties verandert, en het levert software op waarvan 65% niet of nauwelijks wordt gebruikt. Er wordt stevig geëscaleerd, de relatie komt flink onder druk gestaan, maar ondanks 9 jaar ervaring in zakendoen onder aanbestedingen, weet ik dat boetes zelden worden geëffectueerd (de opdrachtgever was zelf nooit van 100% onbesproken gedrag).

Michiel Steltman schrijft in zijn column op rijnstreekbusiness.nl in mei 2014 over Agile:

Die aanpak bleek wel goed te werken. Software bedrijven als Microsoft en Google weten dat ook ontwikkelen al sinds jaren ‘Agile’. Maar zo gaat dat niet bij onze overheid. Die blijft volharden in de illusie van voorspelbaarheid. En roept met de aanbestedingswet in de hand steeds opnieuw dezelfde ellende over zich af. De hoogste tijd dus om het over een andere boeg te gaan gooien. Laten we bij het volgende grote systeem nu eens alleen afspreken om met elkaar een paar duizend uur te gaan klussen aan een probleem. Met als doelstelling ‘we zien wel waar we uitkomen’.

Ik ben voor! Hoe dat moet? Neem een voorbeeld aan deze Agile aanbesteding in het Verenigd Koninkrijk van 40 miljoen Britse Pond.

NB: De ruim 40 developers van LeaseWeb gebruiken de “Agile” methode met veel succes voor de interne software development projecten.

Building software is like building a LEGO house

One of the problems of the software development industry, is non-software people making decisions about software builds. Many of these people believe that building software is like building houses, which it is not. People see the software business, approach it like building a house, specifying upfront and asking when it is finished. Assigning project managers as general contractors and hiring cheap labor to execute the build. I can hear some managers who are reading this think: Wow! Can we do that? That is the answer! No, it is not, we tried it for more than a decade and it failed spectacularly. They even came up with a name for it: “out-sourcing”.

But even though we move from Big Design Up Front (BDUF) to Agile the analogy of building houses is hard to get rid of. Yes, we know we do Agile with sprints, no deadlines just sprint goals and a backlog with user stories and a road-map with epics. But we are still building a house. Well.. no.. we are actually not building a house. The whole house analogy is wrong. We are building a LEGO house. Yes, something like this:

lego-house

Was it a coincidence that Paul Hammant used a LEGO picture in his post about software architecture? I think not. Here are some of the points that show why building LEGO house is a good analogy for building product software:

  • Innovation: Best builds come from experimenting constantly and thinking out-of-the-box.
  • Flexible: LEGO house foundations can be replaced carefully. Anything can be replaced any time.
  • Looks: Building a pretty LEGO house is very hard and requires special skills.
  • Agility: There is no such thing as a “done” LEGO house. It is alive and constantly changing.
  • Requirements: You don’t plan building a LEGO house. You know what you want when it evolves.
  • Specs: LEGO is enjoyed best by building whatever comes to mind, not by following specs.
  • Compatibility: Everything connects to everything, with hardly any effort.
  • Freedom: LEGO can be build without having to comply with all kinds of regulation.
  • Creative joy: Creating software is as much fun as creating with LEGO.
  • Value: People pay for the promise of easy integration and high flexibility.
  • Validation: A real-life test by the target audience tells you things you could not imagine.
  • Mindset: Grown-ups do not understand LEGO, like business does not understand software development.
  • Out-sourcing: If somebody else builds your LEGO house, you will get something you don’t like.
  • Standards: Build with non-standard blocks and you’ll get stuck quickly.

Can you think of other things in which the analogy is right? Use the comments! Don’t comment saying that I do not take software development serious. I believe that software development is a serious business, but it is also seriously fun. To put it stronger: If it does not remind you of playing with LEGO when you were still a kid, you are doing it wrong.