Draining the swamp

Taking the water out of water-scrum-fall.

I do love the term water-scrum-fall. It so aptly  describes a phenomena seen, particularly at scale, where teams are using Scrum for development (and perhaps some level of testing), but the processes before and after development are waterfall-ish.

waterscrumfall_cycletime
Figure 1 – Cycle time with Water-Scrum-Fall

Water-scrum-fall may be a side affect of standard Scrum education, which addresses development ceremonies/practices, but spends very little time on strategy, architecture, requirements management, testing and release at scale.  An organization transforming to agile can get the mistaken impression that Scrum is all about development practices. Too many organizations have thought that training and certifying your scrum masters will get you most of the way to an agile organization.  I’ve already talked about how this leads to cargo culting type failures.

Looking at cycle time, in water-scrum-fall, a typical timeline for processing a requirement looks about like figure 1.

Applying good automated testing and devops can virtually eliminate, or at least substantially reduce, the fall section of the cycle time, resulting in a time line about like figure 2.

waterscrum_cycletime
Figure 2 – Cycle time improved by using automated testing and devops to minimize “Fall” section.

Of course, getting to automated testing and devops is a significant accomplishment that will take months.

But how do we minimize the front end cycle time in water-scrum-fall?
The goal is something like figure 3, where the front end time is greatly reduced (in addition to the fall portion).

drainedwaterscrum
Figure 3 – Cycle time further improved by minimizing “Water” section

Why do we care about improved cycle time?  With dramatically reduced cycle time, software organizations can dramatically improve their responsiveness to change.  When the business climate changes, or you need to catch-up with a new direction taken by a competitor, short cycle time will help immensely.  With short cycle time, you’ll also see overall increased throughput (see video here), thanks to improved focus.

But there are complicating aspects in the front end.  There may be many different roles involved (Enterprise Architecture, UX, Operations) and many different interconnected work items.   There may be dependencies to work between teams.

Traditionally, deriving/deciding on requirements takes time.  Then Architecture decisions are needed.  The PMO can group work into big projects that require lots of planning.  Lots of documents were produced, then reviewed, revised, and communicated.

Scrum education didn’t cover this.  So, in too many organizations, that ‘water’ lead time persists.

Another aspect of the delay is that organizations tend to over multi-task. They explore too many options, working on too many tasks and projects at once.  The lack of focus (i.e too much multitasking) then drags out the *water* phase.

I doubt the approaches I’m going to discuss are controversial, but they are not widely practiced AFAIK. In a nutshell, here’s what I suggest:

  • Use workshops, attended by the right people, to reach decisions in a very short time frame (1/2 an hour to 2 days).  With the right people in the room, an appropriate agenda, and someone working as facilita
    workshopphoto
    Workshop

    tor to keep you on track, a lot can get done in a short time.

  • Reach group decisions by developing visual (not text) models.  The models document and communicate workshop output.  Visual models invite critical input from everyone (What’s wrong?  What’s missing?  What’s extra?)
  • Hold workshops at the last responsible moment (e.g. inside an iteration, after backlog refinement, at project kickoff).
  • Address only substantive concerns. Comprehensive modeling is not appropriate
  • Output may be specific answers, or it may be multiple experiments towards finding a better answer

 

Why I like Visual Models:

  1. Visual models serve well as information radiators.  Models provide a mechanism for readily communicating decisions, and provide a basis for incremental improvement as more eyes review your decisions.
    The models created can then be posted in team areas for future reference, and incremental improvement.
  2. Visual models are readily understood by consumers.  A brief explanation is likely the most that is needed for anyone to understand a well prepared model. Often, no explanation is necessary.

    storymapping
    Updating story map
  3. Visual models continually invite updates, revisions as new knowledge is obtained, and new conclusions reached.
  4. I haven’t heard anyone assert that visual models are not valuable.  Everyone seems to appreciate the value of the finished product.

Learning to draw the models, and knowing which model to use when, is where the difficulty arises.  In my opinion, knowing models simply gives you the appropriate tools you need to do your job.

I suggest you try to learn one model at a time.  Look through a list of models, find one that is meaningful for your current project, and figure out how to use it.  Continue to add to your skills and you’ll soon enough be a confident workshop facilitator.

A good Agile coach, in my opinion, will understand the importance of modeling, and either facilitate herself, or get a facilitator involved (e.g. from a company like Seilevel ).  Some of us can model and facilitate, and since we’re coaches, can show you how.

linoit
Linoit Board
web-whiteboard
webwhiteboard (of course)

Co-location is great of course, since these workshops can be done face to face with stickies and white boards. When not co-located, the workshops will suffer a bit, but web tools like webwhiteboard, Linoit, Padlet and others will substantially help when workshops are not co-located. These tools can also radiate the information after the workshop.

I won’t talk much about how to conduct a workshop (a big digression – see the references below). The basics involve having a clear agenda, inviting the right people, having a facilitator to keep you on track, and walking out with clear agreement on all decisions (documented with visual models). Bear in mind that you can team with others to model and facilitate. You know you have a good workshop when everyone is involved, adding ideas, drawing etc.

There are many models available.  But I’ve found a few to be most helpful.  Don’t overdo. Just use models you need.  Questions with obvious answers probably don’t need workshops/models.  When multiple opinions are in play, use a model to reach agreement, and then to communicate the decision.  .

 
RoleSuggested models
Product Owner/Product ManagerStory Map, Impact Assessment, State Machine, Persona, Glossary, Scenario, Prerequsite Tree,
ArchitectGlossary, Personas, State Diagram, Activity Diagram, Component Diagram, NFR Zone Map, Physical Model/Deployment Model, Data model, Scenarios
Project Manager/Scrum Master/ManagerPrerequisite Tree, Risk Map, Evaporating Cloud

I’ve added a few references in table 2

Title/Amazon Link Author Image
Requirements by Collaboration: Workshops for Defining Needs Ellen Gottesdiener requirementsbycollaboration
Visual Models for Software Requirements Joy Beatty and Anthony Chen  visualmodels
Software Requirements Karl Wiegers and Joy Beatty  softwarereqts
The Logical Thinking Process Scott W. Ambler  storymapping
The Logical Thinking Process H. William Dettmer  logicalthinking
Impact Mapping Gojko Adzic  impactmapping
4 + 1 Model Phillipe Kruchten  41_architectural_view_model-svg
User Story Mapping Jeff Patton  storymapping

Well run workshops can greatly reduce front end cycle time, thereby greatly improving the agility of your organization.  Many projects/sprints need very few models. But good information radiator models are priceless for getting clarity on decisions, and communicating within and across teams.

Your comments are welcome.

About Jim Brisson

After 15 years of agile experience and 35 years in the industry, I've formed a few opinions. Let me know if you want to hire agile coaching.

Agile Transformations need Coaching

The connections between Cargo Culting, Training and the Dreyfus Education Model

Some folks continue to believe that training on Agile is all that should be necessary to transform an organizaton.  I respectfully disagree and will try to explain why in this article.

First, let me briefly describe “Cargo Culting” (Anthropology Guide).

The term “Cargo Culting” may be traced back to World War II, when air bases were created on islands in the Pacific.  To appease native inhabitants, US airmen dropped gifts: perhaps chocolate bars, rib

Hellcats - WW II
Hellcats – WW II

eye steaks or whatever. Of course, the airmen set up control towers and runways and ran aircraft missions from their base.

After the war, the airmen left and the natives missed the gifts dropped from above.  To restore the gifts, the natives built bamboo plane like things, towers, put coconuts on their ears etc … and of course this did not work (Wikipedia).

In Agile Transformations, cargo culting translates to adhering to some of the rules of agile (e.g. ceremonies) without really understanding why, and so expected results never appear.

Let’s next talk about the Dreyfus Model for learning (Wikipedia article). The Dreyfus model applies well to unstructured problems where the number of potentially relevant facts are numerous, and many solutions are possible.  That problem description almost always applies to organization problems and management issues.  This sort of learning is quite different from learning facts in a History Class.  Where many problems and many solutions are possible, varied experience is necessary.  You need to develop a sense for what works in your context and what doesn’t. It’s easier to head off problems when they’re found early,  and experience will help you see problems earlier.  Eventually patterns are uncovered, and general principles may be derived for solutions. The Dreyfus Model clearly applies to Agile Transformations, and that idea is widely accepted.  The concept of “Shu Ha Ri” is very similar to the Dreyfus Model (Martin Fowler has a nice quick description here).

The Dreyfus Model asserts that skill acquisition goes through 5 stages: Novice, Advanced Beginner, Competent, Proficient and Expert:

  1. The Novice simply follows rules, and does not feel responsible for anything but applying the rules.
  2. An Advanced Beginner has found other conditions where the rules apply but still is just applying rules with no personal responsibility (e.g. I’m just doing what training said to do).
  3. When Competent, the rules are starting to become excessive. Some organizing principles are realized that help sort out relevant information to assist in decision making.  Responsibility for decision making appears at this stage.
  4. With Proficiency, multiple organizing principles have been attained from multiple real world experiences, and problem solving becomes more intuitive.
  5. With Expertise, the patterns are intuitively recognized with little or no decomposition needed (Agile Coaches should be expert).

Dreyfus Model

 KnowledgeStandard of workAutonomyCoping with complexityPerception of Context
NoviceMinimal/textbook knowledge not connected to practiceNot satisfactory unless closely supervisedClose supervision/ instruction neededCluelessSees actions in isolation
Advanced BeginnerWorking knowledge of key aspects of practiceStraightforward tasks can be completed in acceptable fashionSome tasks can be completed using own judgement, but supervision needed for manyUnderstands when problems are complex and can apply some analysisSees actions in a series of steps
CompetentGood working and background knowledge of area of practiceFit for purpose but lacking refinementCan achieve most tasks based on own juedgementCan copy with complexity using analysisSees actions partly in terms of broader goals
ProficientDepth of understanding of discipline and area of practiceFully acceptable work routinely producedCan take full responsibility for own workHandles complex situations thoroughly and confidentlySees big picture & understands how actions fit into that big picture
Dreyfus model stages

 

During initial training, people pay close attention to rules, want to learn the rules, and don’t understand and don’t care to listen to principles.  New learners have no foundational experience, and so are not ready for larger concepts, patterns or principles.

Scrum Master, and other Agile basic training follows this pattern.  Students initially want to learn the rules … standup meetings every day, check; maximum of 15 minutes, OK; Standups are not status meetings … hmmm … they sure look like status meetings.

For Agile Training students, the importance of the words in the 17-page Scrum Guide, and the Agile Manifesto are not yet comprehensible.  For new students, talk of principles is just noise.  Students are not ready for principles until they have more experience.  That’s a key point of the Dreyfus Model (and Shu Ha Ri).

I’ve seen an over reliance on initial training to accomplish Agile Transformation.  Let me describe about how this can happen.  Management likes what they hear about agile, and wants to realize the benefits.  So they decide to train everyone and then they’ll be Agile.  Of course training everyone is expensive so they get the Scrum Masters trained (and certified – they’ll be able to tell others what to do if they’re certified).  So Management contracts certified trainers to come to their business to train their employees at perhaps $1300 per.  With some trainers, employees taking the course can easily become Certified Scrum Masters after taking a two day course.  Employees take a certification test (previously no test at all), but the test is easy enough to pass based on the two day course.

Graduates of this training are Beginners (in the Dreyfus sense) …but they may now be certified.

After training, scrum masters start by following the rules they learned in training.   But they encounter difficulties.  Without expert advice, simply following Beginner rules often doesn’t get results.  Because management has made it clear they’re transforming to agile, employees continue to follow the rules, but employees don’t understand how to get the results … they’re now cargo culting.  Without expert advice, Scrum Masters may skip ceremonies they find useless, and invent processes that are Frankenstein blends of old and new practices.

cargocultfail

Training Problem

The “Training Problem” diagram above sums up the scenario.  Starting from the bottom, training produces Novices.  Novices without expert guidance will move into Cargo Culting.  Improvement is not realized, so the transformation is not successful

Management will eventually figure out that the transformation based soled on training is not working.. Some clever managers will then hire Agile Coaches.  The agile coaches have seen all this before. But now their problem may be more difficult.  The coaches have to deal with bad habits and misunderstandings.  Perhaps even worse is dealing with “Certified” employees that are actually Beginners.  Some employees will insist they’re agile, but Agile doesn’t work.

Most coaches would rather work with folks new to agile than work with an organization that developed bad habits and bad attitudes.

The Dreyfus model has it right.  You cannot start with principles and patterns when teaching.  Folks need to first learn the basics, then, through practice learn some patterns.  They need to see multiple examples of things working badly and then working well..  After understanding the patterns, they can move on to principles.    Expert guidance is key to succeeding on this journey.

Initial training need only cover the basics (e.g. the rules) suggesting that deeper explanation will follow.  Then people need to practice the basics, and get coaching from an expert on how they’re doing and how to grow some more.  Regular meetings to discuss problems and learnings go well with this approach.  Advanced training can proceed at a pace appropriate for the learners.

dreyfusproficient

Coaching Solution

This approach is summed up in the “Coaching Solution” diagram. Initial training covers the rules, and may suggest some of the principles.  Learning continues through experience, coaching, and advanced training. The Dreyfus model suggests that you must be at least at Proficient level to be able to coach. I assume Agile Coaches should be at expert level.

After a year or two of education and real work experience, perhaps people are then ready to take a more difficult exam and become certified.

Agile can work.  But please do not train folks in one session and expect success.  The initial training results in knowledge of basics that can lead to cargo culting, Frankenstein practices and transformation failure.

My advice:

  • Training alone won’t work. Expert guidance is needed, and Agile Coaches should be expert.
  • You might well start by hiring coaches, and let them give initial training on the basics. Coaches can then keep employees on track as they gain experience, revealing the principles through multiple examples as employees learn.
  • Be very wary of education that promises certification in just a few days. Certification exams can test the basics on the ceremonies and techniques. But certification tests must also verify that deeper understanding of principles that enable achieving agile goals.

 

About Jim Brisson

After 15 years of agile experience and 35 years in the industry, I've formed a few opinions. Let me know if you want to hire agile coaching.