How to structure our workflow when working in an Agile way for a significant project?

Spamcast 205 - September 2012 Column Transcript Follow me on Twitter: @philippeback ( Hello everyone, this is Philippe Back and I'll be discussing how we came up with a way to structure our workflow in a project where I was a project lead (as this involved two firms, we were two at the helm actually). On the methodological front, we were supposed to use an Agile approach. And a tool. Well, okay, agile and a tool. I have been saying it for ages, say more than 10 years now, I am all for people over processes and tools.

Tools to the rescue

But at one point, people do love to have a tool to help them track their work. Especially when it becomes quite complex, full of little things to remember, and with interdependencies between people that are scattered around quite a number of locations, some very remote. Add to the mix that some resources in a project can become quite hard to reach because they do have a scarce skill, and that fact alone leads to them being unable to be around all the time. In such a situation, so much for post-it notes. So much for a whatever-kanban-or-anything board on the wall. It just isn't sufficient to deal with the situation. There are tools to help here, our happens to be IBM Rational Team Concert. A fine tool if you ask me. Especially when you go through the massive learning curve that comes with it. Let me digress here. It is important to have a tool to help and the tool has a massive learning curve. Given the typical project setup and constraints, nobody in its right mind would go for a massive learning curve when a deadline is near.

Process practicioners and process engineers

That's why we must distinguish process engineers from process practitioners. Process practitioners do not have to care about all the nitty gritty details of the bowels of the beast. They just need a clean workflow that they can participate to. A way to work out their tasks, hopefully in an oranized way that makes sense. And a workflow that will help them. Not turn them into bureaucrats. Process engineers must master the tooling. But you do not need a ton of them. At the same time, I guess we do not need the kind of process engineer who would rather be splitting hairs in 4096 and come up with a process template so convoluted that anyone looking at it would either run for the bar or the door (depending of which was near at the time of that vision from hell). Long story short, here are two key factors for success:
  1. Process practitioners who are convinced that the thing actually helps them
  2. A process engineer (that should be waay enough with solid expertise in keeping things simple and effective (we'll leave efficient later as an efficient ineffective approach kind of leaves me cold).

Structuring 101

Let's get back to our structuring thing. In the agile "ontology" (oops, sorry, one more buzzword, so let say "vocabulary" or "agile-talk", there are several interesting structuring concepts:
  1. Product
  2. Iteration (aka Sprint, well, a Timebox will do anyway)
  3. Release
  4. Backlog
  5. Impediments, blocking factors (I kind of like the impediment thing tough)
  6. Defects
  7. Tasks (almost forgot those, all this management stuff is really growing on us at times)
  8. Stories
  9. Epics
All of these bits and pieces are nice. But actually, they aren't helping much with structuring the work. I kind of like Epics. So much that I used them as the way to represent my PBS (for Product Breakdown Structure). Maybe you know the GDPM (Goal Directed Project Management) method. It is kind of neat. There is one single structuring concept that hit home with me for all those years. The notion that when thinking about a project, one must look at three specific things (and no, I am not talking about the PSO (People, Systems, Organization) acronym). These are:
  1. Enabling Factors
  2. Core Product
  3. Support Functions
What are these in some more details? Well, "Enabling Factors" are the elements that you need to have in place so that you can actually work. No team, no work. No systems, no work. No office space, no work. You see the thing. Under that label, we also find silly things like "Access to the facility is provided". Some learning about unknown areas also do lie under those "Enabling factors". Make no mistake, those aren't all have to be put all upfront in the first iteration of your project. Quite a bunch of them should be there actually but well, not all of them. And "Enabling Factors" is an Epic for me. Then we ge the "Core Product" which is what you are trying to build in the first place. There we have things like Core-Platform, Core-Clients, Core-AccessManagement, Core-Mechanisms, Core-Flows, Core-Monitoring, Core-Reporting. I like have all those things as "Epics" as well, all under one parent "Epic" named "Core-Product". The children of the Core-Product Epic are more often than not structured like what you have as architectural building blocks in you high level design. Don't think of them as cast in stone, but rather as a guide to help in structuring the Core Product. As we learn more by doing, the structure may change. No big deal if you work with a decent enough tool that will allow us to restructure dynamically on the go. And, last but not least, we get the "Support Functions" part. Oops, pardon me, the "Support Functions" Epic. And this one is supposed to be the home for all things preventing the core product from suffering too much decay as it is built and operated. Typical entries include: "Backup is operational", "Source is managed in the source control system", "Bugs are reported", "Estimates are provided". Well, I guess you can picture that. All of this structure of epics provides a quite standard layout into which one can hook the myriad of stories that are in the backlog. Epics One thing is for sure, it is that an Epic is not something that will be confused with a task. An Epic labeled "Configure a queue in the X queue manager" is sending chills down my spine. "The queuing system is configuredas specified" is already better. The fact that it could be a story and not an epic is another matter. Let's now focus on "Tasks". Indeed, there is work to be done, right?

Structuring tasks

Now, we have this set of "tasks" that people will have to carry to reach the destination. What I see is that a lot of times, we do get silly tasks, labeled in wild ways. And that doesn't help in getting a clear picture. I do hate a "fog of war" when it comes to software projects. So, how to write tasks that do hold water? Consider this: You always have three kinds of tasks: Design tasks, Implementation tasks, and Handover tasks. enter image description here To get something working, you need to think it through, get a pass at actually building it, and once done, ensure that what you produced as output or value gets transferred decently to whoever needs it further in the stream. And then they can be pulled by the team members. Well, at the beginning, they may need a little gentle push to get started. And then these can be estimated. And performed. And given actual values. And velocity computed. And a realistic, understandable burndown chart produced.

A little recap

I guess our plate is full for this installment. Let's summarize:
  1. In order to be effective, we need some structure
  2. The basic tenets of Agile are all fine but we need some more structuring concepts. We can get those from Epics, themselves broken down in further epics.
  3. The three top level epics are: Enabling factors, Core-Product, and Support Functions
  4. Get the core product Epic broken down in sub-epics, one for each architectural building block, and one for key mechanims you need.
  5. Get stories out of the backlog and stick them under that top-level, PBS-like structure.
  6. Put tasks in there, along the lines of Design-related tasks, Implementation tasks, and Hand over tasks.
What not to do... Yes, it is important to remember that as well. So, what not to do:
  1. Do not ... Try to be complete up front. That is going to put you into a frozen state in no time.
  2. Do not ... Try to come up with a plan full of dependencies in all directions. A single parent connection from stories to epics and epics to parent epics will do.
  3. Do not ... Expect the team to make all of this happen on its own out of the blue. Get someone working on the structure and be ready to reorganize.
  4. Do not ... Expect to master all of this overnight. It took me years. I'd be glad you can do this in no time, but actually, that's not how it work. So, do not put that pressure on your team as well. Let the concepts permeate the thinking and be ready to reorganize mistakes like, every evening so that the structure is clean for the next day.


I hope you'll find something usable in all of this. Remember that a frame like the one we discussed allows the team to collaborate and transfer their energy and abilities into tangible resutls. A frame like the frame of a bike. A frame that frees you from trouble. Not to be confused by a frame that binds you into limits. But remember that with great power comes great responsibility. Act accordingly and refrain from too grandiose a plan for a single timebox. Reality always wins in the end anyway. See you next time. Bye! --Philippe