How to structure our workflow when working in an Agile way for a significant project?

Spamcast 205 - September 2012 Column Transcript Follow me on Twitter: @philippeback ( Hello everyone, this is Philippe Back and I'll be discussing how we came up with a way to structure our workflow in a project where I was a project lead (as this involved two firms, we were two at the helm actually). On the methodological front, we were supposed to use an Agile approach. And a tool. Well, okay, agile and a tool. I have been saying it for ages, say more than 10 years now, I am all for people over processes and tools.

Tools to the rescue

But at one point, people do love to have a tool to help them track their work. Especially when it becomes quite complex, full of little things to remember, and with interdependencies between people that are scattered around quite a number of locations, some very remote. Add to the mix that some resources in a project can become quite hard to reach because they do have a scarce skill, and that fact alone leads to them being unable to be around all the time. In such a situation, so much for post-it notes. So much for a whatever-kanban-or-anything board on the wall. It just isn't sufficient to deal with the situation. There are tools to help here, our happens to be IBM Rational Team Concert. A fine tool if you ask me. Especially when you go through the massive learning curve that comes with it. Let me digress here. It is important to have a tool to help and the tool has a massive learning curve. Given the typical project setup and constraints, nobody in its right mind would go for a massive learning curve when a deadline is near.

Process practicioners and process engineers

That's why we must distinguish process engineers from process practitioners. Process practitioners do not have to care about all the nitty gritty details of the bowels of the beast. They just need a clean workflow that they can participate to. A way to work out their tasks, hopefully in an oranized way that makes sense. And a workflow that will help them. Not turn them into bureaucrats. Process engineers must master the tooling. But you do not need a ton of them. At the same time, I guess we do not need the kind of process engineer who would rather be splitting hairs in 4096 and come up with a process template so convoluted that anyone looking at it would either run for the bar or the door (depending of which was near at the time of that vision from hell). Long story short, here are two key factors for success:
  1. Process practitioners who are convinced that the thing actually helps them
  2. A process engineer (that should be waay enough with solid expertise in keeping things simple and effective (we'll leave efficient later as an efficient ineffective approach kind of leaves me cold).

Structuring 101

Let's get back to our structuring thing. In the agile "ontology" (oops, sorry, one more buzzword, so let say "vocabulary" or "agile-talk", there are several interesting structuring concepts:
  1. Product
  2. Iteration (aka Sprint, well, a Timebox will do anyway)
  3. Release
  4. Backlog
  5. Impediments, blocking factors (I kind of like the impediment thing tough)
  6. Defects
  7. Tasks (almost forgot those, all this management stuff is really growing on us at times)
  8. Stories
  9. Epics
All of these bits and pieces are nice. But actually, they aren't helping much with structuring the work. I kind of like Epics. So much that I used them as the way to represent my PBS (for Product Breakdown Structure). Maybe you know the GDPM (Goal Directed Project Management) method. It is kind of neat. There is one single structuring concept that hit home with me for all those years. The notion that when thinking about a project, one must look at three specific things (and no, I am not talking about the PSO (People, Systems, Organization) acronym). These are:
  1. Enabling Factors
  2. Core Product
  3. Support Functions
What are these in some more details? Well, "Enabling Factors" are the elements that you need to have in place so that you can actually work. No team, no work. No systems, no work. No office space, no work. You see the thing. Under that label, we also find silly things like "Access to the facility is provided". Some learning about unknown areas also do lie under those "Enabling factors". Make no mistake, those aren't all have to be put all upfront in the first iteration of your project. Quite a bunch of them should be there actually but well, not all of them. And "Enabling Factors" is an Epic for me. Then we ge the "Core Product" which is what you are trying to build in the first place. There we have things like Core-Platform, Core-Clients, Core-AccessManagement, Core-Mechanisms, Core-Flows, Core-Monitoring, Core-Reporting. I like have all those things as "Epics" as well, all under one parent "Epic" named "Core-Product". The children of the Core-Product Epic are more often than not structured like what you have as architectural building blocks in you high level design. Don't think of them as cast in stone, but rather as a guide to help in structuring the Core Product. As we learn more by doing, the structure may change. No big deal if you work with a decent enough tool that will allow us to restructure dynamically on the go. And, last but not least, we get the "Support Functions" part. Oops, pardon me, the "Support Functions" Epic. And this one is supposed to be the home for all things preventing the core product from suffering too much decay as it is built and operated. Typical entries include: "Backup is operational", "Source is managed in the source control system", "Bugs are reported", "Estimates are provided". Well, I guess you can picture that. All of this structure of epics provides a quite standard layout into which one can hook the myriad of stories that are in the backlog. Epics One thing is for sure, it is that an Epic is not something that will be confused with a task. An Epic labeled "Configure a queue in the X queue manager" is sending chills down my spine. "The queuing system is configuredas specified" is already better. The fact that it could be a story and not an epic is another matter. Let's now focus on "Tasks". Indeed, there is work to be done, right?

Structuring tasks

Now, we have this set of "tasks" that people will have to carry to reach the destination. What I see is that a lot of times, we do get silly tasks, labeled in wild ways. And that doesn't help in getting a clear picture. I do hate a "fog of war" when it comes to software projects. So, how to write tasks that do hold water? Consider this: You always have three kinds of tasks: Design tasks, Implementation tasks, and Handover tasks. enter image description here To get something working, you need to think it through, get a pass at actually building it, and once done, ensure that what you produced as output or value gets transferred decently to whoever needs it further in the stream. And then they can be pulled by the team members. Well, at the beginning, they may need a little gentle push to get started. And then these can be estimated. And performed. And given actual values. And velocity computed. And a realistic, understandable burndown chart produced.

A little recap

I guess our plate is full for this installment. Let's summarize:
  1. In order to be effective, we need some structure
  2. The basic tenets of Agile are all fine but we need some more structuring concepts. We can get those from Epics, themselves broken down in further epics.
  3. The three top level epics are: Enabling factors, Core-Product, and Support Functions
  4. Get the core product Epic broken down in sub-epics, one for each architectural building block, and one for key mechanims you need.
  5. Get stories out of the backlog and stick them under that top-level, PBS-like structure.
  6. Put tasks in there, along the lines of Design-related tasks, Implementation tasks, and Hand over tasks.
What not to do... Yes, it is important to remember that as well. So, what not to do:
  1. Do not ... Try to be complete up front. That is going to put you into a frozen state in no time.
  2. Do not ... Try to come up with a plan full of dependencies in all directions. A single parent connection from stories to epics and epics to parent epics will do.
  3. Do not ... Expect the team to make all of this happen on its own out of the blue. Get someone working on the structure and be ready to reorganize.
  4. Do not ... Expect to master all of this overnight. It took me years. I'd be glad you can do this in no time, but actually, that's not how it work. So, do not put that pressure on your team as well. Let the concepts permeate the thinking and be ready to reorganize mistakes like, every evening so that the structure is clean for the next day.


I hope you'll find something usable in all of this. Remember that a frame like the one we discussed allows the team to collaborate and transfer their energy and abilities into tangible resutls. A frame like the frame of a bike. A frame that frees you from trouble. Not to be confused by a frame that binds you into limits. But remember that with great power comes great responsibility. Act accordingly and refrain from too grandiose a plan for a single timebox. Reality always wins in the end anyway. See you next time. Bye! --Philippe

Anybody in for collective team intelligence?

A Sunday at Camp Smalltalk before #esug2012 – neat!

A rainy trip to a shiny place

Today, I went to Camp Smalltalk and it was a great experience. The place was the offices of Lots of desks, power chords, coffee that didn't suck. And a ton of wonderful people that do know their trade. Thanks to Estaban for putting up with my never ending questions on how to get things working with Pharo on the iPad (now things do work) and to Igor for helping me out with a PNG loading that just fails (this still under the microscope). Got a neat talk with Johan Fabry and Ben gave me a quick session on how Spec works. I'll invest more time in that, I promise. Got a chat with Sean, very interesting as well.

Launching images with Pharo Launcher

I got a tweet from HwaJong Oh and asked him about how Pharo Launcher worked. We got it from the App Store and I got a walkthrough of the features. Nice tool, especially when it searches for all images on your box, including into Pharo One-Clicks.

A quest for food

Getting food proven to be somewhat of a journey. At the end I got a kebab with Guillermo and Estaban near a bridge. This is kind of a welcome change as far as I am concerned. But kebabs are better in Brussels!

A look at presenty

I also had a look at presenty, showcased by Dionisiy. Interesting stuff in there, especially for creating stories to be played on an iPad. Some ideas of usage:
  • Business games
  • Kids storylines
  • Game screen sequences

Debuggging iStackVM on iPad

Well, I got my debugging lesson with Igor. I learned interesting little tricks, including the Cmd-D on iTerm2 to split the screen. Also, I am more motivated than ever to put some keybindings for the bluetooth keyboard since that double-touch gesture from hell is really a usability liability. Good thing to remember for the debugger, using arrays to capture several calls. Like in:
  {stream next. stream next. stream next.}
ExploreIt or printIt and things are nice.

Parting Ways

Time flies! So, the day was over. I discussed with Johan the possibility to do a Pharo on iPad thing in my offices, like in November. I can host about 12 people over here. Interested ones, please drop me a line. Also, I'd like to see how we can harness Pharo with CUDA and NativeBoost and Opal. That could be a killer combo. So, thanks for all and see you on wednesday (no other available slot for me, what a shame).

Moving to Markdown

I've been seeing markdown used more and more and decided to give it a try. (Ok this thing exists since 2004, it is about time I move!)
Markdown’s syntax is intended for one purpose: to be used as a format for writing for the web.
I do like wiki syntax and markdown is quite good enough for a lot my uses. Also, it is much easier than SGML-style or XML style DocBook (which I still like a lot) when it comes to teaching people on how to use them (I've seen people freaking out.) Give it a shot. It will improve your speed when it comes to write for the web in no time. Okay, one hour max. Also, check the WP-Markdown plugin. Install, Settings, Enable. Done. Don't forget the toolbar.

NVidia #CUDA deviceQuery on 13″ Macbook Pro Mid 2009: 16 cores

Just in case you wondered, here is what CUDA can give you on a 13 inch MBP from 2009. I am interested in your results PhilMac:release philippeback$ ./deviceQuery [deviceQuery] starting... ./deviceQuery Starting... CUDA Device Query (Runtime API) version (CUDART static linking) Found 1 CUDA Capable device(s) Device 0: "GeForce 9400M" CUDA Driver Version / Runtime Version 4.2 / 4.2 CUDA Capability Major/Minor version number: 1.1 Total amount of global memory: 254 MBytes (265945088 bytes) ( 2) Multiprocessors x ( 8) CUDA Cores/MP: 16 CUDA Cores GPU Clock rate: 1100 MHz (1.10 GHz) Memory Clock rate: 1064 Mhz Memory Bus Width: 128-bit Max Texture Dimension Size (x,y,z) 1D=(8192), 2D=(65536,32768), 3D=(2048,2048,2048) Max Layered Texture Size (dim) x layers 1D=(8192) x 512, 2D=(8192,8192) x 512 Total amount of constant memory: 65536 bytes Total amount of shared memory per block: 16384 bytes Total number of registers available per block: 8192 Warp size: 32 Maximum number of threads per multiprocessor: 768 Maximum number of threads per block: 512 Maximum sizes of each dimension of a block: 512 x 512 x 64 Maximum sizes of each dimension of a grid: 65535 x 65535 x 1 Maximum memory pitch: 2147483647 bytes Texture alignment: 256 bytes Concurrent copy and execution: No with 0 copy engine(s) Run time limit on kernels: Yes Integrated GPU sharing Host Memory: Yes Support host page-locked memory mapping: Yes Concurrent kernel execution: No Alignment requirement for Surfaces: Yes Device has ECC support enabled: No Device is using TCC driver mode: No Device supports Unified Addressing (UVA): No Device PCI Bus ID / PCI location ID: 2 / 0 Compute Mode: < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) > deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 4.2, CUDA Runtime Version = 4.2, NumDevs = 1, Device = GeForce 9400M [deviceQuery] test results... PASSED > exiting in 3 seconds: 3...2...1...done! While we are at it, here is the bandwidth test.   PhilMac:release philippeback$ ./bandwidthTest [bandwidthTest] starting... ./bandwidthTest Starting... Running on... Device 0: GeForce 9400M Quick Mode Host to Device Bandwidth, 1 Device(s), Paged memory Transfer Size (Bytes) Bandwidth(MB/s) 33554432 1563.6 Device to Host Bandwidth, 1 Device(s), Paged memory Transfer Size (Bytes) Bandwidth(MB/s) 33554432 1391.7 Device to Device Bandwidth, 1 Device(s) Transfer Size (Bytes) Bandwidth(MB/s) 33554432 5061.8 [bandwidthTest] test results... PASSED > exiting in 3 seconds: 3...2...1...done!  

Insights into what #bigdata is and how to deal with it (by Sogeti’s VINT)

I attended Sogeti's event yesterday and it was a great. Lots of insights. Watch this vid that covers some of the discussed ground. I'd rather name all this "Big Insights" since that's what we are interested in as outcomes. Data is meaningless per se. My body is full of information but I don't need that at all, it handles itself. Maybe that's what we should target. Recorded Future from Sogeti VINT on Vimeo.