Project Tracking Sheet

I have created and collected quite some excel sheets over time while managing/specifying/estimating/coding/testing/releasing/deploying projects.
I spent a while putting the most important ones together in a single Excel Workbook.
Home page of project tracking sheet

Home page of project tracking sheet

You can replace the <<Project>> value on the front page and all other sheets will get updated. There is a sheet for features with a system for tracking the dev status.
Tracking sheet Features

Tracking sheet Features

And a kind of poor man's issue tracker. This is no Jira but does the job when you share the sheet on a MS Network.
Tracking Sheet - Issues

Tracking Sheet - Issues

 There is more, like a graphical follow-up of functional tests and a follow-up of service disruptions. One sheet that proved very useful is the deployment sheet:  
Deployment Sheet

Deployment Sheet

Get the Excel! If you have such sheets, I am interested in comparing notes.

Sparx EA – Modeling Business Processes with BPMN and EA : A case study of Callataÿ and Wouters

Albert and BPMNBPMN basics are the next topic in the session. The point of model a business process is to capture the organizational way of working. Typically we talk about operations, customer support, and R&D. A notation is needed to talk with each other. BPMN provides a visual notation. It is standard now. Enterprise Architect supports BPMN1.1. The OMG (Object Management Group, altough sometimes they err on the side of « Oh My God » on the complexity front) is maintaining it. Well, actitity diagrams do form the basis for modeling the processes, made of activities, decisions, transitions and so on. So, why BPMN when we have activity diagrams ? BPMN provides a much richer notation, with cleared building blocks (receiving, complex decisions, resources). FYI, there is a core set of diagram elements, namely : Event, Sequence Flow, Activity, Message Flow, Gateway, Association. In the complete set of diagram elements, we ge tthis like special event types : message, timer, exception, cancel, compensations, rule, link, terminate, multiple. Activities also get a fair treatment on that front. Subprocess, task, recurrent task... We can also use Pool et Lane. Pool is a pool of people (think and organization). Inside a pool, you can get multiple lanes. One person can be a lane as well. Some rules do apply. Message flows can only go between pools, because you are not going to control the business processes of other organizations (they won't let you!). All of these things are depicted with a nice little icon on the diagram. This would be a bit short but there are also the tagged values and the specific pick lists in the tagged values windows for supporting that. All in all, the BPMN support is very decent. As usual, EA provides top notch document generation facilities and getting printouts that hold water nicely is a non-issue and work very well. I jumped in (could not resist) and explained a bit about how BPMN and EA work well hand in hand along with the quicklinker, the shortcuts for creating the same item over and over (shift-F3, F3) if you have lots of them, the fact that you can put the tagged value window docked in a nice place. The tagged value window is crafted in such a way that it behaves in an improved way when dealing with BPMN. Case study time ! Mr Pierre-Philippe Delmarcelle presents the case study. He is Enterprise Architect for C&W (I would say tought leader on these issues). Mr Delmarcelle from Callataÿ and Wouters (2) Callataÿ and Wouters faces the challenge of having customers asking about how to solve business issues and not only be a technical provider. Internationalization brought its own challenges : standardization needs(BPMN helps when working with integrators, consulting, ...), explaining how the solution works. Traceability is very important due to regulations. With risk reduction needs, customization is not that hot anymore. But flexibility points are needed still. To support Enterprise Architecture, we need a business process model, an application architecture model and a systems model. The business is modeled with the Business Object Model, the Business Landscape (BIAN aligned), the Business Process Model, the Business Organization Model. These things have been (quickly) created in the previous months of 2009 with the help of Expert-IT. TEA is the name of the enterprise architecture of the Thaler product of C&W. Traceability between the process activity and the components that realize it is maintained. A number of business areas are established (the diagram appears quite busy, showing the complexity of the undertaking). So, this screen supports this or that process, being part of a specific domain. All modeling was done using BPMN in Enterprise Architect. The full metamodel has been created in Enterprise Architect and made available as a custom toolbox, a set of rules governing the model and so on. Analysts do work fully inside Enterprise Architect and output RTF documents from which PDF document can be delivered. Import/export of model parts is done with the XMI (Package Control features) so that analysts can work in a disconnected way. Archimate serves as a solid metamodel as an inspiration source. EA is a very nice tool for Pierre-Philippe Delmarcelle:very flexible, not costly, with lots of exports (XMI allows to move it into other tools (mega, aris)). How to use the business process model in the projects. Before : Customer Questions : How will you do the implementation? Are there gaps? Since time to market is money, this is helping tremendously on the added value front. Before, we conducted demos and looked at screens. This was very fine grained, missed the big picture. Now (with the new methodology) : the thinking with landscape model helps in scoping faster the discussion. With the client, we can now come with our Thaler reference business model in the initial workshops. The business processes of Thaler and the ones of the customer can be compared. We can now explain how our solution works in business terms. Please note that the business processes in the blueprint are clearly highlighting how Thaler shines as a solution there. Some demos are still necessary but much less than before. The outcome of the workshops provide clear guidance on how to perform the implementation in a value-added way, with a high probability of success. The gaps are now made clear and can be addresses with an economy of means. An outcome of the new ways is that the responsibilities are much clearer when it comes to the customer side and the integrator side. The improved quality is a key addition. As a side effect, this brought a new service when it comes to helping in tailoring business processes. Clear communication is really the power of business processes. Lot more gaps identified with the new approach (up to more than 278 in a large project. Reduced to almost zero when looking at the ways that Thaler standard system worked!). Risks are really going down. Key take aways :
  • TEA is now the underpinning of everything
  • A guiding blueprint is useful for :
    • Client facing demand management to create a solution enabling strategy
    • Client facing implementation much less risky and aligned with the strategy
  • Customer and internal projects are accelerated, risk is mitigated
    • Better communication
    • Start form business concepts
    • Use market agreed concepts leading to consensus fast
  • Efficiency and cost reduction achieved
    • Better scoping of requirement and management of development
    • Standardization and resuse of common concepts
      • Enable SOA strategy for the solution
      • Without the bluepring and details, SOA cannot be truly realized because you need a catalog of services that do make sense in real situations
This makes it very clear that the Enterprise Architecture is not only vaporware but a pragmatic, value-deliverying, business accelerating approach. Callataÿ and Wouters proves it and as such is positioned as a market leader in its field. Kudos to Pierre-Philippe Delmarcelle and team for his great presentation and great vision. His achievements of creating more than 120 processes, defining the methodology, producing the reporting, training teams and partners has been defined in less than 6 months. BIAN proved to be a great asset when it comes to mastering complexity and chunking it in manageables pieces that can be assimilated properly. As a bonus, TEAL -- Thaler Enterprise Architecture Language - a special BPMN evolution has been defined, implemented and put to use. Sparx Enterprise Architect served as the central repository that federated everything together. So: BIAN + TEAL + Blueprinting + Components + Enterprise Architecture = Success and Solid Basics! Like in the A-Team, "I love it when a plan comes together".

Sparx Systems EA – new 7.5 features

Phil & Jean-GabI attended a session on the new Sparx Systems Enterprise Architect organized by my friends at Expert-IT. This time, I am in the audience and not presenting anything. Last time, I was showing Ivar Jacobson Essential Unified Process and EssWork. With the new 7.5 version, Sparx systems Enterprise Architect, there is a move further than structural code generation. From now on, STD, sequence, and activity diagrams can be used to generate code. In fact EA now moves into new ground: mindmapping, business processes modeling, org charts. For business users, BPMN is a good way to capture the processes in a strong repository. A forte of the tool is the traceability features. The fact that EA is based upon a strong relational repository helps tremendously on achieving that. Impact analysis is made easier. Also, business rules are now a first class citizen, more on that later. All artifacts|work products can be stored in the repository as well. EA 7..5 is a major release. No 8.0 yet but we can expect a lot more exciting characteristics end of year. New pricing, albeit still reasonable. What is new is that additional editions are available. Business and Software, Systems Engineering, Ultimate Edition. Since the customers attack larger problems with the tool, performance is improved for WAN communication. WAN optimizer is available as well as a lazy loading feature allowing to load data on an as needed basis. You'll need Pro for the lazy loading and Corp for WAN optimizer. On the Business Process modeling front, the beast can generate BPEL from BPMN 1.1 (get your acronym dictionary ready!). DoDAF, TOGAF, MODAF and Zachman Framework are included. There is also Archimate somewhere in the mix When UML is lacking, EA provides the missing pieces, a good point. MDG integration and MDG link are not the same things. About the Systems engineering edition now: SysML and simulation are in, and code generation to hardware languaged works: Verilog and friends. BPEL support takes the form of additional dialogs for example. BPEL is drawn in BPEL diagrams using the BPMN icons. Please stay aware that BPMN is for the 'what' of a process and BPEL is 'what' since we want to get XML for running inside a process engine. The new business rules system is on steroids: a fact model (a kind of class diagram allowing to capture facts that work with the new feature) is created and an 'orchestration' class is needed there. That special class then gets support from an associated ruleflow diagram for process-type rules (this time an activity diagram retooled a bit for BR support). Ruletasks are the activities there. Moving further, a business rule dealing with decisions is present and uses the elements of the fact model to author clean rules in a rule composer. Nice! Then code can be generated by the tool. This is clean and allows bridging the developers and the business analyst. When it comes to impact analysis, this is great. The challenge is to work at the right granularity level where the rules under modeling are going to offer a clear benefit/investment ratio. As always, model with a purpose. Maybe you want to do some simulation. Build and run is inside EA now, so, self containment is achieved. Another acronym: CTF is of use (Code Template Framework). Strategy Maps, Balanced scorecards, value chains. Flow charts, decision trees and orgcharts do complete the new features. This can move EA inside the boardroom in my view. Obviously as an output, not asking C-level people to draw them. But who knows... Scripting is now fully doable inside an EA code editor. And the files are part of the project if you want to do so. Great! Debugging is also available. VBscript and JavaScript are supported. Intellisense works and helps while navigating the object model of Enterprise Architect. An advanced math library is available. No clue on what it provides yet. More advanced wizardry with EA is available with MDG technologies. New toolboxes, items, ... The sky is the limit (well, as always do not confuse the means with the ends. Falling in love with tooling and ending up with a new addiction is not going to put bread on the table).

Gravitation, Accretion and black holes

From Wikipedia: In general relativity, a black hole is a region of space in which the gravitational field is so powerful that nothing, including light, can escape its pull. The black hole has a one-way surface, called the event horizon, into which objects can fall, but never emerge from. It is called "black" because it absorbs all the light that hits it, reflecting nothing, just like a perfect blackbody in thermodynamics. Quantum analysis of black holes shows them to possess a temperature and radiate like black bodies. Despite its invisible interior, a black hole can reveal its presence through interaction with other matter. A black hole can be inferred by tracking the movement of a group of stars that orbit a region in space which looks empty. Alternatively, one can see gas falling into a relatively small black hole, from a companion star. This gas spirals inward, heating up to very high temperature and emitting large amounts of radiation that can be detected from earthbound and earth-orbiting telescopes. Such observations have resulted in the general scientific consensus that, barring a breakdown in our understanding of nature, black holes do exist in our universe. This appears to me as being very similar to some IT projects I do see now and then. Indeed, the gravitational field of these projects is so strong that a lot of individuals get attracted to the project, mostly due to political agendas. There also appears to be an event horizon because people getting in there rarely see the light of day before soon (this usually amounts to several years). The main issue is that the mass of the beast is usually growing, that “companion stars” are sucked in and valuable resources are expended just to create high temperatures and radiations, but little tangible results (not to mention that they are not going to be frequent either). To further the metaphor, one can think of super massive black holes that do make a whole galaxy gravitate around them. One interesting feature of blackholes that also appears to apply here is that space-time also gets massively distorted when it comes to shipping dates and effort estimates. Also note that black holes do reflect nothing, a characteristic that is shared with several large projects where people do seem to have forgotten their brains at home. The one who do not have understood that the longer timeframe and the larger the headcount, the more impressive they do look in terms of “territory” and “army size”. This is really bad when we consider that studies from Larry Putnam and other metrics-gurus tell us for ages that when you go above a headcount of 7 in a project and that a project lasts more than 2 years (1 would be better), you are heading for trouble ! So, a large endeavor should be broken down in pieces that can be addressed by a team of 7 or less. Each team should collaborate with other teams to get things done. Obviously massive efforts will require advance collaboration techniques. Still, the majority of the blackholes that I do encounter are created by accretion of people for not clear reason. These massive monsters can and must be avoided.

What to do ?

Well, first of all, blackholes are usually massive things and you should first think about what your level is in the organization. Just whining will not cut it and only worsen the problem. Furthermore, if you are down the chain, it will just provide management with lame excuses for not achieving results. But if you are in a position of responsibility and if improving your way of working is one of your objectives (since it usually shows on the bottom line),  then it becomes a professional responsibility to do something to remove the blackhole. Not mentioning that doing nothing will result in bad karma, think about the consequences of letting this go:
  • good money is sucked in and never to be retrieved (and the burn rate is usually high), hurting the bottom line severely. In these times of crisis, this amounts to a cardinal sin
  • people who are otherwise motivated in other areas do see the blackhole and reduce their own efficiency, again hurting the bottom line
  • messages are becoming distorted when moving upon command chain and bad decisions are taken in return
  • a heavy politically-laden climate is developing, leading to people reducing productive work and starting to trash more and more
What is to be done is at lease to report the fact to upper management (at least twice). But no whining, just facts and consequences (monetary figures works best). The best way to have an improved situation is to have some kind of indicator able to detect blackholes. As a decision maker, do yourself a favor and add such an indicator to your dashboard, scorecard, KPI's or whatsoever. One of the best indicators will reflect on frequent, tangible, working results. Not reams of paper, not slideware. Ask to see a working prototype, test reports working on real case scenarios against the system delivered, the bug summary reports. Most of all, do not be in denial of problems. Of course, you may not have the time to do these things. But then you need to have them prepared for you and in an unbiased manner. If you are surrounded by "yes-men", this may prove difficult. Get some help there. Another thing to do is to avoid the tunnel effect. This means that the project entered a tunnel at one point and there is no visibility for a long time. Well, this cannot be tolerated anymore these days. Quality assurance (more of a mindset than a phase or a position) must occur and produce metrics and reporting that helps in identifying issues as soon as they do arise. Remember that a project gets 2 years late by a day at a time. Letting it slip is the issue, and reaching for a scapegoat at one point is not a decent approach. Would you drive you car without a dashboard ? If a project is not able to give you a view on quality assurance measurements from a neutral point of view, this is a symptom of dysfunction. Get help there. There are always antibodies in any organism to react to the medicine. Businesses are no different. To close on the issue, please remember that throwing more money at the problem in the hope it will go away will only make it worse. Do resist the temptation since it is well known that hope is not that good as a strategy. Using hope as a strategy is like buying lottery tickets and hoping for the win. It may happen but the probability is pretty weak. Any decent risk manager arouind the block will tell you so. So, if  you suspect you do have such beasts in town, I suggest you avoid digging yourself into a hole (which may end up being more brown than black at the end of day) and and take action to stop the radiations. Just sticking your head in the sand will expose the remaining rear parts to radiations, which may hurt in the long run.

IT Archaeology of existing systems

There are lots of architects around but I do think that we are in dire need of IT archaeologists. Based on my work as a coach to IT teams working with production business systems, I have to saythat the latest and greatest in technology, frameworks, patterns, and what not is not at all the success factor. Too many times, appetite for technology is hiding the real issues. People do prefer to dive in tech and burn time than to address the core business needs and achieve ROI. The characteristics of the systems at hand are that they do often exist for ages, most of the knowledge does lie in the heads of people, there is a firehose producing an endless stream of changes to implement, and existing documentation material lies in tons of solution-centric documents. Typically, early contributors have moved to other jobs without transferring knowledge. Most of the time, when confronted with change requests, the jump is automatic from high level requirements to technical implementation since people in the know are so coupled with the system and change-centric, tactical documentation is produced. Real issues are associated with the current situation: the mobility of personnel is very limited (I will dare to say frozen), the truck number very low, and it is hard to implement visionary changes to system. Also, implementing normal changes takes eons. When it comes to raising the bar in terms of innovation, the defensiveness of personnel is hindering the success of the initiatives. Mostly, this is not related to a competency problem, nor a lack of goodwill. As far as I am concerned, I do think that the personnel is quite stressed out when required to maintaining the current system alive and any improvement initiative has to consider that factor. When approaching the challenge of documenting existing systems and applications, one must ask what documenting means in the first place. More precisely, documenting an existing system must have clear benefits. If nobody is able to articulate then the effort is just adding more work to the backlog and should be avoided. So, always document with a clear purpose. One very important purpose is to be able to navigate the system code from a higher level of abstraction. Navigation usually requires a kind of map. The map is so that a contributor can understand the key concepts, constructs, and mechanisms being in place and perform changes on the system in an efficient way. This means being able to see the system from a logical perspective, devoid of technological jargon (no files, Oracle databases, CICS, JSPs, ...). Technological jargon is obscuring things more often than not. What is important is to know the responsibilities that a piece does provide, not which technologies are used to implement them. Producing such maps is the key output that an IT archaeologist can provide. But producing useful maps requires global vision. It also means that you have to accept to do research (sometimes exhausting research) and not always be sure to find a decent map. You need to be tolerant to ambiguity to be a good mapper. Assembling bits of knowledge is good but will never be providing a good map. Studying them and figuring out the patterns behind the tons of sand requires exposure to lots of systems of various types. You also need to have a good theory to back the thinking. Notations such as the UML can definitely help. Also, awareness of process frameworks such as the Unified Process is good for separating the various disciplines at play. But all the theory in the world is not worth a single sheet of a useful map. Theory can back action but must stay at its place and not in the foreground. The point is that the map must help to clarify the situation and not further confuse it. The process is looking like: Study existing Documentation ? Remix, and Structure ? Produce Useful Documentation For documenting the system, here is our way to perform some archaeology: As a system is a solution to a given problem, we must be clear about the purpose: are we making a map of the problem or are we making a map of the solution? In our case, we do have a system and do want a map of it. The solution is made out of building blocks, let's call them logical components for now (another name would be subsystems). These logical components are often « virtually present » but are not documented as such (most of the time, you'll find folders grouping files). A characteristic of subsystems is that they do have strong cohesive features : the logging subsystem, the calculation subsystem, the dispatching subsystem, the presentation subsystem, the external gateway subsystem, the security subsystem, the audit subsystem etc. Some items are not easily amenable to the grouping into a subsystem but would rather be « aspects ». For example, even if the security subsystem would concentrate the responsibility of implementing the core of security, enabling security across other subsystems would not be possible to concentrate in that subsystem. But we can introduce a security « aspect », that would be spread to whatever subsystem or subsystem interaction that is subject to realizing a security requirement. Finding out these logical components and aspects and creating the base map out of them would already be quite an achievement as an archaeologist. Another output of the archaeologist is a number of typical interactions between these logical components. And in order to make sense of all of the domain-specific words and concepts used in the project, conceptual documents explaining the business that the system automates is part of the job of the archaeologist. IT Archaeologist may not be the sexiest job on the planet at first sight but it adds tons of value and does indeed have the potential to save millions that would have been spent burning cash spinning wheels.