Writing about agile while confined to avoid getting Swine Flu

Arrived to Mexico City on business last April 27 just two hours short of the WHO’s request to close Mexico's borders due to the swine flu. Not surprising, this has been the less productive of all my business trips: For example my last 10-day trip included 12 meetings and 3 presentations. This 14-day trip I ended up with lots of cancelations and so far have had 3 meetings and if nothing else is cancelled will have one meeting and one presentation before returning to the USA in six days. I even got to experience no traffic in Mexico City, wow! Better be safe than sorry, right? Streets are empty and remind me of a scene from the Spanish movie "Abre Los Ojos", remade as "Vanilla Sky" in English. Fortunately, I am staying at a very safe place, away from public contact and have been feeling as well as usual.

On the bright side, this has given me time to take some rest and do some writing, including an article on agile, standards, and models to be published on the Software Guru magazine this summer, and preparing some more training materials.

Mexico City traffic, vendors and self-organization

Everybody who has ever visited Mexico City for sure remembers the traffic. Think New York City traffic is bad? ...think again! A few decades ago the expressways and highways, although small for USA standards, were fast and a joy to ride. Nowadays traffic is so bad that on a recent day while stuck in bumper-to-bumper traffic on Viaducto (an expressway) I saw locals walking in between lanes selling all sorts of things, from sodas to cigarettes, chewing gum, fruit, etc.; they even had banners posted on the expressway’s walls announcing products and prices.

What amazed me the most was how efficient those people were. Keeping an eye on which lanes were moving more slowly to shift from one to the other so maximize the chances of making a sale. Monitoring the overall traffic to determine if they should step aside for a few minutes or stay put and increase sales. They are also distributed along the road so that each person has a “reasonable number of customers”. They knew intuitively that it was better to disperse along the road when cars where moving at a slow steady pace, and to get closer together but in between different lanes when cars were barely moving. I was amazed by the efficiency of their self-organization.

I started thinking about how I’ve seen development teams be less efficient when they are being managed, thus waiting for a decision-maker to come and order changes, and how much more efficient they become when they self-organize. It is better to identify the demands of task-at-hand, assess the situation, determine a course of action, and act.

Moprosoft and agile: can do

Back in 2001, the same year the Agile Manifesto was published, the Secretary of The Economy in Mexico created a national program for software development and called it Prosoft. The program's objective was for the software development industry in Mexico to reach levels that compete internationally; an ambitious goal with no execution plan. The initial efforts consisted on evaluating the ISO-9000, ISO 15504, SW-CMM, CMMI and concluded that none were adequate because "they do not comply with the requirements expressed by the national industry". I think it is good that their conclusion was not to use any of those standards or models although I fail to see what the difference would be between software development in Mexico and anywhere else in the planet, besides cultural aspects, to the point of needing a new model to be developed. Writing good software and being effective in developing it should be a basic premise anywhere. As result, the Mexican Association for Quality in Software Engineering (AMCIS) collaborated with the National University of Mexico to develop their own model for the Mexican small and medium size software companies and called it Moprosoft (http://www.comunidadmoprosoft.org.mx/).

In a nutshell, Moprosoft was developed over several years and in 2005 was declared a Norm to be utilized by the software industry in Mexico; it is documented in four volumes and most of its definition is a mapping of ISO/IEC 15504 and SW-CMM to what they considered adequate for software development within the boundaries Prosoft established. One addition they made was the inclusion of a large set of templates that need to be followed to document the design and implementation of the entire development cycle. There is five levels of compliance similar to those on some standards and models. Moprosoft definition is published in four volumes. Last but not least, the ISO is on its final stage of approval to accept it as one more of its standards. At industry level, this meant that the Secretary of The Economy established it as a standard that software development organizations in Mexico must follow and the ripple effect is on with adoption in several countries in Latin America.

Those of you who do agile might be thinking “oh no, it happened again” or something along those lines with respect to the friction dealing with standards and models. The question is how to do agile development under those circumstances. We have learned how to be agile even under an ISO or a CMM/CMMI environment and even some ISO auditors have learned the benefits of accepting documentation a-la-agile. The trick with Moprosoft is that the model not only establishes what to do but also includes detailed definitions on how to do it in the form of templates, think big-process-up-front.

The intention of this blog is to start a discussion and I’ll begin with some observations. Moprosoft adoption has serious disadvantages such as:

• BPUF. Four volumes of documentation and numerous templates imply a steep learning curve. For a software organization to be ramp up an achieve compliance it will be necessary to delay release dates
• Agility loss. The high level of detail to document imply that changes in the process will require big process adjustment effort, which will further delay projects and discourage software teams to do improvements over the products developed
• Slowness. Technologies evolve more rapidly than we can keep up with and having a heavyweight process in place makes it even harder to stay up to date
• Time to market is compromised. this is critical to business success and the friction that comes with compliance might just be the one thing that makes a company lose against the competition
• Productivity doesn’t improve. Because this is a repeat-for-each-project activity the friction remains no matter how many projects are developed under the model
• Creativity loss. Also, the time spent doing compliance work is time that could be much better spend doing creative work for the product being developed

One other aspect to consider is that in practice most teams use models as if they were standards, and that adds even more friction because standards establish a process to be followed whereas models recommend a methodology to take as starting point from which to customize. So, one interesting aspect with Moprosoft is that it was created as a model but it is now about to become a standard!

The best approach to the problem, IMO, is not to follow the model and instead adopt agile practices. For those teams who already have no alternative but to be in compliance my suggestion would be that instead of aiming for level 5 compliance they should be as lightweight as possible to remain within level 3, which is what is considered a good-practice level of compliance and use agile for the planning, estimating, and implementation/test process. Fit the practice within the templates and show auditors how the agile practice works within compliance and how it is also more effective than traditional practices.

Lean Manufacturing for electric manufacturing companies

My last business trip to Mexico included meetings with two electric manufacturers. One of them had a fairly good level of manufacturing process in place, with an reasonably clean/ordered plant and a good skill set baseline. The other one had a rather chaotic atmosphere: overcrowded in some areas, extremely hot, and with some workers building several different items at the same time. The commonality they have is the quality of their products is hurting them although, and surprisingly, the first factory more than the second one. Although both companies are profitable they lose money in the faulty products they produce and one of them even lost it's most important customer.

Both companies keep ISO compliance but the time/money consuming compliance activities have done little, if anything, to fix their problems. One first one has made efforts to follow JIT and Six Sigma but to no avail. I noticed during the visit to their plants two common denominators. They both hace a genuine interest on improving their processes; and they have received very limited coaching on how to implement and enforce a good manufacturing process.

I gave them a presentation on Lean Manufacturing and how it may help them produce better products and potentially get their workers to be more productive. Their response was enthusiastic and now it is a matter of them deciding if they want to give Lean a shot.

Hell's Kitchen needs some Agile and Lean lessons

I love cooking. Graduate school was overseas and the food at both the university's cafeteria and the local restaurant were less than appetizing despite of the great sea food, vegetables, and other goodies available at the market just a mile away, so I decided to take matters into my own hands and stated cooking. To my own surprise cooking was fun and easy; and I was good at it, thankfully because very few times I ate my food thinking I rather be eating at the cafeteria.

Although it is quite rare for me to watch cooking shows I have fun watching Hell's Kitchen. For those of you who don't know the show: there is this famous Chef Gordon Ramsay who recruits 16 contestants (8 women and 8 men) in two teams to compete cooking for guests at the restaurant under the same name as the show and eliminate one contestant from the losing team at the end of each episode. The winning team is treated big time with an outing to an upscale place, and the losing team has to do some quite nasty chores. The most fun part, from my point of view, is when the restaurant opens and the teams have to cook the orders because the entire kitchen turns chaotic very quickly. Food gets burned, undercooked, ingredients are missed, portions of one same dish are prepared at different times, contestants within the same team disrespect each other, customers get quite unhappy... and Chef Ramsay's shouting madly at the contestants until there is no way to save the night and the restaurant is shutdown for the night ahead of time. I don't watch the show all the time but from the couple dozen episodes I've seen out of five seasons service was completed only twice. That success record is embarrasing at the least.

Why does that happen? Both teams have strong motivation to win: not being eliminated and being treated like royalty. So, why things continuously turn bad? Here's my take on it:
- Poor communication: the team members don't communicate with each other to know who is doing what, keep their timings right, and help each other.
- Upfront planning: it would be a waste to plan how the entire evening will go because such plan would become obsolete within the first three minutes of receiving the first order; however they should all have a high-level strategy in place so that they know what to expect from each other.
- Slow to adapt: once one thing goes wrong problems escalate because they have a hard time adapting to the customer demands and the state of the kitchen
- Movement: as things start going wrong and escalate the contestants start moving around a lot more and more quickly. Such frenzy only delays things further and make it so much easier to make mistakes. They should help each other more instead.
- Lack of self-organization: were they better organized among themselves most problems would not occur.
- Lack of respect: Although it is a competition, if they respected each other they would work better as a team and significantly increase the chances of winning

Does that sound to you like what is going on in your organization or what happened at a previous job? I have many memories of projects going wrong for different combinations of the same reasons.

A very effective way to overcome those issues is by following the Agile and Lean principles. Effective communication facilitates self-organization which significantly reduces movement. High-level upfront planning allows the team predict each others actions more effectively and how to help each other. Continuously adapting to the current situation produces more and better results more quickly. And last but no least is the respect for each other, which not only makes working together more enjoyable but also increases productivity.

More comments on SDWEST´09

Agile vs. Traditional


On Wednesday, March 11, Scott Ambler and Terry Quatrani gave a fabulous keynote about traditional vs. agile sw dev. The setting was Scott playing a ¨hello, I am PC" role representing the traditional approach, and Terry playing the "hello, I am Mac" role representing Agile. Those of you who know Scott Ambler can very well imagine how hard it was for him to play his role, given the strong advocate to Agile that he is. In a very entertaining fashion and with the help of some images they showed aspects of tradional sw dev and how agile approaches the same issues. Terry's catch phrase for the keynote was "...delivering what the customer wants!"

Scott then proceeded to show some of Dr. Dobb's survey results, showing some eye opening results showing that agile teams do more modeling and generate almost as much documentation as their Traditional counterparts. Maybe more surprising were the results showing that team distribution is less affected by agile than by traditional organizations; i.e., Agile teams have a higher success rate.

Note: Source for the following diagrams was taken from Scott Ambler's 2008 survey via Dr Dobbs. The modeling diagram is a manicured version of the table he published.

Strategy for modeling


Documentation


Success rate






Jolt Awards
The oscars of the software industry. For a list of this year's Jolt Awards visit http://www.joltawards.com/winners.html

Software Development West conference and expo 2009, day 1

This year's SDWest (http://www.sdexpo.com/2009/) is being dominated by tutorials and classes on diverse Agile subjects.

Tutorials and Keynotes


Classes, case studies, and panels


The first session I attended started at 8:30 AM and the last activity ended at 8:30 PM. Needless to say I am happy to be back home relaxing at the couch. Here's some notes on what I attended:

a) Gerard Meszaros gave a tutorial on how things that need to be considered when going from concept to product backlog. Most practitioners have tend to pay attention to the activities closer to the implementation such as stories, use cases, and unit tests; and spend less time, if any, on the what needs to be done to translate the customer's concept into the right design. That is no design is as bad as BDUF because we end up with story blinders. Such lack of functionality not only delays projects but are a source for the need to redesign later on; and under-designing is as bad as over-designing. Projects require an iteration zero to do up-front planning. Five levers need to be covered: Product vision; product roadmap; release plan; iteration plan; and daily plan

b) Robert C Martin, a.k.a. uncle Bob, gave a keynote speech in his usual entertaining and engaging way about the state of the art of XP after ten years of existence (actually more since it started in 1995) . He is quite a showman as a presenter. Taking as starting point the xp diagram he pointed out that scrum covers the outer ring only. Scrum has been extremely successful because it talks at a business level in a way that the business people can understand and makes them more likely to buy in the methodology. The innermost layer is the geeky layer and the middle layer is the glue that sticks the other two together. The success of Scrum has been so tremendous that XP has been falling behind in terms of adoption. This is a very dangerous thing because agile loses effectiveness without good practices. The result is that blindly following scrum leads us to think we are moving fast whereas we are not because over time the cycles become harder to keep up with due to the code being harder and harder to maintain.. This doesn't mean agile is bad; it isn't because it exposes the problems and traditional/waterfall doesn't. What it means is that we need to raise the bar as programmers. This brings the central point of his presentation: we need to develop good software, period. Robert also pointed us to the Manifesto for Software Craftsmanship (http://manifesto.softwarecraftsmanship.org/main) which was put together very recently.
Raising the bar.

Manifesto for Software Craftsmanship As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value: Not only working software, but also well-crafted software Not only responding to change, but also steadily adding value Not only individuals and interactions, but also a community of professionals Not only customer collaboration, but also productive partnerships That is, in pursuit of the items on the left we have found the items on the right to be indispensable.
© 2009

c) Scott Ambler gave a tutorial on Agile Model Driven Development (AMDD) with two focus points:
The negative impact that traditional and waterfall approaches to software development have on productivity, quality, and deliverables. And the importance to drive our agile activities based on practice vs theory. Throughout the entire tutorial Scott went over and over again on the importance to plan and act honestly in the best interest of the success of the product and the organization. Regarding modeling he pointed out that the right model to use is the one that best communicates with the target audience keeping focus on the product's construction with the goal to develop a high quality system. Another important factor for large organizations is scaling distributed teams making sure that the minimum necessary up-front design is made. Note that this doesn't mean BDUF but that a larger amount of design is needed for larger teams than for small and for colocalized teams. Also, for remote teams it is more effective to divide the work based on product features than on job function because that reduces the amount of remote communication needed; although strong coordination will still be needed.

First Posting! Catching Up

Hello all,

This is my first blog posting so I am formatting it as a catch-up bullet list.

• Mar 3: Software Guru magazine wants me to write an article on Agile for their special issue to be published this summer. I will be writing about Agile and how it fits with standards and models, particularly with MoProSoft
  
• Feb 1~12: Back to Mexico. Followed up with some companies on deciding to adopt agile. Some of them are still enthusiastic but are yet to decide, in addition that March is the time to plan budget for the year so delays are expected. One government company is going really slow and will need a lot of effort from my side to get them to put the proposal on the right hands for approval. Also had a first meeting with a couple of small companies, both curious to learn more about agile.
  
• Jan 12~22, 2009: First business trip to Mexico. Gave a presentation on Agile to the Asociación Mexicana de Informática (AMIAC) at the National University of Mexico. There was a lot of enthusiasm from the attendants and most of their question were related to how to make Agile function with standards (ISOs) and models such as CMMI. I met with executives from 3 companies, one of them government, to present my services. Also met with Hanna Oktaba, director of ProSoft, to talk about their MoProSoft standard. The central aspect of the trip was to learn that most Mexican industries are very tightly bonded with standards and small software service companies seem to have no much choice but to abide by those rules. What I found somewhat unfortunate is that there is no much awareness of Agile and its benefits; and although there are some big Mexican companies that already do agile (most of them banks) the existence and use of agile is yet to gain momentum there. On the bright side of things some people showed interest and communication with them is ongoing. In conclusion I realized that it will take a lot of effort to get Mexico to adopt agile.