Cargo Cults

"Those who cannot remember the past are condemned to repeat it." - George Santaya

The very mention of the phrase "cargo cult" will elicit either a knowing smirk or confused ignorance from anyone who hears it.  There is no other known reaction to this phrase.  For those in the know, there is a certain condescending superiority in mocking the ignorance of an apparently primitive people. The sense of superiority is not earned as every person exhibits cargo cult behavior at one time or another.  Being able to identify this behavior is truly a rare skill.

For those who have never heard of cargo cults, prepare to learn an interesting quirk of history and, more importantly, how it applies in everyday life.  The cargo cult phenomena can be traced to the later part of the nineteenth century.  As Europeans and Americans began to interact with the islanders of the South Pacific, they misconstrued the mechanism by which the white people gained their apparent wealth.

To the islanders, Whitey performed certain rituals and the gods rendered unto them the gifts of material wealth or cargo.  Of course, the islanders desired cargo from the gods and began to copy the rituals that delivered the cargo.  The Islanders had been known to create mock airstrips, radios out of coconuts, conduct army field maneuvers, and design air traffic towers.  The fetishization of these rituals in the attempt to appease the gods is what makes it so comical and tragic at the same time.  It's laughable to think that talking into a coconut radio in a crudely constructed air traffic control tower beside a fake runway that has never seen an airplane will produce the desired result.

In addition to mimicking the rituals that produce cargo, the largest active cargo cult centers around the myth of John Frum.  John Frum is the god in their religion - a black man who has mastered the art of making cargo appear.  He had promised the islanders that he would return to them and bring bountiful cargo.  On the island of Vanuata, February 15th is known as John Frum day.  Every year, the islanders gather and conduct mock army drills carrying fake rifles with the letters USA written on their bodies anticipating the prophesied return of John Frum. 

Interestingly enough, the name Frum does not appear in any census in the English speaking world prior to the start of the cargo cult.  The theory is that the cult was founded by a GI who introduced himself to the natives as "John from America".  It's as if everything he said after that was completely misinterpreted by the natives.  John Frum day has been celebrated for over seventy years and John has not returned to the natives nor brought any cargo.  Still, year after year the celebrations continue and the belief persists that John's return is imminent.

Lest any Westerner feel too smug, it has been said that a participant in John Frum day was asked how can he go on participating in the drills waiting for John to come back when no one has heard from him in over seventy years.  The cargo cultist simply replied, "We have only been waiting seventy years for John to return and you have been waiting for over two thousand for your Jesus."  You have to admit, he has a point.  There are plenty of everyday examples of dogmatically following an idea without understanding why it is supposed to work.

In the United States, the diet industry is a $40 billion a year juggernaut.  In its simplest form, weight loss can be achieved by eating fewer calories than a person expends.  It really is that simple.  Yet how many people attend a Weight Watcher's meeting and then eat a Krispe Kreme doughnut?  Going to Weight Watchers is not a bad idea to get some support, learn some diet success tips, and learning about nutrition; but it will not result in weight loss in and of itself any more than speaking into coconut radios will bring cargo.  The meeting does not cause the desired result, it is the behavior modification that matters.

The mistake of thinking that attending the meeting results in weight loss can be found all over corporate America.  For example, the application of Six Sigma processes to software development.  Six Sigma originated at Motorola in the 1980's as a way to improve quality.  The desired goal was to have less than 3.4 defects per million units - a standard of quality so high that testing units would actually result in more defects than not testing.  For defined manufacturing processes where defects followed a normal distribution curve, Six Sigma provided great mathematical tools and concepts to improve efficiency.  The success of Motorola's methodology prompted other companies to start applying what was a manufacturing process to just about everything.  At General Electric, under the leadership of Six Sigma loving Jack Welch, Six Sigma methodology was applied to every aspect of the company.  Recruiters in HR were tasked with creating projects to lower the amount of days it took to hire a candidate for a job posting, which begs the question is it more important to hire someone quickly or to hire someone who is a perfect fit for the role?  Would it be better if the job was open for 30 days and filled with a mediocre candidate or if it took 50 days to find the right candidate?  With mis-applications such as Six Sigma projects for HR, the entire movement quickly became a corporate Cargo Cult.

Six Sigma has five distinct phases:  Define, Measure, Analyze, Improve, and Control.  New buzzwords were created to find out what the real problem to be solved for like root causes, voice of the customer, voice of the business, etc.  Templates were created to capture the information and an emphasis was placed on metrics.  In the HR example, the problem was open positions were not being filled quickly.  A goal was set to reduce this time and an elaborate way of measuring the opening's position was created.  Where the new process gets Cargo Cultish is the moment incentives are put in place is the same moment that people will start gaming the system for their own purposes.  What is in the best interest of the project manager, the HR recruiter, and all involved with the project may or may not be in the best interest of the company.  The project manager will claim that due to their hard work, the average number of days it took to fill a position fell by some percentage saving some amount of money (the savings will be based on a series of assumptions which are dubious at best, but the good thing is no one will ever ask for the numbers let alone challenge them), and he or she will probably get promoted.  The cost and time spent on the project will rarely be factored into the program's true cost and everyone will get a promotion.  Unfortunately, the new system will reward hasty hiring and not good hiring.  Mediocre candidates will receive job offers in order to not exceed the arbitrary goal of leaving a position open.  An analysis of the quality of candidates resulting from the new process will likely not be performed and all attention will be placed on the now critical (but once arbitrary) number of days a position was open metric.

At least when it comes to hiring, the consequences of this example are somewhat harmless.  It has been shown time and time again that the way a candidate performs in an interview situation has no correlation to how they perform on the job.  Where it gets really scary (and even more Cargo Cultish) is when processes like this are applied to cost cutting and used to define Key Performance Indicators (KPIs), and worse circulated in executive scorecards.

The Dreaded KPI and the Birth of Offshoring

The offshoring movement has been going on for decades.  Around the year 2000, it picked up momentum starting with call center jobs.  In theory, it was cheaper to hire people in India to handle customer support calls from America.  Since the consumer in America was calling, not interacting face to face, why not send the call to a country where people speak English (sort of), did not have unions, did not make demands for health care, and were willing to work for a fraction of what an American would ask for?  Call centers sort of work to drive costs down and so long as consumers stick to the script, it kind of works.  Sure, investments must be made in building out the infrastructure, rigorous training, linguistic classes to minimize accents, and management.  However, the workers in the call centers eventually believe they are valuable and have no loyalty, so turnover is high.  To counteract high turnover, wages go up, bringing a price equilibrium to about the cost of keeping the job onshore, but like I said, it kind of works for call centers.  It's low end work and at the end of the day, regardless of where the work is done, no one is ever really happy with the resolution that is provided.

Since offshoring call centers sort of worked, there has been a movement to offshore all kinds of other work on the premise that it will be cheaper.  Take, for example, programming of large-scale enterprise systems applications.  In the fury of systems integration work leading up to the Year 2000 and the dreaded Y2K bug, it was said that implementing Enterprise Resource Planning systems was "the corporate equivalent of a root canal".  The projects were complex and very expensive.  In an effort to lower costs, companies started to offshore what was thought to be low level tactical work.

My theory is that someone took a look at the hourly rate that a programmer was making in 1999.  Let us say, for example, billing rates were between $150 and $200 per hour.  To a manager making less than a six figure salary annually, this looks like an exorbitant number.  Most consultants or programmers who were billing this hourly rate were receiving only a fraction of it, but the manager paying the bill could not comprehend how this person they hired was making nearly $400k per year.  An Indian offshoring company walks in and says we will provide you with programmers and we will charge you $20 per hour.  The manager has no problem paying someone forty grand a year and it seems like a bargain.  Instantly, their bill was lowered by 90%.

Unfortunately, the adage you get what you pay for has never been more true.  In systems integration work, the real problem is rarely writing the code.  The problem lies in understanding what exactly the end user really wants as, often times, it is not what they say they want.  Users, Subject Matter Experts, Customers, Clients, Stakeholders, Business Managers or whatever you call them do not think in terms of code, tables, or design.  They think in terms of high level business scenarios and think a fully defined specification is, "JUST convert the purchase orders from our legacy system." (A real spec I have actually handled in a previous life)

Please note, the above "specification" makes no mention as to what validation rules must be performed on all purchase orders, what tables the legacy purchase orders are stored in, how to map any differences in data between systems, etc.  The very high level and non-actionable requirement is then sent offshore to be acted upon by a software engineer who barely speaks English and has zero practical business experience.  The engineer will predictably do one of two things, he or she will either freeze like a deer in headlights or make wild assumptions and code away merrily.

Sadly, the deer in headlights scenario is the lesser of the two offered evils.  Even though no progress is being made on the activity, the engineer will always report that their status is green and there are no problems.  Culturally, it is career suicide to raise one's hand and say, "I don't get it."  Eventually, the deadline will approach and the onshore program manager will suddenly find out that no progress has been made in weeks.  Panic will ensue, but thankfully, real requirements will be created, a new deadline will be established, and eventually the requirement will be satisfied.  Since no code will be written prior to the panic, there will be no fundamental re-architecting or regression testing required.  In this case, doing nothing with bad requirements is far more preferable to making speculative guesses and reworking the solution once requirements are known...

Which leads us to the alternative to doing nothing.  Doing something, paradoxically, is far, far worse, in this scenario, than doing nothing.  The engineer that does something has no relevant business background or understanding of the rules governing the requirement they have received.  In order to feel productive, they simply make up business rules.  When asked for a status, the engineer says that everything is going fine.  When the deadline comes, sure enough, the legacy purchase orders are converted but the actual results are no where near the expected.  Panic ensues and all the business rules governing the conversion are then captured.  The engineer then has to redo all their work, but rarely starts with a clean slate and instead tries to shoehorn in new requirements on top of their made up requirements resulting in buggy and overall poor code.

So where are the savings?  The hourly rate now moves from $200 per hour to $20 per hour.  However, since engineers are now available at bargain basement rates, usually five offshore resources are hired to do the work of what would have been done by one onshore resource.  With this in mind, the cost now is $100 an hour.  But wait, there's more - it usually takes the offshore team twice as long to get the job done when you account for the do nothing and then get requirements, or do something and redo it all over again methodology. Now, we're right back to we we started.  There is no actual savings taking place, the work takes twice as long, and it requires a lot of sacrifice on behalf of the business users.  Their lives will be disrupted by conference calls in the middle of the night, multiple efforts to give requirements to people who do not understand your requirements, and hours pouring through documents that appear to be in English but in fact are not.  When factoring the cost of the extended team's increased commitment to the project the costs begin to skyrocket.

It all comes back to the KPIs and the Scorecards...  A General Manager can report to their Vice President during their review that they have reduced costs.  The GM can say (and how they can do this with a straight face is beyond me) that they have reduced the cost of development from $200 per hour to $20 per hour.  VPs being VPs never think to ask the next question and ask what the overall program costs are with onshore versus offshore or if the number of resources and hours spent in development were kept the same.  The VP shakes their head and congratulates the GM.  No money was saved, the extended team is burnt out, the code sucks, but the GM gets a big raise and that's what really matters.

Vendor Consolidation and Basic Economics?

As if offshoring is not bad enough, large companies have found a different way to screw themselves and their onshore partners in the process.  Procurement people who are used to making consolidated purchases of materials decided that consulting resources could be purchased in the same way.  Analysis was done on just how much money was being spent on consultants and an across the board number was thrown out, probably so it could be tracked on a scorecard.  A bold proclamation was made, such as cut vendor spend by 15%, or else.  Procurement then decides that since the company is big, they can consolidate to a handful of agencies and give them all their work.  In exchange, the agencies will reduce their rates.  At this point, anyone who has had any experience in dealing with consulting companies, staffing agencies, marketing companies, etc. should burst out laughing.  The problem is, people are not commodities.  The right consultant is worth their weight in gold while the wrong consultant is nothing more than a corporate parasite.

The process starts by procurement, who has never hired consultants, taking a look at the companies that provide consulting services.  From there, they make a list of arbitrary requirements as to who should be included or excluded from the final list of "approved vendors".  If any vendor company finds that they are not on the list, they begin an all-out campaign to be included.  Once the list has been established, it is nearly impossible to be included on future versions.  For those who have managed to make the list, a game of politicking begins to see how many competing firms they can convince procurement to kick out in the future.

Before examining the theoretical cost savings that centralized procurement may or may not provide, it is worth a brief examination of how consulting companies work. They hire a person and pay them $X/ hr (even if it is an annual salary, this still applies - just divide the annual amount by 2000 hours and voila - $X /hr). They then charge a client $Y /hr for client services. Their profits are then $Y - $X x billable hours. No matter how complicated some firms try to make it. Just as every fast food restaurant claims to have a secret sauce and uses mayonnaise and ketchup, it’s the same with consulting firms.

With that in mind, what should people expect if the variable Y now takes a 15% hit, competition is all but removed, and most consulting firms have a KPI for margin? Obviously, X goes down with Y which results in attracting lower performing people. Just as with offshoring, the lower rate now requires more people and more hours resulting in no cost savings, increased risk, and a host of other problems. However, if you focus on the pure dollar per hour cost, it has been reduced.

I have seen this approach used at multiple Seattle based companies. Consulting firms that had been in existence for less than two years magically got on the Approved Vendor List (mainly by kissing the right rings in procurement) and started flooding the market with twenty-four year olds freshly laid off from the now defunct Washington Mutual. However, work still had to get done by someone and with no real competition, they were able to get fat and happy all in the name of cost cutting.

Back to Cargo Cults

“Everyone wants out of the box thinking, right up until you give it to them.” -Me

These are just a handful of observations I have made in my two decade long career as an IT professional working as a Business Analyst, Program Manager, Project Manager, Software Engineer, and Solutions Architect as well as armchair economist (not a professional - just a top seeded amateur). Every observation should be apparently obvious, but somehow isn’t.

Given that making software is difficult, expensive, and risky; maybe it’s time to stop doing things simply because that’s the way it’s always been done. Methodology has moved from the waterfall approach to agile, but neither guarantees results.

“That was a great use of my time!” said no software engineer ever following the daily standup.

Just as performing incantations into a pair of coconuts fashioned into a radio will not bring cargo, neither will holding a daily standup produce great software. Process can be a good thing and can be used to keep teams on track. However, every time something is measured and someone is held accountable to a metric, behavior will change to achieve the desired metric. This behavior may be a good or a bad thing, but management by metrics is both lazy and dangerous. Unintended consequences have to be sought out and accounted for, clear vision and leadership provided, and cost cutting can make things more expensive.