My letter to the editor of "The Economist"

[sweetheart]

Following is a LTTE that i'm not bothering to send, as it is too
long for them. Unfortunately, the original article is premium
content of the print edition.

Editor
The Economist
25 St. James St.
London SW1A 1HG

Honourable Sir,

Regarding “Managing Complexity” http://www.economist.com/displaystory.cfm?story_id=S%27%29%28%3C%2EP%213%20%23%40%224%0A&bypass=1&pbuviewed=1

Your magazine article suggests the reasons behind this: “Estimated
that 30% of all software projects are cancelled, nearly half come in
over budget, 60% are considered failures by the organizations that
initiated them, and nine out of 10 come in late.”

I think your economic explanations are weak. The problem with
managing complexity, is the empowerment of the enlightened individual
in corporate society. There is an institutional collusion that is
creating the circumstances, an your article simply introduced the
collusion without telling the root truth, that they are incompetent
to have a track record as you mention, the consultancy software
companies, the development operations and the big 4 accountancy
consultancy, +1 including the re-named one that escaped Arthur
Andersen’s fate. (Accenture)

I have, as project manager and lead developer have taken over
responsibility for multiple failed industry software projects and put
them back on the rails to deliver. In each case, too many “testing
tools” were being used. My first action, in each case, was to ditch
all the tools, and get people talking again, introducing Developers
and designers straight to the end stakeholders and putting them in
direct connection, using the architect as the “spark plug”, like an
orchestra conductor. The system working prototype, is under
continuous daily release to the needs of every stakeholder who are in
direct communication with the architect. If it’s worth preserving in
writing, it is left as comments in the code and the help.

Complexity is managed by putting human minds together, not silly
software testing regiments. Iterative software development around
the user’s every need, is the only way to operate. Indeed, chaotic,
but the human mind is capable of directly managing this complexity,
without intermediate tools.

In this regard, systems design is an art, and an empowered architect
is paid to consider the whole architecture. Clearly the failures are
not employing qualified architects. And it seems the whole software
profession is lacking in this regard. Why is it that the luminary
architects do not work for companies? For the same reason in
physical architecture that Architects have their own operations, and
consult. Independence begets the discrimination necessary to execute
the job effectively, based on the number of agencies that must be
balanced in end-to-end architecture in business software systems
projects.

The industry has not soused this out, and has instead fought against
the mythical man month principal that more people accomplish software
projects better, and armies of IBM people are better than 5 serious
experts in software. An army of monkeys, given enough time, can
write Shakespeare. The millions of lines of code, written on
millions of hours of billing and project timeline justification, are
they way this service-consultancy profession justifies its cost base,
and in an attempt to link cost with value, they have made the error.
An army of monkeys with a big logo is NOT better than a few
programmers who are experts. If those few programmers have to carry
the overhead of a full-service consultancy, they lose their skills as
architects, as to maintain coal-face software development contact
means actually leading the software build. The architect IS the lead
developer, and needs to have a liquid inter-project market to keep
skills honed on current and latest tools by ALL vendors, not roped in
to one vendor product line, as the oligopoly presents itself. This
leads corporatists to focus more on the big 4 consultancies for their
independent software advice, and since when is an accountant a risk
taking software designing architect artist? The corporate cultures
do not mix, the failure to appoint quality architects is a root
failure, and worse yet, a failure for the market to select a
microstructure that sustains its life blood…. Need one mention the
effects outsourcing is having on the knowledge pool.

Rather, what has made the software industry liquid, are the hidden
element of independent consultants. These individual companies have
been the software designers with experience designing and coding
product after product, system after system, and are the grease that
makes projects succeed…. These are the wandering samurai, ronin, of
the software profession. Industry consolidation in post-growth
software, has been devastating to the ronin, and has had the
marketing people trying to replace real expertise with software
testing tools. Real good programmers write perfect code. The crap
code problem, comes with keeping the army of monkeys busy day after
day.

All the emperor’s toys are not equal in complexity management, to one
master programmer. Tools like UML, are only as universal as those
using them, and are too, way too complex, unfortunately, leaving the
industry constantly reinventing the same over-complexity again and
again, when people need simple secure database access, comms, and
user interface systems. You ought to see what an army of monkeys can
do in UML in a week (Rational). It takes a week for another army of
monkeys to decrypt it. Goldman sachs is to be commended in this
regard for paying very high salaries and economically respecting its
software architects. A few companies have the right idea, explaining
your successes stories.

Unfortunately, master programmer independence, is under assault in the
coercive markets of the American neocon revolution, that has with the
1986 tax act (IR35) sought to re-classify this liquid market as
outsourced permanent employment, and forcing architects to “pick a
Company” and give up their technological independence, like physical
tradespersons in the building trades, not having to work for 1
construction company, but arranged on site.

Economists should be looking to the construction trade microstructure
as to how projects should be managed and coherently delivered, that
the software industry be admonished for its failures, not
complemented for its impotence.

The large companies that do press interviews, do not expose the
internal labour markets the make projects run, and in such cases of
gross market negligence, as your article points to with the original
quote, there is fertile ground for lucid journalism.

Sincerely yours,

(sweetheart)