Improving Wetware

Because technology is never the issue

When the Low Cost Supplier Stops Supplying

Posted by Pete McBreen 22 Feb 2011 at 18:59

A parallel to the outsourcing debacles is what is currently happening in the supply of Rare Earths (mainly the transition metals if you remember your periodic table). The rare earth metals are commonly used in high tech components, often for the super strong magnets, but also for lots of other electronic components.

China used to be such a low cost provider that it managed to capture most of the market, all of the other suppliers basically closing their mines. Now China, which produces 97% of the world’s supply of rare earths, slashed its exports to a trickle to feed its growing domestic needs. The link goes to a feel good story about a mine that is trying to ramp up production quickly, but there is still a measure of reality in the storyline

“Bottom line, we fell asleep as a country and as an industry,” Smith said. “We got very used to these really low prices coming out of Asia and never really thought about it from a supply chain standpoint.”

In more and more areas we seem to be bumping up against problems when demand starts to exceed easily available supply. It is not that we are running out, it is just that demand is larger than expected so there is not the production capacity, so the price in the market becomes unstable, with large spikes and then resulting drops as some customers leave the market for alternatives.

Mainstream media is catching up with outsourcing

Posted by Pete McBreen 16 Feb 2011 at 09:41

The LA Times is the latest to report on Boeing’s costly lesson on outsourcing. They have an interesting lead to the story

The biggest mistake people make when talking about the outsourcing of U.S. jobs by U.S. companies is to treat it as a moral issue.

Sure, it’s immoral to abandon your loyal American workers in search of cheap labor overseas. But the real problem with outsourcing, if you don’t think it through, is that it can wreck your business and cost you a bundle.

I don’t agree that many people thought of outsourcing as a “moral issue.” The conversation was more about the balance between short term economics and the long term implications of outsourcing. Short term the numbers can look better, but long term organizations lose the ability to do the work. The end result is that in the end the supplier becomes dominant, captures most of the available profit, and the outsourcer ends up being responsible for the downside risk.

See also Outsourcing too much part II

Time for Intolerance?

Posted by Pete McBreen 16 Feb 2011 at 07:50

Jim Bird recently pointed out that there’s a cost to building good software, so calls for Zero Bug Tolerance are sometimes misguided.

There’s a cost to put in place the necessary controls and practices, the checks and balances, and to build the team’s focus and commitment and discipline, and keep this up over time. To build the right culture, the right skills, and the right level of oversight. And there’s a cost to saying no to the customer: to cutting back on features, or asking for more time upfront, or delaying a release because of technical risks.

He goes on to say …

That’s why I am concerned by right-sounding technical demands for zero bug tolerance. This isn’t a technical decision that can be made by developers or testers or project managers … or consultants. It’s bigger than all of them. It’s not just a technical decision – it’s also a business decision.

Alistair Cockburn has pointed this out in his Crystal set of methodologies, that the cost of an error in different kinds of applications varies, and that an appropriate methodology takes this into account.

  • Life and Safety Critical - things like pacemakers and flight control software
  • Mission Critical - systems that the business depends on, if the system fails you make the evening news (for example a stock exchange)
  • Departmental applications - yes the business depends on it, but a certain amount of downtime can be accepted
  • Comfort - yes, twitter and facebook, I’m looking at you

Processes and verification procedures that are appropriate for a mission critical application would be detrimental to a departmental time booking application. A good test suite would be appropriate for both, but only the mission critical application needs a formal review of the test code to make sure it is covering the appropriate cases.

This is one of the reasons why I support Jim’s idea of being intolerant of the Zero Bug Tolerance mantra.

Wider Intolerance

Some ideas just fail the laugh test, but they are afforded too much deference by people who should know better. The Rugged Software Manifesto fails the laugh test as well.

After reading about the attempts to brand Alberta’s Tar Sands as “Ethical Oil”, maybe it is time to start laughing at politicians that espouse crazy ideas as well.

The UK Government Chief Scientific Adviser John Beddington recently called for scientists to be “grossly intolerant” if science is misused

“I really would urge you to be grossly intolerant,” he said. “We should not tolerate what is potentially something that can seriously undermine our ability to address important problems.”

Beddington also had harsh words for journalists who treat the opinions of non-scientist commentators as being equivalent to the opinions of what he called “properly trained, properly assessed” scientists. “The media see the discussions about really important scientific events as if it’s a bloody football match. It is ridiculous.”

Limits Of Expertise

Posted by Pete McBreen 15 Feb 2011 at 18:09

A common problem in software development and science is that people are often not aware of the limits of their knowledge.

The electric car provides a great example of this. Richard Muller in Physics for future Presidents makes several statements about the feasibility of electric cars that have turned out to be incorrect.

This quote is from Confusing Future Presidents part 2

High-performance batteries are very expensive and need to be replaced after typically 700 charges. Here is a simple way to calculate the numbers. The computer battery for my laptop (on which I am writing this) stores 60 watt-hours of electric energy. It can be recharged about 700 times. That means it will deliver a total of 42,000 watt-hours, or 42 kilowatt-hours, before it has to be replaced for $130.

Muller implies in this post that a car would need to replace the battery after only 700 cycles, but although this might have been the case at the time, car battery suppliers are continually extending the life of a battery.

Better Place which is the pioneer in rapid change for electric batteries in cars has also pointed out that the end of life is not a fixed point. In their blog post batteries can have a second life they state

In the world of batteries, what does “end of life” really mean? According to the industry, end of life is defined as that point in time when a battery has lost 20% of its original energy storage capacity or 25% of its peak power capacity. This implies that an EV battery, with an initial range of 100 miles per charge, will reach its end of life when, years later, it only delivers 80 miles per charge.

That time is likely to be reached only after the battery has carried an electric car about 200,000 miles or 2,000 cycles. But that’s not really the end for an EV battery – it’s just the beginning of a second life that not many people know about.

All this goes to show that Muller’s expertise in a branch of physics does not necessarily extend to expertise in economics, engineering and battery chemistry. Muller had tried to dismiss the Tesla roadster before it was released by claiming that it would cost too much and the batteries would be too heavy Confusing Future Presidents part 1. But the Tesla was launched successfully as a niche sports car. Now the GM Volt and Nissan Leaf are also proving that it is possible to build electric cars.

The Better Place business model of renting the battery through quick change stations that can swap a battery faster than you can refuel at a filling station is likely to be a game changer as well.

So the next time someone states categorically that something is not possible, first check to see if they have the relevant expertise in the appropriate areas before listening too long or hard.

Outsourcing Too Much Part II

Posted by Pete McBreen 08 Feb 2011 at 13:02

To follow on the outsourcing saga, the Seattle Times reported that 10 years later Boeing is finally listening to John Hart-Smith. Note that that PDF link is covered in BOEING PROPRIETARY labels.

One interesting item from my viewpoint is that it took 10 years for the warnings to be heeded, so there are echoes of the climate change issue there.

TL;DR Fourteen pages later …

Although John Hart-Smith’s paper deserves to be read in full, I know few people will, so here are some key excerpts.

Almost all potential suppliers indicated a preference for being subcontractors rather than risk-sharing “partners’. Could they have known more about maximizing profits, minimizing risk, etc. than the prime manufacturer who sought their help?

One must ask the question as to where the skills for writing such specifications will come from if there is no continued in-house production from which to learn.

Outsourcing the generation and distribution of a company’s proprietary intellectual data would seem fraught with opportunities for potential customers to acquire the knowledge they need elsewhere.

The inherent difficulty with such an approach to business is the need to retain and develop the technical skills needed to develop future products. … One must be able to contribute in some way to products one sells to avoid becoming merely a retailer of other people’s products.

The paper does discuss some situations whereby outsourcing is useful. On a personal level this is useful because I do a lot of contract software development, so whenever I am working, my client is in effect outsourcing the work.

It is accepted that prime manufacturers cannot afford to have expensive facilities that are under utilized, even if it would save on downstream rework. It really is better to out-source such work, IF AND ONLY IF the selected supplier has excess capacity on such equipment.

In the software world, this equates to contracting for work that is done infrequently, when the skills necessary are not needed on a constant basis. It is also useful to contract in skills that are to be passed on to the in-house employees, so that the contractor comes in, sets up particular processes/machines and then trains the other staff in how to maintain and use them on a regular basis.

Using Climate Change to study Software Development

Posted by Pete McBreen 04 Feb 2011 at 20:51

Although lots of software is used in studying climate change, for example a python re-implementation of a climate model, there are many issues in climate change that relate to software development.

Parallels between Climate Change and Software Development

  • Things change slowly and then suddenly things are different

    An extra 2ppm of CO2 might not seem much, but over a working lifetime, an extra 60-70ppm has changed lots of things. In software development Moore’s Law was slowly making computers faster and cheaper. Suddenly things changed when we realized that hardware was no longer an expensive item in projects. When I started software development, the developer costs on a project were swamped by the cost of the hardware. Now using cloud machines, the hardware costs of a project can be less than the cost of gas to commute to the office.

  • What we are sure we know is not necessarily true

    In climate change, the distinction between weather and worldwide climate is not well understood. Also it is hard to figure out how a 2C change could matter that much when locally the weather can range over 60C from the depths of winter to the height of summer. In software development, historically it really mattered that the code was as efficient as possible because the machines were slow. So everyone knows that scripting languages are not useful for large scale development. Enterprise systems are built in traditional languages, but most web companies are using some form of a scripting language, or one of the newer functional languages.

  • Fear, Uncertainty and Doubt

    Software development probably lead on this one. IBM was justifiably famous for FUD to ensure that customers followed IBM’s lead and did what they were told. IBM had the market and customers were supposed to do what IBM told them was good for them. With Climate Change, large organizations that will have to change to preserve the climate that is suitable for our current civilization, are spreading as much doubt as possible to delay the realization that business as usual is no longer possible. In Software Development the threat of Open Source is currently the target of lots of FUD and large corporations that are seeing changes on the horizon are doing all they can to preserve their business as usual.

  • Nobody wants to listen to the people who know what is going on

    Software Developers are used to being overruled by people who do not really know what is going on. Sure sometimes it is genuinely a business decision, but often the business people are busy making technical decisions that they are not competent to make. In Climate Change, the scientists doing the work are being challenged by the political decision makers who do not have a clue about science. Realistically the political decision makers should accept the science and then start talking about the political choices that we need to make as a result of that science.

  • The results really matter

    There are two things that our civilization depends on, working software and a livable, stable climate. The news is occasionally enlivened by the story of a major software project that costs hundreds of millions of dollars that fails to deliver the expected benefits. The smaller day to day losses from smaller failures are hidden. Similarly the big storms that are increasing in frequency and severity due to climate change are making headlines, but the smaller impacts are never visible in the news.

Making sense of the parallels

Still working on this one, but I have a sense that the political power structures have a big impact. The techno geeks that do software or science are not part of the larger political power structures that govern our corporate dominated societies. As such the techno geeks are marginalized and can be safely ignored … at least for now … obligatory link to Tim Bray - Doing it Wrong.

Ruby on Rails Gotcha — cannot always omit parenthesis on a method call

Posted by Pete McBreen 03 Feb 2011 at 10:45

When trying to test out some legacy routes I got a really obtuse error from rake while running the following test case

  test "legacy routing OK" do
      assert_recognizes {:controller => 'index', :action => 'contact'},  '/contact.html'
  end

All I was trying to do was make sure that any URLs saved in search engines from the old static site still worked in the new Rails based site, so no issue, just use assert_recognizes and that will make sure the routes are protected by test cases. It is not possible to use assert_routing since I did not want the new code to generate the .html style URLs. Anyway, this is what rake complained about

/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.8/lib/active_support/dependencies.rb:147:in 
   `load_without_new_constant_marking': ./test/functional/index_controller_test.rb:25: 
            syntax error, unexpected tASSOC, expecting '}' (SyntaxError)
          assert_recognizes {:controller => 'index', :action => 'contact'},  '/contact.html'

         ./test/functional/index_controller_test.rb:25: syntax error, unexpected ',', expecting '}'
            assert_recognizes {:controller => 'index', :action => 'contact'},  '/contact.html'

It turns out that although the code documentation uses the normal rails convention of omitting the () around the method arguments, you cannot do that if the first parameter to a method is a hash. So the fix is to put in the parenthesis and the test runs as expected.

  test "legacy routing OK" do
      assert_recognizes( {:controller => 'index', :action => 'contact'},  '/contact.html' )
  end

Fun with Ruby on Rails

Posted by Pete McBreen 01 Feb 2011 at 16:31

Rails can be for fun as well. Rather than going the usual Wordpress/Joomla or PHP route for my local Cochrane Red Rock Running and Tri Club website, I decided to have some fun with Rails and jQuery. Nothing really fancy but it does what it needs to without anyone having to learn about the joys of a CMS control panel.

Overall the site took less than 10 hours of spare time, time that would easily get eaten up by the usual requests for updates to a static site. There are still a few static parts to the site, but those are ones that do not change frequently, so not a big deal to maintain those parts.

Edited Dec-2018, site is now retired, had a long run but now replaced by a wordpress site maintained by someone else. Over the years spent about 40 hours maintaining the site, so not too bad, and zero breaches/security issues in that time