Looking to chemistry this time, here are six proposed Rules of Reproducibility.
- Were studies blinded?
- Were all results shown?
- Were experiments repeated?
- Were positive and negative controls shown?
- Were reagents validated?
- Were the statistical tests appropriate?
Many science papers are unfortunately weak when it comes to these rules, and in many fields #2 is a real problem - only the positive results are shown, the rest are hidden away and never seen.
Found another interesting parallel between software development and running. The field of running and exercise is full of lots of claims about special ideas that will drastically improve performance of athletes. The Science of Sport site has a blog post on How to spot bad science and fads- Determining whether an idea is worthwhile
At a recent track meet I was having a conversation with a friend in college, who made the astute observation that if the coaches inserted random scientific terms to explain things, even if they were totally wrong, the runners seemed to buy into it more enthusiastically. That’s a very common reaction, we all do it. We associate science and complexity with being smart or correct. As I’ve said before…people trying to fool you go from simple to complex…good coaches translate complex things into simple understandable ideas.
In another post the same site talks about the value of research, theory and practice
… I often rely on what one of my Professor’s, Jason Winchester, called the three stool leg test. You have research, theory, and practice. If you have all three, it’s almost certainly a good idea to implement it. If you have 2 of 3, it’s fairly likely that it works and it depends on the strength of the 2. If you’ve only got 1 of 3 going for it, it probably doesn’t work. The beauty of using the 3 stool leg test is it blends science and practice, and compliments it with theory which in itself is a blend of science and practice.
Jim Bird has taken a look at how much is technical debt costing you. Nice to see that he ignores the dollar estimates per line of code that some authors use and just uses a simple $$$ through to $ notation.
$$$ Making a fundamental mistake in architecture or the platform technology – you don’t find out until too late, until you have real customers using the system, that a key piece of technology like the database or messaging fabric doesn’t scale or isn’t reliable, or …
$ Missing or poor error handling and exception handling. It will constantly bite you …
Recently Jim Bird had to point out that Source Code is an Asset, Not a Liability. Unfortunately it means that there are people in the software development community that are not aware of the literature - specifically Howard Baetjer Jr.’s Software as Capital.
Some interesting lessons for Software Development can be obtained form outside our field. I was reminded of this while reading a running blog that looked at what lessons could be gained from outside of the field of running coaching…
Rules of Everything
- When something is new or gains popularity, it is overemphasized until it eventually falls into it’s rightful place. How long that process takes varies greatly.
- Research is only as good as the measurement being used is.
- We overemphasize the importance of what we can measure and what we already know, ignoring that which we can not measure and know little about.
- We think in absolutes and either/ors instead of the spectrum that is really present.
Point 1. helps explain a lot of the original hype/hope surrounding the agile approaches to software development.
Lessons from outside the running world
We go through a cycle of forgetting and remembering what’s been done before us. You see this in the reintroduction or rememphasis in certain training methods in the coaching world. That’s why it is incredibly important to know your history. And if you can, know your history from a primary source where you attempt to look at it through their eyes during that time period. For example, going back and reading Lydiard’s original work gives a greater appreciation of what he was trying to do, then reading someones summary now, 50 years later. We lose a little bit of the original message.
Sometimes there is useful information available from looking back at what worked in the past. Although many on the software field seem to try to forget the past, the pioneers in the field learned a lot, some of which is still applicable to our present circumstances.
Don’t normally link to Dave winer, but his The bosses do everything better is priceless…
When he looked at the code he must have been shocked to find something complex and intricate. Why isn’t the source code as simple as the software? Hah. When you figure that out let me know.
All too often in software development I hear the comment that there must be a “simpler/easier way.”
Unfortunately, although sometimes simple solutions are workable, in most cases the simplest solution is not workable. Or rather the simple solution would be workable in some circumstances, but not for the current project becasue of some fairly obvious deficiencies in the simple solution.
From On Bullshit by Harry G Frankfurt:
In the old days, craftsmen did not cut corners, They worked carefully, and they took care with every aspect of their work. Every part of the product was considered, and each was designed and made to be exactly as it should be. These craftsmen did not relax their thoughtful self-discipline even with respect to features of their work that would ordinarily not be visible. Although no one would notice if those features were not quite right, the craftsmen would be bothered by their consciences. So nothing was swept under the rug. [pp 20-21]
Some interesting parallels to Software Craftsmanship in Shop Class as Soulcraft. Focus is on working in the so called craft trades, specifically as Electrician and Motorcycle Mechanic.
Parallels are uncanny in the way that both books address Scientific Management, but Soulcraft found a very interesting quote from one of Ford’s biographers
So great was labor’s distaste for the new machine system that toward the close of 1913 every time the company wanted to add 100 men to its factory personnel, it was necessary to hire 963. (pg 42)
Small wonder then that Ford was forced to double the wages of the factory staff in order to retain workers. Of course this has since been spun as Ford wanting the workers to be able to afford the cars they were making, but it sure seems like it was a defensive move based on turnover.
Why Software Development Will Never be Engineering
Basic idea in the article is that things like bridge building are now fairly static. The types of bridges we know how to build are well codified and replicable. Not mentioned in the article is that novel bridges still have novel problems, but after a few mistakes the construction engineers seem to resolve most of the issues.
Software development is different because it keeps on changing. The article argues that 10 years ago the future seemed to involve UML and CASE tools, but that the current state of the art of software development (Agile) does not use either of them.