In The Leprechauns of Software Engineering Laurent Bossavit has an interesting take on the folklore of software development.
Software development is a hard problem.
Books like The Mythical Man Month, Set Phasers on Stun and The Inmates are running the Asylum have all pointed out in their own way that creating software is hard. Fred Brooks focused on the problem of large complex projects, and the problems that face project managers, the other two remind us that even small projects can fail because we still are not able to create software that is both easy to use and powerful enough to do that tasks that we want to do with software.
Until we are able to understand why software development is such a hard problem, we are not going to make much beyond incremental improvement. There will always be a few projects that through the operation of blind luck across millions of projects that results in seemingly reproducible improvement, but the normal regression to the mean will correct that eventually.
I didn’t see this when it was first written, but it matches with my recent experiences.
… most programmers simply don’t know where the quality bar is. They don’t know what disciplines they should adopt. They don’t know the difference between good and bad code. And, most importantly, they have not learned that writing good clean code in a disciplined manner is the fastest and best way get the job done well. – Robert Martin
Many software developers do not seem to understand the basics of our craft. Recently I’ve seen
- SQL queries that were massively more complex than they needed to be - that when simplified, without any database changes ran more than 10 times faster
- Client server applications that issue nearly 1000 SQL queries while refreshing what is supposed to be an interactive screen - the end result being that the poor user has to wait 5 to 10 seconds for the screen to refresh after conceptually simple actions
- Supposedly secure web applications that sent Active Directory usernames and passwords in cleartext across HTTP connections
- Code that created connections to external resources but forgot to free them - made for a very effective rate limiting mechanism since the external resource freed unused handles about an hour after they were last used
There have been lots more examples, but most of them fall into the category of being unbelievable if you were not a direct witness to the utter ignorance of the basics of software development that brought them to my attention.
Maybe it is time that we started to focus on the basics of the craft of coding before we get too far into creating overly complex systems that nobody can understand or fix.
But I’m not sure we will learn the right lessons.
In the early 1980’s I worked on applications that had to process 300 transactions per minute. At the time that was considered a heavy load for a Dec Vax to deal with. By the late 1980’s the same applications were dealing with 1,000 transactions per minute because the hardware had got a lot faster. In the mid-1990’s I worked on a small scale credit card processing application, running on a later incarnation of the Dec Vax that was able to handle nearly 20 transactions per second. In the late 1990’s I worked on a stock exchange system that had to deal with what we thought at the time was a stupidly large number of messages per second … little did I know. Fast forward to the early 2000’s and I worked on consumer facing web applications that had to withstand 10,000 requests per second. By 2010 I had the fun of being on a real web scale project, experiencing the joys of being linked to by Digg and CNN and hoping that the ensuing millions of requests per hour would not being the system down.
All this is to say that dealing with internet scale applications is a solved problem, but it seems that whoever was involved in the Healthcare.gov fiasco did not realize that. Dave Winer pointed out that the Government develops software differently, but there is no excuse for building a site that cannot handle the traffic.
I disagree with Bob Goodwin - there is no software engineering crisis - OK I have to say that because I wrote the Software Craftsmanship book. But that is not the real reason I have to disagree - I have to disagree because whoever built the site went about it the wrong way. Dave Winer parodied the approach that big consulting companies take
They’d fully specify the software, the user interface, its internal workings, file formats, even write the user documentation, before a single line of code was written. Then they’d hand the parts off to development teams who would independently of each other create the components. Another team would do the integration.
The sad fact is that the big corporations that are awarded these big government contracts do not have a clue how to build web scale applications that work. They over promise and massively underdeliver. All too often large companies are awarded contracts to build large systems and fail to deliver anything of value except to their own shareholders.
Just had to help someone with Apple’s Pages, not sure what version, but they were not able to save a separate version of the file to keep two slightly different versions of a document. Turns out that Pages no longer has a Save As… file menu option, instead it has a Duplicate and Rename menu option.
Which designer in their right mind thinks that it is a good idea to change a thirty year old idiom that their entire userbase is familiar with?
Best bit of this insanity is that the shortcut key for Duplicate is the same as what used to invoke the Save As… dialog box, but of course the behavior is different.
Hot off the press, there is now a Simplified Chinese Edition of Software Craftsmanship - amazon.cn link. ISBN is 978-7-115-28068-8 for anyone who is interested.
The link is also proof that international alphabets are now supported in URLs - http://www.amazon.cn/软件工艺-Pete-McBreen/dp/B00AAQXL28
But I don’t like the way people at code.org are pitching it. And I don’t like who is doing the pitching, and who isn’t. Out of the 83 people they quote, I doubt if many of them have written code recently, and most of them have never done it, and have no idea what they’re talking about.
These people don’t themselves know how to do what they want you to do. So what they say makes no sense. It won’t make you rich, but it will make them rich. And if you do it, they won’t listen to you. And even worse, if you do what they want you to do, you’ll be tossed out on the street without any way to earn a living when you turn 35 or 40. Even though you’re still a perfectly good programmer.
The Atlantic has an article called When the nerds go marching in that tells a story about the comparative approaches of the Obama and Romney teams and how they built and tested their systems in the run up to the 2012 US presidential election.
Obama team had an interesting approach to the planning - Making it a game
Hatch was playing the role of dungeon master, calling out devilishly complex scenarios that were designed to test each and every piece of their system as they entered the exponential traffic-growth phase of the election. Mark Trammell, an engineer who Reed hired after he left Twitter, saw a couple game days. He said they reminded him of his time in the Navy. “You ran firefighting drills over and over and over, to make sure that you not just know what you’re doing,” he said, “but you’re calm because you know you can handle your shit.”