Thanks to all for the many thoughtful and insightful comments on Whatever happened to programming?
Stupidly, I didn’t realise that comments would be held for moderation, so it’s only this morning that I went through and OK’d them all (even the ones that say I’m just whining :-)). There are too many to respond to individually, but since some consistent themes emerged, I wanted to follow up on some of those.
Popular comments, distilled
1. Poor you, your job must suck!
Sorry, I realise I didn’t really explain this properly. In truth, I am lucky enough to have one of the relatively few jobs out there that really is largely programming. I work for a small open-source software house, and what we mostly do is make tools. So, actually, no, my job doesn’t suck. (Er, except maybe right now, when the dice happen to have fallen in such a way that I am in the middle of my third consecutive documentation project.) I was aiming for a lament at the state of programming in general.
And of course since my employer actually creates and distributes libraries and web-service components, we have an obligation to make them simple and pleasant to use — to avoid leaving people with Library Fatigue. You might say that our goal is to provide people with libraries that Just Work, and that provide narrow enough interfaces that you don’t need a lot of glue.
2. But you wouldn’t want to have to write printf()!
No indeed. I certainly didn’t mean to suggest that there is no place for libraries! It’s great to have a set of existing functionality that we can call on whenever we need it. The problem comes when all you do is glue libraries together. To put it another way, libraries make excellent servants, but terrible masters. Somewhere along the line, we let the libraries take over the asylum. Not good.
Here’s a rule of thumb (which, like all such rules, is often broken and should not be taken too seriously): beware of anything that calls itself a Framework. Anything that, instead of providing stuff that you can call, takes over the wheel and tells you what code to provide for it to call. Not always, but often, that marks the line where this stops being fun.
In my experience of programming with frameworks — and this goes all the way back to writing Windows 2 GUI applications in raw C with the aid of the appalling Microsoft resource compiler — they do keep their promise of making things very quick and easy … so long as you do things in exactly the way the framework author intended. The moment you go off-piste, everything goes Pete Tong, and what should be simple is suddenly not merely hard but often impossible. The framework goes out of its way to prevent you from doing what you intended. Strange side-effects hamper your efforts. The hook that you need to get the functionality you want isn’t there. Your attempt to work around it causes something else, apparently unrelated, to start misbehaving in subtle and unpredictable ways. Congratulations! You are now suffering from Framework Fever! Doctor Taylor prescribes a period of complete rest before building up your strength by working through the exercises in K&R.
(Again: this is not always true of all frameworks. I hope from the bottom of heart that when I start using Rails, it will prove to be among the exceptions. But my experience so far does not make me optimistic.)
You know the real problem with frameworks? They demo too well. Someone shows you their favourite framework and demonstrates how you can build 50% of your application in half an hour! Great! That other 50% can’t be hard, can it? But it turns out that what looked like 50% is actually 5%, and filling in the other 95% gets exponentially more difficult as you approach the 100% mark. Frameworks are great for building toys, and that fools us — again and again — into assuming they’re good for building products.
3. But libraries help us do things more efficiently!
You have problem X, consisting of sub-problems X1, X2, X3. There are readily available solutions for X1 and X3. If you don’t feel like being a bricklayer, code X. I choose to code X2, plug in solutions X1 and X3, and spend the rest of the day investigating and possibly solving problem Y.
I admit this sounds great. Hut here’s the fallacy: we all assume (I know I do) that “plug in solutions X1 and X3″ is going to be trivial. But it never is — it’s a tedious exercise in impedance-matching, requiring lots of time spent grubbing around in poorly-written manuals that tell you everything the code already told you (because it was generated with JavaDoc or Rdoc or whatever), and none of the high-level stuff that you actually need to be told. So maybe my initial complaint was not really so much “whatever happened to programming?” as “why do so many of our libraries suck so much?”
The good thing is that (again, as others have pointed out), we are not just pawns in this game. We are, if not kings and queens, then at least knights and bishops. We are not, or don’t need to be, passive consumers of others’ libraries, but also makers of libraries for the benefit of others as well as ourselves. We can and should raise the bar. We need to make every effort to design our libraries in such a way that the programmer is in control and the library is his or her servant. I think that’s so important that I am now going to go and have it tattooed on the insides of my eyelids, so that it’s always before me when I’m sleeping.
And folks, let’s write useful documentation. I don’t need a 900-page document that is a recapitulation of the method signatures. I need a three-page statement of what the library is for, what shape it is, how the bits fit together, why it uses italics so much and where to start.
4. You should be using Ruby instead!
Seriously, I absolutely agree that different languages, with their different expressive power and especially their different culture, yield very different experiences. Especially, I have learned that anything that has “Enterprise” in its name is so incredibly boring that the people who use it had to shove the name of the Star Trek ship into its title just to keep themselves awake. (I am convinced that this is the case.) Better languages and better environments make for a better and more programmingy programming experience. I’m in favour of that.
The conclusion of the matter
What chills me to the core is this comment from Jesse:
I frequently create custom business and ecommerce applications, and I hardly ever get to do any copy/pasting. There’s always a custom report to create, a custom csv to parse, or a custom xml system to integrate..
That, right there — that’s what I’m afraid of.
Eowyn: I fear neither death nor pain
Aragorn: What do you fear, my lady?
Eoywyn: A cage.
Lord deliver us from a career of creating custom reports, parsing custom CSV and integrating custom XML systems.
I’ve just found the discussion of the original Whatever happened … at Hacker News. Lots of interesting perspectives. And especially this, from jdietrich, which captures exactly what I’ve been trying to say:
Back in ye olden days, most programming tasks I performed felt quite natural and painless, just a quiet little chat between me and the compiler. Sometimes longwinded, sometimes repetitive, but I just sat and though and typed and software happened. The work I do these days feels more like being a dogsbody at the tower of babel. I just don’t seem to feel fluent in anything much any more.
We talk about ‘flow’ quite a lot in software and I just have to wonder what’s happening to us all in that respect. Just like a conversation becomes stilted if the speakers keep having to refer to their phrasebooks and dictionaries, I wonder how much longer it will be possible to retain any sort of flowful state when writing software. Might the idea of mastery disappear forever under a constant torrent of new tools and technologies?
That’s it exactly.