The radius of comprehension — in PragPub

I’m delighted to announce that issue 10 of PragPub is out today, and contains an article by me, Tangled Up in Tools.  For those who like nice typesetting, you can download the whole issue as a PDF, or as epub or mobi (whatever they may be).

PragPub is the monthly programming magazine published by the Pragmatic Bookshelf — that’s the publisher that brought you the definitive books on Ruby and on Rails, among much else.  I like their stuff, and I am pleased to be in their magazine.  Not only that, they let me illustrate the article with photos of — oh, but you’ll see for yourself when you read it.

So what is the article about?  I was commissioned to write it in the wake of Whatever happened to programming? and its followup, and asked to write on related topics, so the tenor of the article will seem familiar to those of you who’ve been reading since back then.  (It’s astonishing to me to think that Whatever happened was published only five weeks ago — it seems much longer.)  This is a completely new article, though, sharing only the Knuth quote that kicked off the original.  Rather than just whining about the state of things, I tried to finish up with two specific suggestions for making things better.  (Don’t worry, though — there is plenty of whining as well.)

Those two suggestions are:

1. Better documentation of libraries, and by “better” I largely mean shorter.  Documentation that tells you not just how to use the library, but why you would want to use it, what it can do for you, and when it’s appropriate.  Documentation that is free of the word “Enterprise”.  Most importantly, overview documents that fit on a single page of A4.  (I think of Mark Ardis’s observation that “a specification that will not fit on one page of 8.5×11 inch paper cannot be understood”.)

2. Trying to reduce what I’ve called the “radius of comprehension”.  It’s a property of a codebase that I defined as follows: if you are looking at a given fragment of code, how far away from that bit of the code do you need to have in your mind at that time in order to understand the fragment at hand?  I think this it potentially a valuable concept (although it’s going to need a lot more thought than I’ve given it so far).  In short: if I want to use a class from your library because it solves a problem for me, do I need to learn eight other classes from your library before I can do that?  If so, then the RoC is too high.

Of course the article itself has a lot more to say about this concept.  I’m keen to get your feedback so don’t forget to come back here to comment after you’ve read it!

13 responses to “The radius of comprehension — in PragPub

  1. It was a pleasure working with you, Mike. Of course, to follow the full “Whatever Happened…” argument, including the PragPub article, readers will now have to duct-tape it together from disparate threads. Seems, I dunno, ironic? -ms

  2. Oddly, I very roughly touched on the concept of Radius of Comprehension while talking to our summer intern today. The context was the line editor – in order to use one effectively, one had to keep most of the code in one’s head.

    And putting my sysadmin hat on for a moment, there’s also the issue of the expanding tree of library dependencies. There are few things worse than trying to install a piece of software that requires an ever-increasing tree of dependencies like some horrible army of Matryoshka dolls. Three layers down you hit a problem and you find yourself trying to patch a Makefile or whatever and you cannot remember how you got there from the relatively straightforward goal of installing a piece of software.

    And don’t get me started on frameworks. I was burned badly by Rails, with vast chunks of functionality suddenly and arbitrarily deprecated with no migration path to replacement code. Unless you were psychic and knew to freeze your version, code written only a year before would not run and there was no straightforward way to replicate your precious snowflake of tightly-coupled library versions.

    I left vocational sysadminnery a year and a half ago and now I’m happily back doing hard engineering, among other things analyzing the thermal characteristics of radioactive sludge. We developed our own code to do the analysis; work started on it in 1996 and it’s written in a sad combination of F77 and F95 with poorly-indented fixed-format source. Yet for as sad of a state this code is in, it’s actively maintained, documented, validated, and QA’d. My guess is that it’ll compile and run as-is in another 15 years. Maybe it’s not a fair comparison, but stepping back, I’d rather develop against a tool that I can rely on not to pull the rug out from under me when I least expect it. My experience is limited, but I’m leery of frameworks, especially those coded by people with a high metabolism.

    Is it just me, or has the notion of a stable release (i.e. not SVN HEAD) disappeared? I started programming on a VIC-20 so it could be that I’m just old and cranky…

  3. The concepts of coupling and cohesion have been around for literally decades. RoC sounds like coupling cast in the semantic domain.

  4. Jeshua Smith

    I enjoyed reading the article, as you’ve hit on a lot of the frustrations I’ve had with programming. Part of the joy of playing with Legos as a kid was being creative with the limited selection of blocks available. Now there are so many different types of blocks that instead of building something you just spend a bunch of time finding prefab blocks and gluing them together.

    I think you’ve got a few typos in the article though:
    wrote out own => wrote our own
    It gave as a whole => It gave us a whole

  5. Rats, dumb typos! Thanks for finding them, but I don’t think there’s much that can be done at this stage. (I found another one myself — at one stage I wrote “The are” when I meant “There are”, or something similar. What a doofus.)

  6. My bad on the typo front. I hate it when they get through my net.

  7. You really hit the nail directly on the head with that article, well done! It articulated exactly my own frustrations.

  8. The documentation thing is something that I really feel the Python community tends to excel at (and hence, I have been trained to expect). I didn’t quite understand it until I watched a talk of defunkt’s[0] from RubyConf where he mentioned it. Docstrings are an integral part of the language (and easily accessible from the python shell), doctest provides cucumber-like BDD testing, and documentation, especially the official Python docs, reads somewhere between an essay and a tutorial[1].

    So, when I was enrolled in an Android development class this last quarter, I got very quickly frustrated with being given almost nothing aside from API docs (and poorly-done ones, at that).

    [0]: http://rubyconf2009.confreaks.com/20-nov-2009-16-20-rippin-off-python-chris-wanstrath.html
    [1]: http://docs.python.org/library/re.html

  9. (BTW, since that’s a rather long video I linked to, you can get to the part I was talking about if you skip ahead to around 11:30.)

  10. These issues ring true for me. With some of the code I maintain I have found myself wondering what the author was thinking. I can’t go and ask him because I was him!

    I think part of the problem is that we don’t have good descriptive tools for code. Yes, the old cry of “Use the source, Luke!”, but if readable things are so easy to understand, then why
    does literary criticism exist?  Why do we have terms for characterisation, plot development, narrative structure, denouement, etc., for something as inherently readable as a novel? I think they help different people see things they’d miss.

    In OO code we have classes for (possibly abstract) objects, methods for actions, and modules, for some types of grouping. In procedural code we just have actions, with no “characters”. We have nothing at the scale of plot structure. We have nothing to connect up usage during initializations, for example.

    There is UML, but I have found this more abstract than electronic circuit diagrams, which leads to the vicious circle that I don’t use it because I don’t find it helpful, because I don’t use it.

    I think these problems, and those of the “expression problem”, would be helped if there were less emphasis on trees and more on tables in computer science. We talk of programming *languages*, but linguists use tables to describe verbs, with their tenses and imperative forms. Some languages have cases which describe how an object is being used. I don’t know how useful that would be to programming.

    There is also literary criticism of poetry, which must have a different set of techniques from those used for novels.

    I think better notation in source code to aid such analysis would help. And we need not fear vague, waffly argumentation for code, because we are dealing with speed, memory usage, and other constraints which affect the design.

  11. Pingback: Frameworks and leaky abstractions « The Reinvigorated Programmer

  12. The crux of your argument seems to fall in this sentence: “It’s a property of a codebase that I defined as follows: if you are looking at a given fragment of code, how far away from that bit of the code do you need to have in your mind at that time in order to understand the fragment at hand?”

    Unfortunately, I think that sentence qualifies as self-referential, since I can’t understand it without looking at the following sentences in the document :)

  13. Pingback: Investing Quote « Rubber Tyres –> Smooth Rides

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s