A hard-to-find bug, and equivalent APIs in low- and high-level languages

I just wrote an entry about a hard-to-find bug on the blog of my employer, Index Data.  It turned out to hinge on the different API expectations of programmers in low- and high-level languages — specifically C and Ruby — and I think it’ll be of interest to most Reinvigorated Programmer readers.  Enjoy!

4 responses to “A hard-to-find bug, and equivalent APIs in low- and high-level languages

  1. Mike–

    I feel a little in the dark. There are both layers and history going on, would you consider laying enough of that out to understand how it worked before, and not later?

    As it is, I’m imagining someone took a C routine that was like

    Resultset *search_in_zoom_db( Conn *conn, char *query ) {

    return resultset;
    }

    and “translated” it to the Ruby function you quote:

    def search_in_zoom_db(query)
    conn = ZOOM::Connection.open(Host, Port)
    return conn.search(query)
    end

    This looks like a plain bug to me. You say, “the Ruby ZOOM binding follows the semantics of the underlying ZOOM-C library,” but the Ruby function doesn’t have what I assume would be a necessary parameter to the equivalent C function. So, not only does the interface not imply that the caller is in charge of connections, but if the caller did have a connection to hold onto, the Ruby function doesn’t give the caller a way to perform a query on that connection!

    Sure, C and Ruby programmers might naturally expect different conventions to handle this issue. But when translating a library from one to the other, somebody was responsible for either keeping the old convention, or translating to the new, and it seems like there would have been clear local clues here…. unless I’m missing something you didn’t explain?

    (It’s a little scary that you have a class for connections, but you also have the Ruby function calling …open(Host, Port) where Host and Port seem to be globals or constants. This seems to imply that there’s only one thing to connect to, and a library that requires you to continuously pretend that someday we might have more than one connection when we all know there’s only one, sounds like an invitation to slip.)

  2. Just a test:

    line
        indented line
    line
    
  3. Besides using angle brackets in that earlier post, I didn’t make explicit:

    When a function (in any language) has a parameter for an open file or connection, it gives a pretty strong clue that the caller is responsible for thinking about the lifetime and scope of the file or connection. When not, it implies the caller is not responsible.

    It would be a bad idea for the function (in any language) to open the connection and pass back the open connection as part of the result, except in such a way that garbage-collecting or closing the result object would close the connection. I hope your C code doesn’t do that.

    Also, the situation you describe where the result object caches results as they come in, but the caller is responsible for knowing when to close the connection, seems a confused division of responsibilities, again, in any language.

  4. Hi, Steve, thanks for your thoughts on this. Evidently I didn’t lay out enough of the background — always a fine line to walk when describing a problem with a specific system.

    No, there was no C routine search_in_zoom_db() — the Ruby function of that name was not translated from a C function, but written from scratch as part of a program that exists only in that language. So the bug wasn’t that application code was ported blindly from C to Ruby, but that the Ruby-level API to the underlying C code was put together in a way that, while it matches the C code that it makes available, isn’t a good match for how such things are generally done in high-level languages. It’s like a too-literal translation of a French phrase into English that doesn’t read well.

    (The C function doesn’t have an additional ‘conn’ parameter, no. The point is that in Ruby/Python/whatever, you naturally assume that the Result Set object hangs on to a reference to the Connection that it was created for; but in fact it doesn’t.)

    BTW., ignore the use of globals for the host and port: I just did that to simplify the code. In the real system, it reads these parameters, and others, from a configuration file. That’s not the issue here. It is indeed possible to have multiple ZOOM connections open at once, as in the Perl application IRSpy, which every night does a big run that keeps fifty connections running at any one time.

    Finally: “It would be a bad idea for the function (in any language) to open the connection and pass back the open connection as part of the result, except in such a way that garbage-collecting or closing the result object would close the connection. I hope your C code doesn’t do that.” No, the C code doesn’t. But the Ruby code should (and also doesn’t). And that is the bug.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s