Rebooting Linux with every new kernel

Back in the early 1990s, when I was working on Sun’s Unix boxes, it was routine for them to be up for multiple years at a time. A reboot was a big event.

Now here we are in 2017, and every single update to the Linux kernel — of which there are several every week — requests a reboot.

As I write, my Ubuntu 14.04 LTS box is running Linux 4.4.0-72. The update process just downloaded Linux 4.4.0-75, and now it’s asking to restart. Linux is more than 25 years old; yet it still can’t install a new sub-patch-level release of the kernel without rebooting? Clearly there can be no API changes between 4.4.0-72 and 4.4.0-75, so why can’t it just slip the new kernel into place without a reboot?

To be clear: I’m not saying that doing this is trivial; I am saying that after 25 years of development, I am amazed no-one’s taken it on. How much time is wasted rebooting millions of Linux boxes every other day, and re-establishing the working environment (window layout, files loaded into the editor, etc.)? This seems like a problem worth solving.

Perhaps part of the problem here is cultural: Windows has taught people that you expect to have to reboot your computer every day. Most Linux users have come to the light side from Windows, and have brought that expectation with them. It seems normal to them.

But it’s not. Or, at least, it shouldn’t be.

(Yes, part of the reason the Suns stayed up for years was that kernel updates were amazingly rare. Seamless kernel upgrade was a problem that didn’t need to be solved then. But it’s a problem that needs to be solved now.)

Instant update

Two responses on Twitter to the stream of tweets that became this post:

  • Tony McMahon points out that there is a thing called Livepatch.
  • Alex Stapleton says that Ubuntu offers kpatch rebootless upgrades.

Very interesting — I will follow these up. (And I will be be asking myself, as I do so, why there are two solutions instead of one, and why this behaviour is not the default.)

Advertisements

13 responses to “Rebooting Linux with every new kernel

  1. Then again, Windows no longer needs rebooting every day, but folklore, myth and legend, especially when brought by those fleeing the enemy, is hard to shift. Sometimes it stays around, poisoning the atmosphere for generations.

  2. Maarten Daalder

    And sometimes reboots are necessary simply because the Windows 10 broke hibernation and sleep modes >_<

  3. Also, perhaps, a shift in usage from when Unix was mainly a mainframe/server OS, and therefore reboots were a big deal as it involved inconvenience to many users, potential disruption of services, etc; to now when I suspect that a fairly large proportion of computers running Linux are single-user desktops or laptops, so rebooting isn’t that much of a big deal (indeed laptops presumably get rebooted all the time as a matter of course; I wonder what proportion of computers running Linux these days are laptops)?

    To be sure it does seem strange that it’s a problem that hasn’t been solved; but if, for example, and just making up numbers, 60% of Linux installations these days are on Laptops, you can see why development effort might go into laptop-related problems (eg, more efficient power-saving mode transitions, etc) than into mainframe-server related problems like reducing reboots.

  4. For what it’s worth, H, I virtually never reboot my laptop. (It’s a MacBook, and the close-the-lid-to-sleep facility Just Works, slipping into hibernation mode on its own when necessary.) As with my Ubuntu desktop box, I basically only reboot when a system upgrade requires it.

  5. There are two solutions instead of one because it’s Linux, of course. I mean, KDE and Gnome? Plus all the others, of course.

  6. Some part of a kernel is always going to require a reboot. Theoretically you can make this part very very small, but the benefits are probably not worth the cost. Still, it should be able to be made orders of magnitude smaller without adverse consequences (e.g. by moving more features in userspace, à la Minix).

  7. I usually let the kernel updates “mature” for awhile before rebooting. With my CentOS 6.8 server that’s a once or twice a year operation. Takes awhile for DKMS to update things like my nVidia and other DKMS enabled drivers.

  8. If I’m not mistaken, the essential obstacle to rebootless kernel updates is migration of kernel data structures from the old image to the new one. Avoiding that migration implies nuking all the userland, which is so close to reboot as to make no essential difference. Implementing such migration sounds like a huge deal, but maybe I’m imagining monsters in the little known territory?

  9. It might be easier to work on the “re-establishing the working environment (window layout, files loaded into the editor, etc.)” problem. I have a Macbook Pro that I reboot maybe half a dozen times a year. It used to be a pain since I’d have to re-establish the working environment. Over the last two to three years MacOS has made this much simpler since more and more applications now automatically remember what windows were open before the quit and reopen them on restart. I’ve stopped dreading security updates that require restarting since I am usually back in business with in a minute or two depending on how much mucking around the update does.

    I know Apple encourages restarts since it doesn’t trust unloading kernel extensions and then reloading their new versions. I gather from online sources that these actually work pretty well, though I had some trouble debugging an FTDI driver and wound up rebooting every few tries. Unloading and reloading extensions seems to be one of those things that works, except when it doesn’t. Apple put its effort into solving the re-establishing problem rather than dealing with tens of millions of users getting bitten by the “except when it doesn’t” problem. (Apple would probably like to let you sit down with any compatible Mac and rebuild your working environment using iCloud, but that’s another issue.)

    Have LINUX distributors decided to take a similarly cautious approach now that the system has been taken up by people who have no idea what a kernel extension is and wouldn’t want to have one? If I were threatened with having to run a user support group, I’d seriously consider it.

  10. rtfp: yes, migrating changed data-structures would be complex and error-prone, and I fully agree that rebooting in such circumstances is much safer. But remember, what we are talking about here is not even a change at the patch level: it’s from 4.4.0-72 to 4.4.0-75. The change is at a version-number facet to trivial I don’t even know its name! It seems pretty clear there can be no changes to data-structures in that change.

    Kaleberg, I don’t want to give up on trying to solve the right problem and concentrate on a workaround instead. Re-establishing state is not just a matter of window positions, but of terminal tabs and their directories and histories, emacs buffers (maybe hundreds of them) and their running processes, browser windows and tabs and their locations, and potentially the states of many JS SPAs running within them. Much better to just not mess with all those units.

    Anyway, the good news is that it looks like there is a nice and simple solution to this problem now: Canonical’s livepatch service at https://www.ubuntu.com/server/livepatch. I have installed this, and await the next kernel update to see how it goes.

  11. Oh there really can be changes to data structures in that change, sometimes quite big ones. The kernel really does have no ABI. (However… enterprise-distro kernels generally impose a guarantee that data structures are not allowed to change incompatibly, so for them, live patching is more practical, or rather, will work in a higher percentage of cases. You can at least reliably tell when it won’t work, thanks to the modversions machinery.)

  12. Then what is the point of a four-faceted version number?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s