SCM_RIGHTS

Tue May 21 '19

SCM_RIGHTS is a feature that allow file descriptors to be duplicated across UNIX domain sockets.[1] This allows you to pass access to resources to other processes.

I wanted to try using this to do some sort of process reloading without dropping connections and to get experience with Rust & mio, a library for selecting on file descriptors (using epoll in Linux).

The source code is available at github.com/sqwishy/scmrights-thingy.

Flow

My program simultaneously listens on a UNIX socket and connects over TCP to a “chat” server (nc -l --chat). Data from the TCP stream is simply copied to stdout. This is the connection we want to hand off and keep open when restarting.

The UNIX socket is used to convey some application-specific control messages (since we might want to extend this protocol to convey other sorts of information in the future) and to pass the file descriptors. The exchange is roughly:

  1. Program A accepts a UNIX connection from program B, which is just starting up.

  2. B sends a message requesting the file descriptors. A reads the message.

  3. A calls sendmsg to send an SCM_RIGHTS UNIX domain socket control message. B reads the message with recvmsg.

  4. B sends a message to A telling it to quit. A reads the message and quits.

  5. After A quits, B’s UNIX connection will be closed so B figures that the exchange went well. It starts up normally but uses the file descriptor acquired to initialize the chat client.

Since we don’t want this little dance to be happening more than once at any given moment, only one connection is accepted from the UNIX socket at a time.

Implementation

I made an example and it put in asciinema.

A thumbnail and link to a terminal session demonstrating a connection being handed off between processes.
In this example we start a chat server, two clients, and show the scmrights-thingy program restarting without the connection dropping. Finally, a sequence of numbers are sent quickly and the exchange happens without information being lost–mainly because it’s not buffered or anything by example program.

Why?

My scenario is pretty naive. In more realistic applications, I’d expect more information than just a handle to a system resource would need to be shared; like buffers or state about the protocol. But these could be serialized and sent across the UNIX socket as a normal packet.

So you might use this as an approach to “hot code reloading”. Maybe you want to run a new process to run a program under a new configuration, upgrade it to run a new version, switch between debug & release binaries, or to show off at a dinner party.