- May 21, 2019
Duplicating file descriptors between processes across a UNIX socket.
SCM_RIGHTS is a feature that allow file descriptors to be
duplicated across UNIX domain sockets.1 This allows you to pass access to resources
to other processes.
Cloudflare’s blag has a nice post explaining it a bit better and how they used it to support TLS 1.3.
I wanted to try using this to do some sort of process reloading without dropping
connections and to get experience with Rust & mio, a library for selecting
on file descriptors (using
epoll in Linux).
The source code is available at github.com/sqwishy/scmrights-thingy.
My program simultaneously listens on a UNIX socket and connects over TCP to a “chat”
nc -l --chat). Data from the TCP stream is simply copied to stdout. This is
the connection we want to hand off and keep open when restarting.
The UNIX socket is used to convey some application-specific control messages (since we might want to extend this protocol to convey other sorts of information in the future) and to pass the file descriptors. The exchange is roughly:
Program A accepts a UNIX connection from program B, which is just starting up.
B sends a message requesting the file descriptors. A reads the message.
sendmsgto send an
SCM_RIGHTSUNIX domain socket control message. B reads the message with
B sends a message to A telling it to quit. A reads the message and quits.
After A quits, B’s UNIX connection will be closed so B figures that the exchange went well. It starts up normally but uses the file descriptor acquired to initialize the chat client.
Since we don’t want this little dance to be happening more than once at any given moment, only one connection is accepted from the UNIX socket at a time.
I made an example and it put in asciinema.
My scenario is pretty naive. In more realistic applications, I’d expect more information than just a handle to a system resource would need to be shared; like buffers or state about the protocol. But these could be serialized and sent across the UNIX socket as a normal packet.
Bonus meme: systemd can do this for you using
the documentation on sd_notify, which also suggests sharing application state
So you might use this as an approach to “hot code reloading”. Maybe you want to run a new process to run a program under a new configuration, upgrade it to run a new version, switch between debug & release binaries, or to show off at a dinner party.