Swap to APIServer for all communications by dcantah · Pull Request #628 · apple/container
Today, we have a sort of odd model all things considered. We talk to the APIServer to do the initial container creation, which mostly all of the work is just registering the runtime helper with launchd. After this registration, almost all communication from a client is talking directly to that runtime helper. This has a couple rather annoying issues in that we now need a sort of event/notification channel that the helper will establish with the APIServer for errors/start/exited cases. If instead of talking to the runtime helper directly, we instead took a detour through the APIServer, the APIServer is clued into exactly what order of operations is occurring. This makes "did starting the container fail? Okay we should clean up" scenarios much simpler, and it also simplifies the clients quite a bit as they don't need this split brained client model, everyone just talks to the APIServer. This change is in pursuit of that. I have reworked our clients, the ContainerService and some of our XPC types to accomplish it. The biggest "contract" changes are in the SandboxService. The first is today we have on the flag when we register any runtime helper that makes any xpc messages wake up the registered process. This isn't great in scenarios where the process may have crashed, or it exited normally and we're just trying to invoke an RPC on it. Today the helper would spawn again and try and answer our request. It'd be much nicer if we have a connection object that will become invalid if the process that vended it to us is gone. To accomplish this, now the runtime helpers will listen on an anonymous xpc connection and vend endpoints from this via only one handler exposed by the SandboxService (createEndpoint). From that point onwards all communication will be through the endpoint the service vended a client. The second change is the runtime helper will not exit on its own when the container exits, and the event mechanism has been removed. Now the APIServer simply calls wait() to listen for container exit in the background, and once we get an exit we will explicitly tell the helper to shutdown. The rationale is if shutdown is driven by the APIServer now, we can be certain we received everything we need from the helpers before they power down.