Home Original page

[Python-ideas] Adding shm_open to mmap?

shibturn shibturn at gmail.com
Thu Feb 16 22:20:55 CET 2012 
On 16/02/2012 1:56am, Greg Ewing wrote:
> I don't know about Windows, but in Unix it's possible to send a
> file descriptor from one process to another over a unix-domain
> socket connection. So a refcounted anonymous mmap handover could
> be achieved this way:
>
> 1. Process A creates a temp file, mmaps it and unlinks it.
> 2. Process A sends the file descriptor to process B over a
> unix-domain socket.
> 3. Process B mmaps it.
>
> Even if process A closes its version of the fd right after
> sending it, the OS should keep it alive while it's in transit,
> I think.

If the receiving process is expecting an fd then that certainly works. 
But making it work transparently with pickle is difficult. 
(multiprocessing.reduction tried making it transparent using a 
background thread to accept requests for fds from unpickling processes. 
  But that functionality has been disabled.)

On Windows one rather cleaner possibility is for the process pickling 
the handle to use DuplicateHandle() to copy the handle to the main 
process.  Then the receiving process can copy the handle from the main 
process, removing it from the main process at the same time by using 
"dwOptions=DUPLICATE_CLOSE_SOURCE".  Since the main process will not 
exit before its descendants, that will solve the keep-alive problem.  (I 
have managed to produce a working example of this scheme for transfering 
a file handle.)

sbt
More information about the Python-ideas mailing list