Home Original page

[Python-Dev] ctypes, memory mapped files and context manager

Nick Coghlan ncoghlan at gmail.com
Wed Jan 4 21:37:20 EST 2017 
On 5 January 2017 at 10:28, Hans-Peter Jansen <hpj at urpla.net> wrote:
> In order to get this working properly, the ctypes mapping needs a method to
> free the mapping actively. E.g.:
>
> @contextmanager
> def map_struct(m, n):
>     m.resize(n * mmap.PAGESIZE)
>     yield T.from_buffer(m)
>          T.unmap_buffer(m)
>
> Other attempts with weakref and the like do not work due to the nature of the
> ctypes types.

I don't know ctypes well enough myself to comment on the idea of
offering fully deterministic cleanup, but the closest you could get to
that without requiring a change to ctypes is to have the context
manager introduce a layer of indirection:

    class _T_data(ctypes.Structure):
        _fields = [("foo", ctypes.c_uint32)]

    class T:
        def __init__(self, buffer):
            self.data = _T_data.from_buffer(buffer)
        def close(self):
            self.data = None

    @contextmanager
    def map_struct(m, n):
        m.resize(n * mmap.PAGESIZE)
        mapped = T(m)
        try:
            yield mapped
        finally:
            mapped.close()

Client code would then need to consistently access the struct through
the data attribute:

    with map_struct(m, 1) as a:
        a.data.foo = 1
    with map_struct(m, 2) as b:
        b.data.foo = 2

Something like http://wrapt.readthedocs.io/en/latest/wrappers.html#object-proxy
would let you make the indirection to a contained object transparent,
but as far as I can tell, wrapt doesn't currently support "closing" a
proxy by replacing the reference to the internal object with a
reference to None (adding that might be possible, but I don't
personally know wrapt well enough to guess the feasibility of doing
so).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
More information about the Python-Dev mailing list