gh-44098: Release the GIL while calling mmap() in the mmap module on Windows by ZackerySpytz · Pull Request #14114 · python/cpython
Right. Releasing the global lock in mmap_resize_method would require redesigning the mmap module to synchronize access with a local lock, similar to what "_io/bufferedio.c" has to do to protect its internal state. resize isn't called frequently enough to justify the potential for introducing new bugs here.
I did a little timing test in C to gauge the relative time taken by the calls needed to create a file, resize it to 128 MiB, get the size, create a section for it, map a view, unmap the view, and close the section and file handles. The following is the result for 1000 trials, keeping only values within a standard deviation for each call, and normalizing the averages against that of CreateFileMapping:
| Function | Average Time |
|---|---|
| CreateFile | 14.90 |
| SetEndOfFile | 4.34 |
| GetFileSize | 0.19 |
| CreateFileMapping | 1.00 |
| MapViewOfFile | 0.87 |
| UnmapViewOfFile | 0.95 |
| CloseHandle, Section | 0.42 |
| CloseHandle, File | 3.72 |
In new_mmap_object, we're not releasing the GIL for GetFileSize, but it's a relatively inexpensive call. But the cost of MapViewOfFile is on the order of CreateFileMapping. The two should be viewed as a unit. For example, given m_obj->data is initially NULL:
Py_BEGIN_ALLOW_THREADS
m_obj->map_handle = CreateFileMapping(m_obj->file_handle,
NULL,
flProtect,
size_hi,
size_lo,
m_obj->tagname);
if (m_obj->map_handle == NULL) {
dwErr = GetLastError();
} else {
m_obj->data = (char *) MapViewOfFile(m_obj->map_handle,
dwDesiredAccess,
off_hi,
off_lo,
m_obj->size);
if (m_obj->data == NULL) {
dwErr = GetLastError();
CloseHandle(m_obj->map_handle);
m_obj->map_handle = NULL;
}
}
Py_END_ALLOW_THREADS
if (m_obj->data == NULL) {
Py_DECREF(m_obj);
PyErr_SetFromWindowsErr(dwErr);
return NULL;
}
return (PyObject *)m_obj;