Primary Purpose

This PR proposes to extend the semaphore_tracker range of action to other system resources.
Currently in python3.8, if a process that created a shared_memory object gets violently terminated (with kill for example), the memory segment will not be properly released until the next machine reboot. This will be problematic for long-running clusters used by many different users.

The primary reason of this PR is thus to extend the semaphore_tracker to track also shared_memory segments. The semaphore_tracker module gets consequently renamed resource_tracker.

Extension to a potentially public API

Additionally, the design of the resource_tracker is more generic, and possibly suggests a new public API to enable the tracking other named-resources. For now, the private _CLEANUP_FUNCS dict references all the types that can possibly be tracked.

To start tracking a new type, the resource_tracker only needs to know which cleanup routine to apply to each leaked instance of this type after the python session ends. The public API would thus consist of a function like resource_tracker.register_resource_type(resource_type, cleanup_func), where:

• resource_type is a string
• cleanup_func must be a callable taking a single string as an argument.

Under the hood, register_resource_type would populate the CLEANUP_FUNCS with the new (type, cleanup_func) record.

Operating System Availability

To ease the review process of this PR. the resource_tracker is now POSIX only. I however have a functional windows implementation that I will provide upon demand :)

pinging @pitrou @ogrisel @tomMoral

@pablogsal this may interest you as well :)

https://bugs.python.org/issue36867