Home Original page

[Python-Dev] (Licensing question) backport of shutil.copyfile() functionality

Giampaolo Rodola' g.rodola at gmail.com
Tue Mar 12 17:55:09 EDT 2019 
On Tue, Mar 12, 2019 at 3:01 AM Glenn Linderman <v+python at g.nevcal.com>
wrote:

> On 3/11/2019 4:35 PM, Giampaolo Rodola' wrote:
>
> Hello,
> some time ago I contributed a couple of patches to speedup shutil.copy*()
> functions:
> https://bugs.python.org/issue33671
> https://bugs.python.org/issue33695
> I would like to backport both functionalities so that they can be used on
> Python 2.7 and <3.8 and put it on PYPI. In order to do so I will basically
> have to copy some parts of shutil module (copytree() function + the
> unit-tests I added in BPO-33671 and a couple of other things). Are there
> constraints regarding this in terms of license? Am I supposed to use GPL?
> (I was thinking about using MIT)
>
> Note: in this package called "zerocopy" I will probably want to expose
> other functionalities such as tee(), splice() and CopyFileEx and
> TransmitFile on Windows, so it's probably gonna be half a backport and half
> a brand new project.
>
> Thanks.
>
>
> Thanks for the contributions. I don't know about the licensing.
>
> I wonder if you should make two packages, though... one just exactly a
> backport of the shutil speedups, and the second containing the new
> functionalities.
>

That was my initial thought as well (a "backports.shutil_copy" module
targeting copy* functions only), but (especially after playing with this
today) I think I have something a bit more ambitious in mind. I'm currently
experimenting with different things which could be baked in a third-party
lib and possibly contributed back to Python later on:

1) on OSX we could use f/copyfile() syscall to copy file attrs/metadata;
that may be useful to speedup shutil.copystat() and shutil.copymode()
2) copytree() on OSX could take advantage of f/copyfile() +
COPYFILE_RECURSIVE (which is probably too platform-specific for inclusion)
3) on Linux we could use copy_file_range() as a replacement for
os.sendfile() in shutil.copyfile() (it's supposed to be faster)
4) on Linux ioctl() + FICLONE could be used to implement CoW (copy on
write) instantaneous copies, and could be added as shutil.cow_copyfile()
(haven't look into Windows yet)
5) I was thinking about backporting socket.socket.sendfile() as well, which
uses os.sendfile() on POSIX but not TransmitFile on Windows (asyncio does
though)
6)  another idea (but I'm not sure if it's possible, as I still have to dig
into that) is a new socket recvfile() function boosted up by tee() /
splice() on Linux, which maybe could be contributed back as
socket.socket.recvfile()

-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190312/abd71102/attachment.html>
More information about the Python-Dev mailing list