Issue34097
Created on 2018-07-11 14:35 by petr.viktorin, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 8270 | merged | Dormouse759, 2018-07-13 11:43 | |
| PR 8725 | merged | Dormouse759, 2018-08-10 11:41 | |
| Messages (16) | |||
|---|---|---|---|
| msg321460 - (view) | Author: Petr Viktorin (petr.viktorin) * ![]() |
Date: 2018-07-11 14:35 | |
The ZIP format cannot handle times before 1980. Issue6090 provided a nice error message for trying to add such files. I'm seeing a system for reproducible builds that sets mtime to 1970 (zero UNIX timestamp), resulting in files that Python can't package into (zip-based) wheels. At least here on Fedora, the `zip` command-line utility silently bumps old timestamps to 1980-01-01. Of course, silently corrupting data would not be good default behavior for Python. But in many cases timestamps don't matter. It would be nice to give ZipFile and ZipFile.write() a `strict_timestamps=True` keyword argument that could be turned off. |
|||
| msg321464 - (view) | Author: Marcel Plch (Dormouse759) * | Date: 2018-07-11 14:40 | |
I'm going to have a closer look at this and try to add the option. |
|||
| msg321595 - (view) | Author: Marcel Plch (Dormouse759) * | Date: 2018-07-13 11:45 | |
I have created a PR for this: https://github.com/python/cpython/pull/8270 |
|||
| msg321597 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * ![]() |
Date: 2018-07-13 12:04 | |
There are ZIP extensions which allow to save timestamps before 1980 and with better than 2-seconds resolution. It would be better to use this feature. But first we need to resolve issue17681. |
|||
| msg321598 - (view) | Author: Petr Viktorin (petr.viktorin) * ![]() |
Date: 2018-07-13 12:22 | |
I'm not sure the extensions will solve this problem fully. If an implementation that doesn't support these extensions, how does it handle them? For example, if Python 3.8 implements the extensions, I use py3.8 to create a zipfile containing old timestamps, and then want to uncompress the file using Python 2.7, how do I avoid losing the timestamp information? |
|||
| msg321600 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * ![]() |
Date: 2018-07-13 12:37 | |
You can't, because Python 2.7 doesn't support it. But you will be able to pack files in a ZIP archive and extract them without a loss on 3.8 and with a loss of a timestamp on 2.7. And you will be able to use a third-party utilities for extracting files without a loss. |
|||
| msg321607 - (view) | Author: Petr Viktorin (petr.viktorin) * ![]() |
Date: 2018-07-13 13:05 | |
Which third-party utilities support these? As I said, AFAIK `zip` on my system does not. I assume the loss of data is the reason we have an error now -- if that wasn't a concern, zipfile could just silently bump the timestamp to 1980. When the extensions are implemented, `strict_timestamps=False` should *additionally* use the extension, but default should still be to raise the error (to prevent metadata loss when decompressing with older/simpler tools). |
|||
| msg322949 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-08-02 13:04 | |
New changeset a2fe1e52eb94c41d9ebce1ab284180d7b1faa2a4 by Victor Stinner (Marcel Plch) in branch 'master': bpo-34097: Add support for zipping files older than 1980-01-01 (GH-8270) https://github.com/python/cpython/commit/a2fe1e52eb94c41d9ebce1ab284180d7b1faa2a4 |
|||
| msg322950 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-08-02 13:05 | |
Thank you Petr Viktorin for the bug report and thanks to Marcel Plch for the implementation of the new strict_timestamps keyword-only parameter! |
|||
| msg322954 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * ![]() |
Date: 2018-08-02 13:23 | |
If add a new option, I prefer to add it to the ZipFile constructor, similarly to allowZip64. Initially allowZip64 was False by default, because not all third-party tools supported ZIP64 extension. Later the default was changed to True, but you still can force raising an error if the archive is not compatible with old tools. I think the same policy should be applied here. Add the ability of writing date before 1980, and add an option for raising an error for date before 1980. |
|||
| msg322957 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-08-02 14:23 | |
Serhiy: > If add a new option, I prefer to add it to the ZipFile constructor, similarly to allowZip64. Aha, that would make sense. I'm not sure that it's useful to control the parameter per added file, it's enough to control the parameter per ZIP archive. Marcel: would you mind to try to move the strict_timestamps parameter from ZipFile.write() to ZipFile constructor? |
|||
| msg322960 - (view) | Author: Marcel Plch (Dormouse759) * | Date: 2018-08-02 14:31 | |
It seems reasonable, I'll have a look at it. |
|||
| msg323023 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * ![]() |
Date: 2018-08-03 05:20 | |
Tests are failed on some buildbots. See issue34325. |
|||
| msg323095 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-08-03 23:09 | |
> Tests are failed on some buildbots. See issue34325. Marcel Plch already fixed it: commit 7b41dbad78c6b03ca2f98800a92a1977d3946643. The two buildbots are back to green. |
|||
| msg324423 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-08-31 14:43 | |
New changeset 77b112cd56a50232abcdbf28f9aba88dc5d33ad3 by Victor Stinner (Marcel Plch) in branch 'master': bpo-34097: Polish API design (GH-8725) https://github.com/python/cpython/commit/77b112cd56a50232abcdbf28f9aba88dc5d33ad3 |
|||
| msg324424 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-08-31 14:44 | |
Thanks Petr Viktorin for reporting this issue and thanks Marcel Plch for the fix! |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:03 | admin | set | github: 78278 |
| 2018-08-31 14:44:46 | vstinner | set | status: open -> closed resolution: fixed messages: + msg324424 stage: patch review -> resolved |
| 2018-08-31 14:43:38 | vstinner | set | messages: + msg324423 |
| 2018-08-10 11:41:39 | Dormouse759 | set | stage: resolved -> patch review pull_requests: + pull_request8209 |
| 2018-08-03 23:09:34 | vstinner | set | messages: + msg323095 |
| 2018-08-03 05:20:43 | serhiy.storchaka | set | messages: + msg323023 |
| 2018-08-02 14:31:42 | Dormouse759 | set | messages: + msg322960 |
| 2018-08-02 14:23:24 | vstinner | set | status: closed -> open resolution: fixed -> (no value) messages: + msg322957 |
| 2018-08-02 13:23:00 | serhiy.storchaka | set | messages: + msg322954 |
| 2018-08-02 13:05:41 | vstinner | set | status: open -> closed resolution: fixed messages: + msg322950 stage: patch review -> resolved |
| 2018-08-02 13:04:56 | vstinner | set | nosy:
+ vstinner messages: + msg322949 |
| 2018-07-13 13:05:01 | petr.viktorin | set | messages: + msg321607 |
| 2018-07-13 12:37:48 | serhiy.storchaka | set | messages: + msg321600 |
| 2018-07-13 12:22:14 | petr.viktorin | set | messages: + msg321598 |
| 2018-07-13 12:04:25 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg321597 |
| 2018-07-13 11:45:19 | Dormouse759 | set | messages: + msg321595 |
| 2018-07-13 11:43:45 | Dormouse759 | set | keywords:
+ patch stage: patch review pull_requests: + pull_request7805 |
| 2018-07-11 14:40:11 | Dormouse759 | set | nosy:
+ Dormouse759 messages: + msg321464 |
| 2018-07-11 14:35:58 | petr.viktorin | create | |
