Issue38251
Created on 2019-09-22 11:29 by Ofer Sadan, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Messages (10) | |||
|---|---|---|---|
| msg352968 - (view) | Author: Ofer Sadan (Ofer Sadan) | Date: 2019-09-22 11:29 | |
running `urllib.request.urlopen` multiple times causes the memory usage to increase with each run, even after calling `close()` on the request or using `del` on the result
To recreate this problem, run code:
import urllib.request
def ip():
r = urllib.request.urlopen('https://api.ipify.org')
b = r.read()
r.close()
print(len(b))
return b
for i in range(1000):
result = ip()
del result
Even though `len(b)` has a maximum value of 15 (for this url at least), the memory increases with each run by 200KB - 1MB
|
|||
| msg352969 - (view) | Author: Ofer Sadan (Ofer Sadan) | Date: 2019-09-22 11:35 | |
Just to note: I'm using Python 3.7.4 on Windows 10 (64bit), the issue exists on both the 32bit/64bit python versions I could test on. |
|||
| msg352975 - (view) | Author: Kumar Akshay (kakshay) * | Date: 2019-09-22 12:19 | |
JFYI, Not reproducible on macOS. |
|||
| msg352976 - (view) | Author: Karthikeyan Singaravelan (xtreak) * ![]() |
Date: 2019-09-22 12:31 | |
Is this related to https://bugs.python.org/issue35941 given that it's related to windows. Similar report https://bugs.python.org/issue37498 |
|||
| msg352978 - (view) | Author: Kumar Akshay (kakshay) * | Date: 2019-09-22 12:42 | |
Yes, https://bugs.python.org/issue37498 is indeed the same issue |
|||
| msg352989 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2019-09-22 22:15 | |
Closing as a duplicate of issue35941 |
|||
| msg352996 - (view) | Author: hongweipeng (hongweipeng) * | Date: 2019-09-23 09:17 | |
My Windows 10 (64bit). There is no such problem in 3.6.1, but 3.7.0 has. And when requesting `http://...`, both of them are normal. By the way, the related issues are all closed, is it supposed to reopen one? |
|||
| msg353000 - (view) | Author: Ofer Sadan (Ofer Sadan) | Date: 2019-09-23 09:58 | |
I can confirm that testing with http instead of https this problem doesn't exist, on 3.7.4. Are the issues closed because this problem is fixed in the upcoming 3.7.5, or 3.8.0? |
|||
| msg353005 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2019-09-23 10:25 | |
Yes, the problem will be fixed by 3.7.5. In the mean time you can work around the issue by creating a single SSLContext object with ssl.create_default_context() and reusing it for all requests. It's also much more efficient to reuse a single context for all requests. |
|||
| msg353006 - (view) | Author: Ofer Sadan (Ofer Sadan) | Date: 2019-09-23 10:34 | |
I appreciate the suggestion and the update, thank you! |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:20 | admin | set | github: 82432 |
| 2019-09-23 10:34:31 | Ofer Sadan | set | messages: + msg353006 |
| 2019-09-23 10:25:50 | christian.heimes | set | nosy:
+ christian.heimes messages: + msg353005 |
| 2019-09-23 09:58:54 | Ofer Sadan | set | messages: + msg353000 |
| 2019-09-23 09:17:07 | hongweipeng | set | messages: + msg352996 |
| 2019-09-22 22:15:46 | steve.dower | set | status: open -> closed superseder: ssl.enum_certificates() regression messages: + msg352989 resolution: duplicate |
| 2019-09-22 14:57:20 | hongweipeng | set | nosy:
+ hongweipeng |
| 2019-09-22 12:42:55 | kakshay | set | messages: + msg352978 |
| 2019-09-22 12:31:49 | xtreak | set | nosy:
+ steve.dower |
| 2019-09-22 12:31:15 | xtreak | set | nosy:
+ xtreak messages: + msg352976 |
| 2019-09-22 12:19:37 | kakshay | set | nosy:
+ kakshay messages: + msg352975 |
| 2019-09-22 11:35:21 | Ofer Sadan | set | messages: + msg352969 |
| 2019-09-22 11:29:48 | Ofer Sadan | create | |
