I tried python3.8 for my project. But it is crashing in the beginning . When debugged , found out that "Line 145 in cloudpickle.py" is returning the value in bytes which is not expected.

Code Snippet: 
 
 _cell_set_template_code = _make_cell_set_template_code()

Error: 

return types.CodeType(
TypeError: an integer is required (got type bytes)

Is this on Python 3.8.0 ? There was a similar report in the past with cloudpickle at https://bugs.python.org/issue36886#msg342182 due to PEP 570 with which CodeType.replace was introduced. cloudpickle also added support for this with https://github.com/cloudpipe/cloudpickle/pull/269 and has Python 3.8 in their CI matrix. Can you please add the version of cloudpickle and python version with which this error is caused? I guess it's more of an issue to be fixed with cloudpickle.

Hi @xtreak ,
Thank you for the reply . Please find below detailed description. 

I was using python 3.8 and installed pyspark 2.4.4 . I encountered error related to cloudpickle file. Please find the attached snapshot. 

I tried to convert the value of co.co_code in cloudpickle.py file (line 132) from bytes to int to check if it is really a issue with cloudpickle file. 

Please let me know if you need more information .

I guess this was fixed upstream with https://github.com/apache/spark/commit/811d563fbf60203377e8462e4fad271c1140b4fa . Please try the latest version as per the commit details. I am closing this as third party since it's not a CPython issue. In future reports please attach the text of the traceback since it's more accessible. Thanks.

resolution: third party
stage: resolved

messages: + msg356582