Hello all,
This 16-year old commit (*) allows an object's state to be updated
using its slots instead of its __dict__ at unpickling time. To use this
functionality, the state keyword-argument of Pickler.save_reduce (which maps to
the third item of the tuple returned by __reduce__) should be a length-2 tuple.
As far as I can tell, this is not mentioned in the documentation (**). I suggest having the docs updated. What do you think?

(*) https://github.com/python/cpython/commit/ac5b5d2e8b849c499d323b0263ace22e56b4f0d9
(**) https://docs.python.org/3.8/library/pickle.html#object.__reduce__

Does it still work?  With both the C and Python pickler?
Can you post an example?

It turns out that both pickle and _pickle implement this feature, but the behavior is inconsistent.

- As a reminder, instances of slotted classes do not have a dict attribute (1)
- On the other side, when pickling slotted class instances, __getstate__ can return a tuple of 2 dicts. The first dict represents the __dict__ attribute. Because of (1), this first dict should simply be a sentinel value. In pickle, the condition is that it evaluates to False, but in _pickle, it should be explicitly None.

(- Finally, The second dict in state contains the slotted attribute. )

Here are the lines in the two files causing the inconsistent behavior:
https://github.com/python/cpython/blob/master/Modules/_pickle.c#L6236
https://github.com/python/cpython/blob/master/Lib/pickle.py#L1549

I included an example that illustrates it.

You can have both a dict and slots by subclassing:

>>> class A: 
...:     __slots__ = ('x',) 
...:                                                                                                                                                                   
>>> class B(A): pass                                                                                                                                                   
>>>                                                                                                                                                                    
>>> b = B()                                                                                                                                                            
>>> b.x = 5                                                                                                                                                            
>>> b.y = 6                                                                                                                                                            
>>> b.__dict__                                                                                                                                                         
{'y': 6}
>>> A.x                                                                                                                                                                
<member 'x' of 'A' objects>
>>> A.x.__get__(b)                                                                                                                                                     
5

Interestingly, you can also put an instance dict in slots:

>>> class A:
	__slots__ = ['x', '__dict__']
	
>>> a = A()
>>> a.x = 5
>>> a.y = 6
>>> a.__dict__
{'y': 6}
>>> a.x
5

Thanks Antoine and Raymond for the feedback.

Indeed, a subclass of a slotted class can have a dict: I enriched the script, pickling_depickling instances of such subclasses, with the length-2 tuple __getstate__ method, and made sure their attributes were properly retrieved.

Apart from the different checks on state carried out in the c load_build and the python load_build, AFAICT, it seems like this feature works :)

A slotted class will have a dict also when it inherits it from a non-slotted class. This is why the base class of slotted class should have slots if you do not want an instance dict.

__getstate__ and __setstate__ for slotted classes are described in PEP 307. Unfortunately this was not copied to the module documentation.

I added a PR with a small patch to document this behavior and reconcile _pickle.c and pickle.py

Some explanations on why I am pushing this forward:
 
Pickling instances of classes/subclasses with slots is done natively for pickle protocol >= 2. Mentioning this behavior in the docs should *not* make the user worry about implementing custom __getstate__ methods just to preserve slots.

Here is the reason why I think this functionality (allowing state and slotstate) is worth documenting: pickle gives us a lot of flexibility for reducing thanks to the dispatch_table. But unpickling is rather rigid: if no __setstate__ exists, we have to deal with the default state updating procedure present in load_build. Might as well document all of it ;)

See also issue26579 which propose to add a function or method for standard implementation of __getstate__ and use it consistently in custom __getstate__ implementations. I am not sure about API yet.

messages: + msg335041