This is on Windows XP SP2. Using Python 2.7.1 and 2.7.2.

While executing the unit tests of the PySerial package, version 2.5 (http://pypi.python.org/pypi/pyserial) one of the tests fails when using the C version of the io module ("io"), but pass if the pure Python version is used ("_pyio.py").

The specific test case in question is: test_iolib.py. When executed like:

    > python test_iolib.py loop://

the test fails with the following traceback:

----
Traceback (most recent call last):
  File "C:\pyserial-2.5\test\test_iolib.py", line 68, in test_hello_raw

    self.failUnlessEqual(hello, unicode("hello\n"))
AssertionError: u'<memory at 0x00C5D940>' != u'hello\n'
----

And Windows shows a crash report dialog, with the following information:

----
AppName: python.exe	 AppVer: 0.0.0.0	 ModName: python27.dll
ModVer: 2.7.2150.1013	 Offset: 00089f57

Exception Information
Code: 0xc0000005	Flags: 0x00000000
Record: 0x0000000000000000	Address: 0x000000001e089f57
----

As said above, if in the said test case, instead of "import io" one does "import _pyio as io", the crash does not happens, and the test succeeds.

Regarding the crash,

From what I can see, the `tp_clear` method of bufferedrwpair is called during Py_Finalize() that leads to garbage collection. This NULLs `self->writer` for the rwpair object.

Later, in the same garbage collection routine, the `tp_clear` of textio (the wrapper) is called. This attempts to finalize its wrapped object by first calling the `closed` property of the wrapped object. This property read is propagated down to the wrapped bufferedrwpair, `bufferedrwpair_closed_get()`.

At this point, `self->writer` for the bufferedrwpair object is already NULL from the previous clear. Hence, the crash happens because a NULL pointer dereferencing is attempted.

Can you produce a self-contained test case?

As for the "<memory at %p>" string, when _io.BufferedWriter prepares the byte buffer into a PyMemoryView wrapper and passes it into the raw IO object:

    res = PyObject_CallMethodObjArgs(self->raw, _PyIO_str_write, memobj, NULL);

For some reason, memobj.__repr__ is called before being passed to the raw IO's write method.

I can't reproduce this odd behavior using a dumb raw IO that extends _io.RawIOBase. Mimicking what pyserial does, however, reproduces both issues (see attached).

Note: Removing threading call (lock acquire, release) would remove the crash, but still triggers the "<memory at %p>" conversion of PyMemoryView.

If `write(self, data)` is expected to receive `data` as a memoryview object, then it's up to pyserial to use `data.tobytes()` instead of `bytes(data)`, though this does not seem to be documented in the io module doc.

The issue hasn't seen an update for over two years. Were you able to solve the issue?

I can't reproduce the access violation on Windows, using Python 2.7.3.

Using the test_bufio.py script by santa4nt, I can see that the fail still happens.

As Santoso Wijaya (santa4nt) said in a previous comment:

The `write(data)` method is getting a `memoryview` object, thus the test code should be using `data.tobytes()` instead of using `bytes(data)`. If we introduce that logic on the test_bufio.py script it passes.

The thing is that if we, instead of doing `import io` (the C version of the module) we do `import _pyio as io` (the pure Python version) the FAIL does not appears at all in the original script!

That difference ("`write` might receive a `memoryview` object if using the C version of this module") is not documented anywhere.

It should, if that's the expected behavior [*], and the code using the io module is responsible for using .tobytes() instead of bytes().

[*] Why "_pyio.py" behaves differently?

Looks as the crash was fixed in one of previous bugfixes. As for memoryview related issue, this is a duplicate of issue20699.

nosy: + serhiy.storchaka
messages: + msg235775

superseder: Document that binary IO classes work with bytes-likes objects

messages: + msg138463