bpo-19382: Adding test cases for module tabnanny. by ultimatecoder · Pull Request #851

bpo-19382: Adding test cases for module tabnanny. by ultimatecoder · Pull Request #851 · python/cpython

Description: Unit tests for standard module tabnanny. Tests are added for almost all the functionality except tabnanny.Whitespace class.
Reason leaving Whitespace:? I found the module contains mathematical calculations. I will try to understand it first and then write tests for it. I think it will be good to do that in another branch.
Testing strategy: Whitebox
BPOs
- https://bugs.python.org/issue19382
- https://bugs.python.org/issue19102

https://bugs.python.org/issue19382

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just write “captures exceptions” (or change arised → raised).

What is “tabsize”? Did you mean “tabnanny.check”?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feeded → fed
dipicting → depicting

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feeded → fed
Expacted → Expected

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verifying → verify the
existance → existence

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few → a few [otherwise, the implication is that most of the files have errors: few are error-free]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

giving

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quiet mode?
“less” looks wrong, unless you meant “Should display less”

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don’t you get CRLF on Windows?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Python 3, b"%r" is equivalent to calling the “ascii” function. I expect the test will fail if the file path has non-ASCII characters. It may be better to only test that an error is being reported, but tolerate variations in the message.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the path is non-ASCII? Again, just check the general form of the output and don’t be too specific.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but the OS can provide a parent directory for temporary files that has non-ASCII path components. It seems relatively common on Windows that the temporary directory is within a user’s profile, and the profile directory can be a localized non-ASCII name. Top Google result: https://bugzilla.mozilla.org/show_bug.cgi?id=260034.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks for affording time for reviewing this. I will agree with you on this point. I have updated code. Will you please review again?

Check out the existing patches and reviews you linked. You may be repeating some of the same problems, e.g. CRLF on Windows.

@vadmium Many thanks for your comments. I am not native English speaker. I have fixed the mistakes. I have added comments to some comments. Please review my fixes. Thanks!

@DimitrisJim Thanks for your comment. I have removed tabnanny from untested modules list. Thanks!

Sorry but I don’t have an easy way to see your fixes relative to the old version I reviewed. I’m not really set up to properly review and push stuff with the new Git Hub setup, so I will leave this to someone else, or for another time in the future.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove one empty line

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this. This is Github glitch.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum, I prefer to write mock.patch, please remove this import

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please import tempfile and write tempfile.NamedTemporaryFile.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this docstring is useless. Usually, we don't comment much tests, they are not intented to end users.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you only care of creating a temporary filename, use tempfile.mktemp() and then open() manually the file to create it. You can use open(tmpname, "x") to prevent race conditions.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of a dict, you can simply use a tuple (args, expected)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think we should kill the readability by converting it to tuple. How about collections.namedtuple?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or How about this way?

        tests = [
         #  (Function arguments , Expected output)
            (['first', 'second'], 'first second\n'),
            (['first']          , 'first\n'),
            ([1, 2, 3]          , '1 2 3\n'),
            ([]                 , '\n')
        ]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep tests simple:

for args, expected in [
    ([1], '1'),
  ]:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, keep them simple. And don't use namedtuples except for keeping backwards-compatibility with an interface that used to take an indexed object and now takes an object with attributes.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that this docstring is super useful.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that it's worth it to declare one test case to test a single class/function. Try to put most or all tests into a single test case called "TabnannyTestCase".

TestCheck deserves its own test case since it has a setUp() method.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this list into the method directly.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unit tests must have zero side effect: you must save the old verbose value and restore it. Example:

self.addCleanup(setattr, tabnanny, 'verbose', tabnanny.verbose)
tabnanny.verbose = 0

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha! a geek way 👍

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to make this method private, remove "_" prefix.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I'm not sure that a long comment is needed for a simple unit test.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is explaining why the method is here. Suggest a way for improving the docstring.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least ditch the whole Arguments section; we don't use that style in the stdlib.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who removes the temporary file? I suggest to put create_tmp_python_file() in the test case (or in a base class if it's needed by multiple test cases) and use self.addCleanup(support.unlink, tmpfile) to make sure that the temporary file is removed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding it to a method will be a poor idea. If I will add it to base class then we will have a base class with one method. Will you accept that?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if the method needs to access information that will be on the instance then a class with a single method is fine. But if you aren't reusing it then just inline the function.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the overall change, but I have a long list of requests :-)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO the docstring is useless.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this assert out of the with block.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't move this out.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not? Is tokenize.generate_tokens() lazy (which isn't documented)?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brettcannon The tokenize.generate_tokens() is a lazy function (generator). Can you help to identify a way which allows putting assertRaises outside the with? Thanks!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use universal_newlines=True to normalize newlines. I also prefer to use text rather than bytes for stdout/stderr.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert_python_ok() already tests that returncode==0.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this method public

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For better readability, you can use textwrap.dedent() and a multiline """...""" indented string. See test_faulthandler for examples.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice suggestion 👍 Are you expecting these multiline strings to convert into a regular expression? I think this way just looks fine.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

witnesses, not whitenesses

To try and help move older pull requests forward, we are going through and backfilling 'awaiting' labels on pull requests that are lacking the label. Based on the current reviews, the best we can tell in an automated fashion is that a core developer requested changes to be made to this pull request.

If/when the requested changes have been made, please leave a comment that says, I have made the requested changes; please review again. That will trigger a bot to flag this pull request as ready for a follow-up review.

@brettcannon Many thanks for your comment. I had tried to comment back at confusing points. I think @vstinner lost his interest in my solution. What efforts should I do to make it mergeable? Have a great day!

@ultimatecoder if you think you have addressed all of the comments left by @vstinner then just leave a comment that says I have made the requested changes; please review again.

@brettcannon I have done the requested changes. There were few which I would like to discuss with @vstinner according to his availability. He has pointed out few changes on which I am confused about. I have tried to write back as a part of reply comment in response to improvement comment. I hope I made a clear statement about the situation. Thanks!

@brettcannon Many thanks for your comments. I am under little work pressure. Please expect answers/improvements at the end of this week. Thanks!

@brettcannon Reminding you for this PR. All the best for sprints 👍

@ultimatecoder no need as this is the top of the list for when I actually get time to start reviewing PRs 😉

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing fundamentally wrong, just some things to help make the tests easier to manage in future years.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use f-strings everywhere. 😄

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make this easier to read, I would create the context managers outside the with statement:

file1 = TemporaryPyFile(...)
file2 = TemporaryPyFile(...)
with file1 as file1_path, file2 as file2_path:
    ...

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this specific case, IMHO "file1 = ..." is better.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the TemporaryPyFile instantiation to its own line so you don't have to spread over multiple lines. (Same of the other lines where you were forced to spread a with statement over multiple lines.)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at the new code and it's fine.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull this up to the next line up (it's okay for it to spill over 80 columns).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't worry too much about the exact output being formatted as it is. For instance, if someone tweaked the grammar of the sentence the test should still pass. Do still check for semantically important info, though, like that the offending line is in the output.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the test is too strict, but IMHO we have been too pendantic on this nice PR which has been proposed 3 months ago. tabnanny has currently no test. If the test fails tomorrow, we will just fix the test. No need to write the perfect test here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't want to tie yourself to exception messages as they do change between versions. Just care about the right exception being raised.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sub tests would be great here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@brettcannon I am a bit busy with present work. I am sure, I will not be able to complete during this sprints. But I will be able to improve according to your comments in few days. Thanks for reviewing :)

@ultimatecoder no rush! I'm not expecting to take quite so long to do future reviews as I only need to review relative future changes.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove Lib/test/.test_tabnanny.py.swo from your PR.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I am sorry, I forgot about your old PR :-( Here is a new review. Your PR is very close to be ready to be merged. Sorry again about the slow review :-(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry but this style is not compliant with PEP 8, please strip spaces before ",".

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should always remove the created file.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may use:

proc = script_helper.spawn_python('-m', 'tabnanny', *args, text=True)
out, err = proc.communicate()
self.assertEqual(out, stdout)
self.assertEqual(err, stderr)
self.assertEqual(proc.returncode, 0)

text=True normalizes newlines to \n (Unix EOL).

@vstinner Thanks for re-observing this. I am working on the improvements suggested by you and @brettcannon 😃

@brettcannon I have updated the changes according to your comments. There is one comment in which you mentioned to assert the exception and not the output. I have added a reply to that comment with a reference. I request to provide your input on that. Thanks.

@vstinner I have improved the code according to your suggestion. I request to re-review.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that returning True is correct here. I just suggest to remove the "return" line.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, returning True isn't making sense here. Thanks for commenting.

@vstinner @brettcannon I have updated the code according to your comments. I request to proceed for further review. Thanks.

@vstinner Thanks for taking time and reviewing this PR.

@vstinner Thank you for reviewing your PR and inspiring for continuing.

I have learned many this at the time of creating, maintaining this Patch.