Python_Basics/Testing.md at master · learnbyexample/Python

assert statement

assert is primarily used for debugging purposes like catching invalid input or a condition that shouldn't occur
An optional message can be passed for descriptive error message than a plain AssertionError
It uses raise statement for implementation
assert statements can be skipped by passing the -O command line option
Note that assert is a statement and not a function

>>> assert 2 ** 3 == 8
>>> assert 3 > 4
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

>>> assert 3 > 4, "3 is not greater than 4"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: 3 is not greater than 4

Let's take a factorial function as an example:

>>> def fact(n):
        total = 1
        for num in range(1, n+1):
            total *= num
        return total

>>> assert fact(4) == 24
>>> assert fact(0) == 1
>>> fact(5)
120

>>> fact(-3)
1
>>> fact(2.3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in fact
TypeError: 'float' object cannot be interpreted as an integer

assert fact(4) == 24 and assert fact(0) == 1 can be considered as sample tests to check the function

Let's see how assert can be used to validate arguments passed to the function:

>>> def fact(n):
        assert type(n) == int and n >= 0, "Number should be zero or positive integer"
        total = 1
        for num in range(1, n+1):
            total *= num
        return total
    
>>> fact(5)
120
>>> fact(-3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in fact
AssertionError: Number should be zero or positive integer
>>> fact(2.3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in fact
AssertionError: Number should be zero or positive integer

The above factorial function can also be written using reduce

>>> def fact(n):
        assert type(n) == int and n >= 0, "Number should be zero or positive integer"
        from functools import reduce
        from operator import mul
        return reduce(mul, range(1, n+1), 1)
    

>>> fact(23)
25852016738884976640000

Above examples for demonstration only, for practical purposes use math.factorial which also gives appropriate exceptions

>>> from math import factorial
>>> factorial(10)
3628800

>>> factorial(-5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: factorial() not defined for negative values
>>> factorial(3.14)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: factorial() only accepts integral values

Further Reading

Using assert to test a program

In a limited fashion, one can use assert to test a program - either within the program (and later skipped using the -O option) or as separate test program(s)

Let's try testing the palindrome program we saw in Docstrings chapter

#!/usr/bin/python3

import palindrome

assert palindrome.is_palindrome('Madam')
assert palindrome.is_palindrome("Dammit, I'm mad!")
assert not palindrome.is_palindrome('aaa')
assert palindrome.is_palindrome('Malayalam')

try:
    assert palindrome.is_palindrome('as2')
except ValueError as e:
    assert str(e) == 'Characters other than alphabets and punctuations'

try:
    assert palindrome.is_palindrome("a'a")
except ValueError as e:
    assert str(e) == 'Less than 3 alphabets'

print('All tests passed')

There are four different cases tested for is_palindrome function
- Valid palindrome string
- Invalid palindrome string
- Invalid characters in string
- Insufficient characters in string
Both the program being tested and program to test are in same directory
To test the main function, we need to simulate user input. For this and other useful features, test frameworks come in handy

$ ./test_palindrome.py 
All tests passed

Using unittest framework

This section requires understanding of classes

#!/usr/bin/python3

import palindrome
import unittest

class TestPalindrome(unittest.TestCase):

    def test_valid(self):
        # check valid input strings
        self.assertTrue(palindrome.is_palindrome('kek'))
        self.assertTrue(palindrome.is_palindrome("Dammit, I'm mad!"))
        self.assertFalse(palindrome.is_palindrome('zzz'))
        self.assertFalse(palindrome.is_palindrome('cool'))

    def test_error(self):
        # check only the exception raised
        with self.assertRaises(ValueError):
            palindrome.is_palindrome('abc123')

        with self.assertRaises(TypeError):
            palindrome.is_palindrome(7)

        # check error message as well
        with self.assertRaises(ValueError) as cm:
            palindrome.is_palindrome('on 2 no')
        em = str(cm.exception)
        self.assertEqual(em, 'Characters other than alphabets and punctuations')

        with self.assertRaises(ValueError) as cm:
            palindrome.is_palindrome('to')
        em = str(cm.exception)
        self.assertEqual(em, 'Less than 3 alphabets')

if __name__ == '__main__':
    unittest.main()

First we create a subclass of unittest.TestCase (inheritance)
Then, different type of checks can be grouped in separate functions - function names starting with test are automatically called by unittest.main()
Depending upon type of test performed, assertTrue, assertFalse, assertRaises, assertEqual, etc can be used
An Introduction to Classes and Inheritance
Python docs - unittest

$ ./unittest_palindrome.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

$ ./unittest_palindrome.py -v
test_error (__main__.TestPalindrome) ... ok
test_valid (__main__.TestPalindrome) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Using unittest.mock to test user input and program output

This section requires understanding of decorators, do check out this wonderful intro

A simple example to see how to capture print output for testing

>>> from unittest import mock
>>> from io import StringIO

>>> def greeting():
        print('Hi there!')

>>> def test():
        with mock.patch('sys.stdout', new_callable=StringIO) as mock_stdout:
            greeting()
            assert mock_stdout.getvalue() == 'Hi there!\n'
    
>>> test()

One can also use decorators

>>> @mock.patch('sys.stdout', new_callable=StringIO)
    def test(mock_stdout):
        greeting()
        assert mock_stdout.getvalue() == 'Hi there!\n'

Now let's see how to emulate input

>>> def greeting():
        name = input('Enter your name: ')
        print('Hello', name)
    
>>> greeting()
Enter your name: learnbyexample
Hello learnbyexample

>>> with mock.patch('builtins.input', return_value='Tom'):
        greeting()
    
Hello Tom

Combining both

>>> @mock.patch('sys.stdout', new_callable=StringIO)
    def test_greeting(name, mock_stdout):
        with mock.patch('builtins.input', return_value=name):
            greeting()
            assert mock_stdout.getvalue() == 'Hello ' + name + '\n'
    
>>> test_greeting('Jo')

Having seen basic input/output testing, let's apply it to main function of palindrome

#!/usr/bin/python3

import palindrome
import unittest
from unittest import mock
from io import StringIO

class TestPalindrome(unittest.TestCase):

    @mock.patch('sys.stdout', new_callable=StringIO)
    def main_op(self, tst_str, mock_stdout):
        with mock.patch('builtins.input', side_effect=tst_str):
            palindrome.main()
        return mock_stdout.getvalue()

    def test_valid(self):
        for s in ('Malayalam', 'kek'):
            self.assertEqual(self.main_op([s]), s + ' is a palindrome\n')

        for s in ('zzz', 'cool'):
            self.assertEqual(self.main_op([s]), s + ' is NOT a palindrome\n')

    def test_error(self):
        em1 = 'Error: Characters other than alphabets and punctuations\n'
        em2 = 'Error: Less than 3 alphabets\n'

        tst1 = em1 + 'Madam is a palindrome\n'
        self.assertEqual(self.main_op(['123', 'Madam']), tst1)

        tst2 = em2 + em1 + 'Jerry is NOT a palindrome\n'
        self.assertEqual(self.main_op(['to', 'a2a', 'Jerry']), tst2)

if __name__ == '__main__':
    unittest.main()

Two test functions - one for testing valid input strings and another to check error messages
Here, side_effect which accepts iterable like list, compared to return_value where only one input value can be mocked
For valid input strings, the palindrome main function would need only one input value
For error conditions, the iterable comes handy as the main function is programmed to run infinitely until valid input is given
Python docs - unittest.mock
- patchers
An Introduction to Mocking in Python
PEP 0318 - decorators
decorators

$ ./unittest_palindrome_main.py 
..
----------------------------------------------------------------------
Ran 2 tests in 0.003s

OK

Other testing frameworks

Test driven development (TDD)