Doctests

Doctests are a tool in Python language that allows the programmer to test a solution to a problem against a number of test cases that are called doctests.

We will start by assuming that we have to solve the following problem to use it as an example:

Design the double function that, given a positive integer, returns twice that number.

Let’s run IDLE, open a new file and write the following function there:

Let’s save the function in a file named double.py.

Manually executing tests

The first way to execute tests for our functions is to do it manually. Assuming that we already have the function programmed and saved in a file, we will use the Run Module option to import the function into the shell. Once imported, we will test it simply by typing calls with different arguments and checking if the result is as expected.

>>> calculate_double(0)
0
>>> calculate_double(3)
6
>>> calculate_double(143)
286

Executing tests with doctest

What is doctest?

doctest is a Python module for executing code tests based on examples of how a function should behave given certain inputs. It allows you to automatically execute a set of tests saved in a file. The goal is to avoid having to manually type all the tests into the shell every time we make a modification to the function.

Test files

A test file for doctest replicates the format of an interactive session with the Python shell. It is as if all the examples that would be tested manually in the shell had been typed into the file. Each example we want to test (a function call with specific arguments) begins with >>>, followed by the line of code we want to execute. On the next line, the expected result is specified.

Since the test file is not executed in the same interactive environment as the shell, it is necessary to previously import the function or module we want to test. This is done by including the import lines at the beginning of the test file. If the correct function is not imported, doctest will not be able to execute the examples correctly. For this reason it is important to always respect the function and file names requested in the exercises.

For example, if we wanted to include the same tests that we have previously tested manually, we would have the following test file:

>>> from doble import calcula_doble

>>> calcula_doble(0)
0
>>> calcula_doble(3)
6
>>> calcula_doble(143)
286

language:

pytho

doctest will automatically execute each example and compare the result obtained with the expected result. If the result does not match the expected one, doctest will display an error indicating where the test failed.

You will not have to write test files, they will be available with the statements of each problem. But you do have to understand what they contain and how they are used.

How to run a test file with doctest from the terminal ~ You can download the previous test file test-doble.txt

To run the doctest module and tell it that we want it to run the examples in a file, we will use the following command in the terminal:

python3 -m doctest test_filename -v -f

For our example, the command would be

python3 -m doctest test-doble.txt -v -f

• The name of the test file is the name of the file with tests that we downloaded from the exercise page that we are solving, it is not the name of the file that contains the function.

• The -v option means that, even if there are no errors, Python displays messages with the result and is clearer.

• The -f option makes doctest stop when it finds the first error. It is very useful because errors must always be corrected in order, starting with the first one and can be difficult to find if all the tests are executed and there are many lines of results on the screen.

After executing this command, we will obtain a result like the following:

language:

python3

Each of the tests specified by the function have been automatically executed. In this case, the function has passed all the tests but, if there were errors, syntax or that the results are not correct, they would be indicated. If the function needs to be corrected, each time a modification is made the changes must be saved and then the command indicated above must be executed to execute doctest.

The tests are examples of correct results to check if a function is correct and where it fails if it is not. Do not use them to “calculate the grade” of an exercise by looking at how many are correct and how many are incorrect, because doctest counts as tests all lines that begin with >>>, but some may contain variable assignments or calculations necessary to check the results that may count as correct even though the function is not.

How to run a test file with doctest from IDLE ~ You can download the previous test file test-doble.txt

With the file containing the function open in the IDLE editing window (in our example, the doble.py file), we execute Run Module to import the function into the shell. This is important to ensure that IDLE sets the working directory to the directory where the file with the function is located.

Now inside the shell, we must import the module doctest and execute the test file as follows:

>>> import doctest
>>> doctest.testfile('test-doble.txt',verbose=True)

• What goes between quotes is always the name of the test file, which we will download from the exercise page we are solving, it is not the name of the file containing the function.

• Using doctest from IDLE, doctest does not stop when it finds the first error, as can be done using it from the terminal, and all the results are displayed.

The results will be the same as when we run doctest from the terminal, but inside the IDLE shell.

Tests with float objects

We will start by assuming that we have to solve the following problem to use it as an example:

Design the area_triangle function that, given the base and height of a triangle, returns the area.

We run IDLE, open a new file, write the following function there and save it in the file area_triangle.py:

def area_triangle(base,altura):
    return base*altura/2

We also assume that we have the following test file that we can download in the file test-atriangle.txt:

>>> from area_triangle import area_triangle

>>> area_triangle(3, 4)
6.0
>>> area_triangle(12, 5)
30.0
>>> area_triangle(3,1.52)
2.28

Problems with the precision of float objects with doctest

We check that the function passes all the tests with the following command:

python3 -m doctest atriangle.txt -v -f

It will give us the following result:

Although the result 2.28 specified in the test by the call area_triangle(3, 1.52) is correct, the test fails. The reason is that the function returns 2.28000000000000002, a small deviation due to precision errors in the representation of float numbers. These types of errors are common and do not indicate that the function is poorly designed.

To prevent these precision mismatches from failing tests that are actually correct, we can use the round() function to round the result before comparing it, limiting it to a fixed number of decimal places. The test would look like this:

>>> round(area_triangle(3,1.52),2)
2.28

You can download the new corrected test file test-atriangle-corregit.txt. Once we use this updated test, the function passes all the tests correctly.