Doctests ======== Doctests are a tool in Python language that allows the programmer to test a solution to a problem against a number of test cases that are called *doctests*. We will start by assuming that we have to solve the following problem to use it as an example: .. code-block:: text Design the double function that, given a positive integer, returns twice that number. Let's run IDLE, open a new file and write the following function there: .. literalinclude:: double.py :language: python3 Let's save the function in a file named ``double.py``. Manually executing tests ------------------------ The first way to execute tests for our functions is to do it manually. Assuming that we already have the function programmed and saved in a file, we will use the *Run Module* option to import the function into the *shell*. Once imported, we will test it simply by typing calls with different arguments and checking if the result is as expected. .. code:: python >>> calculate_double(0) 0 >>> calculate_double(3) 6 >>> calculate_double(143) 286 Executing tests with :mod:`doctest` ----------------------------------- What is :mod:`doctest`? ~~~~~~~~~~~~~~~~~~~~~~~ :mod:`doctest` is a Python module for executing code tests based on examples of how a function should behave given certain inputs. It allows you to automatically execute a set of tests saved in a file. *The goal is to avoid having to manually type all the tests into the shell every time we make a modification to the function*. Test files ~~~~~~~~~~ A test file for :mod:`doctest` replicates the format of an interactive session with the Python *shell*. It is as if all the examples that would be tested manually in the *shell* had been typed into the file. Each example we want to test (a function call with specific arguments) begins with ``>>>``, followed by the line of code we want to execute. On the next line, the expected result is specified. Since the test file is not executed in the same interactive environment as the *shell*, it is necessary to previously import the function or module we want to test. This is done by including the import lines at the beginning of the test file. If the correct function is not imported, :mod:`doctest` will not be able to execute the examples correctly. For this reason *it is important to always respect the function and file names requested in the exercises*. For example, if we wanted to include the same tests that we have previously tested manually, we would have the following test file: .. literalinclude:: test-doble.txt :language: pytho :mod:`doctest` will *automatically execute each example and compare the result obtained with the expected result*. If the result does not match the expected one, :mod:`doctest` will display an error indicating where the test failed. .. warning:: You will not have to write test files, they will be available with the statements of each problem. But you do have to understand what they contain and how they are used. How to run a test file with :mod:`doctest` from the terminal ~ You can download the previous test file :download:`test-doble.txt ` To run the :mod:`doctest` module and tell it that we want it to run the examples in a file, we will use the following command in the terminal: .. code-block:: bash python3 -m doctest test_filename -v -f For our example, the command would be .. code-block:: bash python3 -m doctest test-doble.txt -v -f * The name of the test file is the name of the file with tests that we downloaded from the exercise page that we are solving, *it is not the name of the file that contains the function*. * The ``-v`` option means that, even if there are no errors, Python displays messages with the result and is clearer. * The ``-f`` option makes :mod:`doctest` stop when it finds the first error. It is very useful because *errors must always be corrected in order, starting with the first one* and can be difficult to find if all the tests are executed and there are many lines of results on the screen. After executing this command, we will obtain a result like the following: .. literalinclude:: results.txt :language: python3 Each of the tests specified by the function have been automatically executed. In this case, the function has passed all the tests but, if there were errors, syntax or that the results are not correct, they would be indicated. If the function needs to be corrected, each time a modification is made **the changes must be saved** and then the command indicated above must be executed to execute :mod:`doctest`. .. warning:: The tests are examples of correct results to check if a function is correct and where it fails if it is not. Do not use them to "calculate the grade" of an exercise by looking at how many are correct and how many are incorrect, because :mod:`doctest` counts as tests all lines that begin with ``>>>``, but some may contain variable assignments or calculations necessary to check the results that may count as correct even though the function is not. How to run a test file with :mod:`doctest` from IDLE ~ You can download the previous test file :download:`test-doble.txt ` With the file containing the function open in the IDLE editing window (in our example, the ``doble.py`` file), we execute *Run Module* to import the function into the *shell*. This is important to ensure that IDLE sets the working directory to the directory where the file with the function is located. Now inside the *shell*, we must import the module :mod:`doctest` and execute the test file as follows: .. code:: python >>> import doctest >>> doctest.testfile('test-doble.txt',verbose=True) * What goes between quotes is always the name of the test file, which we will download from the exercise page we are solving, *it is not the name of the file containing the function*. * Using doctest from IDLE, :mod:`doctest` does not stop when it finds the first error, as can be done using it from the terminal, and all the results are displayed. The results will be the same as when we run :mod:`doctest` from the terminal, but inside the IDLE *shell*. Tests with float objects ------------------------ We will start by assuming that we have to solve the following problem to use it as an example: .. code-block:: text Design the area_triangle function that, given the base and height of a triangle, returns the area. We run IDLE, open a new file, write the following function there and save it in the file :file:`area_triangle.py`: .. literalinclude:: area_triangle.py :language: python3 We also assume that we have the following test file that we can download in the file :download:`test-atriangle.txt`: .. literalinclude:: test-atriangle.txt :language: python3 Problems with the precision of float objects with doctest ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We check that the function passes all the tests with the following command: .. code-block:: pycon python3 -m doctest atriangle.txt -v -f It will give us the following result: .. literalinclude:: results-floats.txt :language: python3 Although the result 2.28 specified in the test by the call ``area_triangle(3, 1.52)`` is correct, the test fails. The reason is that the function returns 2.28000000000000002, a small deviation due to precision errors in the representation of float numbers. These types of errors are common and do not indicate that the function is poorly designed. To prevent these precision mismatches from failing tests that are actually correct, we can use the :func:`round` function to round the result before comparing it, limiting it to a fixed number of decimal places. The test would look like this: .. code-block:: pycon >>> round(area_triangle(3,1.52),2) 2.28 You can download the new corrected test file :download:`test-atriangle-corregit.txt`. Once we use this updated test, the function passes all the tests correctly.