{"id":1922,"date":"2019-06-04T17:18:39","date_gmt":"2019-06-04T16:18:39","guid":{"rendered":"https:\/\/rosetta.vn\/short\/?p=1922"},"modified":"2019-06-04T17:18:39","modified_gmt":"2019-06-04T16:18:39","slug":"getting-started-with-testing-in-python-real-python","status":"publish","type":"post","link":"https:\/\/rosetta.vn\/short\/2019\/06\/04\/getting-started-with-testing-in-python-real-python\/","title":{"rendered":"Getting Started With Testing in Python \u2013 Real Python"},"content":{"rendered":"<blockquote><p>This tutorial is for anyone who has written a fantastic application in Python but hasn\u2019t yet written any tests.<\/p>\n<p>Testing in Python is a huge topic and can come with a lot of complexity, but it doesn\u2019t need to be hard. You can get started creating simple tests for your application in a few easy steps and then build on it from there.<\/p>\n<p>In this tutorial, you\u2019ll learn how to create a basic test, execute it, and find the bugs before your users do! You\u2019ll learn about the tools available to write and execute tests, check your application\u2019s performance, and even look for security issues.<\/p>\n<div class=\"alert alert-warning\" role=\"alert\">\n<p><strong>Free Bonus:<\/strong>\u00a0<a class=\"alert-link\" href=\"https:\/\/realpython.com\/python-testing\/\" data-toggle=\"modal\" data-target=\"#modal-python-mastery-course\" data-focus=\"false\" rel=\"noopener noreferrer\">5 Thoughts On Python Mastery<\/a>, a free course for Python developers that shows you the roadmap and the mindset you&#8217;ll need to take your Python skills to the next level.<\/p>\n<\/div>\n<h2 id=\"testing-your-code\">Testing Your Code<\/h2>\n<p>There are many ways to test your code. In this tutorial, you\u2019ll learn the techniques from the most basic steps and work towards advanced methods.<\/p>\n<h3 id=\"automated-vs-manual-testing\">Automated vs. Manual Testing<\/h3>\n<p>The good news is, you\u2019ve probably already created a test without realizing it. Remember when you ran your application and used it for the first time? Did you check the features and experiment using them? That\u2019s known as\u00a0<strong>exploratory testing<\/strong>\u00a0and is a form of manual testing.<\/p>\n<p>Exploratory testing is a form of testing that is done without a plan. In an exploratory test, you\u2019re just exploring the application.<\/p>\n<p>To have a complete set of manual tests, all you need to do is make a list of all the features your application has, the different types of input it can accept, and the expected results. Now, every time you make a change to your code, you need to go through every single item on that list and check it.<\/p>\n<p>That doesn\u2019t sound like much fun, does it?<\/p>\n<p>This is where automated testing comes in. Automated testing is the execution of your test plan (the parts of your application you want to test, the order in which you want to test them, and the expected responses) by a script instead of a human. Python already comes with a set of tools and libraries to help you create automated tests for your application. We\u2019ll explore those tools and libraries in this tutorial.<\/p>\n<h3 id=\"unit-tests-vs-integration-tests\">Unit Tests vs. Integration Tests<\/h3>\n<p>The world of testing has no shortage of terminology, and now that you know the difference between automated and manual testing, it\u2019s time to go a level deeper.<\/p>\n<p>Think of how you might test the lights on a car. You would turn on the lights (known as the\u00a0<strong>test step<\/strong>) and go outside the car or ask a friend to check that the lights are on (known as the\u00a0<strong>test assertion<\/strong>). Testing multiple components is known as\u00a0<strong>integration testing<\/strong>.<\/p>\n<p>Think of all the things that need to work correctly in order for a simple task to give the right result. These components are like the parts to your application, all of those classes, functions, and modules you\u2019ve written.<\/p>\n<p>A major challenge with integration testing is when an integration test doesn\u2019t give the right result. It\u2019s very hard to diagnose the issue without being able to isolate which part of the system is failing. If the lights didn\u2019t turn on, then maybe the bulbs are broken. Is the battery dead? What about the alternator? Is the car\u2019s computer failing?<\/p>\n<p>If you have a fancy modern car, it will tell you when your light bulbs have gone. It does this using a form of\u00a0<strong>unit test<\/strong>.<\/p>\n<p>A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.<\/p>\n<p>You have just seen two types of tests:<\/p>\n<ol>\n<li>An integration test checks that components in your application operate with each other.<\/li>\n<li>A unit test checks a small component in your application.<\/li>\n<\/ol>\n<p>You can write both integration tests and unit tests in Python. To write a unit test for the built-in function\u00a0<code>sum()<\/code>, you would check the output of\u00a0<code>sum()<\/code>\u00a0against a known output.<\/p>\n<p>For example, here\u2019s how you check that the\u00a0<code>sum()<\/code>\u00a0of the numbers\u00a0<code>(1, 2, 3)<\/code>\u00a0equals\u00a0<code>6<\/code>:<\/p>\n<div class=\"highlight python pycon\"><span class=\"repl-toggle\" title=\"Toggle REPL prompts and output\">&gt;&gt;&gt;<\/span><\/p>\n<pre><span class=\"gp\">&gt;&gt;&gt; <\/span><span class=\"k\">assert<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">([<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">])<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span>\r\n<\/pre>\n<\/div>\n<p>This will not output anything on the REPL because the values are correct.<\/p>\n<p>If the result from\u00a0<code>sum()<\/code>\u00a0is incorrect, this will fail with an\u00a0<code>AssertionError<\/code>and the message\u00a0<code>\"Should be 6\"<\/code>. Try an assertion statement again with the wrong values to see an\u00a0<code>AssertionError<\/code>:<\/p>\n<div class=\"highlight python pycon\"><span class=\"repl-toggle\" title=\"Toggle REPL prompts and output\">&gt;&gt;&gt;<\/span><\/p>\n<pre><span class=\"gp\">&gt;&gt;&gt; <\/span><span class=\"k\">assert<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">([<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">])<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span>\r\n<span class=\"gt\">Traceback (most recent call last):<\/span>\r\n  File <span class=\"nb\">\"&lt;stdin&gt;\"<\/span>, line <span class=\"m\">1<\/span>, in <span class=\"n\">&lt;module&gt;<\/span>\r\n<span class=\"gr\">AssertionError<\/span>: <span class=\"n\">Should be 6<\/span>\r\n<\/pre>\n<\/div>\n<p>In the REPL, you are seeing the raised\u00a0<code>AssertionError<\/code>\u00a0because the result of\u00a0<code>sum()<\/code>\u00a0does not match\u00a0<code>6<\/code>.<\/p>\n<p>Instead of testing on the REPL, you\u2019ll want to put this into a new Python file called\u00a0<code>test_sum.py<\/code>\u00a0and execute it again:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"k\">def<\/span> <span class=\"nf\">test_sum<\/span><span class=\"p\">():<\/span>\r\n    <span class=\"k\">assert<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">([<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">])<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span>\r\n\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">\"__main__\"<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">test_sum<\/span><span class=\"p\">()<\/span>\r\n    <span class=\"nb\">print<\/span><span class=\"p\">(<\/span><span class=\"s2\">\"Everything passed\"<\/span><span class=\"p\">)<\/span>\r\n<\/pre>\n<\/div>\n<p>Now you have written a\u00a0<strong>test case<\/strong>, an assertion, and an entry point (the command line). You can now execute this at the command line:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python test_sum.py\r\n<span class=\"go\">Everything passed<\/span>\r\n<\/pre>\n<\/div>\n<p>You can see the successful result,\u00a0<code>Everything passed<\/code>.<\/p>\n<p>In Python,\u00a0<code>sum()<\/code>\u00a0accepts any iterable as its first argument. You tested with a list. Now test with a tuple as well. Create a new file called\u00a0<code>test_sum_2.py<\/code>\u00a0with the following code:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"k\">def<\/span> <span class=\"nf\">test_sum<\/span><span class=\"p\">():<\/span>\r\n    <span class=\"k\">assert<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">([<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">])<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span>\r\n\r\n<span class=\"k\">def<\/span> <span class=\"nf\">test_sum_tuple<\/span><span class=\"p\">():<\/span>\r\n    <span class=\"k\">assert<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">((<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">))<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span>\r\n\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">\"__main__\"<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">test_sum<\/span><span class=\"p\">()<\/span>\r\n    <span class=\"n\">test_sum_tuple<\/span><span class=\"p\">()<\/span>\r\n    <span class=\"nb\">print<\/span><span class=\"p\">(<\/span><span class=\"s2\">\"Everything passed\"<\/span><span class=\"p\">)<\/span>\r\n<\/pre>\n<\/div>\n<p>When you execute\u00a0<code>test_sum_2.py<\/code>, the script will give an error because the\u00a0<code>sum()<\/code>\u00a0of\u00a0<code>(1, 2, 2)<\/code>\u00a0is\u00a0<code>5<\/code>, not\u00a0<code>6<\/code>. The result of the script gives you the error message, the line of code, and the traceback:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python test_sum_2.py\r\n<span class=\"go\">Traceback (most recent call last):<\/span>\r\n<span class=\"go\">  File \"test_sum_2.py\", line 9, in &lt;module&gt;<\/span>\r\n<span class=\"go\">    test_sum_tuple()<\/span>\r\n<span class=\"go\">  File \"test_sum_2.py\", line 5, in test_sum_tuple<\/span>\r\n<span class=\"go\">    assert sum((1, 2, 2)) == 6, \"Should be 6\"<\/span>\r\n<span class=\"go\">AssertionError: Should be 6<\/span>\r\n<\/pre>\n<\/div>\n<p>Here you can see how a mistake in your code gives an error on the console with some information on where the error was and what the expected result was.<\/p>\n<p>Writing tests in this way is okay for a simple check, but what if more than one fails? This is where test runners come in. The test runner is a special application designed for running tests, checking the output, and giving you tools for debugging and diagnosing tests and applications.<\/p>\n<h3 id=\"choosing-a-test-runner\">Choosing a Test Runner<\/h3>\n<p>There are many test runners available for Python. The one built into the Python standard library is called\u00a0<code>unittest<\/code>. In this tutorial, you will be using\u00a0<code>unittest<\/code>\u00a0test cases and the\u00a0<code>unittest<\/code>\u00a0test runner. The principles of\u00a0<code>unittest<\/code>\u00a0are easily portable to other frameworks. The three most popular test runners are:<\/p>\n<ul>\n<li><code>unittest<\/code><\/li>\n<li><code>nose<\/code>\u00a0or\u00a0<code>nose2<\/code><\/li>\n<li><code>pytest<\/code><\/li>\n<\/ul>\n<p>Choosing the best test runner for your requirements and level of experience is important.<\/p>\n<h4 id=\"unittest\"><code>unittest<\/code><\/h4>\n<p><code>unittest<\/code>\u00a0has been built into the Python standard library since version 2.1. You\u2019ll probably see it in commercial Python applications and open-source projects.<\/p>\n<p><code>unittest<\/code>\u00a0contains both a testing framework and a test runner.\u00a0<code>unittest<\/code>has some important requirements for writing and executing tests.<\/p>\n<p><code>unittest<\/code>\u00a0requires that:<\/p>\n<ul>\n<li>You put your tests into classes as methods<\/li>\n<li>You use a series of special assertion methods in the\u00a0<code>unittest.TestCase<\/code>\u00a0class instead of the built-in\u00a0<code>assert<\/code>\u00a0statement<\/li>\n<\/ul>\n<p>To convert the earlier example to a\u00a0<code>unittest<\/code>\u00a0test case, you would have to:<\/p>\n<ol>\n<li>Import\u00a0<code>unittest<\/code>\u00a0from the standard library<\/li>\n<li>Create a class called\u00a0<code>TestSum<\/code>\u00a0that inherits from the\u00a0<code>TestCase<\/code>\u00a0class<\/li>\n<li>Convert the test functions into methods by adding\u00a0<code>self<\/code>\u00a0as the first argument<\/li>\n<li>Change the assertions to use the\u00a0<code>self.assertEqual()<\/code>\u00a0method on the\u00a0<code>TestCase<\/code>\u00a0class<\/li>\n<li>Change the command-line entry point to call\u00a0<code>unittest.main()<\/code><\/li>\n<\/ol>\n<p>Follow those steps by creating a new file\u00a0<code>test_sum_unittest.py<\/code>\u00a0with the following code:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">import<\/span> <span class=\"nn\">unittest<\/span>\r\n\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">TestSum<\/span><span class=\"p\">(<\/span><span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_sum<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"nb\">sum<\/span><span class=\"p\">([<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">]),<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span><span class=\"p\">)<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_sum_tuple<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"nb\">sum<\/span><span class=\"p\">((<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">)),<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span><span class=\"p\">)<\/span>\r\n\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'__main__'<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">main<\/span><span class=\"p\">()<\/span>\r\n<\/pre>\n<\/div>\n<p>If you execute this at the command line, you\u2019ll see one success (indicated with\u00a0<code>.<\/code>) and one failure (indicated with\u00a0<code>F<\/code>):<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python test_sum_unittest.py\r\n<span class=\"go\">.F<\/span>\r\n<span class=\"go\">======================================================================<\/span>\r\n<span class=\"go\">FAIL: test_sum_tuple (__main__.TestSum)<\/span>\r\n<span class=\"go\">----------------------------------------------------------------------<\/span>\r\n<span class=\"go\">Traceback (most recent call last):<\/span>\r\n<span class=\"go\">  File \"test_sum_unittest.py\", line 9, in test_sum_tuple<\/span>\r\n<span class=\"go\">    self.assertEqual(sum((1, 2, 2)), 6, \"Should be 6\")<\/span>\r\n<span class=\"go\">AssertionError: Should be 6<\/span>\r\n\r\n<span class=\"go\">----------------------------------------------------------------------<\/span>\r\n<span class=\"go\">Ran 2 tests in 0.001s<\/span>\r\n\r\n<span class=\"go\">FAILED (failures=1)<\/span>\r\n<\/pre>\n<\/div>\n<p>You have just executed two tests using the\u00a0<code>unittest<\/code>\u00a0test runner.<\/p>\n<div class=\"alert alert-primary\" role=\"alert\">\n<p><strong>Note:<\/strong>\u00a0Be careful if you\u2019re writing test cases that need to execute in both Python 2 and 3. In Python 2.7 and below,\u00a0<code>unittest<\/code>\u00a0is called\u00a0<code>unittest2<\/code>. If you simply import from\u00a0<code>unittest<\/code>, you will get different versions with different features between Python 2 and 3.<\/p>\n<\/div>\n<p>For more information on\u00a0<code>unittest<\/code>, you can explore the\u00a0<a href=\"https:\/\/docs.python.org\/3\/library\/unittest.html\">unittest Documentation<\/a>.<\/p>\n<h4 id=\"nose\"><code>nose<\/code><\/h4>\n<p>You may find that over time, as you write hundreds or even thousands of tests for your application, it becomes increasingly hard to understand and use the output from\u00a0<code>unittest<\/code>.<\/p>\n<p><code>nose<\/code>\u00a0is compatible with any tests written using the\u00a0<code>unittest<\/code>\u00a0framework and can be used as a drop-in replacement for the\u00a0<code>unittest<\/code>\u00a0test runner. The development of\u00a0<code>nose<\/code>\u00a0as an open-source application fell behind, and a fork called\u00a0<code>nose2<\/code>\u00a0was created. If you\u2019re starting from scratch, it is recommended that you use\u00a0<code>nose2<\/code>\u00a0instead of\u00a0<code>nose<\/code>.<\/p>\n<p>To get started with\u00a0<code>nose2<\/code>, install\u00a0<code>nose2<\/code>\u00a0from PyPI and execute it on the command line.\u00a0<code>nose2<\/code>\u00a0will try to discover all test scripts named\u00a0<code>test*.py<\/code>and test cases inheriting from\u00a0<code>unittest.TestCase<\/code>\u00a0in your current directory:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> pip install nose2\r\n<span class=\"gp\">$<\/span> python -m nose2\r\n<span class=\"go\">.F<\/span>\r\n<span class=\"go\">======================================================================<\/span>\r\n<span class=\"go\">FAIL: test_sum_tuple (__main__.TestSum)<\/span>\r\n<span class=\"go\">----------------------------------------------------------------------<\/span>\r\n<span class=\"go\">Traceback (most recent call last):<\/span>\r\n<span class=\"go\">  File \"test_sum_unittest.py\", line 9, in test_sum_tuple<\/span>\r\n<span class=\"go\">    self.assertEqual(sum((1, 2, 2)), 6, \"Should be 6\")<\/span>\r\n<span class=\"go\">AssertionError: Should be 6<\/span>\r\n\r\n<span class=\"go\">----------------------------------------------------------------------<\/span>\r\n<span class=\"go\">Ran 2 tests in 0.001s<\/span>\r\n\r\n<span class=\"go\">FAILED (failures=1)<\/span>\r\n<\/pre>\n<\/div>\n<p>You have just executed the test you created in\u00a0<code>test_sum_unittest.py<\/code>from the\u00a0<code>nose2<\/code>\u00a0test runner.\u00a0<code>nose2<\/code>\u00a0offers many command-line flags for filtering the tests that you execute. For more information, you can explore the\u00a0<a href=\"https:\/\/nose2.readthedocs.io\/\">Nose 2 documentation<\/a>.<\/p>\n<h4 id=\"pytest\"><code>pytest<\/code><\/h4>\n<p><code>pytest<\/code>\u00a0supports execution of\u00a0<code>unittest<\/code>\u00a0test cases. The real advantage of\u00a0<code>pytest<\/code>\u00a0comes by writing\u00a0<code>pytest<\/code>\u00a0test cases.\u00a0<code>pytest<\/code>\u00a0test cases are a series of functions in a Python file starting with the name\u00a0<code>test_<\/code>.<\/p>\n<p><code>pytest<\/code>\u00a0has some other great features:<\/p>\n<ul>\n<li>Support for the built-in\u00a0<code>assert<\/code>\u00a0statement instead of using special\u00a0<code>self.assert*()<\/code>\u00a0methods<\/li>\n<li>Support for filtering for test cases<\/li>\n<li>Ability to rerun from the last failing test<\/li>\n<li>An ecosystem of hundreds of plugins to extend the functionality<\/li>\n<\/ul>\n<p>Writing the\u00a0<code>TestSum<\/code>\u00a0test case example for\u00a0<code>pytest<\/code>\u00a0would look like this:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"k\">def<\/span> <span class=\"nf\">test_sum<\/span><span class=\"p\">():<\/span>\r\n    <span class=\"k\">assert<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">([<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">])<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span>\r\n\r\n<span class=\"k\">def<\/span> <span class=\"nf\">test_sum_tuple<\/span><span class=\"p\">():<\/span>\r\n    <span class=\"k\">assert<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">((<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">))<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">6<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Should be 6\"<\/span>\r\n<\/pre>\n<\/div>\n<p>You have dropped the\u00a0<code>TestCase<\/code>, any use of classes, and the command-line entry point.<\/p>\n<p>More information can be found at the\u00a0<a href=\"https:\/\/docs.pytest.org\/en\/latest\/\">Pytest Documentation Website<\/a>.<\/p>\n<h2 id=\"writing-your-first-test\">Writing Your First Test<\/h2>\n<p>Let\u2019s bring together what you\u2019ve learned so far and, instead of testing the built-in\u00a0<code>sum()<\/code>\u00a0function, test a simple implementation of the same requirement.<\/p>\n<p>Create a new project folder and, inside that, create a new folder called\u00a0<code>my_sum<\/code>. Inside\u00a0<code>my_sum<\/code>, create an empty file called\u00a0<code>__init__.py<\/code>. Creating the\u00a0<code>__init__.py<\/code>\u00a0file means that the\u00a0<code>my_sum<\/code>\u00a0folder can be imported as a module from the parent directory.<\/p>\n<p>Your project folder should look like this:<\/p>\n<div class=\"highlight\">\n<pre>project\/\r\n\u2502\r\n\u2514\u2500\u2500 my_sum\/\r\n    \u2514\u2500\u2500 __init__.py\r\n<\/pre>\n<\/div>\n<p>Open up\u00a0<code>my_sum\/__init__.py<\/code>\u00a0and create a new function called\u00a0<code>sum()<\/code>, which takes an iterable (a list, tuple, or set) and adds the values together:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"k\">def<\/span> <span class=\"nf\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">arg<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"n\">total<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\r\n    <span class=\"k\">for<\/span> <span class=\"n\">val<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">arg<\/span><span class=\"p\">:<\/span>\r\n        <span class=\"n\">total<\/span> <span class=\"o\">+=<\/span> <span class=\"n\">val<\/span>\r\n    <span class=\"k\">return<\/span> <span class=\"n\">total<\/span>\r\n<\/pre>\n<\/div>\n<p>This code example creates a variable called\u00a0<code>total<\/code>, iterates over all the values in\u00a0<code>arg<\/code>, and adds them to\u00a0<code>total<\/code>. It then returns the result once the iterable has been exhausted.<\/p>\n<h3 id=\"where-to-write-the-test\">Where to Write the Test<\/h3>\n<p>To get started writing tests, you can simply create a file called\u00a0<code>test.py<\/code>, which will contain your first test case. Because the file will need to be able to import your application to be able to test it, you want to place\u00a0<code>test.py<\/code>above the package folder, so your directory tree will look something like this:<\/p>\n<div class=\"highlight\">\n<pre>project\/\r\n\u2502\r\n\u251c\u2500\u2500 my_sum\/\r\n\u2502   \u2514\u2500\u2500 __init__.py\r\n|\r\n\u2514\u2500\u2500 test.py\r\n<\/pre>\n<\/div>\n<p>You\u2019ll find that, as you add more and more tests, your single file will become cluttered and hard to maintain, so you can create a folder called\u00a0<code>tests\/<\/code>\u00a0and split the tests into multiple files. It is convention to ensure each file starts with\u00a0<code>test_<\/code>\u00a0so all test runners will assume that Python file contains tests to be executed. Some very large projects split tests into more subdirectories based on their purpose or usage.<\/p>\n<div class=\"alert alert-primary\" role=\"alert\">\n<p><strong>Note:<\/strong>\u00a0What if your application is a single script?<\/p>\n<p>You can import any attributes of the script, such as classes, functions, and variables by using the built-in\u00a0<code>__import__()<\/code>function. Instead of\u00a0<code>from my_sum import sum<\/code>, you can write the following:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"n\">target<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">__import__<\/span><span class=\"p\">(<\/span><span class=\"s2\">\"my_sum.py\"<\/span><span class=\"p\">)<\/span>\r\n<span class=\"nb\">sum<\/span> <span class=\"o\">=<\/span> <span class=\"n\">target<\/span><span class=\"o\">.<\/span><span class=\"n\">sum<\/span>\r\n<\/pre>\n<\/div>\n<p>The benefit of using\u00a0<code>__import__()<\/code>\u00a0is that you don\u2019t have to turn your project folder into a package, and you can specify the file name. This is also useful if your filename collides with any standard library packages. For example,\u00a0<code>math.py<\/code>\u00a0would collide with the\u00a0<code>math<\/code>module.<\/p>\n<\/div>\n<h3 id=\"how-to-structure-a-simple-test\">How to Structure a Simple Test<\/h3>\n<p>Before you dive into writing tests, you\u2019ll want to first make a couple of decisions:<\/p>\n<ol>\n<li>What do you want to test?<\/li>\n<li>Are you writing a unit test or an integration test?<\/li>\n<\/ol>\n<p>Then the structure of a test should loosely follow this workflow:<\/p>\n<ol>\n<li>Create your inputs<\/li>\n<li>Execute the code being tested, capturing the output<\/li>\n<li>Compare the output with an expected result<\/li>\n<\/ol>\n<p>For this application, you\u2019re testing\u00a0<code>sum()<\/code>. There are many behaviors in\u00a0<code>sum()<\/code>\u00a0you could check, such as:<\/p>\n<ul>\n<li>Can it sum a list of whole numbers (integers)?<\/li>\n<li>Can it sum a tuple or set?<\/li>\n<li>Can it sum a list of floats?<\/li>\n<li>What happens when you provide it with a bad value, such as a single integer or a string?<\/li>\n<li>What happens when one of the values is negative?<\/li>\n<\/ul>\n<p>The most simple test would be a list of integers. Create a file,\u00a0<code>test.py<\/code>with the following Python code:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">import<\/span> <span class=\"nn\">unittest<\/span>\r\n\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">my_sum<\/span> <span class=\"k\">import<\/span> <span class=\"nb\">sum<\/span>\r\n\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">TestSum<\/span><span class=\"p\">(<\/span><span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_list_int<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"sd\">\"\"\"<\/span>\r\n<span class=\"sd\">        Test that it can sum a list of integers<\/span>\r\n<span class=\"sd\">        \"\"\"<\/span>\r\n        <span class=\"n\">data<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">]<\/span>\r\n        <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">result<\/span><span class=\"p\">,<\/span> <span class=\"mi\">6<\/span><span class=\"p\">)<\/span>\r\n\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'__main__'<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">main<\/span><span class=\"p\">()<\/span>\r\n<\/pre>\n<\/div>\n<p>This code example:<\/p>\n<ol>\n<li>Imports\u00a0<code>sum()<\/code>\u00a0from the\u00a0<code>my_sum<\/code>\u00a0package you created<\/li>\n<li>Defines a new test case class called\u00a0<code>TestSum<\/code>, which inherits from\u00a0<code>unittest.TestCase<\/code><\/li>\n<li>Defines a test method,\u00a0<code>.test_list_int()<\/code>, to test a list of integers. The method\u00a0<code>.test_list_int()<\/code>\u00a0will:\n<ul>\n<li>Declare a variable\u00a0<code>data<\/code>\u00a0with a list of numbers\u00a0<code>(1, 2, 3)<\/code><\/li>\n<li>Assign the result of\u00a0<code>my_sum.sum(data)<\/code>\u00a0to a\u00a0<code>result<\/code>\u00a0variable<\/li>\n<li>Assert that the value of\u00a0<code>result<\/code>\u00a0equals\u00a0<code>6<\/code>\u00a0by using the\u00a0<code>.assertEqual()<\/code>\u00a0method on the\u00a0<code>unittest.TestCase<\/code>\u00a0class<\/li>\n<\/ul>\n<\/li>\n<li>Defines a command-line entry point, which runs the\u00a0<code>unittest<\/code>\u00a0test-runner\u00a0<code>.main()<\/code><\/li>\n<\/ol>\n<p>If you\u2019re unsure what\u00a0<code>self<\/code>\u00a0is or how\u00a0<code>.assertEqual()<\/code>\u00a0is defined, you can brush up on your object-oriented programming with\u00a0<a href=\"https:\/\/realpython.com\/python3-object-oriented-programming\/\">Python 3 Object-Oriented Programming<\/a>.<\/p>\n<h3 id=\"how-to-write-assertions\">How to Write Assertions<\/h3>\n<p>The last step of writing a test is to validate the output against a known response. This is known as an\u00a0<strong>assertion<\/strong>. There are some general best practices around how to write assertions:<\/p>\n<ul>\n<li>Make sure tests are repeatable and run your test multiple times to make sure it gives the same result every time<\/li>\n<li>Try and assert results that relate to your input data, such as checking that the result is the actual sum of values in the\u00a0<code>sum()<\/code>\u00a0example<\/li>\n<\/ul>\n<p><code>unittest<\/code>\u00a0comes with lots of methods to assert on the values, types, and existence of variables. Here are some of the most commonly used methods:<\/p>\n<div class=\"table-responsive\">\n<table class=\"table table-hover\">\n<thead>\n<tr>\n<th>Method<\/th>\n<th>Equivalent to<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code>.assertEqual(a, b)<\/code><\/td>\n<td><code>a == b<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>.assertTrue(x)<\/code><\/td>\n<td><code>bool(x) is True<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>.assertFalse(x)<\/code><\/td>\n<td><code>bool(x) is False<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>.assertIs(a, b)<\/code><\/td>\n<td><code>a is b<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>.assertIsNone(x)<\/code><\/td>\n<td><code>x is None<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>.assertIn(a, b)<\/code><\/td>\n<td><code>a in b<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>.assertIsInstance(a, b)<\/code><\/td>\n<td><code>isinstance(a, b)<\/code><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><code>.assertIs()<\/code>,\u00a0<code>.assertIsNone()<\/code>,\u00a0<code>.assertIn()<\/code>, and\u00a0<code>.assertIsInstance()<\/code>\u00a0all have opposite methods, named\u00a0<code>.assertIsNot()<\/code>, and so forth.<\/p>\n<h3 id=\"side-effects\">Side Effects<\/h3>\n<p>When you\u2019re writing tests, it\u2019s often not as simple as looking at the return value of a function. Often, executing a piece of code will alter other things in the environment, such as the attribute of a class, a file on the filesystem, or a value in a database. These are known as\u00a0<strong>side effects<\/strong>\u00a0and are an important part of testing. Decide if the side effect is being tested before including it in your list of assertions.<\/p>\n<p>If you find that the unit of code you want to test has lots of side effects, you might be breaking the\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Single_responsibility_principle\">Single Responsibility Principle<\/a>. Breaking the Single Responsibility Principle means the piece of code is doing too many things and would be better off being refactored. Following the Single Responsibility Principle is a great way to design code that it is easy to write repeatable and simple unit tests for, and ultimately, reliable applications.<\/p>\n<h2 id=\"executing-your-first-test\">Executing Your First Test<\/h2>\n<p>Now that you\u2019ve created the first test, you want to execute it. Sure, you know it\u2019s going to pass, but before you create more complex tests, you should check that you can execute the tests successfully.<\/p>\n<h3 id=\"executing-test-runners\">Executing Test Runners<\/h3>\n<p>The Python application that executes your test code, checks the assertions, and gives you test results in your console is called the\u00a0<strong>test runner<\/strong>.<\/p>\n<p>At the bottom of\u00a0<code>test.py<\/code>, you added this small snippet of code:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'__main__'<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">main<\/span><span class=\"p\">()<\/span>\r\n<\/pre>\n<\/div>\n<p>This is a command line entry point. It means that if you execute the script alone by running\u00a0<code>python test.py<\/code>\u00a0at the command line, it will call\u00a0<code>unittest.main()<\/code>. This executes the test runner by discovering all classes in this file that inherit from\u00a0<code>unittest.TestCase<\/code>.<\/p>\n<p>This is one of many ways to execute the\u00a0<code>unittest<\/code>\u00a0test runner. When you have a single test file named\u00a0<code>test.py<\/code>, calling\u00a0<code>python test.py<\/code>\u00a0is a great way to get started.<\/p>\n<p>Another way is using the\u00a0<code>unittest<\/code>\u00a0command line. Try this:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python -m unittest <span class=\"nb\">test<\/span>\r\n<\/pre>\n<\/div>\n<p>This will execute the same test module (called\u00a0<code>test<\/code>) via the command line.<\/p>\n<p>You can provide additional options to change the output. One of those is\u00a0<code>-v<\/code>\u00a0for verbose. Try that next:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python -m unittest -v <span class=\"nb\">test<\/span>\r\n<span class=\"go\">test_list_int (test.TestSum) ... ok<\/span>\r\n\r\n<span class=\"go\">----------------------------------------------------------------------<\/span>\r\n<span class=\"go\">Ran 1 tests in 0.000s<\/span>\r\n<\/pre>\n<\/div>\n<p>This executed the one test inside\u00a0<code>test.py<\/code>\u00a0and printed the results to the console. Verbose mode listed the names of the tests it executed first, along with the result of each test.<\/p>\n<p>Instead of providing the name of a module containing tests, you can request an auto-discovery using the following:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python -m unittest discover\r\n<\/pre>\n<\/div>\n<p>This will search the current directory for any files named\u00a0<code>test*.py<\/code>\u00a0and attempt to test them.<\/p>\n<p>Once you have multiple test files, as long as you follow the\u00a0<code>test*.py<\/code>naming pattern, you can provide the name of the directory instead by using the\u00a0<code>-s<\/code>\u00a0flag and the name of the directory:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python -m unittest discover -s tests\r\n<\/pre>\n<\/div>\n<p><code>unittest<\/code>\u00a0will run all tests in a single test plan and give you the results.<\/p>\n<p>Lastly, if your source code is not in the directory root and contained in a subdirectory, for example in a folder called\u00a0<code>src\/<\/code>, you can tell\u00a0<code>unittest<\/code>where to execute the tests so that it can import the modules correctly with the\u00a0<code>-t<\/code>\u00a0flag:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python -m unittest discover -s tests -t src\r\n<\/pre>\n<\/div>\n<p><code>unittest<\/code>\u00a0will change to the\u00a0<code>src\/<\/code>\u00a0directory, scan for all\u00a0<code>test*.py<\/code>\u00a0files inside the the\u00a0<code>tests<\/code>\u00a0directory, and execute them.<\/p>\n<h3 id=\"understanding-test-output\">Understanding Test Output<\/h3>\n<p>That was a very simple example where everything passes, so now you\u2019re going to try a failing test and interpret the output.<\/p>\n<p><code>sum()<\/code>\u00a0should be able to accept other lists of numeric types, like fractions.<\/p>\n<p>At the top of the\u00a0<code>test.py<\/code>\u00a0file, add an import statement to import the\u00a0<code>Fraction<\/code>\u00a0type from the\u00a0<code>fractions<\/code>\u00a0module in the standard library:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">from<\/span> <span class=\"nn\">fractions<\/span> <span class=\"k\">import<\/span> <span class=\"n\">Fraction<\/span>\r\n<\/pre>\n<\/div>\n<p>Now add a test with an assertion expecting the incorrect value, in this case expecting the sum of 1\/4, 1\/4, and 2\/5 to be 1:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">import<\/span> <span class=\"nn\">unittest<\/span>\r\n\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">my_sum<\/span> <span class=\"k\">import<\/span> <span class=\"nb\">sum<\/span>\r\n\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">TestSum<\/span><span class=\"p\">(<\/span><span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_list_int<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"sd\">\"\"\"<\/span>\r\n<span class=\"sd\">        Test that it can sum a list of integers<\/span>\r\n<span class=\"sd\">        \"\"\"<\/span>\r\n        <span class=\"n\">data<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">]<\/span>\r\n        <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">result<\/span><span class=\"p\">,<\/span> <span class=\"mi\">6<\/span><span class=\"p\">)<\/span>\r\n\r\n<span class=\"hll\">    <span class=\"k\">def<\/span> <span class=\"nf\">test_list_fraction<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n<\/span><span class=\"hll\">        <span class=\"sd\">\"\"\"<\/span>\r\n<\/span><span class=\"hll\"><span class=\"sd\">        Test that it can sum a list of fractions<\/span>\r\n<\/span><span class=\"hll\"><span class=\"sd\">        \"\"\"<\/span>\r\n<\/span><span class=\"hll\">        <span class=\"n\">data<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"n\">Fraction<\/span><span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">4<\/span><span class=\"p\">),<\/span> <span class=\"n\">Fraction<\/span><span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">4<\/span><span class=\"p\">),<\/span> <span class=\"n\">Fraction<\/span><span class=\"p\">(<\/span><span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">5<\/span><span class=\"p\">)]<\/span>\r\n<\/span><span class=\"hll\">        <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">)<\/span>\r\n<\/span><span class=\"hll\">        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">result<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">)<\/span>\r\n<\/span>\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'__main__'<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">main<\/span><span class=\"p\">()<\/span>\r\n<\/pre>\n<\/div>\n<p>If you execute the tests again with\u00a0<code>python -m unittest test<\/code>, you should see the following output:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python -m unittest <span class=\"nb\">test<\/span>\r\n<span class=\"go\">F.<\/span>\r\n<span class=\"go\">======================================================================<\/span>\r\n<span class=\"go\">FAIL: test_list_fraction (test.TestSum)<\/span>\r\n<span class=\"go\">----------------------------------------------------------------------<\/span>\r\n<span class=\"go\">Traceback (most recent call last):<\/span>\r\n<span class=\"go\">  File \"test.py\", line 21, in test_list_fraction<\/span>\r\n<span class=\"go\">    self.assertEqual(result, 1)<\/span>\r\n<span class=\"go\">AssertionError: Fraction(9, 10) != 1<\/span>\r\n\r\n<span class=\"go\">----------------------------------------------------------------------<\/span>\r\n<span class=\"go\">Ran 2 tests in 0.001s<\/span>\r\n\r\n<span class=\"go\">FAILED (failures=1)<\/span>\r\n<\/pre>\n<\/div>\n<p>In the output, you\u2019ll see the following information:<\/p>\n<ol>\n<li>The first line shows the execution results of all the tests, one failed (<code>F<\/code>) and one passed (<code>.<\/code>).<\/li>\n<li>The\u00a0<code>FAIL<\/code>\u00a0entry shows some details about the failed test:\n<ul>\n<li>The test method name (<code>test_list_fraction<\/code>)<\/li>\n<li>The test module (<code>test<\/code>) and the test case (<code>TestSum<\/code>)<\/li>\n<li>A traceback to the failing line<\/li>\n<li>The details of the assertion with the expected result (<code>1<\/code>) and the actual result (<code>Fraction(9, 10)<\/code>)<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>Remember, you can add extra information to the test output by adding the\u00a0<code>-v<\/code>\u00a0flag to the\u00a0<code>python -m unittest<\/code>\u00a0command.<\/p>\n<h3 id=\"running-your-tests-from-pycharm\">Running Your Tests From PyCharm<\/h3>\n<p>If you\u2019re using the PyCharm IDE, you can run\u00a0<code>unittest<\/code>\u00a0or\u00a0<code>pytest<\/code>\u00a0by following these steps:<\/p>\n<ol>\n<li>In the Project tool window, select the\u00a0<code>tests<\/code>\u00a0directory.<\/li>\n<li>On the context menu, choose the run command for\u00a0<code>unittest<\/code>. For example, choose\u00a0<em>Run \u2018Unittests in my Tests\u2026\u2019<\/em>.<\/li>\n<\/ol>\n<p>This will execute\u00a0<code>unittest<\/code>\u00a0in a test window and give you the results within PyCharm:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/files.realpython.com\/media\/py_run_test_folder.b0e61b478c81.png?ssl=1\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" loading=\"lazy\" class=\"img-fluid mx-auto d-block w-75\" src=\"https:\/\/i0.wp.com\/rosetta.vn\/short\/wp-content\/uploads\/sites\/3\/2019\/06\/py_run_test_folder.b0e61b478c81.png?resize=744%2C232&#038;ssl=1\" alt=\"PyCharm Testing\" width=\"744\" height=\"232\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>More information is available on the\u00a0<a href=\"https:\/\/www.jetbrains.com\/help\/pycharm\/performing-tests.html\">PyCharm Website<\/a>.<\/p>\n<h3 id=\"running-your-tests-from-visual-studio-code\">Running Your Tests From Visual Studio Code<\/h3>\n<p>If you\u2019re using the Microsoft Visual Studio Code IDE, support for\u00a0<code>unittest<\/code>,\u00a0<code>nose<\/code>, and\u00a0<code>pytest<\/code>\u00a0execution is built into the Python plugin.<\/p>\n<p>If you have the Python plugin installed, you can set up the configuration of your tests by opening the Command Palette with\u00a0<span class=\"keys\"><kbd class=\"key-control\">Ctrl<\/kbd>+<kbd class=\"key-shift\">Shift<\/kbd>+<kbd class=\"key-p\">P<\/kbd><\/span>\u00a0and typing \u201cPython test\u201d. You will see a range of options:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/files.realpython.com\/media\/vscode-test-capture.dfefa1d20789.PNG?ssl=1\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" loading=\"lazy\" class=\"img-fluid mx-auto d-block w-100\" src=\"https:\/\/i0.wp.com\/rosetta.vn\/short\/wp-content\/uploads\/sites\/3\/2019\/06\/vscode-test-capture.dfefa1d20789.png?resize=750%2C302&#038;ssl=1\" alt=\"Visual Studio Code Step 1\" width=\"750\" height=\"302\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>Choose\u00a0<em>Debug All Unit Tests<\/em>, and VSCode will then raise a prompt to configure the test framework. Click on the cog to select the test runner (<code>unittest<\/code>) and the home directory (<code>.<\/code>).<\/p>\n<p>Once this is set up, you will see the status of your tests at the bottom of the window, and you can quickly access the test logs and run the tests again by clicking on these icons:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/files.realpython.com\/media\/vscode-test-results.951be75c3d3b.PNG?ssl=1\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" loading=\"lazy\" class=\"img-fluid mx-auto d-block w-50\" src=\"https:\/\/i0.wp.com\/rosetta.vn\/short\/wp-content\/uploads\/sites\/3\/2019\/06\/vscode-test-results.951be75c3d3b.png?resize=483%2C112&#038;ssl=1\" alt=\"Visual Studio Code Step 2\" width=\"483\" height=\"112\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>This shows the tests are executing, but some of them are failing.<\/p>\n<h2 id=\"testing-for-web-frameworks-like-django-and-flask\">Testing for Web Frameworks Like Django and Flask<\/h2>\n<p>If you\u2019re writing tests for a web application using one of the popular frameworks like Django or Flask, there are some important differences in the way you write and run the tests.<\/p>\n<h3 id=\"why-theyre-different-from-other-applications\">Why They\u2019re Different From Other Applications<\/h3>\n<p>Think of all the code you\u2019re going to be testing in a web application. The routes, views, and models all require lots of imports and knowledge about the frameworks being used.<\/p>\n<p>This is similar to the car test at the beginning of the tutorial: you have to start up the car\u2019s computer before you can run a simple test like checking the lights.<\/p>\n<p>Django and Flask both make this easy for you by providing a test framework based on\u00a0<code>unittest<\/code>. You can continue writing tests in the way you\u2019ve been learning but execute them slightly differently.<\/p>\n<h3 id=\"how-to-use-the-django-test-runner\">How to Use the Django Test Runner<\/h3>\n<p>The Django\u00a0<code>startapp<\/code>\u00a0template will have created a\u00a0<code>tests.py<\/code>\u00a0file inside your application directory. If you don\u2019t have that already, you can create it with the following contents:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">from<\/span> <span class=\"nn\">django.test<\/span> <span class=\"k\">import<\/span> <span class=\"n\">TestCase<\/span>\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">MyTestCase<\/span><span class=\"p\">(<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"c1\"># Your test methods<\/span>\r\n<\/pre>\n<\/div>\n<p>The major difference with the examples so far is that you need to inherit from the\u00a0<code>django.test.TestCase<\/code>\u00a0instead of\u00a0<code>unittest.TestCase<\/code>. These classes have the same API, but the Django\u00a0<code>TestCase<\/code>\u00a0class sets up all the required state to test.<\/p>\n<p>To execute your test suite, instead of using\u00a0<code>unittest<\/code>\u00a0at the command line, you use\u00a0<code>manage.py test<\/code>:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python manage.py <span class=\"nb\">test<\/span>\r\n<\/pre>\n<\/div>\n<p>If you want multiple test files, replace\u00a0<code>tests.py<\/code>\u00a0with a folder called\u00a0<code>tests<\/code>, insert an empty file inside called\u00a0<code>__init__.py<\/code>, and create your\u00a0<code>test_*.py<\/code>\u00a0files. Django will discover and execute these.<\/p>\n<p>More information is available at the\u00a0<a href=\"https:\/\/docs.djangoproject.com\/en\/2.1\/topics\/testing\/overview\/\">Django Documentation Website<\/a>.<\/p>\n<h3 id=\"how-to-use-unittest-and-flask\">How to Use\u00a0<code>unittest<\/code>\u00a0and Flask<\/h3>\n<p>Flask requires that the app be imported and then set in test mode. You can instantiate a test client and use the test client to make requests to any routes in your application.<\/p>\n<p>All of the test client instantiation is done in the\u00a0<code>setUp<\/code>\u00a0method of your test case. In the following example,\u00a0<code>my_app<\/code>\u00a0is the name of the application. Don\u2019t worry if you don\u2019t know what\u00a0<code>setUp<\/code>\u00a0does. You\u2019ll learn about that in the\u00a0<a href=\"https:\/\/realpython.com\/python-testing\/#more-advanced-testing-scenarios\">More Advanced Testing Scenarios<\/a>\u00a0section.<\/p>\n<p>The code within your test file should look like this:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">import<\/span> <span class=\"nn\">my_app<\/span>\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">unittest<\/span>\r\n\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">MyTestCase<\/span><span class=\"p\">(<\/span><span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">setUp<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"n\">my_app<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">testing<\/span> <span class=\"o\">=<\/span> <span class=\"kc\">True<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span> <span class=\"o\">=<\/span> <span class=\"n\">my_app<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">test_client<\/span><span class=\"p\">()<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_home<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">get<\/span><span class=\"p\">(<\/span><span class=\"s1\">'\/'<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"c1\"># Make your assertions<\/span>\r\n<\/pre>\n<\/div>\n<p>You can then execute the test cases using the\u00a0<code>python -m unittest discover<\/code>\u00a0command.<\/p>\n<p>More information is available at the\u00a0<a href=\"http:\/\/flask.pocoo.org\/docs\/0.12\/testing\/\">Flask Documentation Website<\/a>.<\/p>\n<h2 id=\"more-advanced-testing-scenarios\">More Advanced Testing Scenarios<\/h2>\n<p>Before you step into creating tests for your application, remember the three basic steps of every test:<\/p>\n<ol>\n<li>Create your inputs<\/li>\n<li>Execute the code, capturing the output<\/li>\n<li>Compare the output with an expected result<\/li>\n<\/ol>\n<p>It\u2019s not always as easy as creating a static value for the input like a string or a number. Sometimes, your application will require an instance of a class or a context. What do you do then?<\/p>\n<p>The data that you create as an input is known as a\u00a0<strong>fixture<\/strong>. It\u2019s common practice to create fixtures and reuse them.<\/p>\n<p>If you\u2019re running the same test and passing different values each time and expecting the same result, this is known as\u00a0<strong>parameterization<\/strong>.<\/p>\n<h3 id=\"handling-expected-failures\">Handling Expected Failures<\/h3>\n<p>Earlier, when you made a list of scenarios to test\u00a0<code>sum()<\/code>, a question came up: What happens when you provide it with a bad value, such as a single integer or a string?<\/p>\n<p>In this case, you would expect\u00a0<code>sum()<\/code>\u00a0to throw an error. When it does throw an error, that would cause the test to fail.<\/p>\n<p>There\u2019s a special way to handle expected errors. You can use\u00a0<code>.assertRaises()<\/code>\u00a0as a context-manager, then inside the\u00a0<code>with<\/code>\u00a0block execute the test steps:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">import<\/span> <span class=\"nn\">unittest<\/span>\r\n\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">my_sum<\/span> <span class=\"k\">import<\/span> <span class=\"nb\">sum<\/span>\r\n\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">TestSum<\/span><span class=\"p\">(<\/span><span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_list_int<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"sd\">\"\"\"<\/span>\r\n<span class=\"sd\">        Test that it can sum a list of integers<\/span>\r\n<span class=\"sd\">        \"\"\"<\/span>\r\n        <span class=\"n\">data<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">]<\/span>\r\n        <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">result<\/span><span class=\"p\">,<\/span> <span class=\"mi\">6<\/span><span class=\"p\">)<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_list_fraction<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"sd\">\"\"\"<\/span>\r\n<span class=\"sd\">        Test that it can sum a list of fractions<\/span>\r\n<span class=\"sd\">        \"\"\"<\/span>\r\n        <span class=\"n\">data<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"n\">Fraction<\/span><span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">4<\/span><span class=\"p\">),<\/span> <span class=\"n\">Fraction<\/span><span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">4<\/span><span class=\"p\">),<\/span> <span class=\"n\">Fraction<\/span><span class=\"p\">(<\/span><span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"mi\">5<\/span><span class=\"p\">)]<\/span>\r\n        <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">result<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">)<\/span>\r\n\r\n<span class=\"hll\">    <span class=\"k\">def<\/span> <span class=\"nf\">test_bad_type<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n<\/span><span class=\"hll\">        <span class=\"n\">data<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">\"banana\"<\/span>\r\n<\/span><span class=\"hll\">        <span class=\"k\">with<\/span> <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertRaises<\/span><span class=\"p\">(<\/span><span class=\"ne\">TypeError<\/span><span class=\"p\">):<\/span>\r\n<\/span><span class=\"hll\">            <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">)<\/span>\r\n<\/span>\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'__main__'<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">main<\/span><span class=\"p\">()<\/span>\r\n<\/pre>\n<\/div>\n<p>This test case will now only pass if\u00a0<code>sum(data)<\/code>\u00a0raises a\u00a0<code>TypeError<\/code>. You can replace\u00a0<code>TypeError<\/code>\u00a0with any exception type you choose.<\/p>\n<h3 id=\"isolating-behaviors-in-your-application\">Isolating Behaviors in Your Application<\/h3>\n<p>Earlier in the tutorial, you learned what a side effect is. Side effects make unit testing harder since, each time a test is run, it might give a different result, or even worse, one test could impact the state of the application and cause another test to fail!<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/files.realpython.com\/media\/YXhT6fA.d277d5317026.gif?ssl=1\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" loading=\"lazy\" class=\"img-fluid mx-auto d-block w-80\" src=\"https:\/\/i0.wp.com\/rosetta.vn\/short\/wp-content\/uploads\/sites\/3\/2019\/06\/YXhT6fA.d277d5317026.gif?resize=480%2C270&#038;ssl=1\" alt=\"Testing Side Effects\" width=\"480\" height=\"270\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>There are some simple techniques you can use to test parts of your application that have many side effects:<\/p>\n<ul>\n<li>Refactoring code to follow the Single Responsibility Principle<\/li>\n<li>Mocking out any method or function calls to remove side effects<\/li>\n<li>Using integration testing instead of unit testing for this piece of the application<\/li>\n<\/ul>\n<p>If you\u2019re not familiar with mocking, see\u00a0<a href=\"https:\/\/realpython.com\/python-cli-testing\/#mocks\">Python CLI Testing<\/a>\u00a0for some great examples.<\/p>\n<h3 id=\"writing-integration-tests\">Writing Integration Tests<\/h3>\n<p>So far, you\u2019ve been learning mainly about unit testing. Unit testing is a great way to build predictable and stable code. But at the end of the day, your application needs to work when it starts!<\/p>\n<p>Integration testing is the testing of multiple components of the application to check that they work together. Integration testing might require acting like a consumer or user of the application by:<\/p>\n<ul>\n<li>Calling an HTTP REST API<\/li>\n<li>Calling a Python API<\/li>\n<li>Calling a web service<\/li>\n<li>Running a command line<\/li>\n<\/ul>\n<p>Each of these types of integration tests can be written in the same way as a unit test, following the Input, Execute, and Assert pattern. The most significant difference is that integration tests are checking more components at once and therefore will have more side effects than a unit test. Also, integration tests will require more fixtures to be in place, like a database, a network socket, or a configuration file.<\/p>\n<p>This is why it\u2019s good practice to separate your unit tests and your integration tests. The creation of fixtures required for an integration like a test database and the test cases themselves often take a lot longer to execute than unit tests, so you may only want to run integration tests before you push to production instead of once on every commit.<\/p>\n<p>A simple way to separate unit and integration tests is simply to put them in different folders:<\/p>\n<div class=\"highlight\">\n<pre>project\/\r\n\u2502\r\n\u251c\u2500\u2500 my_app\/\r\n\u2502   \u2514\u2500\u2500 __init__.py\r\n\u2502\r\n\u2514\u2500\u2500 tests\/\r\n    |\r\n    \u251c\u2500\u2500 unit\/\r\n    |   \u251c\u2500\u2500 __init__.py\r\n    |   \u2514\u2500\u2500 test_sum.py\r\n    |\r\n    \u2514\u2500\u2500 integration\/\r\n        \u251c\u2500\u2500 __init__.py\r\n        \u2514\u2500\u2500 test_integration.py\r\n<\/pre>\n<\/div>\n<p>There are many ways to execute only a select group of tests. The specify source directory flag,\u00a0<code>-s<\/code>, can be added to\u00a0<code>unittest discover<\/code>\u00a0with the path containing the tests:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> python -m unittest discover -s tests\/integration\r\n<\/pre>\n<\/div>\n<p><code>unittest<\/code>\u00a0will have given you the results of all the tests within the\u00a0<code>tests\/integration<\/code>\u00a0directory.<\/p>\n<h3 id=\"testing-data-driven-applications\">Testing Data-Driven Applications<\/h3>\n<p>Many integration tests will require backend data like a database to exist with certain values. For example, you might want to have a test that checks that the application displays correctly with more than 100 customers in the database, or the order page works even if the product names are displayed in Japanese.<\/p>\n<p>These types of integration tests will depend on different test fixtures to make sure they are repeatable and predictable.<\/p>\n<p>A good technique to use is to store the test data in a folder within your integration testing folder called\u00a0<code>fixtures<\/code>\u00a0to indicate that it contains test data. Then, within your tests, you can load the data and run the test.<\/p>\n<p>Here\u2019s an example of that structure if the data consisted of JSON files:<\/p>\n<div class=\"highlight\">\n<pre>project\/\r\n\u2502\r\n\u251c\u2500\u2500 my_app\/\r\n\u2502   \u2514\u2500\u2500 __init__.py\r\n\u2502\r\n\u2514\u2500\u2500 tests\/\r\n    |\r\n    \u2514\u2500\u2500 unit\/\r\n    |   \u251c\u2500\u2500 __init__.py\r\n    |   \u2514\u2500\u2500 test_sum.py\r\n    |\r\n    \u2514\u2500\u2500 integration\/\r\n        |\r\n        \u251c\u2500\u2500 fixtures\/\r\n        |   \u251c\u2500\u2500 test_basic.json\r\n        |   \u2514\u2500\u2500 test_complex.json\r\n        |\r\n        \u251c\u2500\u2500 __init__.py\r\n        \u2514\u2500\u2500 test_integration.py\r\n<\/pre>\n<\/div>\n<p>Within your test case, you can use the\u00a0<code>.setUp()<\/code>\u00a0method to load the test data from a fixture file in a known path and execute many tests against that test data. Remember you can have multiple test cases in a single Python file, and the\u00a0<code>unittest<\/code>\u00a0discovery will execute both. You can have one test case for each set of test data:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"kn\">import<\/span> <span class=\"nn\">unittest<\/span>\r\n\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">TestBasic<\/span><span class=\"p\">(<\/span><span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">setUp<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"c1\"># Load test data<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span> <span class=\"o\">=<\/span> <span class=\"n\">App<\/span><span class=\"p\">(<\/span><span class=\"n\">database<\/span><span class=\"o\">=<\/span><span class=\"s1\">'fixtures\/test_basic.json'<\/span><span class=\"p\">)<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_customer_count<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">customers<\/span><span class=\"p\">),<\/span> <span class=\"mi\">100<\/span><span class=\"p\">)<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_existence_of_customer<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"n\">customer<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">get_customer<\/span><span class=\"p\">(<\/span><span class=\"nb\">id<\/span><span class=\"o\">=<\/span><span class=\"mi\">10<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">customer<\/span><span class=\"o\">.<\/span><span class=\"n\">name<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"Org XYZ\"<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">customer<\/span><span class=\"o\">.<\/span><span class=\"n\">address<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"10 Red Road, Reading\"<\/span><span class=\"p\">)<\/span>\r\n\r\n\r\n<span class=\"k\">class<\/span> <span class=\"nc\">TestComplexData<\/span><span class=\"p\">(<\/span><span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">TestCase<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">setUp<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"c1\"># load test data<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span> <span class=\"o\">=<\/span> <span class=\"n\">App<\/span><span class=\"p\">(<\/span><span class=\"n\">database<\/span><span class=\"o\">=<\/span><span class=\"s1\">'fixtures\/test_complex.json'<\/span><span class=\"p\">)<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_customer_count<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">customers<\/span><span class=\"p\">),<\/span> <span class=\"mi\">10000<\/span><span class=\"p\">)<\/span>\r\n\r\n    <span class=\"k\">def<\/span> <span class=\"nf\">test_existence_of_customer<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\r\n        <span class=\"n\">customer<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">get_customer<\/span><span class=\"p\">(<\/span><span class=\"nb\">id<\/span><span class=\"o\">=<\/span><span class=\"mi\">9999<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">customer<\/span><span class=\"o\">.<\/span><span class=\"n\">name<\/span><span class=\"p\">,<\/span> <span class=\"sa\">u<\/span><span class=\"s2\">\"\u30d0\u30ca\u30ca\"<\/span><span class=\"p\">)<\/span>\r\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">assertEqual<\/span><span class=\"p\">(<\/span><span class=\"n\">customer<\/span><span class=\"o\">.<\/span><span class=\"n\">address<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"10 Red Road, Akihabara, Tokyo\"<\/span><span class=\"p\">)<\/span>\r\n\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'__main__'<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"n\">unittest<\/span><span class=\"o\">.<\/span><span class=\"n\">main<\/span><span class=\"p\">()<\/span>\r\n<\/pre>\n<\/div>\n<p>If your application depends on data from a remote location, like a remote API, you\u2019ll want to ensure your tests are repeatable. Having your tests fail because the API is offline or there is a connectivity issue could slow down development. In these types of situations, it is best practice to store remote fixtures locally so they can be recalled and sent to the application.<\/p>\n<p>The\u00a0<code>requests<\/code>\u00a0library has a complimentary package called\u00a0<code>responses<\/code>\u00a0that gives you ways to create response fixtures and save them in your test folders. Find out more\u00a0<a href=\"https:\/\/github.com\/getsentry\/responses\">on their GitHub Page<\/a>.<\/p>\n<h2 id=\"testing-in-multiple-environments\">Testing in Multiple Environments<\/h2>\n<p>So far, you\u2019ve been testing against a single version of Python using a virtual environment with a specific set of dependencies. You might want to check that your application works on multiple versions of Python, or multiple versions of a package. Tox is an application that automates testing in multiple environments.<\/p>\n<h3 id=\"installing-tox\">Installing Tox<\/h3>\n<p>Tox is available on PyPI as a package to install via\u00a0<code>pip<\/code>:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> pip install tox\r\n<\/pre>\n<\/div>\n<p>Now that you have Tox installed, it needs to be configured.<\/p>\n<h3 id=\"configuring-tox-for-your-dependencies\">Configuring Tox for Your Dependencies<\/h3>\n<p>Tox is configured via a configuration file in your project directory. The Tox configuration file contains the following:<\/p>\n<ul>\n<li>The command to run in order to execute tests<\/li>\n<li>Any additional packages required before executing<\/li>\n<li>The target Python versions to test against<\/li>\n<\/ul>\n<p>Instead of having to learn the Tox configuration syntax, you can get a head start by running the quickstart application:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> tox-quickstart\r\n<\/pre>\n<\/div>\n<p>The Tox configuration tool will ask you those questions and create a file similar to the following in\u00a0<code>tox.ini<\/code>:<\/p>\n<div class=\"highlight ini\">\n<pre><span class=\"k\">[tox]<\/span>\r\n<span class=\"na\">envlist<\/span> <span class=\"o\">=<\/span> <span class=\"s\">py27, py36<\/span>\r\n\r\n<span class=\"k\">[testenv]<\/span>\r\n<span class=\"na\">deps<\/span> <span class=\"o\">=<\/span>\r\n\r\n<span class=\"na\">commands<\/span> <span class=\"o\">=<\/span>\r\n<span class=\"s\">    python -m unittest discover<\/span>\r\n<\/pre>\n<\/div>\n<p>Before you can run Tox, it requires that you have a\u00a0<code>setup.py<\/code>\u00a0file in your application folder containing the steps to install your package. If you don\u2019t have one, you can follow\u00a0<a href=\"https:\/\/packaging.python.org\/tutorials\/packaging-projects\/#setup-py\">this guide<\/a>\u00a0on how to create a\u00a0<code>setup.py<\/code>before you continue.<\/p>\n<p>Alternatively, if your project is not for distribution on PyPI, you can skip this requirement by adding the following line in the\u00a0<code>tox.ini<\/code>\u00a0file under the\u00a0<code>[tox]<\/code>\u00a0heading:<\/p>\n<div class=\"highlight ini\">\n<pre><span class=\"k\">[tox]<\/span>\r\n<span class=\"na\">envlist<\/span> <span class=\"o\">=<\/span> <span class=\"s\">py27, py36<\/span>\r\n<span class=\"hll\"><span class=\"na\">skipsdist<\/span><span class=\"o\">=<\/span><span class=\"s\">True<\/span>\r\n<\/span><\/pre>\n<\/div>\n<p>If you don\u2019t create a\u00a0<code>setup.py<\/code>, and your application has some dependencies from PyPI, you\u2019ll need to specify those on a number of lines under the\u00a0<code>[testenv]<\/code>\u00a0section. For example, Django would require the following:<\/p>\n<div class=\"highlight ini\">\n<pre><span class=\"k\">[testenv]<\/span>\r\n<span class=\"na\">deps<\/span> <span class=\"o\">=<\/span> <span class=\"s\">django<\/span>\r\n<\/pre>\n<\/div>\n<p>Once you have completed that stage, you\u2019re ready to run the tests.<\/p>\n<p>You can now execute Tox, and it will create two virtual environments: one for Python 2.7 and one for Python 3.6. The Tox directory is called\u00a0<code>.tox\/<\/code>. Within the\u00a0<code>.tox\/<\/code>\u00a0directory, Tox will execute\u00a0<code>python -m unittest discover<\/code>\u00a0against each virtual environment.<\/p>\n<p>You can run this process by calling Tox at the command line:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> tox\r\n<\/pre>\n<\/div>\n<p>Tox will output the results of your tests against each environment. The first time it runs, Tox takes a little bit of time to create the virtual environments, but once it has, the second execution will be a lot faster.<\/p>\n<h3 id=\"executing-tox\">Executing Tox<\/h3>\n<p>The output of Tox is quite straightforward. It creates an environment for each version, installs your dependencies, and then runs the test commands.<\/p>\n<p>There are some additional command line options that are great to remember.<\/p>\n<p>Run only a single environment, such as Python 3.6:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> tox -e py36\r\n<\/pre>\n<\/div>\n<p>Recreate the virtual environments, in case your dependencies have changed or\u00a0<a href=\"https:\/\/docs.python.org\/3\/install\/#how-installation-works\">site-packages<\/a>\u00a0is corrupt:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> tox -r\r\n<\/pre>\n<\/div>\n<p>Run Tox with less verbose output:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> tox -q\r\n<\/pre>\n<\/div>\n<p>Running Tox with more verbose output:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> tox -v\r\n<\/pre>\n<\/div>\n<p>More information on Tox can be found at the\u00a0<a href=\"https:\/\/tox.readthedocs.io\/en\/latest\/\">Tox Documentation Website<\/a>.<\/p>\n<h2 id=\"automating-the-execution-of-your-tests\">Automating the Execution of Your Tests<\/h2>\n<p>So far, you have been executing the tests manually by running a command. There are some tools for executing tests automatically when you make changes and commit them to a source-control repository like Git. Automated testing tools are often known as CI\/CD tools, which stands for \u201cContinuous Integration\/Continuous Deployment.\u201d They can run your tests, compile and publish any applications, and even deploy them into production.<\/p>\n<p><a href=\"https:\/\/travis-ci.com\/\">Travis CI<\/a>\u00a0is one of many available CI (Continuous Integration) services available.<\/p>\n<p>Travis CI works nicely with Python, and now that you\u2019ve created all these tests, you can automate the execution of them in the cloud! Travis CI is free for any open-source projects on GitHub and GitLab and is available for a charge for private projects.<\/p>\n<p>To get started, login to the website and authenticate with your GitHub or GitLab credentials. Then create a file called\u00a0<code>.travis.yml<\/code>\u00a0with the following contents:<\/p>\n<div class=\"highlight yaml\">\n<pre><span class=\"nt\">language<\/span><span class=\"p\">:<\/span> <span class=\"l l-Scalar l-Scalar-Plain\">python<\/span>\r\n<span class=\"nt\">python<\/span><span class=\"p\">:<\/span>\r\n  <span class=\"p p-Indicator\">-<\/span> <span class=\"s\">\"2.7\"<\/span>\r\n  <span class=\"p p-Indicator\">-<\/span> <span class=\"s\">\"3.7\"<\/span>\r\n<span class=\"nt\">install<\/span><span class=\"p\">:<\/span>\r\n  <span class=\"p p-Indicator\">-<\/span> <span class=\"l l-Scalar l-Scalar-Plain\">pip install -r requirements.txt<\/span>\r\n<span class=\"nt\">script<\/span><span class=\"p\">:<\/span>\r\n  <span class=\"p p-Indicator\">-<\/span> <span class=\"l l-Scalar l-Scalar-Plain\">python -m unittest discover<\/span>\r\n<\/pre>\n<\/div>\n<p>This configuration instructs Travis CI to:<\/p>\n<ol>\n<li>Test against Python 2.7 and 3.7 (You can replace those versions with any you choose.)<\/li>\n<li>Install all the packages you list in\u00a0<code>requirements.txt<\/code>\u00a0(You should remove this section if you don\u2019t have any dependencies.)<\/li>\n<li>Run\u00a0<code>python -m unittest discover<\/code>\u00a0to run the tests<\/li>\n<\/ol>\n<p>Once you have committed and pushed this file, Travis CI will run these commands every time you push to your remote Git repository. You can check out the results on their website.<\/p>\n<h2 id=\"whats-next\">What\u2019s Next<\/h2>\n<p>Now that you\u2019ve learned how to create tests, execute them, include them in your project, and even execute them automatically, there are a few advanced techniques you might find handy as your test library grows.<\/p>\n<h3 id=\"introducing-linters-into-your-application\">Introducing Linters Into Your Application<\/h3>\n<p>Tox and Travis CI have configuration for a test command. The test command you have been using throughout this tutorial is\u00a0<code>python -m unittest discover<\/code>.<\/p>\n<p>You can provide one or many commands in all of these tools, and this option is there to enable you to add more tools that improve the quality of your application.<\/p>\n<p>One such type of application is called a linter. A linter will look at your code and comment on it. It could give you tips about mistakes you\u2019ve made, correct trailing spaces, and even predict bugs you may have introduced.<\/p>\n<p>For more information on linters, read the\u00a0<a href=\"https:\/\/realpython.com\/python-code-quality\/\">Python Code Quality tutorial<\/a>.<\/p>\n<h4 id=\"passive-linting-with-flake8\">Passive Linting With\u00a0<code>flake8<\/code><\/h4>\n<p>A popular linter that comments on the style of your code in relation to the\u00a0<a href=\"https:\/\/www.youtube.com\/watch?v=Hwckt4J96dI\">PEP 8<\/a>\u00a0specification is\u00a0<code>flake8<\/code>.<\/p>\n<p>You can install\u00a0<code>flake8<\/code>\u00a0using\u00a0<code>pip<\/code>:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> pip install flake8\r\n<\/pre>\n<\/div>\n<p>You can then run\u00a0<code>flake8<\/code>\u00a0over a single file, a folder, or a pattern:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> flake8 test.py\r\n<span class=\"go\">test.py:6:1: E302 expected 2 blank lines, found 1<\/span>\r\n<span class=\"go\">test.py:23:1: E305 expected 2 blank lines after class or function definition, found 1<\/span>\r\n<span class=\"go\">test.py:24:20: W292 no newline at end of file<\/span>\r\n<\/pre>\n<\/div>\n<p>You will see a list of errors and warnings for your code that\u00a0<code>flake8<\/code>\u00a0has found.<\/p>\n<p><code>flake8<\/code>\u00a0is configurable on the command line or inside a configuration file in your project. If you wanted to ignore certain rules, like\u00a0<code>E305<\/code>\u00a0shown above, you can set them in the configuration.\u00a0<code>flake8<\/code>\u00a0will inspect a\u00a0<code>.flake8<\/code>\u00a0file in the project folder or a\u00a0<code>setup.cfg<\/code>\u00a0file. If you decided to use Tox, you can put the\u00a0<code>flake8<\/code>\u00a0configuration section inside\u00a0<code>tox.ini<\/code>.<\/p>\n<p>This example ignores the\u00a0<code>.git<\/code>\u00a0and\u00a0<code>__pycache__<\/code>\u00a0directories as well as the\u00a0<code>E305<\/code>\u00a0rule. Also, it sets the max line length to 90 instead of 80 characters. You will likely find that the default constraint of 79 characters for line-width is very limiting for tests, as they contain long method names, string literals with test values, and other pieces of data that can be longer. It is common to set the line length for tests to up to 120 characters:<\/p>\n<div class=\"highlight ini\">\n<pre><span class=\"k\">[flake8]<\/span>\r\n<span class=\"na\">ignore<\/span> <span class=\"o\">=<\/span> <span class=\"s\">E305<\/span>\r\n<span class=\"na\">exclude<\/span> <span class=\"o\">=<\/span> <span class=\"s\">.git,__pycache__<\/span>\r\n<span class=\"na\">max-line-length<\/span> <span class=\"o\">=<\/span> <span class=\"s\">90<\/span>\r\n<\/pre>\n<\/div>\n<p>Alternatively, you can provide these options on the command line:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> flake8 --ignore E305 --exclude .git,__pycache__ --max-line-length<span class=\"o\">=<\/span><span class=\"m\">90<\/span>\r\n<\/pre>\n<\/div>\n<p>A full list of configuration options is available on the\u00a0<a href=\"http:\/\/flake8.pycqa.org\/en\/latest\/user\/options.html\">Documentation Website<\/a>.<\/p>\n<p>You can now add\u00a0<code>flake8<\/code>\u00a0to your CI configuration. For Travis CI, this would look as follows:<\/p>\n<div class=\"highlight yaml\">\n<pre><span class=\"nt\">matrix<\/span><span class=\"p\">:<\/span>\r\n  <span class=\"nt\">include<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"p p-Indicator\">-<\/span> <span class=\"nt\">python<\/span><span class=\"p\">:<\/span> <span class=\"s\">\"2.7\"<\/span>\r\n      <span class=\"nt\">script<\/span><span class=\"p\">:<\/span> <span class=\"s\">\"flake8\"<\/span>\r\n<\/pre>\n<\/div>\n<p>Travis will read the configuration in\u00a0<code>.flake8<\/code>\u00a0and fail the build if any linting errors occur. Be sure to add the\u00a0<code>flake8<\/code>\u00a0dependency to your\u00a0<code>requirements.txt<\/code>\u00a0file.<\/p>\n<h4 id=\"aggressive-linting-with-a-code-formatter\">Aggressive Linting With a Code Formatter<\/h4>\n<p><code>flake8<\/code>\u00a0is a passive linter: it recommends changes, but you have to go and change the code. A more aggressive approach is a code formatter. Code formatters will change your code automatically to meet a collection of style and layout practices.<\/p>\n<p><code>black<\/code>\u00a0is a very unforgiving formatter. It doesn\u2019t have any configuration options, and it has a very specific style. This makes it great as a drop-in tool to put in your test pipeline.<\/p>\n<div class=\"alert alert-primary\" role=\"alert\">\n<p><strong>Note:<\/strong>\u00a0<code>black<\/code>\u00a0requires Python 3.6+.<\/p>\n<\/div>\n<p>You can install\u00a0<code>black<\/code>\u00a0via pip:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> pip install black\r\n<\/pre>\n<\/div>\n<p>Then to run\u00a0<code>black<\/code>\u00a0at the command line, provide the file or directory you want to format:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> black test.py\r\n<\/pre>\n<\/div>\n<h3 id=\"keeping-your-test-code-clean\">Keeping Your Test Code Clean<\/h3>\n<p>When writing tests, you may find that you end up copying and pasting code a lot more than you would in regular applications. Tests can be very repetitive at times, but that is by no means a reason to leave your code sloppy and hard to maintain.<\/p>\n<p>Over time, you will develop a lot of\u00a0<a href=\"https:\/\/martinfowler.com\/bliki\/TechnicalDebt.html\">technical debt<\/a>\u00a0in your test code, and if you have significant changes to your application that require changes to your tests, it can be a more cumbersome task than necessary because of the way you structured them.<\/p>\n<p>Try to follow the\u00a0<strong>DRY<\/strong>\u00a0principle when writing tests:\u00a0<strong>D<\/strong>on\u2019t\u00a0<strong>R<\/strong>epeat\u00a0<strong>Y<\/strong>ourself.<\/p>\n<p>Test fixtures and functions are a great way to produce test code that is easier to maintain. Also, readability counts. Consider deploying a linting tool like\u00a0<code>flake8<\/code>\u00a0over your test code:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> flake8 --max-line-length<span class=\"o\">=<\/span><span class=\"m\">120<\/span> tests\/\r\n<\/pre>\n<\/div>\n<h3 id=\"testing-for-performance-degradation-between-changes\">Testing for Performance Degradation Between Changes<\/h3>\n<p>There are many ways to benchmark code in Python. The standard library provides the\u00a0<code>timeit<\/code>\u00a0module, which can time functions a number of times and give you the distribution. This example will execute\u00a0<code>test()<\/code>\u00a0100 times and\u00a0<code>print()<\/code>\u00a0the output:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"k\">def<\/span> <span class=\"nf\">test<\/span><span class=\"p\">():<\/span>\r\n    <span class=\"c1\"># ... your code<\/span>\r\n\r\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'__main__'<\/span><span class=\"p\">:<\/span>\r\n    <span class=\"kn\">import<\/span> <span class=\"nn\">timeit<\/span>\r\n    <span class=\"nb\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">timeit<\/span><span class=\"o\">.<\/span><span class=\"n\">timeit<\/span><span class=\"p\">(<\/span><span class=\"s2\">\"test()\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">setup<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"from __main__ import test\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">number<\/span><span class=\"o\">=<\/span><span class=\"mi\">100<\/span><span class=\"p\">))<\/span>\r\n<\/pre>\n<\/div>\n<p>Another option, if you decided to use\u00a0<code>pytest<\/code>\u00a0as a test runner, is the\u00a0<code>pytest-benchmark<\/code>\u00a0plugin. This provides a\u00a0<code>pytest<\/code>\u00a0fixture called\u00a0<code>benchmark<\/code>. You can pass\u00a0<code>benchmark()<\/code>\u00a0any callable, and it will log the timing of the callable to the results of\u00a0<code>pytest<\/code>.<\/p>\n<p>You can install\u00a0<code>pytest-benchmark<\/code>\u00a0from PyPI using\u00a0<code>pip<\/code>:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> pip install pytest-benchmark\r\n<\/pre>\n<\/div>\n<p>Then, you can add a test that uses the fixture and passes the callable to be executed:<\/p>\n<div class=\"highlight python\">\n<pre><span class=\"k\">def<\/span> <span class=\"nf\">test_my_function<\/span><span class=\"p\">(<\/span><span class=\"n\">benchmark<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"n\">benchmark<\/span><span class=\"p\">(<\/span><span class=\"n\">test<\/span><span class=\"p\">)<\/span>\r\n<\/pre>\n<\/div>\n<p>Execution of\u00a0<code>pytest<\/code>\u00a0will now give you benchmark results:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/files.realpython.com\/media\/pytest-bench-screen.6d83bffe8e21.png?ssl=1\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" loading=\"lazy\" class=\"img-fluid mx-auto d-block w-80\" src=\"https:\/\/i0.wp.com\/rosetta.vn\/short\/wp-content\/uploads\/sites\/3\/2019\/06\/pytest-bench-screen.6d83bffe8e21.png?resize=750%2C238&#038;ssl=1\" alt=\"Pytest benchmark screenshot\" width=\"750\" height=\"238\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>More information is available at the\u00a0<a href=\"https:\/\/pytest-benchmark.readthedocs.io\/en\/latest\/\">Documentation Website<\/a>.<\/p>\n<h3 id=\"testing-for-security-flaws-in-your-application\">Testing for Security Flaws in Your Application<\/h3>\n<p>Another test you will want to run on your application is checking for common security mistakes or vulnerabilities.<\/p>\n<p>You can install\u00a0<code>bandit<\/code>\u00a0from PyPI using\u00a0<code>pip<\/code>:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> pip install bandit\r\n<\/pre>\n<\/div>\n<p>You can then pass the name of your application module with the\u00a0<code>-r<\/code>\u00a0flag, and it will give you a summary:<\/p>\n<div class=\"highlight sh\">\n<pre><span class=\"gp\">$<\/span> bandit -r my_sum\r\n<span class=\"go\">[main]  INFO    profile include tests: None<\/span>\r\n<span class=\"go\">[main]  INFO    profile exclude tests: None<\/span>\r\n<span class=\"go\">[main]  INFO    cli include tests: None<\/span>\r\n<span class=\"go\">[main]  INFO    cli exclude tests: None<\/span>\r\n<span class=\"go\">[main]  INFO    running on Python 3.5.2<\/span>\r\n<span class=\"go\">Run started:2018-10-08 00:35:02.669550<\/span>\r\n\r\n<span class=\"go\">Test results:<\/span>\r\n<span class=\"go\">        No issues identified.<\/span>\r\n\r\n<span class=\"go\">Code scanned:<\/span>\r\n<span class=\"go\">        Total lines of code: 5<\/span>\r\n<span class=\"go\">        Total lines skipped (#nosec): 0<\/span>\r\n\r\n<span class=\"go\">Run metrics:<\/span>\r\n<span class=\"go\">        Total issues (by severity):<\/span>\r\n<span class=\"go\">                Undefined: 0.0<\/span>\r\n<span class=\"go\">                Low: 0.0<\/span>\r\n<span class=\"go\">                Medium: 0.0<\/span>\r\n<span class=\"go\">                High: 0.0<\/span>\r\n<span class=\"go\">        Total issues (by confidence):<\/span>\r\n<span class=\"go\">                Undefined: 0.0<\/span>\r\n<span class=\"go\">                Low: 0.0<\/span>\r\n<span class=\"go\">                Medium: 0.0<\/span>\r\n<span class=\"go\">                High: 0.0<\/span>\r\n<span class=\"go\">Files skipped (0):<\/span>\r\n<\/pre>\n<\/div>\n<p>As with\u00a0<code>flake8<\/code>, the rules that\u00a0<code>bandit<\/code>\u00a0flags are configurable, and if there are any you wish to ignore, you can add the following section to your\u00a0<code>setup.cfg<\/code>\u00a0file with the options:<\/p>\n<div class=\"highlight ini\">\n<pre><span class=\"k\">[bandit]<\/span>\r\n<span class=\"na\">exclude: \/test<\/span>\r\n<span class=\"na\">tests: B101,B102,B301<\/span>\r\n<\/pre>\n<\/div>\n<p>More details are available at the\u00a0<a href=\"https:\/\/github.com\/PyCQA\/bandit\">GitHub Website<\/a>.<\/p>\n<h2 id=\"conclusion\">Conclusion<\/h2>\n<p>Python has made testing accessible by building in the commands and libraries you need to validate that your applications work as designed. Getting started with testing in Python needn\u2019t be complicated: you can use\u00a0<code>unittest<\/code>\u00a0and write small, maintainable methods to validate your code.<\/p>\n<p>As you learn more about testing and your application grows, you can consider switching to one of the other test frameworks, like\u00a0<code>pytest<\/code>, and start to leverage more advanced features.<\/p>\n<p>Thank you for reading. I hope you have a bug-free future with Python!<\/p>\n<div class=\"border rounded p-3 card mb-2\">\n<p class=\"mb-0\"><span class=\"badge badge-pill badge-success\"><i class=\"fa fa-play-circle\" aria-hidden=\"true\"><\/i>\u00a0Watch Now<\/span>\u00a0This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding:\u00a0<a class=\"stretched-link text-success\" href=\"https:\/\/realpython.com\/courses\/test-driven-development-pytest\/\"><strong>Test-Driven Development With PyTest<\/strong><\/a><\/p>\n<\/div>\n<\/blockquote>\n<p>Source: <em><a href=\"https:\/\/realpython.com\/python-testing\/\">Getting Started With Testing in Python \u2013 Real Python<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This tutorial is for anyone who has written a fantastic application in Python but hasn\u2019t yet written any tests. Testing in Python is a huge topic and can come with a lot of complexity, but it doesn\u2019t need to be<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false},"categories":[30,215],"tags":[1262,1261,1263,222,1260],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p8jhJx-v0","_links":{"self":[{"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/posts\/1922"}],"collection":[{"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/comments?post=1922"}],"version-history":[{"count":1,"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/posts\/1922\/revisions"}],"predecessor-version":[{"id":1928,"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/posts\/1922\/revisions\/1928"}],"wp:attachment":[{"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/media?parent=1922"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/categories?post=1922"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rosetta.vn\/short\/wp-json\/wp\/v2\/tags?post=1922"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}