You are signed in. Close

You are signed out. Close

Thank you. Your subscription has started. Close

Thank you. You will get an email when a new comic comes out. Close

We could not confirm your payment. If you paid, use the link "Sign in" at the bottom of the page. Close

Something went wrong on our side. Please try again in a moment. Close

Previous episode Continuous integration: why merging every day hurts less than once a month

How CI/CD works: why releasing code every day is safer than once a month

Automated tests: how software checks itself after every change

A developer can check by hand that an app still works once or twice a day. A small program can check it hundreds of times in a few seconds. What an automated test really is, and what it can never prove.

  1. 1 To make sure the double booking of release day never comes back, you could open the app, book machine 3, and try to book it a second time. It takes a few minutes, after every merge, forever.youBook 3. Now book 3again. Refused? Good.4 minutes

    To make sure the double booking of release day never comes back, you could open the app, book machine 3, and try to book it a second time. It takes a few minutes, after every merge, forever.

  2. 2 So you write a few lines of code that do it for you. They book machine 3, try to book it again, and check that the app says no. This is an automated test.a testbookingrulemachine 3?machine 3 again?refused

    So you write a few lines of code that do it for you. They book machine 3, try to book it again, and check that the app says no. This is an automated test.

  3. 3 When the app refuses the second booking, the test passes. If a change ever brings the bug back, the test fails and says which check went wrong.testpasstestfailsecond bookingwas accepted

    When the app refuses the second booking, the test passes. If a change ever brings the bug back, the test fails and says which check went wrong.

  4. 4 The team writes a test for each thing the app must do: book, cancel, pay, send the message. Hundreds of them run in a few seconds. Each one checks only the thing it was written for.bookcancelpaymessagebookcancelpaymessagebookcancelpayno test here8 s

    The team writes a test for each thing the app must do: book, cancel, pay, send the message. Hundreds of them run in a few seconds. Each one checks only the thing it was written for.

  5. 5 There is a catch. The tests run only when someone runs them on their laptop, and on Friday, Rono was in a hurry and forgot.17:50testsRono

    There is a catch. The tests run only when someone runs them on their laptop, and on Friday, Rono was in a hurry and forgot.

  6. Quick check

    A change to the price.Rono changes theprice of a wash. Thedouble-booking teststill passes. Does thatprove his changehas no bug?

    A change to the price.

    Show the answer

    No. That test only checks that a second booking is refused. A bug in the price needs a test that checks the price.

  7. Go further

    More about tests.test suiteall the tests of an apptogether, like the rowsof the pictureunit testa test of one small piece,like the booking rule.Most of the hundredsno prooftests find many bugs,never all of them: only theones someone checks for

    More about tests.

Next episode The CI pipeline: what happens on a server each time you push code

Share this comic

More comics