Tests are subjectives

There are many motivations to writes tests:

• Think about the design, as it is done in TDD
• Provide examples
• Ensuring a bug is fixed for good, as it is done in regression tests
• Validating a feature respects some contract, as in acceptance tests
• Specifying edge cases

I have worked on a wide range of code bases, with various testing disciplines:

• Almost no tests
• Few automated tests, and manual testings
• A lot of unit tests, and one or two integration tests
• Some end-to-end tests
• A large set of contract tests

Non of them followed test desiderata, worse than this, none of them were enforcing test readability.

To give a motivational example, we can have a look at the following test:

describe "Game Of Life" $ do
  it "should be true when it is true and has two neighours" $
    gameOfLife True 2 `shouldBe` True

It brings no value, no meaning about the business is conveyed, and the worst part of it: it is the case for 99.99% of the tests I have seen in real-world projects.

In most teams I have worked, tests are mostly a social convention, we do it because everyone do it, hoping they will be useful for the long term maintenance.

Which leads to structurally deficient test-suits:

• Tests without assertions
• Long tests chaining unrelated assertions without a clear purpose
• Flaky or probabilistic tests which destroy the confidence we should gain with tests
• Highly coupled tests which slows subsequent code changes

And, despite huge efforts, bugs keep arising, because, tests do not prevent bugs, they simply prove execution paths exist.

The core issue is the lack of share intent behind tests, if team had, expectations could be managed.