How to Stop Hating Your Tests by Justin Sears
How to Stop Hating Your Tests on Vimeo
Isolation of tests
the better we isolate tests the better it communicates what the units job is
Structure
- Big objects are hard to work with. People who practice TDD tend to hate big objects, they are harder to deal with
Tests make big objects even harder do deal with (they have many logical branches, many dependencies, etc.)
Rule of product if you have a function that takes four arguments (
valid(a,b,c,d)) then the number of test cases = a ×b × c × d "I'll just add one more argument..." will double your test cases
suggests: "Stop the bleeding" don't add new stuff to your object
"Limit new object to 1 public method and at most 3 dependencies"
=> write lots of small units
Off Script
Tests can and should only do three things:
- Sets stuff up
- Invokes a thing
- Verifies behavior
- Arrange
- Act
- Assert
or
- Given
- When
- Then
always respect this order
Minimize each phase to 1 action per line
jasmine-given, mocha-given
let > given
before -> when
then -> expect
Test design smells:
- Lots of "Given" steps? -> Too many dependencies or too complex arguments
- Multiple "When" steps? -> The API is too complex or hard to invoke
- Many "Then" steps -> The code is doing too much
Hard to read, hard to skim code
"Logic in tests confuses the story of what's being tested"
Test code is untested code (might contain errors) and is hard to read
passing green is "fantasy green"
If you refactor your tests to make them more terse you make them more brittle and harder to understand
The Sim tests
uses context for logical branches
Arrange, Act, Assert should easy by found
Tests that are too magic (or not magic enough)
Balance between expressiveness (small or big API)
spec-given reduced the API to
given
when
Then
and
Invariant
natural assertions
smaller testing apis are easier to understand, but create bigger tests and more one-off helper methods that you will have to carry
bigger test apis yield terse tests, but look like magic to outsiders
Tests that are accidently creative (are very bad)
always uses the subject and result consistently
inconsistent test can have a message (only if 1 in 10, otherwise it becomes a mess)
Make unimportant test code obvious to the reader. Don't use real-life looking values, but words like "pants" - it has an important meaning to the reader
Test Isolation
Unfocused test suites
Defining success: Is the purpose of the test readily apparent & does its test sutie promote consistency
The testing pyramid
Some tests call to real db, some fake third parties, some use third-party apis, etc.etc.
Start with two suites, each approaching one extreme:
One test suite(A): as realistic as possible, as integrated as we can manage (top of the pyramid)
Other suite(B): as isolated as possible (bottom of the pyramid)
Recently worked on an Ember project.
Had to write "components tests". They established the rules:
- Fake all APIs
- no test doubles objects
- trigger actions, not UI events
- Verifiy app state, not HTML templates
it bought them consistency
Too realistic
The boundaries of what exactly is considered "realistic" is very implicit in many projects (this is not good). They test for some scenarios, but not EVERYTHING, so if something blows up in production, they reply by increasing realism even more.
some practical issues with realistic tests:
- are slower
- take more time to write/change/debug
- higher cognitive load
- fail for more reasons
Solution: have clear boundaries
Increases the focus on what's tested and what's controlled
Clear Boundaries:
if something blows up in production - they can analyse better the situation and just (as one solution) write one focused test for that scenario
"Less integrates tests offer more design feedback & failurs are easier to understand"
Redundant code coverage
You change one unit, but then you realise that you need to change all the surrounding units as well now. Kills team morale.
How to detect redundent coverage?
Look at your coverage tool and see the column that shows how many times a certain line was hit.
One approach: identify a clear set of layers to test through
or
Try at outside-in TDD, isolating units via test doubles (you isolate each of your units that stuff that are underneath it) <- called as London-School TD, or mockists, or discovery_testing
Careless mocking
Test double
a fake, a stub, a spy, a mock
His process (strategy) on using test doubles:
Given the subject of his test, that use three dependencies -> he fakes those dependencies and tests how easy/convienent is it to use their APIs.
does the data flow is sound -> it's still easy use because they actually don't exist
In real-life most people don't have this approach: they mimic/replace existing dependencies so that their tests will pass. They use mocks to "shut up" dependencies that are causing them pains
They make tests confusing to read
Application frameworks
Most problems (tasks) we face: "how do I make my application talk to X?" <- it's integration issue
Talks about different strategies people use when working with frameworks (integrate them at every line <-> try to dodge them)
The framework dilemma
- Frameworks focus on integration problems
- Framework test helpers are very integrated
- Users only write integration tests
If some of our code doesn't rely on the framework, why sould our tests?
Answer is: if you have a lot of domain-specific logic in your code, decouple the tests from the framework
Test feedback
Useless error messages
very wasteful, it adds a lot of friction and waste, you have to somewhow print out where the test fails exactly
"use assertion libraries based on the messages they give"
Feedback loops
Speed test is extremely important can cut your productivity into a fraction: if a feedback loop time increases by two -> you just halved your productivity (count in distractions, context switch, decide action)
The mythical 10x developer is found!!! :D (a 30second feedback loop vs. 480 second feedback loop)
Painful test data
versions
- inline (for testing modules)
- fixtures (for integration tests)
- data dump (smoke tests)
- self-priming tests (staging, production)
You don't have to pick just one version
Data setup seems the biggest contributor to slow tests ("citation needed" :) )
profile your slow tests
Linear build slow down
Lot of time in a test is acutally spent in either "app code" or "setup, teardown" and not in "test code"
So the total time is not linear, but can almost be exponential
"Early on set a firm cap on build duration - and then enforce it!" -> delete tests or make stuff faster when approaching the limit
False negative
In real life, true negatives are depressingly rare (when you catch a real bug)
False negatives erode confidence in our test
top causes:
- redundant coverage
- slow tests (it only fails in CI not in development)
ie. lots of integration tests
analyse this data