We do that for some of the more complex business logic. We wrote libraries, which are used by our tests, and we wrote tests which test the library functions to ensure they provide correct results.
What always worries me is that WE came up with that. It wasn’t some higher up, or business unit, or anything. Only because we cared to do our job correctly. If we didn’t - nobody would. Nobody is watching the testers (in my experience).
Mutation testing is quite cool. Basically it analyzes you code and makes changes that should break something. For example if you have if (foo) { ... } it will remove the branch or make the branch run every time. It then runs your tests and sees if anything fails. If the tests don’t fail then either you should add another test, or that code was truly dead and should be removed.
Of course this has lots of “false positives”. For example you may be checking if an allocation succeeded and don’t need to test if every possible allocation in your code fails, you trust that you can write if (!mem) abort() correctly.
Who tests the tests
The Testmen?
I’ve written some tests that got complex enough that I also wrote tests for the logic within the tests.
We do that for some of the more complex business logic. We wrote libraries, which are used by our tests, and we wrote tests which test the library functions to ensure they provide correct results.
What always worries me is that WE came up with that. It wasn’t some higher up, or business unit, or anything. Only because we cared to do our job correctly. If we didn’t - nobody would. Nobody is watching the testers (in my experience).
Create tests to test the tests. Create tests to test those. Recurse to infinity
Mutation testing is quite cool. Basically it analyzes you code and makes changes that should break something. For example if you have
if (foo) { ... }it will remove the branch or make the branch run every time. It then runs your tests and sees if anything fails. If the tests don’t fail then either you should add another test, or that code was truly dead and should be removed.Of course this has lots of “false positives”. For example you may be checking if an allocation succeeded and don’t need to test if every possible allocation in your code fails, you trust that you can write
if (!mem) abort()correctly.Right,too much coverage is also a bad thing. It leads to having to work on the silly tests every time you change som implementation detail.
Good tests let the insides of the unit change without breaking, as long as the behave the same to the outside world.
Who tests the tests for the tests
Unfortunately, if anyone, I do.