Some curmudgeonly advice for writing unit tests

I'm a seasoned developer who has successfully released code for many years without automated unit testing. The way I accomplished this was through frequent unautomated testing, during the development process.

In the past few years, automated unit testing has become very popular, to the point where it is sometimes viewed as a panacea, in much the same way that object-oriented-programming was embraced in the early 90s.

In my opinion, automated unit testing is incredibly useful, and is much preferable to not testing at all. However it can be (and probably often is) easily abused. It can create a false sense of security if the tests are poorly written, and it can waste more time than it saves if the tests are brittle.

It is probably a bad idea to substitute automated unit testing for unautomated or live testing (e.g. actually running and observing your production code). Automated testing should be done in addition to live tests.

Here are my suggestion on ways to make automated unit testing more robust.

1. Ideally, unit tests should break when code breaks. Ideally, they shouldn't break when code doesn't break.
2. A unit test should not be like a motion detector. It is not an alarm to detect that someone has merely modified code. It should detect broken code. If inserting an innocuous line in a method will cause a test to fail, the test is brittle and will waste developers time.
3. A unit test should test results, not how the results are obtained. Otherwise, the test is liable to break if the method of obtaining the result is changed to a different, but equally valid one.
4. A unit test should not care about the order or sequence in which actions are performed by the code under test, unless that order or sequence is truly important to obtaining the correct result. Otherwise, it functions as a motion detector and wastes time.
5. If the code in a unit test is three times larger than the code it is testing, than statistically, the unit test is three times more likely to be the cause of a testing failure than the code it is testing.
6. If a unit test reproduces functionality (rather than testing actual functionality) then it is useful as a proof of concept, but not as an automated test. Subsequently, if the actual code breaks, the unit test won't detect it, because it is not exercising the actual code.
7. A one line method, whose one line cannot actually be run during the unit test should not have a unit test written for it. See #5 and #6.
8. The statement "it is better to test than not to test" is only true if the tests are robust. A highly brittle or ineffectual unit test will be more expensive to the cost of a project, than no test at all.
9. The statement "Automated tests give you justified confidence in your code" assumes the tests are robust and valid. Unit tests are not a panacea. Brittle automated tests give you unjustified confidence in your code, and will likely waste more time than they save.