[u-u] On testing Re: jobs @ rakuten/kobo

[Anthony de Boer]

Ken Burtch wrote:
> * As Dave suggests, unit tests are often written to get the test 
> coverage green, rather than to ensure the program works as intended.  
> This is one of the risks of hiring someone to write tests separate from 
> the programmer, as the tester may not know what was the program is 
> supposed to do.

You'd better communicate the spec to the tester at least as clearly as
you tell the programmer!

Independently developing the tests from the spec is (in part) a separate
implementation that makes it much more likely that the code will pass
the tests only if everyone came to the same understanding of the spec.

Granted, sometimes it felt like the project manager was keeping the spec
a secret from the programmer too.  :-(

> * Some types of programming are cost-prohibitive to unit test, such as 
> features that are temporary or are in constant flux.  Such things may 
> not be worth unit testing.  Other features, like handling out-of-memory 
> or out-of-disk-space errors, may not be worth simulating because the 
> operating environment itself becomes unstable and unpredictable under 
> these conditions, and the test results may not mean much.

A past boss was adamant that our code was untestable because the human in
the loop was too dynamic a thing to replace.  But I think that running a
few captured inputs to the same outputs could have helped.  Certainly
it's a lot easier to test internal code running in the dark away from UI
code.  And if testing obliges you to write a clear internal "input"
interface that can take UI input or a test case and do all the processing
and deliver it to a clear "output" interface that can go to either an
output screen or a test-result capture, then the whole works may end up
cleaner as a result.

There's a good chance that a test harness should log (and perhaps also be
able to simulate) low-memory and low-disk situations.  No sense spending
hours debugging a crash or a broken output blind to the fact that the OOM
killer or a full disk were responsible.  And if the code had acquired too
much of the bloat nature, knowing that you need to put it on a diet
and/or specify more hardware is very useful.

> * TDD itself has many claims such as better design, faster development, 
> eliminating documentation (since the tests themselves theoretically 
> describe the project requirements) but I have not found these to be true.
> 
> In regards to documentation, in my last couple of jobs, the developers 
> argued that documentation slowed them down ...

I've heard good things about Documentation-Driven Development, where you
essentially write the manpage first, make sure it makes sense and
describes a useful thing, and only then start writing an implementation.

-- 
Anthony de Boer