Unit Testing Machine Learning Code

Question

I am writing a fairly complicated machine learning program for my thesis in computer vision. It's working fairly well, but I need to keep trying out new things out and adding new functionality. This is problematic because I sometimes introduce bugs when I am extending the code or trying to simplify an algorithm.

Clearly, the correct thing to do is to add unit tests, but it is not clear how to do this. Many components of my program produce a somewhat subjective answer, and I cannot automate sanity checks.

For example, I had some code that approximated a curve with a lower-resolution curve, so that I could do computationally intensive work on the lower-resolution curve. I accidentally introduced a bug into this code, and only found it through a painstaking search when my the results of my entire program got slightly worse.

But, when I tried to write a unit test for it, it was unclear what I should do. If I make a simple curve that has a clearly correct lower-resolution version, then I'm not really testing out everything that could go wrong. If I make a simple curve and then perturb the points slightly, my code starts producing different answers, even though this particular piece of code really seems to work fine now.

Answer 1

The job of unit testing is not meant to test everything that could go wrong.

Unit testing is to test the right thing, give specific inputs and specific expected results. The important part here is the specific visible, external requirements are satisfied by specific test cases. Not every possible thing in your code that could go wrong is somehow prevented.

Hope this answer helps.
 

If you wish to learn about Machine Learning visit this Machine Learning Course.