12 January 2009

Agnostic Debugging

One of the perverse pitfalls of doing computing research in emergent systems is that it is devilishly difficult to know if your system isn't working because your model is bad or because you just haven't programmed it correctly.

If you're writing a ray tracer you can run it and see it either trace rays or not. But when I run some of my neural networks I can't tell if they're spitting out gibberish because I made a mistake or because the model they're using is rubbish.* So that leaves me sitting here, left eyelid twitching, trying to figure out if my oscillating, temporally asynchronous Hopfield network doesn't work, or just isn't working yet.

* Or the third, and sometimes even more frustrating possibility: the models are good but you haven't staked out a claim to the right section of parameter space yet. Curses.

1 comment:

  1. I'm really feeling with you. I once programmed a whole general neural networking framework with most of the algorithms programmed by myself...same problems.