Most entertainig part was to write a script that generates the solution

I was just talking that if bug can happen, increasing number of same tests, with random data set and values in same domain doesn't actually help that much to catch it. It can help in some cases, but one needs to be lucky :) What can help for sure is to add more edge-cases covering for tests.

So for example to catch bug with sum called w/o arguments we need to change tests so that they will call it w/o arguments :) The thing is that, again, we need to extend data domain, not it's quantity. Generating current random test-cases in for loop won't help to catch this at all.

I just don't like relying on random in tests. Suffered from this a lot during E2E and integration testing, due to their nature. You're talking about thing called "fuzz testing" (https://en.wikipedia.org/wiki/Fuzzing), and still it's more about picking random data domain, than random data in just in bigger quantities. This kind of tests doesn't guarantee 100% coverage of control flow. In practice I've learned a lesson that if thing can happen, but the probability is extremely low - it will happen anyway, just give it a time. So it doesn't makes sense to lower this probability, you need to eliminate it.

So it's not about amount of data, but about it's quality. Currently I test only against positive integers. My point was that increasing number of tests with current data will not reveal subtle bugs for sure, It'll only increase probability of this. And since it's random, user/tester can be lucky enough. On the other hand, putting "special" numbers like +/-Infinity and NaNs (also -0 :P) into the game will reveal some bugs in some solutions. That's a good argument. But also forcing coders to workaround edge-cases usually makes kata less enjoyable since solutiuns become ugly.

How about this: I'll put criteria into description that the sum function will be tested only against natural numbers and zero?

Agree on first point, but as for random testing: what's the profit of multiple random test-cases? As for me, the only thing random tests needed for, is to prevent "hard-code" solutions from passing. And for this purpose, single test with random dataset is completely sufficient.

Sorry guys, I spent a bit of time lurking on katas list and didn't found anything too similar to this. If you all think that it doesn't worth time spent on it (e.g. due to fact that it's a duplicate somehow), I certainly should remove it.

Lessons learned:

1. Do not post kata solution until you're absolutely confident with it;
2. Do not code at 5 AM;
3. Sleep more.

In JS version tokenizer can be improved a bit:

  var regex = /(?=\s*)([-+*/\(\)\[\]]|[A-Za-z]+|[0-9]+)(?=\s*)/g;
  return program.match(regex).map( function (tok) {
    return isNaN(tok) ? tok : tok|0;
  });

Note the (?=) group.

The kata is overrated. It should be 4kyu.

HAAAAAAAX!