why post that?

The main problem I see is that, if there is one single input for all users, there is no way to know if people submitting solutions have built their own solution or if they are just copy/pasting something found on the web (a lot of... users... post solutions to kata on CW, there is no chances this doesn't happen with this kind of kata too. Imo it will actually be worse if there is one single input for all users... :// ).

One way out of this would be something a bit like AoC: a specific input generated and registered for one user, then he gives back his answer once he's done.
Resetting the trainer would allow him to get a new input.
...problem being... this would need a rework of the way codewars' working :/

For me, I think the idea is interesting and at least worth a try. The fact that "there are no such problems on CW" is not a really good argument to me because the way you described it, it's possible to set up a kata in a way it provides the user with input, requirements, and is able to test for correctness of submitted solution. "Check if a binary number is divisible by [some fixed n]" kata are based on a very similar premise.

I am not sure about one thing though: the fact that user was able to solve one particular instance of the problem, does not really mean they managed to implement some nice, decent, general solution. Their solution might be able to solve the problem for one input, and might fail/return bad answers for almost every other one. I am not sure if limiting the input to only one data set is a good idea, but I am also not sure if it will be easy to create the kata in a way which would test for more than one such data set.

You'd also have to explicitly state that it's allowed/possible/expected to calculate the answer(s) offline.

I think that for such kind of kata, random tests would not really matter. The kata would anyway boil down to precalculating one, specific answer on the user's machine.

My main concern would be that this family of solutions can give different results depending on details of implementation, algorithm used, initialization parameters etc. I think (correct me if I am wrong) that you'd need datasets created in some very specific way to ensure that different solutions return consistent answers.

Or not?

Read a note about random tests: https://github.com/codewars/codewars.com/wiki/Tutorial%3A-Create-Your-First-Kata#write-good-tests-and-utilize-both-fixed-and-random-test-cases

Random tests are a reason why you cannot have outer calculations on Codewars.

yes, and same goes the other way around (because, hey, we have... "people" ranking everything 1 kyu too... XD )

I cannot tell about the rank, since I didn't do it. 2 kyu seems doubtful, tho. But that's an opinion out of the blue, so far...

Yes, I have countered the rank to get a more appropriate average. I believe this kata is kyu 2-3. I'm not sure the system can be improved by allowing people that have made several attempts to be able to vote, or to ignore outliers in votes. Perhaps a whole new array of issues might arise from it. But I would never publish such a difficult kata at 4-5.

I believe you forget one thing. Your kata is obviously not white, for sure. But if people can underrank, they can overrank too. Meaning those who ranked it 1 maybe do not actaully believe it's 1 and they are just compensating for what they believe is... stupid ranking?

Other than that, yes, the estimation of the difficulty of a kata will always have to deal with opinions, unfortunately.

All in all, what I mean is: yes, I didn't complete it yet (mostly by lack of motivation to dig more), but 4 seems roughly about right to me.

your vision of the problem is too much partial (talking about your stats). You're forgetting a lot of things:

• age of the kata
• for how long has it been out of beta?
• is the kata a "copycats attractor" or not? (for instance, something like 50% of the solutions to insane coloured triangles aren't original. Same for gerrymander and some others)
• ...

All of this is too much of a multifactorial problem to base your reasonning on those stats only.

The system IS flawed, yeah, but what you want to see here isn't what is actually there.

It implies that averages can be skewed by outliers.

Average rank is pretty fine: 4 - 5 kyu. Please see the recently published 4 kyus on the site.

(Well, seems you're fleeting with the topic. FYI : I'd solved that after you've fixed but it failed in grouping, check again, my solution is there)

Now, coming to topic: Why your rank estimate is 2 kyu?

Nope, I ain't changing the subject(just lookahead of 2-3 steps).

The fact so many can't solve is weighing against its difficulty rating and that is a flaw in the beta process

This assumes that there was a intended difficulty rating and with respect to that the deviation is stated.

Why you think it is 2 kyu?

Here are some example of 2 kyu katas:
Three dots
Assorted Rectangular Pieces Puzzle
Optical Character Recognition

Do you really think that solutions of your kata has complexity anywhere near these?

BigTest completed in 436 milliseconds.

I spotted the same optimization in Python, which eventually barely made me pass that one.