Great, thanks for your insights.

I am on a mission to achieve something here.. Firstly, working on the following:

• Crafting random tests && fuzz testing.
• Append many more test cases to cover all corner/edge cases, passing inputs with already pre-set or turned on switches to ensure the solution mutates the input properly and accurately. And also because the number of test cases provided is still very little. It was for experimental purposes and due to lack of time.

One small thing I wanted to share with you from beginning, is that the kata was initially designed to provided inputs as slices/arrays so that we can easily calculate the consumed space excluding the inputs sizes. But just converted them to strings instead, to simplify writing test cases for myself :D as fixed test cases mandate to write huge slices inputs while I wanted to start with fixed tests and delegate random tests for later.

Secondly to ensure the worst case scenarios for time & space complexities so we can stay informed about how much we exactly need to provide for the solution to consume. Then be able to restrict.

My guess about your trial allocating same amount of mem used by test, is that it's a global variable which is allocated once, while tests can run normaly with no relation to that globally scoped variable across the entire package. Additionally and more importantly, the solution you're proposing to fail the test is still using bitwise ops. Our mission/battle is to prevent against the solutions that don't utilize bitwise ops "allocate in-memory data structures to loop over and calculate the final result". I've already crafted one such a solution that should be an example of these kind of solutions we need to reject or be proof against.

Also, one more interesting phenomena about GoLang that I wanted you to consider by drawing your attention to it, is that stack sizes are flexible in Go (can grow & shrink as per the need at runtime). Since our problem is meant to be mainly utilizing stack as opposed to heap allocation, it becomes a bit more intricate in this particular language "if not all other langs" with no fixed stack size that starts very small (~2KiB) and then grows. Hence, the trial/attempt you proposed to fail tests succeed because the solution is stack-based and the allocation is package scoped not function scoped "lexically talking". We need to be fighting against solutions that allocate from within the stack.

Gimme a couple of days or a bit more to finish some busy work, then get back here to finish this and let ya know :)

Thanks @Mednoob once again for your participation. Really appreciated!

Sincerely & stay tuned 🙏,

Speaking of solutions that would fail memory-wise, Yeah any solution that would try to allocate data structures which consume memory should be sub-optimal and would fail most of the time. Still tryna figure out the percentage that might pass to prevent against them as well.

You sure? A solution that takes the same amount as the tests used passed:
https://www.codewars.com/kata/reviews/66ec5e683c2958327731cf6d/groups/66f273f3bb75c683a122b5e1

I guess that makes it 0% fail. I don't think these type of performance constraint can be done in testing environment like Codewars have. It just doesn't work. My suggestion would be to just remove the performance constraint and let the kata to only be just about switching bits.

Wonderful, thank you for the feedback and thoughtful insights. To the point, you're! Well played.

Could you please check now?
I've adjusted the performance test slightly to pick up the solutions that take care of un-necessary allocations and/or iterations.

Speaking of solutions that would fail memory-wise, Yeah any solution that would try to allocate data structures which consume memory should be sub-optimal and would fail most of the time. Still tryna figure out the percentage that might pass to prevent against them as well.

Look, let me explain to you my POV. Since beginning I wanted to state that this is supposed to be an embedded system. As you know I believe such systems are embedded into devices with few MB of memory. Hence, the constraint to come up with a solution that utilizes only few bytes, the right/required approach does that. Any other approach, allocating space to represent the states, altering them, and finally reflect on the variable that's supposed to be mutated should fail.

The test approach is weird of course, you nailed it. But the issue is that codewars is community driven, thus they can't allow authors to pre-determine the memory consumption per solution as it varies. Some other platforms allow such possibility.
This is why such a weird work-around I invented to make the resources virtually busy with some useless stuff, so the remaining space can be hardly sufficient for solutions that consider such a constraint. If some solution would play around with slices/arrays/maps to represent the switches states using booleans, numbers, etc, this would need more space than available, simply because the test already ate up most of the available memory to leave the least amount for the solution to run in a simulation of little space similar to the real world where the system should be running in production.

Well, now the kata is, eh... kinda interesting ig.

Eating up 14 gigs for performance constraint is pretty weird though. Have you tried making a solution that will fail memory-wise with this test approach?

By mutating the input, we can get a bit closer to the main purpose/goal of the kata, so the solution provider can train/excercise bitwise operations!

Does that make sense?

Guys, I've re-designed the kata for the sake of enforcing one approach, which was the initial purpose of the creating it in the first place.

Kindly review once again and lemme know your thoughts.

Guys, I've re-designed the kata for the sake of enforcing one approach, which was the initial purpose of the creating it in the first place.

Kindly review once again and lemme know your thoughts.

Yes, I got that just few hours ago that the 1.5MB limit is all about STDOUT Buffer Size instead of space complexity. My bad!

You're right, it seems we can't measure the space complexity per execution of the solution provided per each test case without a custom test cases implementation, which is something I'm working on currently, tryna find a way to achieve that! If something arises I am gonna share with you soon.

But have you solved it @hobovsky?

the 1.5MB refers to allowed max length of test output, and not amount of memory. I am not sure how you'd judge space complexity based on amount of information presented by tests.

Thanks for your valuable feedback!

May I ask a couple of questions to better understand your concerns & address them:

• Is it mandatory to have random tests?
• In the case of this particular kata, bigger inputs are not relevant unfortunately. I've already gone through the docs, read them carefully to grasp my head around the rules & best practices .. And yes it says what you say here, However I wondered how can bigger inputs fit here.
• The kata is tagged with performance for space complexity not for time complexity. An approach that exceeds the set Buffer size (1.5MiB) would result in Maximum Buffer Size consumed or similar error.

I really appreciate your review & constructive feedback, as I am trying to optimize and make it at its best.

• No random tests.
• Doing 12k tests is not ok. Performance constraint should be bigger inputs, not more tests.
• Why is this kata tagged with Performance? So far, I haven't seen any approach that would results in time out.

It seems the issue is marked as resolved, but I get the same problem:

Large random tests
Expected
: 18446744073709551615
to equal
: 15041117249284103333
Completed in 0.4287ms

Even the expected result set by test "15041117249284103333" is not a reverse number, is it?

All other test cases are passed!

Please let me know what's wrong.

You mean that 111,121 are considered reverse numbers, don't you?