Lol I literally only just figured out that they don't want actual Ruby objects (which doesn't make much sense). No idea how to make dot notation work for Ruby with hashes..

IIRC changing the sample tests will not trigger re-validation. Only changing the actual tests does (which is why it's locked after 500 solves).

Though I guess you should also blame me for stirring up the top 5 leaderboards ;-)

Still when you press Run Sample Tests the tests will not be meaningful so I shall take the path of least resistance. Plus, I'm also hesitant to republish and trigger revalidation/python 2/3 issues.

IMO its better to comment all sample tests (to give an example how test could be implemented) and leave the decision to write them or not to the user as it done in most katas.

There are comments in the sample tests that explain this.

# since these are hard coded, you should substitute with the current top 5

Sample tests are static, there is no network request in the sample tests. I can't update them dynamically without exposing a solution to the user so the user must update them manually or ignore them and use the real tests(Attempt button). The real tests aren't static so they do check against current leaderboard data, allowing you to pass if your code is correct. You can look at the test fixtures after you've passed.

Sample tests for ruby is invalid. Test are failing because open(url) performs the actual network request, getting the most recent data and tests is based on old data. Also if you try to submit solution (by pressing attempt button) even if sample test failing, it will be accepted (if it working properly), this is very confusing.

I think the solution should be to stub network requests or use hidden tests (i'm not sure however is it possible to hide the body of process_data_sol and build_leaderboard from user until he solves this kata).