Reduce stress on "do not use a reference solution"

content/authoring/guidelines/submission-tests.md

@@ -42,7 +42,8 @@ Random tests are uncommon in "real life" coding and are somewhat specific to Cod
 - **Random tests should generate test cases for all scenarios** which cannot be completely tested with fixed tests. If necessary, build different kinds of random input generators. If a specific kind of input has a very low chance of occurring purely at random (e.g. generating a palindrome), it's better to build a specific random generator that can enforce this kind of input rather than rely on 1000 random tests and just pray for the specific case to come up. Sometimes it can be a good idea to keep one fully random generator, because it may generate cases you didn't think about.
 - **Random tests should ensure that it's infeasible to pass tests by counting test cases.** Cases shouldn't be grouped by output type or behavior, especially if the expected output is a boolean variable (e.g. checking that some input satisfies some criteria), or when it comes to error checking (solution throwing an exception in some specific situations). The order of tested scenarios should be unpredictable. One possible way to achieve this is to generate and collect a set of random inputs for all required scenarios and shuffle them before the actual testing. If there are some fixed tests for particularly tricky scenarios which can be skipped by counting, they should be shuffled into the set of random inputs.
 - **Keep the amount of random tests reasonable if random tests are not used to verify the performance of the user's solution.** Keep their amount as small as possible as long as a good coverage is still guaranteed: in some situations, only a bunch of inputs are actually testable, so no need to tests each of them 10 times - use randomized testing instead. Otherwise, 100 random tests are generally enough, or maybe less, depending on the task/situation.
-- Under some rare circumstances, it is allowed to use so-called [**randomized tests**][randomized-tests] instead of fully random ones. For some types of problems (for example, kata simulating chess, grid puzzles like sudoku or nonograms) it may be difficult, or even impossible, to generate valid input configurations randomly. A problem with a small set of possible inputs does not need randomly generated inputs, if its whole input domain can be iterated. In such rare cases, it may be acceptable to use a predefined, hard-coded set of inputs (possibly with expected outputs too). Before the tests are run, the set of inputs should be shuffled or randomly sampled at each run to make the hard-coding of results more tedious for the user. If possible, some additional transformations can be randomly applied to the inputs if it can be easily accounted for in the result of the test (for example arrays can be reversed, game boards can be rotated, sides can be flipped, etc.)
+- For some types of problems (for example, kata simulating chess, or grid puzzles like sudoku or nonograms) it may be difficult, or even impossible, to generate valid input configurations randomly. In such rare cases, it may be acceptable to use a predefined, hard-coded set of inputs (possibly with expected outputs too). Before the tests are run, the set of inputs should be shuffled or randomly sampled at each run to make the hard-coding of results more tedious for the user. If possible, some additional transformations can be randomly applied to the inputs if it can be easily accounted for in the result of the test (for example arrays can be reversed, game boards can be rotated, sides can be flipped, etc.)
+- Some problems have a small set of possible inputs, and it's possible to easily enumerate their whole input domain. For this kind of tasks, it's not necessary to randomly generate inputs. It is allowed to pregenerate or hard-code all possible inputs, and before the tests are run, shuffle or randomly sample them at each run to render solutions which abuse order of inputs infeasible.
 - **Random tests should be run after fixed tests.** Not all testing frameworks allow for easy ordering of tests, but fixed tests should run, and eventually fail, before random tests.
 - **Use appropriate random utilities available in your language**. Know how to use random number generators, how to randomly generate types of inputs you need, be it numbers in a range, large numbers, strings, big integers, floating-point values, collections, etc. Know how to do random sampling of collections, how to shuffle them, how to avoid duplicates. See this [obligatory XKCD comic](https://xkcd.com/221/) for how NOT to do random tests.
 - **Difficulty should be consistent between test runs.** When ranges of random inputs are very large, it becomes possible for some users to receive many small, easy inputs while other users receive the exact opposite. Make sure that your random tests are built in a way that minimizes the chances for such a situation to occur. If you want to test difficult inputs, split the test cases into a set of easy ones and a set of difficult ones, and test them separately.