You would enable this feature, then set it to rerandomize the problem after 1 attempt.
The disadvantage of using the built-in WeBWorK re-randomization is that it doesn't give the student an opportunity to see the correct answer to the current version before moving on to a new version.
Of course you are always free to do whatever you want with locally created problems. I am more thinking about sharing problems with the OPL. When I assign a problem from the OPL, I make a few assumptions about how it operates. In particular, I expect that the problem will respect the settings available in the homework set, such as:
- Number of attempts - the student should not be able to get any feedback until they submit their answers, which will count as an attempt.
- Random seed - the problem shouldn't change unless the problem seed is changed, which can be done using the re-randomize feature.
These are my thoughts on problem authoring, and may not reflect the WeBWorK community at large.