WeBWorK Problems

Problem Numbers keep changing

Problem Numbers keep changing

by Joseph Pedersen -
Number of replies: 4

I have authored about 25 questions at this point, and haven't experienced this, but for my recent problem the numbers change:

1) every time I select "Save", even though the random seed is not changing

2) When I select "Show Correct Answers" from edit mode.

Is there something in my code (attached) that is causing that?  This is probably the most complicated problem that I've coded, and I have only been using Perl for about two months, so I don't know if I'm breaking some Perl/WeBWorK rule that would cause the numbers to keep changing.

When I am editing a question, and have it show the correct answers, the answers it shows match what I know the answers should be, based on the new - changed - numbers, which are shown in the graph. 

But when I attempt a problem as a student, the answers that I submit are marked incorrect.  When I select to show the correct answers, the answers shown do not match the numbers shown in the graph (which do not change), making me think that numbers have changed behind the scenes.

In reply to Joseph Pedersen

Re: Problem Numbers keep changing

by Alex Jordan -

I haven't examined your code. But it sounds like hashes in random order. In perl, a hash is not an ordered array. The same hash will have its keys in a random order every run.

Is there something in your code that depends on the order if something in a hash?

In reply to Alex Jordan

Re: Problem Numbers keep changing

by Alex Jordan -

I'm looking at the code now. One thing to know about is that images are cached. The first time you run the problem for a given user for a given seed, it creates an image. After that, this same image will be shown even though perl's hash randomization is changing the underlying numbers. Near the top of the problem you can put `$refreshCachedImages = 1;` while debugging. But you should remove that when done so the server doesn't build the same tikz image on every run when it's the same user+seed.

If you change line 109 to:

for $k (lex_sort(keys %edges)) {

and line 140 to:

$edges{$k} = (random_subset(1, lex_sort(keys %canbe)))[0];

then I start getting consistency. That is, wrap lex_sort() around each place you are making an array of keys for some hash, so the array has a consistent order when it's used each run.

Since I was searching for 'keys', I also saw line 108:  `keys %edges;`. I don't think this line does anything. It might be sending the keys array out into the ether.

In reply to Alex Jordan

Re: Problem Numbers keep changing

by Joseph Pedersen -

Thanks!  That fixed it!

Interesting - I assumed that the code only ran one time - to generate the problem.  Is there some reason that the code is rerun every time?  There isn't some option to cache EVERYTHING, not just images, is there?  I cannot imagine a scenario in which someone would want the underlying numbers to potentially change each time, especially if the images are cached.

In reply to Joseph Pedersen

Re: Problem Numbers keep changing

by Glenn Rice -

The code is only run once to generate the problem on a given request.  However, each time the problem is requested the code is run again to generate the problem.  Images are cached for a given user and seed (since they don't change), but the result of the code is different with each request (depending on if answers were submitted, a preview is requested, etc).  The seed for the problem is cached, and from that the problem is generated.  The numbers don't change if the seed doesn't change.  That is how pseudo random number generators work.

The issue is that the order of the keys in a hash don't depend on the seed, but rather on how perl creates hashes, and the order of the keys in the hash will be different each time.  It is designed to be random access memory for efficiency reasons.