This summer we upgraded to WebWork 2.10, and since then we have been having an undesirable occurrence with the gateway quizzes. In particular, many, many versions of the gateway quizzes are generated with the exact same problem seeds: 4730, 1425, 1357, 36, 4959, etc.
This is occurring regardless of the assignment, regardless of the student, regardless of the course. But it is happening intermittently (i.e., not every single version generated has these seeds, but a majority seem to). It doesn't seem to matter how many problems there are in the assignment. If there are two problems, the seeds will be 4730 and 1425. If there are 4 problems, the seeds are 4730, 1425, 1357, and 36.
The settings for the gateway quizzes that I am using are:
Test Time Limit: 15000
Cap Test Time at Set Due Date: Yes
Number of Graded Submissions per Test: 3
Time Interval for New Test Versions: 0=infinite
Number of Tests per Time Interval: 0=infinite
Order Problems Randomly: Yes
Number of Problems per Page: 0=all
I am also using grouping sets for many of the questions on the gateway quizzes if that is relevant.
The real issue is that since I allow the students unlimited versions of the assignment, they are able to use up their 3 attempts on one version (which more often than not seems to be using seeds 4730, 1425, etc), see the answers, and then generate a new version--which more often than not seems to be seeds 4730, 1425, etc--and hence just copy over the answers from the first attempt.
As a temporary fix, I have disabled the showing of the problems on completed quizzes until after the due date.
I am reluctant to switch everything to regular homework (which seem to be functioning properly in regard to seeds) since I heavily using the grouping sets in these assignments.
By the way, I was using this exact setup for gateway quizzes with WebWork 2.9, and this issue did not occur as far as I am aware. (I went back and looked through the seeds assigned to the problems in those classes from WW 2.9, and I do not see this sequence of seeds appearing in the gateway quizzes that I checked.)
Has any one else encountered this issue, and if so, is there a fix?
In reply to Chris Ahrendt
Re: Issues with gateway quizzes generating with the same seeds
by Gavin LaRose -
Hi Chris,
Looking at some courses I have running, I'm seeing some duplication, but not huge amounts; in 5159 tests given for ten of our course-sections, I'm seeing 3234 different seeds. The seeds showing the highest frequencies are:
| seed | number |
|---|---|
| 4338 | 7 |
| 1962 | 6 |
| 50 | 5 |
| 54 | 5 |
| 519 | 5 |
| 899 | 5 |
| 1035 | 5 |
| 1394 | 5 |
| 1639 | 5 |
| 1641 | 5 |
| 1809 | 5 |
| 2195 | 5 |
| 2206 | 5 |
| 2369 | 5 |
| 2902 | 5 |
| 3099 | 5 |
| 3723 | 5 |
| 3793 | 5 |
| 4112 | 5 |
| 4389 | 5 |
| 4532 | 5 |
| 4992 | 5 |
All of these tests have seven problems, each drawn from a grouping set containing on the order of 100 problems.
I have to look at the Gateway code sometime soon for another reason, and will try to notice if there is anything odd going there that would suggest a reason for the behavior you're seeing.
Gavin
In reply to Gavin LaRose
Re: Issues with gateway quizzes generating with the same seeds
by Chris Ahrendt -
Thanks Gavin! I have been trying to pinpoint the issue, but I really can't put my finger on it and it is occurring frequently. For example, a recent gateway quiz assignment had 3 questions. It was made available the morning of 9/24 and was due 9/28 at 11:59p.
The first list here are all the version generated by each student, sorted by student.
Student 1
v1 - opened: 9/28 8:30p, seeds: 3186, 1049, 4664
Student 2
v1 - opened: 9/28 7:42p, seeds: 3186, 1049, 4664
Student 3
v1 - opened: 9/28 5:46p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 7:21p, seeds: 3186, 1049, 4664
Student 4
v1 - opened: 9/24 8:29p, seeds: 4246, 1308, 3582
v2 - opened: 9/24 11:35p, seeds: 944, 2953, 115
v3 - opened: 9/28 1:31p, seeds: 3186, 1049, 4664
Student 5
v1 - opened: 9/24 8:30p, seeds: 86, 892, 736
Student 6
v1 - opened: 9/24 9:14p, seeds: 944, 2953, 115
Student 7
v1 - opened: 9/28 1:50p, seeds: 2777, 1478, 436
Student 8
v1 - opened: 9/28 9:28p, seeds: 1889, 489, 3050
v2 - opened: 9/28 11:41p, seeds: 2777, 1478, 436
Student 9
v1 - opened: 9/28 2:24p, seeds: 944, 2953, 115
Student 10
v1 - opened: 9/28 3:40p, seeds: 1374, 4241, 2278
Student 11
v1 - opened: 9/28 9:46p, seeds: 3186, 1049, 4664
Student 12
v1 - opened: 9/24 9:03p, seeds: 944, 2953, 115
v2 - opened: 9/28 11:51p, seeds: 3186, 1049, 4664
Student 13
v1 - opened: 9/24 12:03p, seeds: 944, 2953, 115
Student 14
v1 - opened: 9/28 8:53a, seeds: 1240, 2777, 1428
v2 - opened: 9/28 10:52p, seeds: 3186, 1049, 4664
Student 15
v1 - opened: 9/25 8:07a, seeds: 4134, 3757, 650
v2 - opened: 9/26 2:23p, seeds: 4134, 3757, 650
Student 16
v1 - opened: 9/28 11:24a, seeds: 3186, 1049, 4664
v2 - opened: 9/28 12:24p, seeds: 3186, 1049, 4664
Student 17
v1 - opened: 9/24 10:23p, seeds: 944, 2953, 115
v2 - opened: 9/25 12:32a, seeds: 3412, 211, 1130
Student 18
v1 - opened: 9/25 3:54a, seeds: 4107, 2602, 3500
v2 - opened: 9/28 1:33p, seeds: 1719, 2756, 4774
v3 - opened: 9/28 1:50p, seeds: 3186, 1049, 4664
Student 19
v1 - opened: 9/28 9:29p, seeds: 3186, 1049, 4664
Student 20
v1 - opened: 9/28 8:28p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 9:49p, seeds: 973, 49, 4810
v3 - opened: 9/28 10:00p, seeds: 2777, 1478, 436
Student 21
v1 - opened: 9/28 10:44p, seeds: 1610, 2777, 1280
Student 22
v1 - opened: 9/24 2:16p, seeds: 944, 2953, 115
Student 23
v1 - opened: 9/24 1:13p, seeds: 944, 2953, 115
v2 - opened: 9/24 1:50p, seeds: 944, 2953, 115
v3 - opened: 9/24 1:55p, seeds: 944, 2953, 115
Student 24
v1 - opened: 9/24 6:38p, seeds: 944, 2953, 115
Student 25
v1 - opened: 9/27 10:49a, seeds: 2720, 4490, 2853
Student 26
v1 - opened: 9/24 6:12p, seeds: 1620, 3412, 1664
v2 - opened: 9/28 2:17p, seeds: 3186, 1049, 4664
Student 27
v1 - opened: 9/25 9:04a, seeds: 4134, 3757, 650
Student 28
v1 - opened: 9/28 6:10p, seeds: 3186, 1049, 4664
Student 29
v1 - opened: 9/28 11:59a, seeds: 3186, 1049, 4664
v2 - opened: 9/28 1:03p, seeds: 1562, 3688, 3716
v3 - opened: 9/28 1:08p, seeds: 2777, 1478, 436
v4 - opened: 9/28 1:26p, seeds: 3186, 1049, 4664
v5 - opened: 9/28 3:16p, seeds: 455, 3246, 2040
*******
Notice that seeds seem to repeat themselves somewhat depending on the time each version was opened. So here are the same list of versions, just sorted by the time opened:
v1 - opened: 9/24 12:03p, seeds: 944, 2953, 115
v1 - opened: 9/24 1:13p, seeds: 944, 2953, 115
v2 - opened: 9/24 1:50p, seeds: 944, 2953, 115
v3 - opened: 9/24 1:55p, seeds: 944, 2953, 115
v1 - opened: 9/24 2:16p, seeds: 944, 2953, 115
v1 - opened: 9/24 6:12p, seeds: 1620, 3412, 1664
v1 - opened: 9/24 6:38p, seeds: 944, 2953, 115
v1 - opened: 9/24 8:29p, seeds: 4246, 1308, 3582
v1 - opened: 9/24 8:30p, seeds: 86, 892, 736
v1 - opened: 9/24 9:03p, seeds: 944, 2953, 115
v1 - opened: 9/24 9:14p, seeds: 944, 2953, 115
v1 - opened: 9/24 10:23p, seeds: 944, 2953, 115
v2 - opened: 9/24 11:35p, seeds: 944, 2953, 115
v2 - opened: 9/25 12:32a, seeds: 3412, 211, 1130
v1 - opened: 9/25 9:04a, seeds: 4134, 3757, 650
v1 - opened: 9/25 3:54a, seeds: 4107, 2602, 3500
v1 - opened: 9/25 8:07a, seeds: 4134, 3757, 650
v2 - opened: 9/26 2:23p, seeds: 4134, 3757, 650
v1 - opened: 9/27 10:49a, seeds: 2720, 4490, 2853
v1 - opened: 9/28 8:53a, seeds: 1240, 2777, 1428
v1 - opened: 9/28 11:24a, seeds: 3186, 1049, 4664
v1 - opened: 9/28 11:59a, seeds: 3186, 1049, 4664
v2 - opened: 9/28 12:24p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 1:03p, seeds: 1562, 3688, 3716
v3 - opened: 9/28 1:08p, seeds: 2777, 1478, 436
v4 - opened: 9/28 1:26p, seeds: 3186, 1049, 4664
v3 - opened: 9/28 1:31p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 1:33p, seeds: 1719, 2756, 4774
v3 - opened: 9/28 1:50p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 1:50p, seeds: 2777, 1478, 436
v2 - opened: 9/28 2:17p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 2:24p, seeds: 944, 2953, 115
v5 - opened: 9/28 3:16p, seeds: 455, 3246, 2040
v1 - opened: 9/28 3:40p, seeds: 1374, 4241, 2278
v1 - opened: 9/28 5:46p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 6:10p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 7:21p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 7:42p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 8:28p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 8:30p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 9:28p, seeds: 1889, 489, 3050
v1 - opened: 9/28 9:29p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 9:46p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 9:49p, seeds: 973, 49, 4810
v3 - opened: 9/28 10:00p, seeds: 2777, 1478, 436
v1 - opened: 9/28 10:44p, seeds: 1610, 2777, 1280
v2 - opened: 9/28 10:52p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 11:41p, seeds: 2777, 1478, 436
v2 - opened: 9/28 11:51p, seeds: 3186, 1049, 4664
*******
The seeds 944, 2953, 115 were repeated quite a bit on 9/24, but there was one version opened on 9/28 that also had those three seeds. Of course these are the results from my one class. We have about 20 different sections with over 700 students using WebWork this semester for various courses, so there were any number of other assignments generated in other sections in these time intervals.
The first list here are all the version generated by each student, sorted by student.
Student 1
v1 - opened: 9/28 8:30p, seeds: 3186, 1049, 4664
Student 2
v1 - opened: 9/28 7:42p, seeds: 3186, 1049, 4664
Student 3
v1 - opened: 9/28 5:46p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 7:21p, seeds: 3186, 1049, 4664
Student 4
v1 - opened: 9/24 8:29p, seeds: 4246, 1308, 3582
v2 - opened: 9/24 11:35p, seeds: 944, 2953, 115
v3 - opened: 9/28 1:31p, seeds: 3186, 1049, 4664
Student 5
v1 - opened: 9/24 8:30p, seeds: 86, 892, 736
Student 6
v1 - opened: 9/24 9:14p, seeds: 944, 2953, 115
Student 7
v1 - opened: 9/28 1:50p, seeds: 2777, 1478, 436
Student 8
v1 - opened: 9/28 9:28p, seeds: 1889, 489, 3050
v2 - opened: 9/28 11:41p, seeds: 2777, 1478, 436
Student 9
v1 - opened: 9/28 2:24p, seeds: 944, 2953, 115
Student 10
v1 - opened: 9/28 3:40p, seeds: 1374, 4241, 2278
Student 11
v1 - opened: 9/28 9:46p, seeds: 3186, 1049, 4664
Student 12
v1 - opened: 9/24 9:03p, seeds: 944, 2953, 115
v2 - opened: 9/28 11:51p, seeds: 3186, 1049, 4664
Student 13
v1 - opened: 9/24 12:03p, seeds: 944, 2953, 115
Student 14
v1 - opened: 9/28 8:53a, seeds: 1240, 2777, 1428
v2 - opened: 9/28 10:52p, seeds: 3186, 1049, 4664
Student 15
v1 - opened: 9/25 8:07a, seeds: 4134, 3757, 650
v2 - opened: 9/26 2:23p, seeds: 4134, 3757, 650
Student 16
v1 - opened: 9/28 11:24a, seeds: 3186, 1049, 4664
v2 - opened: 9/28 12:24p, seeds: 3186, 1049, 4664
Student 17
v1 - opened: 9/24 10:23p, seeds: 944, 2953, 115
v2 - opened: 9/25 12:32a, seeds: 3412, 211, 1130
Student 18
v1 - opened: 9/25 3:54a, seeds: 4107, 2602, 3500
v2 - opened: 9/28 1:33p, seeds: 1719, 2756, 4774
v3 - opened: 9/28 1:50p, seeds: 3186, 1049, 4664
Student 19
v1 - opened: 9/28 9:29p, seeds: 3186, 1049, 4664
Student 20
v1 - opened: 9/28 8:28p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 9:49p, seeds: 973, 49, 4810
v3 - opened: 9/28 10:00p, seeds: 2777, 1478, 436
Student 21
v1 - opened: 9/28 10:44p, seeds: 1610, 2777, 1280
Student 22
v1 - opened: 9/24 2:16p, seeds: 944, 2953, 115
Student 23
v1 - opened: 9/24 1:13p, seeds: 944, 2953, 115
v2 - opened: 9/24 1:50p, seeds: 944, 2953, 115
v3 - opened: 9/24 1:55p, seeds: 944, 2953, 115
Student 24
v1 - opened: 9/24 6:38p, seeds: 944, 2953, 115
Student 25
v1 - opened: 9/27 10:49a, seeds: 2720, 4490, 2853
Student 26
v1 - opened: 9/24 6:12p, seeds: 1620, 3412, 1664
v2 - opened: 9/28 2:17p, seeds: 3186, 1049, 4664
Student 27
v1 - opened: 9/25 9:04a, seeds: 4134, 3757, 650
Student 28
v1 - opened: 9/28 6:10p, seeds: 3186, 1049, 4664
Student 29
v1 - opened: 9/28 11:59a, seeds: 3186, 1049, 4664
v2 - opened: 9/28 1:03p, seeds: 1562, 3688, 3716
v3 - opened: 9/28 1:08p, seeds: 2777, 1478, 436
v4 - opened: 9/28 1:26p, seeds: 3186, 1049, 4664
v5 - opened: 9/28 3:16p, seeds: 455, 3246, 2040
*******
Notice that seeds seem to repeat themselves somewhat depending on the time each version was opened. So here are the same list of versions, just sorted by the time opened:
v1 - opened: 9/24 12:03p, seeds: 944, 2953, 115
v1 - opened: 9/24 1:13p, seeds: 944, 2953, 115
v2 - opened: 9/24 1:50p, seeds: 944, 2953, 115
v3 - opened: 9/24 1:55p, seeds: 944, 2953, 115
v1 - opened: 9/24 2:16p, seeds: 944, 2953, 115
v1 - opened: 9/24 6:12p, seeds: 1620, 3412, 1664
v1 - opened: 9/24 6:38p, seeds: 944, 2953, 115
v1 - opened: 9/24 8:29p, seeds: 4246, 1308, 3582
v1 - opened: 9/24 8:30p, seeds: 86, 892, 736
v1 - opened: 9/24 9:03p, seeds: 944, 2953, 115
v1 - opened: 9/24 9:14p, seeds: 944, 2953, 115
v1 - opened: 9/24 10:23p, seeds: 944, 2953, 115
v2 - opened: 9/24 11:35p, seeds: 944, 2953, 115
v2 - opened: 9/25 12:32a, seeds: 3412, 211, 1130
v1 - opened: 9/25 9:04a, seeds: 4134, 3757, 650
v1 - opened: 9/25 3:54a, seeds: 4107, 2602, 3500
v1 - opened: 9/25 8:07a, seeds: 4134, 3757, 650
v2 - opened: 9/26 2:23p, seeds: 4134, 3757, 650
v1 - opened: 9/27 10:49a, seeds: 2720, 4490, 2853
v1 - opened: 9/28 8:53a, seeds: 1240, 2777, 1428
v1 - opened: 9/28 11:24a, seeds: 3186, 1049, 4664
v1 - opened: 9/28 11:59a, seeds: 3186, 1049, 4664
v2 - opened: 9/28 12:24p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 1:03p, seeds: 1562, 3688, 3716
v3 - opened: 9/28 1:08p, seeds: 2777, 1478, 436
v4 - opened: 9/28 1:26p, seeds: 3186, 1049, 4664
v3 - opened: 9/28 1:31p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 1:33p, seeds: 1719, 2756, 4774
v3 - opened: 9/28 1:50p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 1:50p, seeds: 2777, 1478, 436
v2 - opened: 9/28 2:17p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 2:24p, seeds: 944, 2953, 115
v5 - opened: 9/28 3:16p, seeds: 455, 3246, 2040
v1 - opened: 9/28 3:40p, seeds: 1374, 4241, 2278
v1 - opened: 9/28 5:46p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 6:10p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 7:21p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 7:42p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 8:28p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 8:30p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 9:28p, seeds: 1889, 489, 3050
v1 - opened: 9/28 9:29p, seeds: 3186, 1049, 4664
v1 - opened: 9/28 9:46p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 9:49p, seeds: 973, 49, 4810
v3 - opened: 9/28 10:00p, seeds: 2777, 1478, 436
v1 - opened: 9/28 10:44p, seeds: 1610, 2777, 1280
v2 - opened: 9/28 10:52p, seeds: 3186, 1049, 4664
v2 - opened: 9/28 11:41p, seeds: 2777, 1478, 436
v2 - opened: 9/28 11:51p, seeds: 3186, 1049, 4664
*******
The seeds 944, 2953, 115 were repeated quite a bit on 9/24, but there was one version opened on 9/28 that also had those three seeds. Of course these are the results from my one class. We have about 20 different sections with over 700 students using WebWork this semester for various courses, so there were any number of other assignments generated in other sections in these time intervals.
In reply to Chris Ahrendt
Re: Issues with gateway quizzes generating with the same seeds
by Chris Ahrendt -
One more quick note. Looking at a colleagues course; she had a gateway quiz around the same time that had 5 questions on it. A quick sample of the seeds:
9/23 2:47p, seeds 2574, 944, 2953, 1568, 4900
9/28 10:52a, seeds: 3425, 3186, 1049, 1324, 1240
9/28 11:05p, seeds: 3425, 3186, 1049, 1324, 1240
The 2nd and 3rd seeds match the seeds used for questions 1 and 2 in my section around the same times...
9/23 2:47p, seeds 2574, 944, 2953, 1568, 4900
9/28 10:52a, seeds: 3425, 3186, 1049, 1324, 1240
9/28 11:05p, seeds: 3425, 3186, 1049, 1324, 1240
The 2nd and 3rd seeds match the seeds used for questions 1 and 2 in my section around the same times...
In reply to Chris Ahrendt
Re: Issues with gateway quizzes generating with the same seeds
by Chris Ahrendt -
FYI:
I updated the apache config file with the fix given here:
https://github.com/openwebwork/webwork2/pull/645
I added this in on Wednesday, and have checked various gateway quiz assignments that have been generated since then. At least for the two dozen or so assignments that I have checked, there have not been any repeated seeds in the fashion that was happening before. So, I am cautiously optimistic that this has fixed the issue for us.
Chris
I updated the apache config file with the fix given here:
https://github.com/openwebwork/webwork2/pull/645
I added this in on Wednesday, and have checked various gateway quiz assignments that have been generated since then. At least for the two dozen or so assignments that I have checked, there have not been any repeated seeds in the fashion that was happening before. So, I am cautiously optimistic that this has fixed the issue for us.
Chris