Gateway Tests and Quizzes

GatewayQuiz Assignments

GatewayQuiz assignments in WeBWorK have many of the characteristics that one might expect of on-line "quizzes" or "tests". In particular, they

• have a time limit on the assignment,
• are versioned, and therefore may (depending on how the assignment is defined) be taken by a student multiple times; each version of the assignment will be different in the same way that problems on homework sets differ for different students,
• may require proctor authorization to be taken and graded,
• may present more than one question from the assignment on a single page,
• allow problems in the assignment to be described as a group of problems from which one (or more) problem(s) is(are) drawn,
• allow restriction of the grade and related data that is shown to students when they have completed an assignment,
• allow specification of specific computers or labs from which the assignment may be completed, or from which it may not be completed, and
• may be configured to hide students' scores or work from them after they have completed the assignment.

In the following we consider an instructor's view and the students' view of these assignments, addressing how these different aspects are incorporated into the management and use of a WeBWorK assignment. There is also a page of technical information and philosophy about these types of assignments; this last gives some of the technical information about how the assignments work that may be useful to know as you work with them.

Up to: table of contents || gateway assignments

Instructor's View

The information in this section is split between the following, which seeks to be more of a description of how to do things, and a technical info and philosophy page, that delves into more of how things are set up underneath the outer skin of WeBWorK. As you use GatewayQuiz assignments, it may be worth stopping to read that page as well to understand why some things work as they do.

Setting Up GatewayQuiz Assignments

It is possible to set up GatewayQuiz assignments by editing an existing set or by importing a set from a set definition file. In addition, there are some behaviors of these assignments that are set in the global.conf file. Other than having more settings than a usual homework assignment, GatewayQuiz assignments are essentially the same as other homework sets. It is also possible to restrict where students may take a test from, as described in the assignment access restrictions documentation.

Note that there are some restrictions on what types of problems may be used in GatewayQuiz assignments, and that problems for the assignments may be drawn from a group of problems, as is explained in the GatewayQuiz assignment problems page.

Up to: table of contents || instructors view

GatewayQuiz Assignments by Editing

To create a GatewayQuiz assignment by editing an existing set, you first create a set by using the Library Browser, or from the Instructor Tools or Homework Sets Editor page. Once the set is created, select it in the Instructor Tools menu and click "Edit one set". Under "General Information", change "Assignment type" to "gateway/quiz" or "proctored gateway/quiz" and click the "Save Changes" button.

You will then be able to set the parameters that determine the characteristics of the GatewayQuiz assignment, as described in the table below. Note that the Set Password parameter only appears for proctored sets. In addition to these specific parameters, it is also possible to set restrictions on where assignments may be completed, as described in the assignment access restrictions documentation.

GatewayQuiz Parameters

Test Time Limit	The length of time the student is allowed to work on each version of the assignment. Note that the expiry of the time limit has implications for the recording of student answers, as noted below in the Due Dates section of the documentation. Also note that there is a grace period at the end of the test in which students are still able to submit their answers that is set in the global.conf file.
Cap Test Time at Set Due Date	Whether the time limit for a student's assignment should be truncated at the due date of the course assignment (e.g., if a 20 minute quiz must be completed by noon and a student starts it at 11:50---if the cap is set to "yes", this student will only get 10 minutes in which to complete the quiz). See also the information about Due Dates, below. (Available in WeBWorK 2.4 and later.)
Number of Graded Submissions	The number of times a student may submit each version of the assignment for a grade. In most cases, this will be 1.
Time Interval for New Test Versions	If students are allowed to complete multiple versions of the assignment (e.g., if it is a skills test that may be taken multiple times for practice), this time interval gives the time interval in which those versions must be taken. For example, if a student may take the assignment once per day, we might set this to 1440 (minutes: 24 hours). Note that the time interval is a rolling interval, so that specifying one version per hour means that a student who starts a test/quiz at 11:58 may start a second at 12:58:01. To implement a daily restriction (e.g., two tests per day) it is generally sufficient to specify the number of tests in a 12 or 18 hour interval. In WeBWorK 2.4 and later, entering "0" for the time interval for new versions gives an infinite interval.
Number of Tests per Time Interval	This is explained above; entering "0" puts no limit on the number of allowed versions. If the time interval for new versions is infinite, this gives the total number of test/quizzes that may be taken.
Order Problems Randomly	If this is true, the problems on the test/quiz will be presented in a random order. It is worth noting that this has implications for the manner in which student results are displayed in the Student Progress and Grades pages.
Number of Problems per Page (0=all)	How many of the questions in the test/quiz to show on one page. For example, specifying "3" for a 5 question quiz will result in a first page of 3 questions and a second of 2. Specifying "0" problems per page puts all problems on one page.
Show score on finished assignments	Whether students should be able to see their score on versions of the test/quiz on which they have no remaining attempts, or no remaining time. This is described below in the Showing Student Scores and Work section of the documentation. (Available in WeBWorK 2.4 and later.)
Show student work on finished assignments	Similarly, whether students should see the problems and their work after finishing a test version. This is also described below in the Showing Student Scores and Work section of the documentation. (Available in WeBWorK 2.4 and later.)
Set-level Password	Set Password: (for proctored assignments only, and only available in WeBWorK 2.4 and later) A password for the set that is the required password to start the assignment. This password will not authorize grading, for which a regular proctor login is required. See the proctored tests section below and discussion of proctors and WeBWorK users in the technical information and philosophy documentation.

Note that there are also several settings in the global.conf file that determine the behavior of GatewayQuiz assignments (as noted below). These include settings in the permissionLevels hash and the gatewayGracePeriod variable.

Up to: table of contents || instructors view

GatewayQuiz Assignments by Importing

Gateway/Quiz assignments may be imported from set definition files in the same manner as other assignments. To specify the gateway/quiz parameters, however, additional options must be indicated in the definition file. These refer to the gateway/quiz parameters that can be set by editing an assignment, as is suggested in the following example file:

openDate = 01/24/2008 at 8:00 AM
dueDate = 02/23/2008 at 11:59 PM
answerDate = 02/23/2008 at 11:59 PM
paperHeaderFile = integralGWpaperHeader.pg
assignmentType = gateway
attemptsPerVersion = 1
timeInterval = 43200
versionsPerInterval = 0
versionTimeLimit = 1800
problemRandOrder = 1
problemsPerPage = 0
hideScore = N
hideWork = N
capTimeLimit = 0
restrictIP = RestrictTo
restrictLocation = ProctoredLabs
relaxRestrictIP = No
#
problemList=
(etc.)

All time limits and intervals are given in seconds (note that this is different than the way they are entered in the edit menu). Note some restriction variables have specific valid values:

• assignmentType may be gateway or proctored_gateway (just omit assignmentType for non-gateway sets);
• probRandOrder and capTimeLimit may be 0 or 1 (for don't/do display problems in a random order or cap the time available on the test);
• hideScore and hideWork may be N, Y, or BeforeAnswerDate;
• restrictIP may be No, RestrictTo or DenyFrom;
• relaxRestrictIP may be No, AfterAnswerDate, or AfterVersionAnswerDate.

It may also be worth observing that the input data for the hideScore and hideWork parameters are the negation of the data that are selected on the set editor page.

When importing sets from a set definition file, it is not currently possible to set IP address restrictions on sets to more than one location. The value given by restrictLocation must be a valid location as defined in the WeBWorK environment into which the set is being imported. It is also not currently possible to create a set-level proctor password from a set definition file. See also the assignment access restrictions documentation.

Up to: table of contents || instructors view

GatewayQuiz Parameters in global.conf

There are a number settings in the permissionLevels hash in the global.conf file that determine the behavior of GatewayQuiz assignments:

• proctor_quiz_login - this is the permission level required to allow students to start a proctored_gateway_quiz assignment.
• proctor_quiz_grade - this is the permission level required to allow students to submit (grade) a proctored_gateway_quiz assignment.
• view_proctored_tests - this setting determines whether proctored_gateway_quiz assignments will appear in the set list page for students. If you set up an entry page that directs them to a proctored assignment (instead of having them navigate to it), you may want to set this permission level to ta so that they aren't confused by seeing the proctored tests showing up in their set lists.
• create_new_set_version_when_acting_as_student - (new in WeBWorK versions > 2.4.5) this determines who (if anyone) can create a new set version when acting as a student. If this is set to something other than undef, e.g., professor, then it will be possible for them to create a new version of a set when acting as a student. Note that the created version will count against the student's allowed number of versions, so this is usually not desirable.
• record_set_version_answers_when_acting_as_student - (new in WeBWorK versions > 2.4.5) this determines who can record answers for a new set version when acting as a student. In general, this should be set to undef, as is record_answers_when_acting_as_student, but if it is necessary or desirable for an instructor to be able to create and submit student GatewayQuiz assignments it can be reset.

For the last two of these, there is additional explanation below under Acting as Students on Gateway Assignments.

In addition, there is the setting

• view_ip_restricted_sets - which determines who is able to ignore the location restrictions on sets.

Finally, there is one system variable that is set in global.conf and which helps determine the behavior of GatewayQuiz assignments. $gatewayGracePeriod gives the number of seconds after an assignment's due time that students will still be allowed to submit it for a grade. This only applies to gateway_quiz and proctored_gateway_quiz assignments.

Up to: table of contents || instructors view

Editing Existing GatewayQuiz Assignments

When editing GatewayQuiz assignments, we may want to edit the global set (the set that is assigned to all users), or the user set (which determines the parameters---such as open date, due date, time limit, etc., associated with a specific user's, or student's, assigned set), or a particular set version (the version of the set that the user, or student, actually took). The distinction between these three is described in somewhat greater detail on the technical info and philosophy page.

Editing the Global GatewayQuiz Assignment

Edit the global GatewayQuiz assignment the same way you would the global set for a normal homework assignment: select the set on the Instructor Tools page, and click "Edit one set." This allows you to change the characteristics of the set, as described in the set creation by editing section. As with any assignment, the settings for the global set will propagate to all user sets and set versions---with the exception of those settings that are overridden in the user set or set version. In particular, the open, due and answer dates are automatically set for any set version (because they determine the test time, as noted in the discussion of set versions and time limits in the technical and philosophical information), and the assignment type for set versions may also be set (because finished proctored tests get converted to non-proctored ones so that students can see their results). In particular, if you reset the visibility of a global set, it will change the visibility of all user sets and versions of the assignment that students have taken. Similarly, if you "mark correct" a problem on the global set it will mark that problem correct for all versions of the set that students have taken.

Note that for versions of WeBWorK <= 2.4.5, changes in the global or user sets do not propagate to versions of the set that students have already taken (this includes problems that are marked correct). For all versions of WeBWorK they do propagate to new versions that are taken after the changes are saved.

Up to: table of contents || instructors view

Editing a User Set for GatewayQuiz Assignments

This is again the same as editing a user set for a regular homework assignment. Select the set and the user(s) on the Instructor Tools page, and click "Edit one set for users". Any parameter values set for the user set will override those for the global set. Values set for a user will propagate to versions of the assignment that users take, and to those versions that are already taken (except for parameter values that are overridden by the set version, as indicated above).

Note that for versions of WeBWorK <= 2.4.5, changes in the global or user sets do not propagate to versions of the set that students have already taken. For all versions of WeBWorK they do propagate to new versions that are taken after the changes are saved.

Up to: table of contents || instructors view

Editing a Set Version for a User for GatewayQuiz Assignments

(This is only available in versions of WeBWorK > 2.4.5.) There are some cases in which it may be useful to change parameters for a version of a GatewayQuiz assignment that a student has already started. Because set versions are (necessarily) not listed in the Instructor Tools page, this function is available instead by editing all sets for a user: on the Instructor Tools page, select the user, and click "View/Edit all sets for one user". This will list all sets, including set versions. Clicking the name of a set version will allow editing of (some) set parameters for the set version (parameters that are most properly determined from the user or global sets may not be available here).

Up to: table of contents || instructors view

Student Progress, Statistics and Scoring for GatewayQuiz Assignments

Students' progress on and statistics for GatewayQuiz assignments may be examined in the same manner as for homework assignments: click Student Progress or Statistics on the left side bar, under Instructor Tools. There are only a couple of notes about this for GatewayQuiz assignments.

Student Progress: For GatewayQuiz assignments, there are a number of additional display options, which determine what data is shown for the assignments. In particular, display can be restricted to only the best version of the assignment (that is, only a student's set with the highest score will be shown), and many of the other columns that are normally displayed for homework assignments may be omitted. For GatewayQuiz assignments, the TestTime column will indicate if a student's assignment ran over time or is still in progress. Note: for versions of WeBWorK <= 2.4.5, assignments that were still being taken displayed as being over time.

In early versions of the Gateway/Quiz module, students who have completed no versions of a gateway/quiz assignment do not appear when student results are viewed by set. In version 2.4 and later, students who are not assigned the gateway/quiz do not appear, but all others do, with students who have taken no versions of the assignment being indicated in the results listing.

There is one more subtlety on the Grades and StudentProgress pages. When viewing students' scores by problem on gateway/quiz assignments for which the problems are ordered randomly, there are two orders in which the problem scores could be presented. The first is the instructor's order, that is, the order in which the problems are defined in the set, and the second is the test/quiz order, that is, the order in which the problems are shown on a particular test/quiz. On the student Grades page, and on the instructor's StudentProgress page when viewing results for a student, problem results are shown in the test/quiz order.

Statistics: For GatewayQuiz assignments (and versions of WeBWorK > 2.4.5), the statistics page shows the statistics for all versions of the assignment that have been taken by students. (For versions of WeBWorK <= 2.4.5, the Statistics page fails to display the statistics for the GatewayQuiz assignments.)

Scoring: When scoring sets that include a GatewayQuiz assignment, the score for a student's best version is reported when there are multiple versions that were taken. (Note also that any test that a student does not submit before the test time limit expires is recorded with a score of zero, as noted in the discussion of due dates and time limits in the technical and philosophical information document.)

Up to: table of contents || instructors view

Acting as Students on GatewayQuiz Assignments

In most cases, professors will only want to act as a student to see what a set version that the student has completed looks like. This is done by acting as the student (but not selecting a set) and then clicking into the version of the set that is of interest.

If a professor selects a student and a GatewayQuiz assignment on the Instructor Tools page and clicks "Act as a user (on one set)", the system will return an error indicating that because the student is being acted as, new sets cannot be created (though if the professor has the permission to create_new_set_version_when_acting_as_student, as described in the global.conf file configuration section, s/he will have the option to continue to do so). A similar behavior is, of course, elicited by acting as a student and then clicking the "Take new" link for a GatewayQuiz assignment from that student's problem sets list page.

(In versions of WeBWorK <= 2.4.5, selecting a student and a GatewayQuiz assignment on the Instructor Tools page and clicking "Act as" will return the error "Requested set 'SetName' is a test/quiz assignment but the regular homework assignment content generator problem_list was called..." The error message may be construed as indicated above.)

Up to: table of contents || instructors view

Proctored GatewayQuiz Assignments

Proctored GatewayQuiz assignments require that students get a proctor's authorization to start the assignment, and then similarly get authorization before the assignment can be graded. As discussed in the technical info and philosophy documentation's proctoring section, any user with a permissionLevel of ta or professor can give authorization to start or submit such an assignment, as can any user with a grade_proctor permissionLevel. A user with a permissionLevel of login_proctor may authorize the start of a test but not its grading.

Set Level Proctor Passwords

It is also possible to set a set-level proctor password for a proctored GatewayQuiz assignment. If this is done, then that password is the only one that will allow the assignment to be started. This can be useful, for example, if the assignment is given as a quiz in a computer lab and all students should be given the password at the same time to allow them to start the assignment.

Up to: table of contents || instructors view

Students' View

By default, all GatewayQuiz assignments that are assigned to a student appear at the bottom of the ProblemSets list s/he sees when logging in to WeBWorK. Available GatewayQuiz assignments will be indicated with a link called "Take SetName test". If the view_proctored_tests permissionLevel in the global.conf file is set to "ta" or higher (as indicated in the global.conf file discussion above, links for proctored GatewayQuiz assignments will not show up for students. This link will start a new version of the gateway/quiz for the student, if s/he has available versions (that is, if s/he hasn't completed all of her/his versions for the given time interval; otherwise, an error will be returned). If the student has a version of the assignment that s/he has not completed (that is, which s/he can still submit for a grade, because s/he hasn't submitted it as many times as s/he is allowed and still has time left on the assignment), clicking the "Take SetName test" link will cause the student to resume work on that version of the test.

Once a student has completed a version of a gateway/quiz assignment, the version will show up on the ProblemSets page as a link "SetName (test N)", where N indicates the Nth test that s/he has taken. Note that once a student has completed a proctored test, that version of the test is redefined for the student as an unproctored test. Thus a student is able to review completed proctored gateway/quiz assignments without needing a proctor. Note that completed proctored assignments therefore appear on the ProblemSets page regardless of the value of view_proctored_tests in the permissionLevels hash in global.conf.

See also the discussion of editing set assignments for notes about how setting the "hidden" property of a GatewayQuiz assignment will effect those sets that students see.

Test Timing and Restarting Tests

Once a student starts a new GatewayQuiz assignment version, the timer for that set version is started---that is, all elapsed time counts against the amount of time the student is allowed. In particular, this means that if the student returns to her/his problem sets list page (or if her/his browser crashes) without submitting the test, the allowed time is still being used up. A GatewayQuiz with available time will appear in the problem sets list with a notation indicating it is open and the due time. Clicking that the set version link (that labeled "SetName (test N)") will allow the student to continue work on the assignment, as will clicking the "Take new" link. Once the available time on the set version has expired, clicking the set version link will allow the student to look at and work that set version (assuming that the set isn't configured to hide student work), and clicking the "Take new" link will start a new set version (assuming that there are versions left to take).

Up to: table of contents || instructors view