CheckAnswers SubmitAnswers design

What is the difference between Check Answers and Submit Answers?

• Check Answers does not record the answer in the database.
  • While answers that you have entered will remain when you check an answer (short term sticky answers) they will not remain for long. If you leave the problem page and go to the next page and then return, your previous answers will be erased.
• Submit Answers does record answers in the database.
  • Whenever you return to a problem page your previous answers are retrieved from the database. So the most recent answers you have submitted are sticky long term.
• With the current database and infra structure there is no way to record answers in the database

but to have them not appear in Student Progress, Student Statistics, Past Answers, the Grades and so forth.

• Use case for Check Answers
  • You want the student to be able to practice after the due date and see if their answers are correct but you don't want this to count toward their grades.
  • You are viewing a student's problem acting as a student and you want to check if the computer is checking the answer properly -- but you don't want to answer the question so that the student gets credit without having done the problem, and you don't want your answer appearing the past answer record of the student. In this case pushing the "Check Answers" button is the right thing to do.
  • You would like your colleagues or students not in the class to be able to see homework sets and try them but you don't want their answers to pollute the grading of the course or the collection of past answers. Assigning these users as guests will allow this under the

When are Check Answers and Submit Answers buttons shown?

• A full answer to this depends on the configuration chosen or overridden in default.config, localOverrides.conf, course.conf and simple.conf(the Course Configuration page).

• With the default configuration guests cannot see the checkAnswer button for open sets. They can see them for sets which are past their due date.
  • See bug #2940 at http://bugs.webwork.maa.org/.
  • Why? Because of the linear permission system. Setting permissions so that guests can see the checkAnswer button for open sets means that students (who have a higher permission level) can also see the check Answer button. This is clearly a bad thing since students could then use the checkAnswer button to check their answers without being recorded until they got everything right and then submit their answers.
  • Setting check_answers_after_open_date_with_attempts=>"guest" instead of "ta" will allow both guests and students to check answers. Students will also be able to submit answers.
• When are checkAnswers and submitAnswers buttons both shown?
  • The main use case is to show both for "ta" and "prof". Then the ta or professor can check the students answer when acting as the student (to see if the computer is checking the answer properly) using the checkAnswer button without disturbing the state of the problem -- i.e. no score or past answer is recorded. They can also submit an answer for the student when acting as the student (this is more rarely desired) using the submitAnswer button. The default setting in default.config is not to show the submitAnswer button when acting as the student to prevent accidental submission. If the default is changed then an instructor will see both buttons when acting as a student but the submit button will have a label giving the student_id of the student whose answer will be changed.
  • By default instructors always see both checkAnswer and submitAnswer buttons when viewing their own open problem sets. For closed problem sets only the checkAnswer button will be shown.

• I've logged in as guest (practice1) and seen the submit button
  • Some sites give practice1, practice2, etc. student permissions so that on those sites they have all the privileges of regular students.

Discussion about the best default configuration for Check Answers/Submit Answers

• We may need more than one example configuration here since different people will have different requirements.

The default settings for showing checkAnswers (from default.config)

       check_answers_before_open_date                  => "ta", # ta's and higher are shown the checkAnswer button before the open date
       check_answers_after_open_date_with_attempts     => "ta", # ta's and higher are shown the checkAnswer button before the number of attempts is exceeded 
                                                                # (but after the open date) 
       check_answers_after_open_date_without_attempts  => "guest", # guests, students and higher are shown the checkAnswer button after the number of attempts 
                                                                   # is exceeded. (typically it replaces the SubmitAnswer button)
       check_answers_after_due_date                    => "guest", # guests, students and higher can see the checkAnswer button after the due date
       check_answers_after_answer_date                 => "guest", # guests, students and higher can see the checkAnswer button after the answer date

The gritty details of showing checkAnswers (Problem.pm)

sub can_checkAnswers {
       my ($self, $User, $EffectiveUser, $Set, $Problem, $submitAnswers) = @_;
       my $authz = $self->r->authz;
       my $thisAttempt = $submitAnswers ? 1 : 0;
       if (before($Set->open_date)) {
               return $authz->hasPermissions($User->user_id,"check_answers_before_open_date");
       } elsif (between($Set->open_date, $Set->due_date)) {
               my $max_attempts = $Problem->max_attempts;
               my $attempts_used = $Problem->num_correct +$Problem->num_incorrect + $thisAttempt;
               if ($max_attempts == -1 or $attempts_used < $max_attempts) {
                       return $authz->hasPermissions($User->user_id, "check_answers_after_open_date_with_attempts");
               } else {
                       return $authz->hasPermissions($User->user_id, "check_answers_after_open_date_without_attempts");
               }
       } elsif (between($Set->due_date, $Set->answer_date)) {
               return $authz->hasPermissions($User->user_id, "check_answers_after_due_date");
       } elsif (after($Set->answer_date)) {
               return $authz->hasPermissions($User->user_id, "check_answers_after_answer_date");
       }
}