WeBWorK Main Forum

Error with answer checker in Gateway (WW 2.8)

Error with answer checker in Gateway (WW 2.8)

by Teri Christiansen -
Number of replies: 9
Hi:

Please see the screenshot available here:  


I had a student complete a gateway - and upon grading it, green bars showed up on every problem indicating that "all answers are correct".  But, on the problem shown in the screen capture, you can see that one part of one answer is incorrect.  

Can this be easily fixed?

Thanks!

In reply to Teri Christiansen

Re: Error with answer checker in Gateway (WW 2.8)

by Davide Cervone -
You will need to indicate what problem file (from the Problem Library) is being used in order for us to be able to say anything about this. It may have to do with the programming of that problem.
In reply to Davide Cervone

Re: Error with answer checker in Gateway (WW 2.8)

by Teri Christiansen -
Oh - sorry about that!  Here is the problem file source:

# DESCRIPTION
# Gives a function of the form ax^2 + b or ax^2 + bx + c (factorable) and asks to find f(0) and solve 
# f(x) = 0.
# ENDDESCRIPTION

## DBsubject('Algebra')
## DBchapter('Functions')
## DBsection('Evaluation and Solving')
## KEYWORDS('functions', 'function notation', 'function operations', 'function evaluation')
## Author('Rick Lynch')
## Institution('University of Missouri-Columbia')


###########################################################################
# initialization 
###########################################################################
DOCUMENT();
loadMacros(
  "MathObjects.pl",
  "PeriodicRerandomization.pl",
  "PGstandard.pl",
  "fracListChecker.pl", # contains custom checker and restricts context
  "MUHelp.pl",
);

TEXT(beginproblem());
PeriodicRerandomization("3");
$showPartialCorrectAnswers = 1;


###########################################################################
# setup contexts and variables 
###########################################################################
Context("Numeric");

$case = random(0,1,1);
if ($case == 0) {
  Context()->strings->add("no solutions"=>{}, "infinitely many"=>{}, "no solution"=>{alias=>"no   solutions"}, "none"=>{alias=>"no solutions"}, "all reals"=>{alias=>"infinitely many"});
  Context()->variables->are(x=>"Real", X=>"Real", f=>"Real", F=>"Real");
  Context()->flags->set(reduceConstantFunctions=>0);
  $a = non_zero_random(-7,7);
  $n = list_random(2,3,5,7,11);
  $b = -$n*$a;
  $f0 = $b;
  $solns = List(Formula("-sqrt($n)"), Formula("sqrt($n)"));
  $dsolns = "x = -\sqrt{$n}, \sqrt{$n}";
  $dispf = Formula("$a x^2 + $b")->reduce;
} else {
  do {
    $a = random(1,3);
    $b = non_zero_random(-5,5);
    $c = random(1,3);
    $d = non_zero_random(-5,5);
    $c1 = $a*$d + $b*$c;
  } until ($c1 != 0);
  $c0 = $b*$d;
  $c2 = $a*$c;
  $f0 = $c0;
  $dispf = Formula("$c2 x^2 + $c1 x + $c0")->reduce;
  setFracListFunctions(); # set up context for custom checker
  Context()->strings->add("no solutions"=>{}, "infinitely many"=>{}, "no solution"=>{alias=>"no   solutions"}, "none"=>{alias=>"no solutions"}, "all reals"=>{alias=>"infinitely many"});
  Context()->variables->are(x=>"Real", X=>"Real", f=>"Real", F=>"Real");
  $solns = List(Fraction(-$b,$a)->reduce, Fraction(-$d,$c)->reduce);
  $dsolns = "\displaystyle x = $solns";
}


###########################################################################
# state the problem 
###########################################################################
Context()->texStrings;
BEGIN_TEXT
Given that \(f(x) = $dispf\), find \(f(0)\) and solve \(f(x) = 0\).
$PAR
(a) \(f(0) = \) \{ans_rule(10)\} $BR
(b) \(f(x) = 0\) whenever \(x = \) \{ans_rule(20)\}
END_TEXT
Context()->normalStrings;


###########################################################################
# check the answer  
###########################################################################
ANS(Compute($f0)->cmp());
if ($case == 0) {
  ANS($solns->cmp())
} else {
  ANS($solns->cmp(checkFracList()));
}


###########################################################################
# use PeriodicRerandomization to write the answer and generate a new
# version of the problem
###########################################################################
Context()->texStrings;
if ($attempts_modp == 0 && $actualAttempts != 0) {
  BEGIN_TEXT
  $PAR
  ${BBOLD}Answer:${EBOLD} (a) \(f(0) = $f0\), (b) \($dsolns\)
  $PAR
  END_TEXT
} else {
  BEGIN_TEXT
  $PAR
  ${BBOLD}Help:${EBOLD} \{ MUHelp("quadeqns") \}
  $BR
  END_TEXT
}
Context()->normalStrings;
PeriodicStatus(); 

COMMENT('Features Periodic Rerandomization. Edited and updated in 2012/2013.
$BR
Desc: Gives a function of the form ax^2 + b or ax^2 + bx + c (factorable) and asks to find f(0) and solve f(x) = 0.');
ENDDOCUMENT();
In reply to Teri Christiansen

Re: Error with answer checker in Gateway (WW 2.8)

by Gavin LaRose -
Hi Teri,

Three other questions: can you duplicate this error using the same seed for the problem if it is in a regular homework set? What settings to you have for the gateway/quiz assignment (in particular, number of allowed attempts)? And was this on the first submission of the problem?

Thanks,
Gavin
In reply to Gavin LaRose

Re: Error with answer checker in Gateway (WW 2.8)

by Teri Christiansen -
Hi Gavin:

I'm finally getting back to this issue after some time.  I have had this issue come up on a few other problems in the past few weeks, and here are the answers to some of your questions from before:

The error does not duplicate when a problem is part of a homework set.  That is, everything works as expected on a homework set.  It only displays the wrong "All of these answers are correct" green bar on the gateway/quiz, even when not all answers are marked as correct by the grader.

For my gateways, I allow students to have 2 submissions per version.  But, the error is independent of which submission they use - it will show up on the first or second second submission, or even if they continue to just "check" their answers after their score is submitted.

Comparing the different problems that this has occurred on has led me to some more information tonight.  It seems that the issue only happens for the following situation:

(a)  A problem has more than one answer blank.  (Leading to the red/green messages that are printed which state "At least one of these answers is incorrect"/"All of these answers are correct", as opposed to "This answer is not correct"/"This answer is correct")
(b)  One of the correct answers is a list of values (for example, solutions to the equation x^2=4)
(c)  A student types in at least one correct list value into each answer expecting a list (for example, typing 2 or -2 into the answer for the list)

In other words, the green message of "All of these answers are correct" seems to appear if there is anything correct (even one part of a list) in each answer blank, even though WW correctly diagnoses an incorrect answer (missing/wrong parts of the list) by marking off points and indicating this when you ask it to show correct answers.

I hope this makes sense...if you need more information, please let me know. 

Thanks!
-Teri Christiansen
In reply to Teri Christiansen

Re: Error with answer checker in Gateway (WW 2.8)

by Davide Cervone -
The GatewayQuiz module was not taking partial credit into account. I believe that this was taken care of in August at the Portland code camp. The develop branch includes the fix (it was part of the fixes to get the colors applied to the answer blanks properly).
In reply to Davide Cervone

Re: Error with answer checker in Gateway (WW 2.8)

by Patrick Spencer -
David,

Were you referencing commit bae9bd3 on this page:
https://github.com/openwebwork/webwork2/commits/develop?page=2 ?

It seems to me like any fix to this problem would have to change the attemptResults subroutine in the GatewayQuiz.pm file. Specifically this line:

https://github.com/openwebwork/webwork2/blob/master/lib/WeBWorK/ContentGenerator/GatewayQuiz.pm

but this has not changed from version 2.8 to version 2.9. Was the fix you are referring to a javascript fix?

- Patrick
In reply to Patrick Spencer

Re: Error with answer checker in Gateway (WW 2.8)

by Davide Cervone -
The change is in the develop branch, not version 2.9. It has not been included in a release yet, but will be in the 2.10 release, when that is made. The particular commit I'm talking about is

https://github.com/openwebwork/webwork2/commit/a2d96577e99e7ccecc11c11314c672d3e989adda

which does alter the attemptResults subroutine in GatewyaQuiz.pm (see the lines in the 400-450 range).

There are several other commits that also affect the GatewayQuiz module at around that same time. See

https://github.com/openwebwork/webwork2/commits/develop?page=1

for August 11 and 12.