WeBWorK Problems

Can't get rid of "This answer is equivalent to the one you just submitted" message

Can't get rid of "This answer is equivalent to the one you just submitted" message

by Peter Selinger -
Number of replies: 3
Hello,

the problem Library/NAU/setLinearAlgebra/P3span.pg always produces the message "This answer is equivalent to the one you just submitted", if and only if the student enters an answer that is (a) correct, and (b) different from the previous answer (even if, for example, the previous answer was incorrect).

The message is obviously produced in error; for example, x^2 and x^2+1 might both be correct, but are most certainly not equivalent. And a correct answer that follows an incorrect one is certainly not equivalent to it.

The original problem author even put a comment in the .pg file pointing out this issue, but also could not figure out what causes it.

I tried various things, including checking for

$self->{_filter_name} ne 'produce_equivalence_message',

setting or unsetting showEqualErrors, or setting $ansHash->{prev_ans}='';

None of them made any difference. I have not had the same issue in other problems that are coded similarly.

How can I get rid of this warning message?

Thanks, -- Peter
In reply to Peter Selinger

Re: Can't get rid of "This answer is equivalent to the one you just submitted" message

by Peter Selinger -

Here is an update on this issue. Reminder: in problems that involve a custom answer evaluator for MathObjects of type function, the message "This answer is equivalent to the one you just submitted" is consistently issued whether or not the answer is actually equivalent. An OPL problem that illustrates the issue is Library/NAU/setLinearAlgebra/P3span.pg, but the same problem occurs in other handmade problems.

The bug clearly seems to be in PGfunctionevaluators.pl, but the logic there is so complicated that I cannot figure out what is supposed to happen, much less what actually happens. The part of the code that checks whether the answer is equivalent to the previous answer seems to be broken.

I would like to fix this bug, or better, to simply turn off this post-filter altogether, as I am not sure why anybody would care whether their answer is equivalent to one they just submitted, especially if they are both correct.

The only workaround I have found is to invoke the answer checker like this:

$ans = Formula('x^2 + x + 1');

$ans_eval = $ans->cmp(checker => ~~&checker);
$ans_eval->install_post_filter(
   sub {
     my $rh_ans = shift;
     if ($rh_ans->{ans_message} eq "This answer is equivalent to the one you just submitted.") {
       $rh_ans->{ans_message} = "";
     }
     return $rh_ans;
   }
);
ANS($ans_eval);

It is a very brute force way of working around the bug, but it's the only thing I can apparently do from the problem author's side of things, as there doesn't seem to be any other flag I can set to get rid of that message.

Maybe this will be useful to someone. Better yet if we manage to fix the bug in PGfunctionevaluators.pl.

In reply to Peter Selinger

Re: Can't get rid of "This answer is equivalent to the one you just submitted" message

by Davide Cervone -
This is not a actually bug, it is a feature (as described below). Also, the code in PGfunctionevaluators.pl that produces this message is not used any longer, as the function there is the original answer checker before it was replaced by a MathObject-based one. The code that actually produces the message is the post-filter in the pg/lib/Value/AnswerChecker.pm file. The key line is line 1731, which calls Value::cmp_compare() to compare the student's current answer to the previous one. This uses the exact same test that is done to see if a student answer matches the correct one (but considering the previous answer as the correct one in this case).

Note that if a custom checker is provided, the custom checker is used in checking the previous answer and the current one as well (that is because the custom checker is what tells if the answer is right, i.e., it matches the correct answer, so that is what tells whether the current answer matches previous one).

In your problem, however, the custom checker doesn't use the previous answer in any way; it simply checks if a certain matrix based on the student answer has a certain rank. So if the student's answer is correct, it will be counted as correct no matter what it is compared to, including the (possibly incorrect) previous answer. So the checking thinks the answer evaluator thinks the current answer matches the previous one, because the custom checker says it does.

So how do you prevent that message? Well, the check against previous answers is done in a post-filter for the answer evaluator, so you simply have to remove that. Unfortunately, PG doesn't give you a mechanism of removing just one post-filter (you can remove all of them at once, but that is it -- it would be nice to be able to remove one by its filter name). On the other hand, it turns out that this check is the last post-filter installed, so you can get rid of it by popping it off the list, as in the following: 

$ans = Formula('x^2 + x + 1');
$ans_eval = $ans->cmp(checker => ~~&checker);
pop(@{$ans_eval->{post_filters}});
This, of course, will mean that the message is never presented. An better solution might be to install a new post-filter that does a straight formula comparison (rather than Value::cmp_compare()) to decide if the current answer matches the previous one.

In any case, this is the correct behavior for the problem as it is currently written. The problem needs to be modified in order to remove (or replace) the post-filter since the custom checker does not actually compare the two answers it receives.

In reply to Peter Selinger

Re: Can't get rid of "This answer is equivalent to the one you just submitted" message

by Nathan Wallach -
I created a pull request with a small change to PG to enable easily disabling the equivalence message by passing "bypass_equivalence_test => 1" as part of the arguments to the cmp call.

See: https://github.com/openwebwork/pg/pull/497

Hopefully it will make its way into the develop branch and then into WW 2.16.
People with access to modify the main PG code on their server can just make the minor change in the PR to lib/Value/AnswerChecker.pm and try it out.