WeBWorK Problems

Merging custom answer checkers with PostFilter Answer hints

Merging custom answer checkers with PostFilter Answer hints

by tim Payer -
Number of replies: 14
We are trying to combine to aspects of answer checkers: PostFilter AnswerHints with a custom answer checker and are having trouble with the syntax.

Can anyone help with this?

The following code block below (suggested by Danny Glin) works fine. In this block an accuracy score, "$cor", is passed from a GeoGebra applet as a string could be read as an integer score and divided by 100 to be converted into a webwork score ($s/100) for the given assignment.

Context("ArbitraryString");
NAMED_ANS( 'answerBox' => String($cor)->cmp( checker=>sub {
my( $c, $s, $self) =@_;
$c =~ s/> //; ## Stripping "> " of "> 95%", leaving "95%"
$c =~ s/%//; ## Stripping "%" of "95%", leaving "95"
$s =~ s/%//; ## Stripping "%" of the student accuracy, leaving an integer
return $s/100;
}));

This code block permitted integer percentages instead of rounded scores in 5% blocks for the student. But we would like to include the comments that we had previously had along with this integer score.
What is the syntax that permits a custom answer checker and PostFilter answer hint to work together?

From the original, for example

Context("ArbitraryString");
NAMED_ANS( 'answerBox' => String($cor)->cmp->withPostFilter(AnswerHints(
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //; ## Stripping "> " of "> 95%", leaving "95%"
$c =~ s/%//; ## Stripping "%" of "95%", leaving "95"
$s =~ s/%//; ## Stripping "%" of the student accuracy, leaving an integer
return (($s >= 0) && ($s <= 50)); } => ["Oh no! You must score more than 50%, for any credit. Try again.", score => 0],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
#$sc = Real($s);
return (($s > 50) && ($s <= 60)); } => ["You could do better with another attempt, yes?", score => 0.60],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
#$sc = Real($s);
return (($s > 60) && ($s <= 70)); } => ["Try again. You can manage better than this.", score => 0.70],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
#$sc = Real($s);
return (($s > 70) && ($s <= 75)); } => ["This is okay, But you could do better, yes?", score => 0.75],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
#$sc = Real($s);
return (($s > 75) && ($s <= 80)); } => ["You are getting better, Try another?", score => 0.80],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
#$sc = Real($s);
return (($s > 80) && ($s <= 85)); } => ["Nice Work!", score => 0.85],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
#$sc = Real($s);
return (($s > 85) && ($s <= 90)); } => ["Very Good!", score => 0.90],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
#$sc = Real($s);
return (($s > 90) && ($s <= 95)); } => ["Excellent!", score => 0.95],
sub {
my( $c, $s, $self) =@_;
$c =~ s/> //;
$c =~ s/%//;
$s =~ s/%//;
return $s > $c; } => ["Perfect!", score => 1],
)));

Note that replacing "score => 0.95" with "score =>$s/100" is not accepted by Webwork.

How could we keep the scoring option of "$s/100", while still giving the feedback to the student in these rounded 5% blocks based on their results?

Thanks for your continuing help!

Tim
In reply to tim Payer

Re: Merging custom answer checkers with PostFilter Answer hints

by tim Payer -
Correction" "Two" aspects not "to"..

Note that the phrase in the previous post "not accepted" by Webwork should be more specific:

The integer percentage does come across, but Webwork treats all answers less than 95% as "incorrect". We would like all answers above 60% to be "Correct" but then the student can decide if they want to keep trying for a higher score.

Here is the file if interested....Thanks, Tim
In reply to tim Payer

Re: Merging custom answer checkers with PostFilter Answer hints

by Davide Cervone -
I would not use answerHints for this, but instead just use a more sophisticated answer checker. Here is an example: 

loadMacros("contextArbitraryString.pl");

Context("ArbitraryString");

#
#  Messages in the form [test_score, message, new_score]
#    where the message will be used when the student score is less than
#    test_score, and the score will be set to new_score in that case, when
#    new_score is given.
#
@messages = (
  [.50, "Oh no! You must score more than 50%, for any credit. Try again.", 0],
  [.60, "You could do better with another attempt, yes?"],
  [.70, "Try again. You can manage better than this."],
  [.75, "This is okay, But you could do better, yes?"],
  [.80, "You are getting better, Try another?"],
  [.85, "Nice Work!"],
  [.90, "Very Good!"],
  [.95, "Excellent!"],
  [1.0, "Perfect!"],
);

TEXT(ans_rule(10));

ANS(String("100%")->cmp(
  checker => sub {
    my ($c, $s, $ans) = @_;
    $s =~ s/%//;
    my $score = $s/100;
    for my $data (@messages) {
      my ($percent, $msg, $nscore) = @$data;
      if ($score <= $percent) {
        $ans->{ans_message} = $msg;
        $score = $nscore if defined $nscore;
        last;
      }
    }
    return $score;
  }
));

This uses a list of cut-off scores and their associated messages (plus an optional new score to use when that message is given), and the answer checker loops through these scores and selected the appropriate message, which is places in the answer hash by hand, modifying the score to the new score, if needed.

Hope that does the trick.

In reply to Davide Cervone

Re: Merging custom answer checkers with PostFilter Answer hints

by tim Payer -
Thank you Micheal,

But there There is an error with the use of this custom checker:


The complaint is that there was no evaluator for question labeled 'answerBox', which is what the javascript is using to carry value from ggb to ww.

I should have included the problem to give more context.

I tried inserting " 'answerBox' =>" before "String" but this did not work.

How does one incorporate 'answerBox' with this custom answer checker?

Thank you so much for answering this specific case!

Tim


In reply to tim Payer

Re: Merging custom answer checkers with PostFilter Answer hints

by Davide Cervone -
You would need to use NAMED_ANS() not ANS(), and add answerBox => before the String() call.
In reply to Davide Cervone

Re: Merging custom answer checkers with PostFilter Answer hints

by tim Payer -
Unfortunately, I still don't have the right syntax...

Here is what I had used:

@messages = (
[.50, "Oh no! You must score more than 50%, for any credit. Try again.", 0],
[.60, "You could do better with another attempt, yes?"],
[.70, "Try again. You can manage better than this."],
[.75, "This is okay, But you could do better, yes?"],
[.80, "You are getting better, Try another?"],
[.85, "Nice Work!"],
[.90, "Very Good!"],
[.95, "Excellent!"],
[1.0, "Perfect!"],
);

TEXT(ans_rule(10));

NAMED_ANS('answerBox' =>String($cor)->cmp(
checker => sub {
my ($c, $s, $ans) = @_;
$s =~ s/%//;
my $score = $s/100;
for my $data (@messages) {
my ($percent, $msg, $nscore) = @$data;
if ($score <= $percent) {
$ans->{ans_message} = $msg;
$score = $nscore if defined $nscore;
last;
}
}
return $score;
}
));

Thanks for the help!

Tim
In reply to tim Payer

Re: Merging custom answer checkers with PostFilter Answer hints

by Michael Gage -
I don't think you need 

TEXT(ans_rule(10));

That puts an extra answer rule in (named AnSwEr0001 since you didn't explicitly name it) and causes the error since you didn't create an answer evaluator for it using ANS(). I'm assuming you didn't want any obvious answer blank in the problem.


The geogebra applet handler creates a hidden answer blank explicitly

named 'answerBox'.

Incidentally setting one of the parameters debug=>1 (this can be done in a couple of places in the code) will show all of the hidden answer blanks. This can make the behavior of the geogebra applet less mysterious.





In reply to Michael Gage

Re: Merging custom answer checkers with PostFilter Answer hints

by tim Payer -
Thank you Michael,

That worked!

We had to insert " 'answerBox' => " before the String() command below and that did the trick:

Context("ArbitraryString");
@messages = (
[.50, "Oh no! You must score more than 50%, for any credit. Try again.", 0],
[.60, "You could do better with another attempt, yes?"],
[.70, "Try again. You can manage better than this."],
[.75, "This is okay, But you could do better, yes?"],
[.80, "You are getting better, Try another?"],
[.85, "Nice Work!"],
[.90, "Very Good!"],
[.95, "Excellent!"],
[1.0, "Perfect!"],
);


NAMED_ANS('answerBox' =>String("100%")->cmp(
checker => sub {
my ($c, $s, $ans) = @_;
$s =~ s/%//;
my $score = $s/100;
for my $data (@messages) {
my ($percent, $msg, $nscore) = @$data;
if ($score <= $percent) {
$ans->{ans_message} = $msg;
$score = $nscore if defined $nscore;
last;
}
}
return $score;
}
));

Thanks for the customized answer checker! I think we can adapt this for other applications too.

Best, Tim


In reply to tim Payer

Re: Merging custom answer checkers with PostFilter Answer hints

by Andrew Parker -
I've folded everyone's suggestions into a working example for this GeoGebra applet. I think this makes for an excellent showcase of what's possible in WeBWorK:

* The applet consistently displays a randomized polynomial based on the WeBWorK random seed (no longer re-randomizes each time a student submits their answer);

* A static image is dynamically generated for each randomization (along with a solution image as well);

* The answer checker recognizes (and gives an error message) when a student submits a response with the solution already displayed in the applet;

* Partial credit is granted according to the accuracy score, with any accuracy > 90% receiving full credit

The only thing I think this problem could improve on is somehow delaying students' ability to display the correct answer within the applet (e.g. by forcing them to get to at least 50% accuracy before this option appears).

Also, the minimal answer checker from my previous sample code wasn't quite airtight (perhaps you noticed), but a few minor adjustments have been made - along with adding Davide's very clever messaging loop.

-Andrew
In reply to Andrew Parker

Re: Merging custom answer checkers with PostFilter Answer hints

by tim Payer -
Thank you Andrew!

We will use your version to explore further!

We are still experimenting to see if permitting new functions into the model (less persistence) is workable to provide more practice to the students.

I appreciate all the work you put in setting this up!

Tim


In reply to tim Payer

Re: Merging custom answer checkers with PostFilter Answer hints

by Michael Gage -
There are also two mechanisms in webwork for controlling persistence when you want students to practice different versions.

which can be set from the Course Configuration and HomeworkSetEditor pages.

and

let's you determine randomization from within the problem itself.

There is also ShowMeAnother. http://webwork.maa.org/wiki/Show_Me_Another

and the whole Just-In-Time (JIT) capability.
and

There are a lot of choices here and I don't think anyone has figured which one is best (or even if there is a best one).


In reply to Michael Gage

Re: Merging custom answer checkers with PostFilter Answer hints

by Danny Glin -
I'll throw in my opinion here...

I think that WeBWorK problems should show consistent output when loaded with the same seed. Since there are mechanisms for re-randomizing problems within a homework set, I don't think it is necessary to perform the re-randomization within the problem. Also, for me the expected behaviour is that when I load a WeBWorK problem it will always show the same version unless I've specifically requested that it be re-randomized.

This is the way that PG behaves, so it only gets complicated when using external integrations. The R server connection has been set up so that the problem seed from WeBWorK is passed to R, which means that the behaviour is consistent with PG. I would suggest that for other integrations (Sage, GeoGebra, etc.) that either the randomization be done in PG then passed to the external system, or that the problem seed be used to initialize the environment in the external system.

Not sure if this warrants its own thread?
In reply to Andrew Parker

Re: Merging custom answer checkers with PostFilter Answer hints

by Davide Cervone -
Thanks for your work on this. Two small comments:

1) You say 90% gets full credit, but only marked full credit on 95% and above. (Easily fixed).

2) I think you want to use \% rather than \text{%} in your correct_ans_latex_string, as I think that is used in the PDF output when correct answers are requested, and the % will act as a comment character (losing the rest of the line) in actual LaTeX. I haven't checked it, but that would be my suspicion.

Otherwise, good work.
In reply to Davide Cervone

Re: Merging custom answer checkers with PostFilter Answer hints

by Andrew Parker -
1) Doesn't the logic for this comparison essentially say that if you're <= $percent, you get your partial credit replaced by $nscore? So at the .95 level, you'll hit that range with anything >.90

2) I didn't consider that - I now have some answer graders that need updating ;)
In reply to Andrew Parker

Re: Merging custom answer checkers with PostFilter Answer hints

by Davide Cervone -
1) Doesn't the logic for this comparison essentially say that if you're <= $percent, you get your partial credit replaced by $nscore?

You are right, of course. My bad! Sorry!