Forum archive 2000-2006

Gavin LaRose - using existing answer evaluators as filters

Gavin LaRose - using existing answer evaluators as filters

by Arnold Pizer -
Number of replies: 0
inactiveTopicusing existing answer evaluators as filters topic started 8/27/2003; 3:32:38 PM
last post 8/28/2003; 9:20:00 AM
userGavin LaRose - using existing answer evaluators as filters  blueArrow
8/27/2003; 3:32:38 PM (reads: 1186, responses: 4)
Hi all,

It seems to me that it ought to be possible to use existing answer evaluators in new ones. However, my limited brain power is insufficient to accomplish this. I tried the simplest thing I could think of, which is the following.

sub newEvaluator {
my $cAns = shift();
my %opts = @_;
my $evaluator = new AnswerEvaluator();
$evaluator->install_evaluator(~~&fun_cmp, %opts);
return $evaluator;

And then called my new evaluator with ANS(newEvaluator($answer)). But when I do this, I get the error message

Error in /cgi-bin/webwork-cgi/cgi-scripts/
Can't locate object method "error_flag" via package AnswerEvaluator
at /opt/www/webwork/system/lib// line 939.

Which seems to say that my simple-minded approach is, well, too simple-minded. If anyone could tell me how to do this correctly, I'd be appropriately grateful.


<| Post or View Comments |>

userMichael Gage - Re: using existing answer evaluators as filters  blueArrow
8/27/2003; 5:39:34 PM (reads: 1421, responses: 0)
Hi Gavin,

It should work more or less this way, but it doesn't quite yet. Give me a few hours to give you an example of using one answer evaluator inside another and I'll post it here. See you later tonight (or tomorrow morning)


<| Post or View Comments |>

userMichael Gage - Re: using existing answer evaluators as filters  blueArrow
8/27/2003; 9:54:48 PM (reads: 1408, responses: 0)
I've written some simple code that illustrates what has to be done:

Here's the source

## A very simple first problem


# This should be the first executable line in the problem.


$showPartialCorrectAnswers = 1;

$a = random(-10,-1,1);
$b = random(1,11,1);
$c = random(1,11,1);
$d = random(1,11,1);

This problem demonstrates how you enter numerical answers into WeBWorK. $PAR
Evaluate the expression \(3($a )($b -$c -2($d ))\):
\{ ans_rule(10) \}


$ans = 3*($a)*($b-$c-2*($d));

# Here are the important lines where we build a new
# answer evaluator using the old.
# The new answer evaluator doesn't do much. Just adds
# the message 'experimental'.

$old_ans_eval = num_cmp($ans);

# notice that num_cmp() is a function which PRODUCES an answer evaluator
# or technically a pointer to the answer evaluator object. I prefer to think
# of $old_ans_eval as actually containing the answer evaluator object.

$new_eval = new AnswerEvaluator;

# now we have a new, empty answer evaluator

$new_eval->install_evaluator(sub {my $rh_ans=shift;
$rh_ans = $old_ans_eval->evaluate($rh_ans->{student_ans});

# We have added a new filter to the new answer evaluator.
# The structure of a filter is that it is supposed to be a subroutine
# which accepts an answerHash object and returns an answerHash object.
# The first problem is that an AnswerEvaluator is not subroutine
# (it's more complicated) so one can't use
# &$old_ans_eval($answer_hash) to process an answer hash, you need to say
# $old_ans_eval->evaluate($answer_hash).
# For this routine we wrap a subroutine around the old answer evaluator
# so that it will behave correctly.
# [Note: The evaluate routine in AnswerEvaluator should (IMHO) be modified
# so it can accept filters which are answer evaluators as well as ones
# which are ordinary subroutines. This won't be too hard, only a few hours work. ]

# Secondly, even that might not work -- most answer evaluators, the ones built
# with all of the AnswerEvaluator tools accept either
# a string input, OR an AnswerHash input. Some of the older answer evalatuors,
# may not do this reliably
# -- there has not yet been a lot of reuse of answer evaluators so
# there has not been much testing in this area.
# That appears to be the case with num_cmp type evaluators, so
# instead of feeding it an answerHash
# we find the student answer inside the AnswerHash and feed it that. It will
# take some work, but
# eventually we should be able to refit all answer evaluators
# so that they will accept either a
# string or an AnswerHash and can therefore be used one inside the other.
# At the moment the evaluators produced by str_cmp aren't
# even AnswerEvaluator objects, they are subroutines!

# The final thing the filter does is to add the string 'experimental' to the
# answer message of the AnswerHash, just to prove that we've actually done
# something. You could do much more.

# Final note -- I often use $rh_ans, the $rh_ means it's a pointer (reference) to a
# hash value, this way I can remember that the scalar variable holds something
# more complicated than a single value. I also use $r_ for reference, and $ra for
# a reference to an array. Unfortunately I'm not consistent about this, but I use
# technique when I'm likely to get confused as to what is a value and what is a pointer.


# This should be the last executable line in the problem.

Here is what the problem looks like.


(1 pt) rochesterLibrary/setSampleAnswers/
This problem demonstrates how you enter numerical answers into WeBWorK.

Evaluate the expression 3(-4 )(2 -3 -2(3 )):





<| Post or View Comments |>

userGavin LaRose - Re: using existing answer evaluators as filters  blueArrow
8/28/2003; 8:38:52 AM (reads: 1387, responses: 0)
Hi Mike,

Thanks, as usual, for your rapid and extremely useful help. The essence of your solution was my second attempt to get this to work, but it didn't quite get to where it needed to before I decided that I couldn't afford more time to play with it (I lost way more time than I intended on forgetting that backslashes in problem code are actually double tildes...).

The end result? What seems as if it ought to be a much simpler answer evaluator than I've arrived at to evaluate answers involving differentials. For example, in the problem "If the area of a rectangle is A(x,y) = xy, find the differential of this function: dA = [ ]." The correct answer is dA = y dx + x dy, and we want to evaluate the differentials (dx, dy) as single variables. To do this, I wrote an evaluator that first filters the dx, dy (actually, filters d[var] for all variables [var] specified in the answer evaluator options) into new variables P,R, etc. (I skipped Q because that's used for the function evaluator up to a constant), then uses fun_cmp to evaluate the resulting function, and then filters P,R, etc. back to the original variables. I'm pasting it in below. Comments for making it more streamlined or elegant are welcomed.

sub diffl_fun_cmp {
my $correctAns = shift();
my %opts = @_;

my @cAnsList =
((ref($correctAns) eq 'ARRAY')? @{$correctAns}:($correctAns));

# we rely on $opts{'var'} being defined as an array reference
if ( defined( $opts{'var'} ) ) {
$opts{'var'} =
( ref($opts{'var'}) ? $opts{'var'} : [$opts{'var'}] );
} else {
$opts{'var'} = [ 'x' ];

my @outEvaluators = ();
foreach my $cAns ( @cAnsList ) {
push(@outEvaluators, diffl_eval($cAns, %opts));
return (wantarray) ? @outEvaluators : $outEvaluators[0];
sub diffl_eval {
my $cAns = shift();
my %opts = @_;

my $evaluator = new AnswerEvaluator('correct_ans' => $cAns,
'type' => 'diffl_fun_cmp',
'original_correct_ans' =>
$evaluator->install_pre_filter( ~~&replace_differentials_filter,
%opts );
$evaluator->install_evaluator( ~~&diffl_fun_eval, %opts );
$evaluator->install_post_filter( ~~&restore_vars_filter, %opts );
return $evaluator;
sub diffl_fun_eval {
my $rh_ans=shift;
my %opts = @_;

my $cAns = $rh_ans->{'correct_ans'};

my @vars = ( @{$opts{'var'}}, @{$rh_ans->{'added_vars'}} );
$opts{'var'} = [ @vars ];

my $func_eval = fun_cmp($cAns, %opts);
$rh_ans = $func_eval->evaluate( $rh_ans->{'student_ans'} );

return $rh_ans;
sub replace_differentials_filter {
my $rh_ans = shift();
my %opts = @_;

my $student_input = $rh_ans->input();
my $correct_answer = $rh_ans->{'correct_ans'};

my @subs = ( 'P', 'R', 'S', 'M', 'N', 'O' );
my @addedVars = ();
for ( my $i=0; $i<@{$opts{'var'}}; $i++ ) {
$student_input =~ s/d$opts{'var'}->[$i]/$subs[$i]/g;
$correct_answer =~ s/d$opts{'var'}->[$i]/$subs[$i]/g;
push( @addedVars, $subs[$i] );
$rh_ans->{'correct_ans'} = $correct_answer;
$rh_ans->{'added_vars'} = [ @addedVars ];

return $rh_ans;
sub restore_vars_filter {
my $rh_ans = shift();
my %opts = @_;

my $student_input = $rh_ans->input();
my $correct_answer = $rh_ans->{'correct_ans'};
my $original_input = $rh_ans->{'original_student_ans'};
my $text_preview = $rh_ans->{'preview_text_string'};
my $latex_preview = $rh_ans->{'preview_latex_string'};

my @subs = ( 'P', 'R', 'S', 'M', 'N', 'O' );
for (my $i=0; $i<@{$opts{'var'}}; $i++) {
$student_input =~ s/$subs[$i]/d$opts{'var'}->[$i]/g;
$correct_answer =~ s/$subs[$i]/d$opts{'var'}->[$i]/g;
$original_input =~ s/$subs[$i]/d$opts{'var'}->[$i]/g;
$text_preview =~ s/$subs[$i]/d$opts{'var'}->[$i]/g;
$latex_preview =~ s/$subs[$i]/d$opts{'var'}->[$i]/g;

$rh_ans->{'correct_ans'} = $correct_answer;
$rh_ans->{'original_student_ans'} = $original_input;
$rh_ans->{'preview_text_string'} = $text_preview;
$rh_ans->{'preview_latex_string'} = $latex_preview;
delete( $rh_ans->{'added_vars'} );

return $rh_ans;

Thanks again, Mike

<| Post or View Comments |>

userMichael Gage - Re: using existing answer evaluators as filters  blueArrow
8/28/2003; 9:20:00 AM (reads: 1399, responses: 0)
Hi Gavin,

Yeah, I've lost a lot of time over the years on the double tilde problem as well (and I wrote the interface!).  Unfortunately it seems still seems the best compromise between the requirements of using backslashes in tex and backslashes in perl. 

The possibility of reusing code to modify existing answer evaluators is one I'm very interested in and your
needs provide a good example of what facilities need to be made available.  The implementation of the
"filter" framework for answer evaluators is a good step in this direction -- but I'm afraid it stalled a  bit once I
had satisfied my immediate needs. 

What's most needed is documentation and tutorial examples. At the moment
there is only a little documentation in AnswerHash and the rest of the purpose and functionality has to be inferred from some of the better code examples, such as fun_cmp and reading the code in AnswerHash and AnswerEvaluator.

The other two needs I've been aware of  are to (1) modify the answer evaluator method so that filters can be either anonymous subroutines (for simple procedures) or answer evaluators (when more complicated processes are involved) and (2) to rewrite all of the current answer evaluators to correspond to the new scheme.

I know that John Jones has been pretty active in writing new answer evaluators using the "filter" framework.  Perhaps he, you and others have further suggestions for the answer evaluator framework.

I'm in the midst of setting up fall courses, so it will be a week or so before I can look more closely and what you have working above. I would like to call attention to two subroutines in  for handling options in a  uniform way:




both in

It's my current thinking that it is a good idea to  include these in  any but the most trivial of  filters or
answer evaluators.  They may not be important right away, but as people modify the filter, adding options
etc. , the presence of those subroutines will encourage the programmers to use them.  In the future these subroutines themselves might be modified to allow discovery of available options, better debugging features,
and other improvements, so it would be helpful if they were in widespread use so that new features are immediately available to old answer evaluators.

Glad you're playing with this feature.


<| Post or View Comments |>