Forum archive 2000-2006

Michael Gage - Annotated example of an answer evaluator

Michael Gage - Annotated example of an answer evaluator

by Arnold Pizer -
Number of replies: 0
inactiveTopicAnnotated example of an answer evaluator topic started 4/26/2003; 8:48:23 PM
last post 4/26/2003; 8:48:23 PM
userMichael Gage - Annotated example of an answer evaluator  blueArrow
4/26/2003; 8:48:23 PM (reads: 670, responses: 0)

The structure of the AnswerEvaluator created by fun_cmp.

(The AnswerEvaluator and AnswerHash objects are defined in the file

For historical reasons, and backwords compatibility, the code for creating the fun_cmp AnswerEvaluator is contained in the subroutine FUNCTION_CMP in

	my $answer_evaluator = new AnswerEvaluator;
rf_correct_ans => $rh_correct_ans->{rf_correct_ans},
evaluation_points => \@evaluation_points,
ra_param_vars => \@PARAMS,
ra_vars => \@VARS,
type => 'function',
correct_ans => $originalCorrEqn,
debug => $func_params{debug}
ra_vars => \@VARS,debug=>$func_params{debug},);
# @VARS has been guaranteed to be an array,
$var might be a single string.
%func_params, param_vars => \@PARAMS);
$answer_evaluator->install_evaluator(\&calculate_difference_vector, %func_params);
$answer_evaluator->install_evaluator(\&is_zero_array, tolerance => $tol );
$answer_evaluator->install_post_filter(sub {
my $rh_ans = shift; $rh_ans->clear_error('SYNTAX'); $rh_ans;
} );
sub {my $rh_ans = shift;
if ($rh_ans->catch_error('EVAL') ) {
$rh_ans->{ans_message} = $rh_ans->{error_message};

First we create a new, empty, AnswerEvaluator object.

The student's answer is evaluated by a series of filters. The first filter might check syntax, the next would compile a subroutine from the string entered by the student, etc. Each filter is passed the AnswerHash in which the intermediate results have been stored, calculates using this data, perhaps modifies the AnswerHash and then returns the AnswerHash so that it can be passed on to the next filter.

We start by initializing the AnswerHash object.

We store a reference to the compiled subroutine which computes the correct answer. (This subroutine has been compiled from the answer string entered by the instructor. See the description of the Answer Evaluator factory to see how these constants and the ones below are prepared from the answers and options written in the PG problem.)

Next we enter a reference to the array of points where the instructors and student's answers are to be evaluated and compared.

If there are instructor parameters that need to be adapted to the student's answer (for example a constant of integration) then this is entered in the ra_param_vars list.

Next is the list of names of the independent variables for the function (e.g. 'x', 'y').

Finally we have an identifier for the type of answer evaluator, and a copy of the string entered by the instructor (the correct answer reported to the student after the due date for the problem.)

The construction of the AnswerEvaluator continues by adding filters to the AnswerEvaluator's filter queue. The queue is in three parts: prefilters, evaluators, and postfilters. The post filters process errors. The prefilters and evaluators behave in the same way, but conceptually they perform different functions.

The first filtercheck_syntax checks the student's string for syntax errors. If there is an error (indicated by a flag in the AnswerHash) then the rest of the prefilters and evaluators are skipped, and the AnswerHash goes directly to the postfilter queue.

Next the function_from_string2 filter creates a subroutine from the student's string.

The correct values for the parameters are determined in the next pre_filter. This prefilter has access to the key/value pairs in %func_params.

The first evaluator calculates the difference between the student's function and the professor's function at each of the evaluation points.

The is_zero_array filter examines this list of points and determines whether it is close enough to zero. (This is done by making sure that the maximum value is below some tolerance. A different filter could use an L^2 norm, or it could make sure that 4 out of 5 of the values were small.)

The first postfilter is an anonymous subroutine which removes the "SYNTAX" error flag. The check_syntax() filter has already stored an error message in the AnswerHash, so no more cleanup work needs to be done.

The second postfilter will only be activated if there was an error during evaluation. It makes sure that the error_message is passed on to the student (in the ans_message slot) and clears the error.

The AnswerEvaluator could be modified by changing or adding filters. For example, the postFilter handling the syntax error could be modified to add an answer message which suggested a table where allowable symbols are defined. It could do further checks on the student's answer and tell it the correct label to use for the independent variable (if that was incorrect).

We study more about the internal workings of the filters from which an AnswerEvaluator can be constructed in the how to write a new filter.

<| Post or View Comments |>