PARSER-BASED ANSWER CHECKERS
The new parser is designed to be used in two ways. First, you can use
it within your perl code when writing problems as a means of making it
easier to handle formulas, and in particular, to genarate be able to
use a single object to produce numeric values, TeX output and answer
strings. This avoids having to type a function three different ways
(which makes maintaining a problem much harder). Since the parser
also included vector and complex arthimatic, it is easier to work with
these types of values as well.
The second reason for the parser is to use it to process student
input. This is accomplished through special answer checkers that are
part of the Parser package (rather than the traditional WeBWorK answer
checkers). Checkers are available for all the types of values that
the parser can produce (numbers, complex numbers, infinities, points,
vectors, intervals, unions, formulas, lists of numbers, lists of
points, lists of intervals, lists of formulas returning numbers, lists
of formulas returning points, and so on).
To use one of these checkers, simply call the ->cmp method of the
object that represents the correct answer. For example:
$n = Real(sqrt(2));
ANS($n->cmp);
will produce an answer checker that matches the square root of two.
Similarly,
ANS(Vector(1,2,3)->cmp);
matches the vector <1,2,3> (or any computation that produces it, e.g.,
i+2j+3k, or <4,4,4>-<3,2,1>), while
ANS(Interval("(-inf,3]")->cmp);
matches the given interval. Other examples include:
ANS(Infinity->cmp);
ANS(String('NONE')->cmp);
ANS(Union("(-inf,$a) U ($a,inf)")->cmp);
and so on.
Formulas are handled in the same way:
ANS(Formula("x+1")->cmp);
$a = random(-5,5,1); $b = random(-5,5,1); $x = random(-5,5,1);
$f = Formula("x^2 + $a x + $b")->reduce;
ANS($f->cmp);
ANS($f->eval(x=>$x)->cmp);
$x = Formula('x');
ANS((1+$a*$x)->cmp);
Context("Vector")->variables->are(t=>'Real');
$v = Formula("<t,t^2,t^3>"); $t = random(-5,5,1);
ANS($v->cmp);
ANS($v->eval(t=>$t)->cmp);
and so on.
Lists of items can be checked as easily:
ANS(List(1,-1,0)->cmp);
ANS(List(Point($a,$b),Point($a,-$b))->cmp);
ANS(List(Vector(1,0,0),Vector(0,1,1))->cmp);
ANS(Compute("(-inf,2),(4,5)")->cmp); # easy way to get list of intervals
ANS(Formula("x, x+1, x^2-1")->cmp);
ANS(Formula("<x,2x>,<x,-2x>,<0,x>")->cmp);
ANS(List('NONE')->cmp);
and so on. The last example may seem strange, as you could have used
ANS(String('NONE')->cmp), but there is a reason for using this type
of construction. You might be asking for one or more numbers (or
points, or whatever) or the word 'NONE' of there are no numbers (or
points). If you used String('NONE')->cmp, the student would get an
error message about a type mismatch if he entered a list of numbers,
but with List('NONE')->cmp, he will get appropriate error messages for
the wrong entries in the list.
It is often appropriate to use the list checker in this way even when
the correct answer is a single value, if the student might type a list
of answers.
On the other hand, using the list checker has its disadvantages. For
example, if you use
ANS(Interval("(-inf,3]")->cmp);
and the student enters (-inf,3), she will get a message indicating
that the type of interval is incorrect, while that would not be the
case if
ANS(List(Interval("(-inf,3]"))->cmp);
were used. (This is because the student doesn't know how many
intervals there are, so saying that the type of interval is wrong
would inform her that there is only one.)
The rule of thumb is: the individual checkers can give more detailed
information about what is wrong with the student's answer; the list
checker allows a wider range of answers to be given without giving
away how many answers there are. If the student knows there's only
one, use the individual checker; if there may or may not be more than
one, use the list checker.
Note that you can form lists of formulas as well. The following all
produce the same answer checker:
ANS(List(Formula("x+1"),Formula("x-1"))->cmp);
ANS(Formula("x+1,x-1")->cmp); # easier
$f = Formula("x+1"); $g = Formula("x-1");
ANS(List($f,$g)->cmp);
$x = Formula('x');
ANS(List($x+1,$x-1)->cmp);
See the files in webwork2/doc/parser/problems for more
examples of using the parser's answer checkers.
CONTROLLING THE DETAILS OF THE ANSWER CHECKERS:
The action of the answer checkers can be modified by passing flags to
the cmp() method. For example:
ANS(Real(pi)->cmp(showTypeWarnings=>0));
will prevent the answer checker from reporting errors due to the
student entering in the wrong type of answer (say a vector rather than
a number).
There are a number of flags common to all the checkers:
showTypeWarnings=>1 or 0 show/don't show messages about student
answers not being of the right type.
(default: 1)
showEqualErrors=>1 or 0 show/don't show messages produced by
trying to compare the professor and
student values for equality, e.g.,
conversion errors between types.
(default: 1)
ignoreStrings=>1 or 0 show/don't show type mismatch errors
produced by strings (so that 'NONE' will
not cause a type mismatch in a checker
looking for a list of numbers, for example).
(default: 1)
In addition to these, the individual types have their own flags:
Real()->cmp:
ignoreInfinity=>1 or 0 Don't report type mismatches if the
student enters an infinity.
(default: 1)
String()->cmp:
typeMatch=>value Specifies the type of object that
the student should be allowed to enter
(in addition the string).
(default: 'Value::Real')
Point()->cmp:
showDimensionHints=>1 or 0 show/don't show messages about the
wrong number of coordinates.
(default: 1)
showCoordinateHints=>1 or 0 show/don't show message about
which coordinates are right.
(default: 1)
Vector()->cmp:
showDimensionHints=>1 or 0 show/don't show messages about the
wrong number of coordinates.
(default: 1)
showCoordinateHints=>1 or 0 show/don't show message about
which coordinates are right.
(default: 1)
promotePoints=>1 or 0 do/don't allow the student to
enter a point rather than a vector.
(default: 1)
parallel=>1 or 0 Mark the answer as correct if it
is parallel to the professor's answer.
Note that a value of 1 forces
showCoordinateHints to be 0.
(default: 0)
sameDirection=>1 or 0 During a parallel check, mark the
answer as correct only if it is in
the same (not the opposite)
direction as the professor's answer.
(default: 0)
Matrix()->cmp:
showDimensionHints=>1 or 0 show/don't show messages about the
wrong number of coordinates.
(default: 1)
The default for showEqualErrors is set to 0 for Matrices, since
these errors usually are dimension errors, and that is handled
separately (and after the equality check).
Interval()->cmp:
showEndpointHints=>1 or 0 do/don't show messages about which
endpoints are correct.
(default: 1)
showEndTypeHints=>1 or 0 do/don't show messages about
whether the open/closed status of
the enpoints are correct (only
shown when the endpoints themselves
are correct).
(default: 1)
Union()->cmp and
List()->cmp:
all the flags from the Real()->cmp, plus:
showHints=>1 or 0 do/don't show messages about which
entries are incorrect.
(default: $showPartialCorrectAnswers)
showLengthHints=>1 or 0 do/don't show messages about having the
correct number of entries (only shown
when all the student answers are
correct but there are more needed, or
all the correct answsers are among the
ones given, but some extras were given).
(default: $showPartialCorrectAnswers)
partialCredit=>1 or 0 do/don't give partial credit for when
some answers are right, but not all.
(default: $showPartialCorrectAnswers)
(currently the default is 0 since WW
can't handle partial credit properly).
ordered=>1 or 0 give credit only if the student answers
are in the same order as the
professor's answers.
(default: 0)
entry_type=>'a (name)' The string to use in error messages
about type mismatches.
(default: dynamically determined from list)
list_type=>'a (name)' The string to use in error messages
about numbers of entries in the list.
(default: dynamically determined from list)
typeMatch=>value Specifies the type of object that
the student should be allowed to enter
in the list (determines what
constitutes a type mismatch error).
(default: dynamically determined from list)
requireParenMatch=>1 or 0
Do/don't require the parentheses in the
student's answer to match those in the
professor's answer exactly.
(default: 1)
removeParens=>1 or 0 Do/don't remove the parentheses from the
professor's list as part of the correct
answer string. This is so that if you
use List() to create the list (which
doesn't allow you to control the parens
directly), you can still get a list
with no parentheses.
(default: 0 for List() and 1 for Formula())
Formula()->cmp:
The flags for formulas are dependent on the type of the result of
the formula. If the result is a list or union, it gets the flags
for that type above, otherwise it gets that flags of the Real
type above.
More flags need to be added in order to allow more control over the
answer checkers to give the full flexibility of the traditional
WeBWorK answer checkers. Note that some things, like whether trig
functions are allowed in the answer, are controlled through the
Context() rather than the answer checker itself. For example,
Context()->functions->undefine('sin','cos','tan');
would remove those three functions from use. (One would need to remove
cot, sec, csc, arcsin, asin, etc., to do this properly; there could be
a function call to do this.)
Similarly, which arithmetic operations are available is controlled
through Context()->operations.
The tolerances used in comparing numbers are part of the Context as
well. You can set these via:
Context()->flags->set(
tolerance => .0001, # the relative or absolute tolerance
tolType => 'relative', # or 'absolute'
zeroLevel => 1E-14, # when to use zeroLevelTol
zeroLevelTol => 1E-12, # smaller than this matches zero
# when one of the two is less
# than zeroLevel
limits => [-2,2], # limits for variables in formulas
num_points => 5, # the number of test points
);
[These need to be handled better.]
Note that for testing formulas, you can override the limits and
num_points settings by setting these fields of the formula itself:
$f = Formula("sqrt(x-10)");
$f->{limits} = [10,12];
$f = Formula("log(xy)");
$f->{limits} = [[.1,2],[.1,2]]; # x and y limits
You can also specify the test points explicitly:
$f = Formula("sqrt(x-10)");
$f->{test_points} = [[11],[11.5],[12]];
$f = Formula("log(xy)");
$f->{test_points} = [[.1,.1],[.1,.5],[.1,.75],
[.5,.1],[.5,.5],[.5,.75]];
[There still needs to be a means of handling the tolerances similarly,
and through the ->cmp() call itself.]
The new parser is designed to be used in two ways. First, you can use
it within your perl code when writing problems as a means of making it
easier to handle formulas, and in particular, to genarate be able to
use a single object to produce numeric values, TeX output and answer
strings. This avoids having to type a function three different ways
(which makes maintaining a problem much harder). Since the parser
also included vector and complex arthimatic, it is easier to work with
these types of values as well.
The second reason for the parser is to use it to process student
input. This is accomplished through special answer checkers that are
part of the Parser package (rather than the traditional WeBWorK answer
checkers). Checkers are available for all the types of values that
the parser can produce (numbers, complex numbers, infinities, points,
vectors, intervals, unions, formulas, lists of numbers, lists of
points, lists of intervals, lists of formulas returning numbers, lists
of formulas returning points, and so on).
To use one of these checkers, simply call the ->cmp method of the
object that represents the correct answer. For example:
$n = Real(sqrt(2));
ANS($n->cmp);
will produce an answer checker that matches the square root of two.
Similarly,
ANS(Vector(1,2,3)->cmp);
matches the vector <1,2,3> (or any computation that produces it, e.g.,
i+2j+3k, or <4,4,4>-<3,2,1>), while
ANS(Interval("(-inf,3]")->cmp);
matches the given interval. Other examples include:
ANS(Infinity->cmp);
ANS(String('NONE')->cmp);
ANS(Union("(-inf,$a) U ($a,inf)")->cmp);
and so on.
Formulas are handled in the same way:
ANS(Formula("x+1")->cmp);
$a = random(-5,5,1); $b = random(-5,5,1); $x = random(-5,5,1);
$f = Formula("x^2 + $a x + $b")->reduce;
ANS($f->cmp);
ANS($f->eval(x=>$x)->cmp);
$x = Formula('x');
ANS((1+$a*$x)->cmp);
Context("Vector")->variables->are(t=>'Real');
$v = Formula("<t,t^2,t^3>"); $t = random(-5,5,1);
ANS($v->cmp);
ANS($v->eval(t=>$t)->cmp);
and so on.
Lists of items can be checked as easily:
ANS(List(1,-1,0)->cmp);
ANS(List(Point($a,$b),Point($a,-$b))->cmp);
ANS(List(Vector(1,0,0),Vector(0,1,1))->cmp);
ANS(Compute("(-inf,2),(4,5)")->cmp); # easy way to get list of intervals
ANS(Formula("x, x+1, x^2-1")->cmp);
ANS(Formula("<x,2x>,<x,-2x>,<0,x>")->cmp);
ANS(List('NONE')->cmp);
and so on. The last example may seem strange, as you could have used
ANS(String('NONE')->cmp), but there is a reason for using this type
of construction. You might be asking for one or more numbers (or
points, or whatever) or the word 'NONE' of there are no numbers (or
points). If you used String('NONE')->cmp, the student would get an
error message about a type mismatch if he entered a list of numbers,
but with List('NONE')->cmp, he will get appropriate error messages for
the wrong entries in the list.
It is often appropriate to use the list checker in this way even when
the correct answer is a single value, if the student might type a list
of answers.
On the other hand, using the list checker has its disadvantages. For
example, if you use
ANS(Interval("(-inf,3]")->cmp);
and the student enters (-inf,3), she will get a message indicating
that the type of interval is incorrect, while that would not be the
case if
ANS(List(Interval("(-inf,3]"))->cmp);
were used. (This is because the student doesn't know how many
intervals there are, so saying that the type of interval is wrong
would inform her that there is only one.)
The rule of thumb is: the individual checkers can give more detailed
information about what is wrong with the student's answer; the list
checker allows a wider range of answers to be given without giving
away how many answers there are. If the student knows there's only
one, use the individual checker; if there may or may not be more than
one, use the list checker.
Note that you can form lists of formulas as well. The following all
produce the same answer checker:
ANS(List(Formula("x+1"),Formula("x-1"))->cmp);
ANS(Formula("x+1,x-1")->cmp); # easier
$f = Formula("x+1"); $g = Formula("x-1");
ANS(List($f,$g)->cmp);
$x = Formula('x');
ANS(List($x+1,$x-1)->cmp);
See the files in webwork2/doc/parser/problems for more
examples of using the parser's answer checkers.
CONTROLLING THE DETAILS OF THE ANSWER CHECKERS:
The action of the answer checkers can be modified by passing flags to
the cmp() method. For example:
ANS(Real(pi)->cmp(showTypeWarnings=>0));
will prevent the answer checker from reporting errors due to the
student entering in the wrong type of answer (say a vector rather than
a number).
There are a number of flags common to all the checkers:
showTypeWarnings=>1 or 0 show/don't show messages about student
answers not being of the right type.
(default: 1)
showEqualErrors=>1 or 0 show/don't show messages produced by
trying to compare the professor and
student values for equality, e.g.,
conversion errors between types.
(default: 1)
ignoreStrings=>1 or 0 show/don't show type mismatch errors
produced by strings (so that 'NONE' will
not cause a type mismatch in a checker
looking for a list of numbers, for example).
(default: 1)
In addition to these, the individual types have their own flags:
Real()->cmp:
ignoreInfinity=>1 or 0 Don't report type mismatches if the
student enters an infinity.
(default: 1)
String()->cmp:
typeMatch=>value Specifies the type of object that
the student should be allowed to enter
(in addition the string).
(default: 'Value::Real')
Point()->cmp:
showDimensionHints=>1 or 0 show/don't show messages about the
wrong number of coordinates.
(default: 1)
showCoordinateHints=>1 or 0 show/don't show message about
which coordinates are right.
(default: 1)
Vector()->cmp:
showDimensionHints=>1 or 0 show/don't show messages about the
wrong number of coordinates.
(default: 1)
showCoordinateHints=>1 or 0 show/don't show message about
which coordinates are right.
(default: 1)
promotePoints=>1 or 0 do/don't allow the student to
enter a point rather than a vector.
(default: 1)
parallel=>1 or 0 Mark the answer as correct if it
is parallel to the professor's answer.
Note that a value of 1 forces
showCoordinateHints to be 0.
(default: 0)
sameDirection=>1 or 0 During a parallel check, mark the
answer as correct only if it is in
the same (not the opposite)
direction as the professor's answer.
(default: 0)
Matrix()->cmp:
showDimensionHints=>1 or 0 show/don't show messages about the
wrong number of coordinates.
(default: 1)
The default for showEqualErrors is set to 0 for Matrices, since
these errors usually are dimension errors, and that is handled
separately (and after the equality check).
Interval()->cmp:
showEndpointHints=>1 or 0 do/don't show messages about which
endpoints are correct.
(default: 1)
showEndTypeHints=>1 or 0 do/don't show messages about
whether the open/closed status of
the enpoints are correct (only
shown when the endpoints themselves
are correct).
(default: 1)
Union()->cmp and
List()->cmp:
all the flags from the Real()->cmp, plus:
showHints=>1 or 0 do/don't show messages about which
entries are incorrect.
(default: $showPartialCorrectAnswers)
showLengthHints=>1 or 0 do/don't show messages about having the
correct number of entries (only shown
when all the student answers are
correct but there are more needed, or
all the correct answsers are among the
ones given, but some extras were given).
(default: $showPartialCorrectAnswers)
partialCredit=>1 or 0 do/don't give partial credit for when
some answers are right, but not all.
(default: $showPartialCorrectAnswers)
(currently the default is 0 since WW
can't handle partial credit properly).
ordered=>1 or 0 give credit only if the student answers
are in the same order as the
professor's answers.
(default: 0)
entry_type=>'a (name)' The string to use in error messages
about type mismatches.
(default: dynamically determined from list)
list_type=>'a (name)' The string to use in error messages
about numbers of entries in the list.
(default: dynamically determined from list)
typeMatch=>value Specifies the type of object that
the student should be allowed to enter
in the list (determines what
constitutes a type mismatch error).
(default: dynamically determined from list)
requireParenMatch=>1 or 0
Do/don't require the parentheses in the
student's answer to match those in the
professor's answer exactly.
(default: 1)
removeParens=>1 or 0 Do/don't remove the parentheses from the
professor's list as part of the correct
answer string. This is so that if you
use List() to create the list (which
doesn't allow you to control the parens
directly), you can still get a list
with no parentheses.
(default: 0 for List() and 1 for Formula())
Formula()->cmp:
The flags for formulas are dependent on the type of the result of
the formula. If the result is a list or union, it gets the flags
for that type above, otherwise it gets that flags of the Real
type above.
More flags need to be added in order to allow more control over the
answer checkers to give the full flexibility of the traditional
WeBWorK answer checkers. Note that some things, like whether trig
functions are allowed in the answer, are controlled through the
Context() rather than the answer checker itself. For example,
Context()->functions->undefine('sin','cos','tan');
would remove those three functions from use. (One would need to remove
cot, sec, csc, arcsin, asin, etc., to do this properly; there could be
a function call to do this.)
Similarly, which arithmetic operations are available is controlled
through Context()->operations.
The tolerances used in comparing numbers are part of the Context as
well. You can set these via:
Context()->flags->set(
tolerance => .0001, # the relative or absolute tolerance
tolType => 'relative', # or 'absolute'
zeroLevel => 1E-14, # when to use zeroLevelTol
zeroLevelTol => 1E-12, # smaller than this matches zero
# when one of the two is less
# than zeroLevel
limits => [-2,2], # limits for variables in formulas
num_points => 5, # the number of test points
);
[These need to be handled better.]
Note that for testing formulas, you can override the limits and
num_points settings by setting these fields of the formula itself:
$f = Formula("sqrt(x-10)");
$f->{limits} = [10,12];
$f = Formula("log(xy)");
$f->{limits} = [[.1,2],[.1,2]]; # x and y limits
You can also specify the test points explicitly:
$f = Formula("sqrt(x-10)");
$f->{test_points} = [[11],[11.5],[12]];
$f = Formula("log(xy)");
$f->{test_points} = [[.1,.1],[.1,.5],[.1,.75],
[.5,.1],[.5,.5],[.5,.75]];
[There still needs to be a means of handling the tolerances similarly,
and through the ->cmp() call itself.]
Last modified: Saturday, 22 February 2020, 3:11 PM