MathObjectsbased Answer Checkers
From WeBWorK
 MathObjectsbased Answer Checkers
 Controlling the Details of the Answer Checkers
 Flags common to all answer checkers
 Flags for Real()>cmp
 Flags for String()>cmp
 Flags for Point()>cmp
 Flags for Vector()>cmp
 Flags for Matrix()>cmp
 Flags for Interval()>cmp
 Flags for Set()>cmp, Union()>cmp, and List()>cmp
 Flags for Formula()>cmp
 The Context and Answer Checkers
 Tolerances and Limits
 Custom Answer Checkers
MathObjectsbased Answer Checkers
The MathObject library is designed to be used in two ways. First, you can use it within your perl code when writing problems as a means of making it easier to handle formulas, and in particular, to be able to use a single object to produce numeric values, TeX output, and answer strings from a single formula entry. This avoids having to type a function three different ways (which makes maintaining a problem much harder). Since MathObjects also includes vector and complex arthimetic, it is easier to work with these types of values as well.
Secondly using MathObjects improves the processing of student input. This is accomplished through special answer checkers that are part of the MathObjects package (rather than the traditional WeBWorK answer checkers). Each of these checkers has error checking customized to the type of input expected from the student and can provide helpful feedback if the syntax of the student's entry is incorrect.
Checkers are available for each of the types of values that the parser can produce (numbers, complex numbers, infinities, points, vectors, intervals, sets, unions, formulas, lists of numbers, lists of points, lists of intervals, lists of formulas returning numbers, lists of formulas returning points, and so on).
To use one of these checkers, simply call the cmp
method of the
object that represents the correct answer. For example:
$n = Real(sqrt(2)); ANS($n>cmp);
will produce an answer checker that matches the square root of two. Similarly,
ANS(Vector(1,2,3)>cmp);
matches the vector <1,2,3>
(or any computation that produces it, e.g.,
i+2j+3k
, or <4,4,4><3,2,1>
), while
ANS(Interval("(inf,3]")>cmp);
matches the given interval. Other examples include:
ANS(Infinity>cmp); ANS(String('NONE')>cmp); ANS(Union("(inf,$a) U ($a,inf)")>cmp);
and so on.
Formulas are handled in the same way:
ANS(Formula("x+1")>cmp); $a = random(5,5,1); $b = random(5,5,1); $x = random(5,5,1); $f = Formula("x^2 + $a x + $b")>reduce; ANS($f>cmp); ANS($f>eval(x=>$x)>cmp); $x = Formula('x'); ANS((1+$a*$x)>cmp); Context("Vector")>variables>are(t=>'Real'); $v = Formula("<t,t^2,t^3>"); $t = random(5,5,1); ANS($v>cmp); ANS($v>eval(t=>$t)>cmp);
and so on.
Lists of items can be checked as easily:
ANS(List(1,1,0)>cmp); ANS(List(Point($a,$b),Point($a,$b))>cmp); ANS(List(Vector(1,0,0),Vector(0,1,1))>cmp); ANS(Compute("(inf,2),(4,5)")>cmp); # easy way to get list of intervals ANS(Formula("x, x+1, x^21")>cmp); ANS(Formula("<x,2x>,<x,2x>,<0,x>")>cmp); ANS(List('NONE')>cmp);
and so on. The last example may seem strange, as you could have used
ANS(String('NONE')>cmp)
, but there is a reason for using this
type of construction. You might be asking for one or more numbers (or
points, or whatever) or the word NONE
if there are no numbers (or
points). If you used String('NONE')>cmp
, the student would
get an error message about a type mismatch if he entered a list of
numbers, but with List('NONE')>cmp
, he will get appropriate
error messages for the wrong entries in the list.
In the case of a list of points where the answer is NONE
, you should tell MathObject what the expected type of answer is, so that appropriate messages can be given. E.g.,
ANS(List('NONE')>cmp(typeMatch=>Value::Point);
would give appropriate messages for points rather than numbers (the default type).
It is often appropriate to use the list checker in this way even when the correct answer is a single value, if the student might type a list of answers.
On the other hand, using the list checker has its disadvantages. For example, if you use
ANS(Interval("(inf,3]")>cmp);
and the student enters (inf,3), she will get a message indicating that the type of interval is incorrect, while that would not be the case if
ANS(List(Interval("(inf,3]"))>cmp);
were used. (This is because the student doesn't know how many intervals there are, so saying that the type of interval is wrong would inform her that there is only one.)
The rule of thumb is: the individual checkers can give more detailed information about what is wrong with the student's answer; the list checker allows a wider range of answers to be given without giving away how many answers there are. If the student knows there's only one, use the individual checker; if there may or may not be more than one, use the list checker.
Note that you can form lists of formulas as well. The following all produce the same answer checker:
ANS(List(Formula("x+1"),Formula("x1"))>cmp);
ANS(Formula("x+1,x1")>cmp); # easier
$f = Formula("x+1"); $g = Formula("x1"); ANS(List($f,$g)>cmp);
$x = Formula('x'); ANS(List($x+1,$x1)>cmp);
See the files in pg/doc/MathObjects/problems for more examples of using the parser's answer checkers.
Controlling the Details of the Answer Checkers
The action of the answer checkers can be modified by passing flags to
the cmp()
method. For example:
ANS(Real(pi)>cmp(showTypeWarnings=>0));
will prevent the answer checker from reporting errors due to the student entering the wrong type of answer (say a vector rather than a number).
Flags common to all answer checkers
There are a number of flags common to all the checkers:
showTypeWarnings => 1 or 0

Show/don't show messages about student answers not being of the right type. (default: 1)
showEqualErrors => 1 or 0

Show/don't show messages produced by trying to compare the professor and student values for equality, e.g., conversion errors between types. (default: 1)
ignoreStrings => 1 or 0

Show/don't show type mismatch errors produced by strings (so that
NONE
will not cause a type mismatch in a checker looking for a list of numbers, for example). (default: 1) studentsMustReduceUnions => 1 or 0

Controls whether or not students answers are allowed to include overlapping, redundant, or uncombined intervals. (default: 1)
showUnionReduceWarnings => 1 or 0

Do/don't show warnings for unreduced unions. This includes warnings about overlapping sets, uncombined intervals (like
(0,1] U [1,infinity]
), and repeated elements in sets. (default: 1)
In addition to these, the individual MathObject types have their own flags:
Flags for Real()>cmp
ignoreInfinity => 1 or 0

Do/don't report type mismatches if the student enters an infinity. (default: 1)
Flags for String()>cmp
typeMatch => value

Specifies the type of object that the student should be allowed to enter (in addition the string). (default:
'Value::Real'
)
Flags for Point()>cmp
showDimensionHints => 1 or 0

Show/don't show messages about the wrong number of coordinates. (default: 1)
showCoordinateHints => 1 or 0

Show/don't show message about which coordinates are right. (default: 1)
Flags for Vector()>cmp
showDimensionHints => 1 or 0

Show/don't show messages about the wrong number of coordinates. (default: 1)
showCoordinateHints => 1 or 0

Show/don't show message about which coordinates are right. (default: 1)
promotePoints => 1 or 0

Do/don't allow the student to enter a point rather than a vector. (default: 1)
parallel => 1 or 0

Mark the answer as correct if it is parallel to the professor's answer. Note that a value of 1 forces
showCoordinateHints
to be 0. (default: 0) sameDirection => 1 or 0

During a parallel check, mark the answer as correct only if it is in the same (not the opposite) direction as the professor's answer. (default: 0)
Flags for Matrix()>cmp
showDimensionHints => 1 or 0

Show/don't show messages about the wrong number of coordinates. (default: 1)
The default for showEqualErrors
is set to 0 for Matrices, since
these errors usually are dimension errors, and that is handled
separately (and after the equality check).
Flags for Interval()>cmp
showEndpointHints => 1 or 0

Do/don't show messages about which endpoints are correct. (default: 1)
showEndTypeHints => 1 or 0

Do/don't show messages about whether the open/closed status of the endpoints are correct (only shown when the endpoints themselves are correct). (default: 1)
requireParenMatch => 1 or 0

Do/don't require that the open/closed status of the endpoints be correct. (default: 1)
Flags for Set()>cmp, Union()>cmp, and List()>cmp
All the flags from the Real()>cmp
, plus:
showHints => 1 or 0

Do/don't show messages about which entries are incorrect. (default:
$showPartialCorrectAnswers
) showLengthHints => 1 or 0

Do/don't show messages about having the correct number of entries (only shown when all the student answers are correct but there are more needed, or all the correct answsers are among the ones given, but some extras were given). (default:
$showPartialCorrectAnswers
) partialCredit => 1 or 0

Do/don't give partial credit for when some answers are right, but not all. (default:
$showPartialCorrectAnswers
) ordered => 1 or 0

Give credit only if the student answers are in the same order as the professor's answers. (default: 0)
entry_type => 'a (name)'

The string to use in error messages about type mismatches. (default: dynamically determined from list)
list_type => 'a (name)'

The string to use in error messages about numbers of entries in the list. (default: dynamically determined from list)
extra => $object

A MathObject to use for comparison to studentprovided entries that do not match any of the correct answers in the list. Use in particular when the correct answer is a list of a single String (like
NONE
), and the expected type of elements in the list are complicated and need special syntax checking (e.g., for size of vectors). typeMatch => value

Specifies the type of object that the student should be allowed to enter in the list (determines what constitutes a type mismatch error). (default: dynamically determined from list)
requireParenMatch => 1 or 0

Do/don't require the parentheses in the student's answer to match those in the professor's answer exactly. (default: 1)
removeParens => 1 or 0

Do/don't remove the parentheses from the professor's list as part of the correct answer string. This is so that if you use
List()
to create the list (which doesn't allow you to control the parens directly), you can still get a list with no parentheses. (default: 0 forList()
and 1 forFormula()
) implicitList => 1 or 0

Force/don't force single entry answers to be lists (even if they are already lists). This way, if you are asking for a list of lists, and the student enters a single list, it will be turned into a list of a single list, and will be checked properly against the correct answer. (default: 1)
Flags for Formula()>cmp
The flags for Formulas are dependent on the type of the result of the formula; the flags are taken from the class of the result as described above. In addition, Formulas take the following flags:
upToConstant => 1 or 0

For realvalued formulas, this controls whether the student's answer only needs to match the correct answer up to addition of a constant. (default: 0)
showDomainErrors => 1 or 0

If the test points for the function comparison reveals that the student answer is defined on a different domain from the correct answer (e.g., the student's answer is not defined at one of the test points), then this value determines whether or not a message is issued alerting the student to that fact. (default: 1)
The Context and Answer Checkers
Some things, like whether trig functions are allowed in the answer,
are controlled through the Context()
rather than the answer checker
itself. For example,
Context()>functions>disable('sin','cos','tan');
would remove those three functions from use. One would need to remove
cot
, sec
, csc
, arcsin
, asin
, etc., to do this properly.
Categories of functions can be removed all at once, e.g.
Context()>functions>disable('Trig');
would disable all trig functions, while
Context()>functions>disable('All'); Context()>functions>enable('sqrt');
would allow only the sqrt
function to be used in student answers. The available categories are the following:
SimpleTrig
sin
,cos
,tan
,sec
,csc
,cot
InverseTrig
asin
,acos
,atan
,asec
,acsc
,acot
,arcsin
,arccos
,arctan
,arcsec
,arccsc
,arccot
,atan2
SimpleHyperbolic
sinh
,cosh
,tanh
,sech
,csch
,coth
InverseHyperbolic
asinh
,acosh
,atanh
,asech
,acsch
,acoth
,arcsinh
,arccosh
,arctanh
,arcsech
,arccsch
,arccoth
Numeric
log
,log10
,exp
,sqrt
,abs
,int
,sgn
,ln
,logten
Vector
norm
,unit
Complex
arg
,mod
,Re
,Im
,conj
Hyperbolic
all of
SimpleHyperbolic
andInverseHyperbolic
Trig
all of
SimpleTrig
,InverseTrig
, andHyperbolic
All
Note that some functions can be obtained via operators (e.g.,
abs(x)
can be obtained via x
and sqrt(x)
can be obtained by
x^(1/2)
, so you might need to remove more than just the named
functions to limit these operations).
Which arithmetic operations are available is controlled
through Context()>operations
. For example,
Context()>operations>undefine('^','**');
would disable the ability for students to enter powers. Note that
multiplication and division have several forms (in order to make a
nonstandard precedence that allows things like sin(2x)
to be
entered as sin 2x
). So if you want to disable them you need to
include all of them. E.g.,
Context()>operations>undefine('*',' *','* '); Context()>operations>undefine('/',' /','/ ','//');
would be required in order to make multiplication and division unavailable.
Finally, absolute values are treated as a specialized form of parenthesis, so to remove them, use
Context()>parens>remove('');
The pg/macros directory contains a number of predefined contexts that limit the operations that can be performed in a student answer. For example, the contextLimitedNumeric.pl file defines contexts in which students can enter numbers, but no operations, so they would have to reduce their answer to a single number by hand. There are limited contexts for complex numbers, points, and vectors, and there are also specialized contexts for entering polynomials, or where powers are restricted in various ways.
Tolerances and Limits
The tolerances used in comparing numbers are part of the Context as well. You can set these via:
Context()>flags>set( tolerance => .0001, # the relative or absolute tolerance tolType => 'relative', # or 'absolute' zeroLevel => 1E14, # when to use zeroLevelTol zeroLevelTol => 1E12, # smaller than this matches zero # when one of the two is less # than zeroLevel limits => [2,2], # limits for variables in formulas num_points => 5, # the number of test points );
Note that for testing formulas, you can override these values by setting these fields of the formula itself:
$f = Formula("sqrt(x10)"); $f>{limits} = [10,12];
$f = Formula("log(xy)"); $f>{limits} = [[.1,2],[.1,2]]; # x and y limits
You can also specify the test points explicitly:
$f = Formula("sqrt(x10)"); $f>{test_at} = [[11],[11.5],[12]]; # use these plus random ones
$f = Formula("log(xy)"); $f>{test_points} = [[.1,.1],[.1,.5],[.1,.75], [.5,.1],[.5,.5],[.5,.75]]; # test only at these
You can specify the value at the same time you create the object, as in
$f = Formula("sqrt(x1)")>with(limits=>[10,12]);
It is also possible to set the limits of variables in the context itself,
Context()>variables>set(x => {limits => [10,12]});
or when a variable is created in the Context,
Context()>variables>add(t => ['Real',limits=>[1,2]]);
or even when the answer checker is specified,
ANS($f>cmp(limits=>[10,12]));
Custom Answer Checkers
While the builtin answer checkers do a good job of testing a student response for agreement with a given correct answer, sometimes an answer may require a more sophisticated or customized check. For example, you might ask a student to provide a solution to a given implicit equation for which there are infinitely many solutions and would like to count any of them as correct. You can do this by providing your own answer checker as a parameter to a MathObject answer checker. This lets you get the advantage of the MathObject parsing, syntax checking, type checking, and error messages, while still giving you the control of deciding when the student's answer is correct.
You do this by providing a checker
subroutine, as in the following example:
Context("Point"); $a = random(2,10,1); $x = random(5,5,1); $y = $a  $x; BEGIN_TEXT Find a point \((x,y)\) that is a solution to \(x+y=$a\). $PAR \((x,y)\) = \{ans_rule(15)\} END_TEXT ANS(Point($x,$y)>cmp( showCoordinateHints => 0, # doesn't make sense to give these checker => sub { my ($correct,$student,$ansHash) = @_; # get correct and student MathObjects my ($sx,$sy) = $student>value; # get coordinates of student answer return ($sx + $sy == $a ? 1 : 0); # return 1 of correct, 0 otherwise } ));
The subroutine that is given as the value of the checker
parameter
is called when the student has typed an answer that parses properly
and is compatible with a point, so you don't have to worry about
typechecking the student answer yourself, and are guaranteed to have
a MathObject to work with.
Note that your answer checker is tied to a given MathObject, so the
type checking and error messages are appropriate for that type of
object. Also, this is what will be shown as the correct answer if the
student requests answers after the due date, so you should be sure
that you provide an actual correct answer, even if you don't use
$correct
within your checker.
Your checker subroutiune is passed three items, the correct answer (as
a MathObject), the student's answer (as a MathObject), and a reference
to the AnswerHash that is being used to process this answer. In the
example above, $correct
will be the Point($x,$y)
that was used
to create the original MathObject for the answer checker, and
$student
will be the point that the student typed.
The answer hash ($ansHash
), holds additional data about the answer
checker and the student's answer. That data includes all the flags
passed to the answer checker; e.g., in the example above,
$ansHash>{showCoordinateHints}
will be 0
, while
$ansHash>{showTypeWarnings}
will be 1 (the default). Other
fields that are useful include
$ansHash>{correct_ans}

The correct answer string
$ansHash>{isPreview}

Whether the "Preview" button was pressed or not (you might want to limit error messages when this is true).
$ansHash>{original_student_ans}

The unmodified string originally typed by the student. It has not been processed in any way.
$ansHash>{student_formula}

The Formula obtained from parsing the student answer. If it is a constant value, then
$student
is the result of evaluating this formula (i.e., it will be a Real or Point or some other MathObject rather than a Formula); if it is not constant, then$student
will be this Formula. $ansHash>{student_value}

The value passed to
$student
. $ansHash>{correct_value}

The value passed to
$correct
.
The return value of your checker should be a value between 0 and 1 that indicates how much credit the student's answer should receive. Use 1 for full credit, .5 for 50% credit, and so on.
Error messages can be generated via the Value>Error()
function. For example,
Value>Error("Your value should be positive") if $student <= 0;
Note that this function will cause the checker to return with an error condition, so there is no need to return a score in this case.
Listbased Answer Checkers
The builtin answer checkers for the List, Union, and Set classes are
a bit more complicated than those for the other classes, because the
order of elements in the List (or Intervals in a Union, or elements of
a Set) usually doesn't matter. If you provide a checker
routine
for one of these types of MathObjects, it does not refer to the list
as a whole, but to the individual elements in the list. Management of
the list and determining how many elements are matched, what the
partial credit should be, and what error messages to produce is all
handled by the main list answer checker. You only provide a routine
that determines when a student answer matches a correct answer. Your
checker will be called repeatedly on the various combinations of
student and correct answers to determine if the student list matches
the correct list (regardless of order).
When the checker
is called from the list checker, it has two
additional paramters:
checker => sub { my ($correct,$student,$ansHash,$nth,$value) = @_; ... }
Here $nth
is a word indicating which student answer is being tested
(e.g., for the third element in the student's list, $nth
will be
" third"
(note the leading space). This can be used in error
messages, for example, to help the student identify where an error
occurred. The $value
is the name of the type of object that is in
the list, e.g., "point"
. So you could use "Your$nth $value is
not correct"
as an error message in order to get something like
"Your third point is incorrect"
as the output.
As usual, you can use Value>Error()
to generate an error
message. The list checker will trap it and put it int he message area
preceded by "There is a problem with your nth answer:" (where "nth" is
replaced by the proper word for the answer that produced the error
message. If you want to produce a message that doesn't include this prefix, use
$correct>{context}>setError("message","",undef,undef,$Value::CMP_WARNING); return 0;
where "message" is the error message you want to produce.
If you want to act on the List (or Union or Set) as a whole, then you
need to use the list_checker
parameter instead. This specifies a
subtroutine that will handle checking of the entire list, overriding
the default list checking. That means you will be responsible for
dealing with the possibly different order of the student answers from
the correct answer, providing error messages about individual entries
in the list, and so on.
The list_checker
gets called with four parameters: a reference to
an array containing the list of correct answers, a reference to an
array of the student answers, a reference to the AnswerHash, and a
string containing the name of the type of elements expected (for use
in error messages). So a list checker looks like
list_checker => sub { my ($correct,$student,$ansHash,$value) = @_; ... }
You can refer to the individual entries in $correct
or $student
as $correct>[$i]
or $student>[$i]
, where $i
is an
integer representing the position within the list (the first entry is
numbered 0 not 1).
Note that in a list_checker
for a List object, you will have to do
type checking yourself to check if the types of the entries are
correct, since a list can consist of any type of elements.
The list_checker
should return an array consisting of the nuber of
student entries that are correct followed by any error messages that
should be displayed (each on a separate line in the messages section).
Here is an example of a list checker:
Context("Point"); BEGIN_TEXT Three distinct points \((x,y)\) that satisfy the equation \(x+y=5\) are: \{ans_rule(20)\} END_TEXT ANS(List("(0,5),(1,4),(2,3)")>cmp(list_checker => sub { my ($correct,$student,$ansHash,$value) = @_; my $n = scalar(@$student); # number of student answers my $score = 0; # number of correct student answers my @errors = (); # stores error messages my $i, $j; # loop counters # # Loop though the student answers ## for ($i = 0; $i < $n; $i++) { my $ith = Value::List>NameForNumber($i+1); my $p = $student>[$i]; # ith student answer # # Check that the student's answer is a point # if ($p>type ne "Point") { push(@errors,"Your $ith entry is not a point"); next; } # # Check that the point hasn't been given before # for ($j = 0, $used = 0; $j < $i; $j++) { if ($student>[$j]>type eq "Point" && $student>[$j] == $p) { push(@errors,"Your $ith point is the same as a previous one") unless $ansHash>{isPreview}; $used = 1; last; } } # # If not already used, check that it satisfies the equation # and increase the score if so. # if (!$used) { my ($a,$b) = $p>value; if ($a + $b == 5) {$score++} else { push(@errors,"Your $ith point is not correct") unless $ansHash>{isPreview} } } } # # Check that there are the right number of answers # if (!$ansHash>{isPreview}) { push(@errors,"You need to provide more points") if $i < 3; push(@errors,"You have given too many points") if $score > 3 && $i != $score; } return ($score,@errors); }));
As you can see, list checkers are more complicated than checkers for the other types of data. But for certain situations like the one above, they can be indispensible.
Set and Union Answer Checkers
Since Sets and Unions are unordered lists of Reals or Intervals, they
act a lot like lists. If you provide a checker
for one of these,
it works like the List checker: it is applied to the individual
entries in the Set or the individual intervals in a Union. You may
want to write a checker that works with the entire Set or Union at
once, however. In that case, you need to use a list_checker
in
which you obtain the full Set or Union from $ansHash
. For example:
Context("Interval"); BEGIN_TEXT Find a union of disjoint intervals that properly contains the numbers 0 and 5: \{ans_rule(20)\} END_TEXT ANS(Union("(1,1) U (4,infinity)")>cmp(list_checker => sub { my ($correct,$student,$ansHash) = @_; my $U = $ansHash>{student_value}; # complete student answer as MathObject if ($U>type ne "Union") {return (0,"Your answer should be a union of intervals")} my $n = scalar(@$student); # number of intervals my $S = Set(0,5); return ($U>contains($S) && $U != $S ? $n : 0); # number of intervals correct }));
Note that the return value is the score followed by any error
messages, and that the score is the number of intervals in the
student's answer that are correct, which is why this checker returns
$n
$ when the answer is correct.
More error checking might be desired, here. For example, since a
Union can be of Intervals or Sets, we might want to check that all the
student's entries in the Union are actually intervals (the current
version allows Sets). Note thta because the
studentsMustReduceUnions
parameter is 1 by default, we don't have
to check for disjoint intervals. On the other hand, that flag also
requires intervals to be merged into one if they could be, e.g.,
(0,1] U (1,2)
. We might want to work harder to allow that, since
it is a disjoint union. We could also give more error messages to
help the student work through a wrong answer. For example, we could
check which point isn't covered and report that.