PARSER-BASED ANSWER CHECKERS

The new parser is designed to be used in two ways. First, you can use

it within your perl code when writing problems as a means of making it

easier to handle formulas, and in particular, to genarate be able to

use a single object to produce numeric values, TeX output and answer

strings. This avoids having to type a function three different ways

(which makes maintaining a problem much harder). Since the parser

also included vector and complex arthimatic, it is easier to work with

these types of values as well.

The second reason for the parser is to use it to process student

input. This is accomplished through special answer checkers that are

part of the Parser package (rather than the traditional WeBWorK answer

checkers). Checkers are available for all the types of values that

the parser can produce (numbers, complex numbers, infinities, points,

vectors, intervals, unions, formulas, lists of numbers, lists of

points, lists of intervals, lists of formulas returning numbers, lists

of formulas returning points, and so on).

To use one of these checkers, simply call the ->cmp method of the

object that represents the correct answer. For example:

$n = Real(sqrt(2));

ANS($n->cmp);

will produce an answer checker that matches the square root of two.

Similarly,

ANS(Vector(1,2,3)->cmp);

matches the vector <1,2,3> (or any computation that produces it, e.g.,

i+2j+3k, or <4,4,4>-<3,2,1>), while

ANS(Interval("(-inf,3]")->cmp);

matches the given interval. Other examples include:

ANS(Infinity->cmp);

ANS(String('NONE')->cmp);

ANS(Union("(-inf,$a) U ($a,inf)")->cmp);

and so on.

Formulas are handled in the same way:

ANS(Formula("x+1")->cmp);

$a = random(-5,5,1); $b = random(-5,5,1); $x = random(-5,5,1);

$f = Formula("x^2 + $a x + $b")->reduce;

ANS($f->cmp);

ANS($f->eval(x=>$x)->cmp);

$x = Formula('x');

ANS((1+$a*$x)->cmp);

Context("Vector")->variables->are(t=>'Real');

$v = Formula("<t,t^2,t^3>"); $t = random(-5,5,1);

ANS($v->cmp);

ANS($v->eval(t=>$t)->cmp);

and so on.

Lists of items can be checked as easily:

ANS(List(1,-1,0)->cmp);

ANS(List(Point($a,$b),Point($a,-$b))->cmp);

ANS(List(Vector(1,0,0),Vector(0,1,1))->cmp);

ANS(Compute("(-inf,2),(4,5)")->cmp); # easy way to get list of intervals

ANS(Formula("x, x+1, x^2-1")->cmp);

ANS(Formula("<x,2x>,<x,-2x>,<0,x>")->cmp);

ANS(List('NONE')->cmp);

and so on. The last example may seem strange, as you could have used

ANS(String('NONE')->cmp), but there is a reason for using this type

of construction. You might be asking for one or more numbers (or

points, or whatever) or the word 'NONE' of there are no numbers (or

points). If you used String('NONE')->cmp, the student would get an

error message about a type mismatch if he entered a list of numbers,

but with List('NONE')->cmp, he will get appropriate error messages for

the wrong entries in the list.

It is often appropriate to use the list checker in this way even when

the correct answer is a single value, if the student might type a list

of answers.

On the other hand, using the list checker has its disadvantages. For

example, if you use

ANS(Interval("(-inf,3]")->cmp);

and the student enters (-inf,3), she will get a message indicating

that the type of interval is incorrect, while that would not be the

case if

ANS(List(Interval("(-inf,3]"))->cmp);

were used. (This is because the student doesn't know how many

intervals there are, so saying that the type of interval is wrong

would inform her that there is only one.)

The rule of thumb is: the individual checkers can give more detailed

information about what is wrong with the student's answer; the list

checker allows a wider range of answers to be given without giving

away how many answers there are. If the student knows there's only

one, use the individual checker; if there may or may not be more than

one, use the list checker.

Note that you can form lists of formulas as well. The following all

produce the same answer checker:

ANS(List(Formula("x+1"),Formula("x-1"))->cmp);

ANS(Formula("x+1,x-1")->cmp); # easier

$f = Formula("x+1"); $g = Formula("x-1");

ANS(List($f,$g)->cmp);

$x = Formula('x');

ANS(List($x+1,$x-1)->cmp);

See the files in webwork2/doc/parser/problems for more

examples of using the parser's answer checkers.

CONTROLLING THE DETAILS OF THE ANSWER CHECKERS:

The action of the answer checkers can be modified by passing flags to

the cmp() method. For example:

ANS(Real(pi)->cmp(showTypeWarnings=>0));

will prevent the answer checker from reporting errors due to the

student entering in the wrong type of answer (say a vector rather than

a number).

There are a number of flags common to all the checkers:

showTypeWarnings=>1 or 0 show/don't show messages about student

answers not being of the right type.

(default: 1)

showEqualErrors=>1 or 0 show/don't show messages produced by

trying to compare the professor and

student values for equality, e.g.,

conversion errors between types.

(default: 1)

ignoreStrings=>1 or 0 show/don't show type mismatch errors

produced by strings (so that 'NONE' will

not cause a type mismatch in a checker

looking for a list of numbers, for example).

(default: 1)

In addition to these, the individual types have their own flags:

Real()->cmp:

ignoreInfinity=>1 or 0 Don't report type mismatches if the

student enters an infinity.

(default: 1)

String()->cmp:

typeMatch=>value Specifies the type of object that

the student should be allowed to enter

(in addition the string).

(default: 'Value::Real')

Point()->cmp:

showDimensionHints=>1 or 0 show/don't show messages about the

wrong number of coordinates.

(default: 1)

showCoordinateHints=>1 or 0 show/don't show message about

which coordinates are right.

(default: 1)

Vector()->cmp:

showDimensionHints=>1 or 0 show/don't show messages about the

wrong number of coordinates.

(default: 1)

showCoordinateHints=>1 or 0 show/don't show message about

which coordinates are right.

(default: 1)

promotePoints=>1 or 0 do/don't allow the student to

enter a point rather than a vector.

(default: 1)

parallel=>1 or 0 Mark the answer as correct if it

is parallel to the professor's answer.

Note that a value of 1 forces

showCoordinateHints to be 0.

(default: 0)

sameDirection=>1 or 0 During a parallel check, mark the

answer as correct only if it is in

the same (not the opposite)

direction as the professor's answer.

(default: 0)

Matrix()->cmp:

showDimensionHints=>1 or 0 show/don't show messages about the

wrong number of coordinates.

(default: 1)

The default for showEqualErrors is set to 0 for Matrices, since

these errors usually are dimension errors, and that is handled

separately (and after the equality check).

Interval()->cmp:

showEndpointHints=>1 or 0 do/don't show messages about which

endpoints are correct.

(default: 1)

showEndTypeHints=>1 or 0 do/don't show messages about

whether the open/closed status of

the enpoints are correct (only

shown when the endpoints themselves

are correct).

(default: 1)

Union()->cmp and

List()->cmp:

all the flags from the Real()->cmp, plus:

showHints=>1 or 0 do/don't show messages about which

entries are incorrect.

(default: $showPartialCorrectAnswers)

showLengthHints=>1 or 0 do/don't show messages about having the

correct number of entries (only shown

when all the student answers are

correct but there are more needed, or

all the correct answsers are among the

ones given, but some extras were given).

(default: $showPartialCorrectAnswers)

partialCredit=>1 or 0 do/don't give partial credit for when

some answers are right, but not all.

(default: $showPartialCorrectAnswers)

(currently the default is 0 since WW

can't handle partial credit properly).

ordered=>1 or 0 give credit only if the student answers

are in the same order as the

professor's answers.

(default: 0)

entry_type=>'a (name)' The string to use in error messages

about type mismatches.

(default: dynamically determined from list)

list_type=>'a (name)' The string to use in error messages

about numbers of entries in the list.

(default: dynamically determined from list)

typeMatch=>value Specifies the type of object that

the student should be allowed to enter

in the list (determines what

constitutes a type mismatch error).

(default: dynamically determined from list)

requireParenMatch=>1 or 0

Do/don't require the parentheses in the

student's answer to match those in the

professor's answer exactly.

(default: 1)

removeParens=>1 or 0 Do/don't remove the parentheses from the

professor's list as part of the correct

answer string. This is so that if you

use List() to create the list (which

doesn't allow you to control the parens

directly), you can still get a list

with no parentheses.

(default: 0 for List() and 1 for Formula())

Formula()->cmp:

The flags for formulas are dependent on the type of the result of

the formula. If the result is a list or union, it gets the flags

for that type above, otherwise it gets that flags of the Real

type above.

More flags need to be added in order to allow more control over the

answer checkers to give the full flexibility of the traditional

WeBWorK answer checkers. Note that some things, like whether trig

functions are allowed in the answer, are controlled through the

Context() rather than the answer checker itself. For example,

Context()->functions->undefine('sin','cos','tan');

would remove those three functions from use. (One would need to remove

cot, sec, csc, arcsin, asin, etc., to do this properly; there could be

a function call to do this.)

Similarly, which arithmetic operations are available is controlled

through Context()->operations.

The tolerances used in comparing numbers are part of the Context as

well. You can set these via:

Context()->flags->set(

tolerance => .0001, # the relative or absolute tolerance

tolType => 'relative', # or 'absolute'

zeroLevel => 1E-14, # when to use zeroLevelTol

zeroLevelTol => 1E-12, # smaller than this matches zero

# when one of the two is less

# than zeroLevel

limits => [-2,2], # limits for variables in formulas

num_points => 5, # the number of test points

);

[These need to be handled better.]

Note that for testing formulas, you can override the limits and

num_points settings by setting these fields of the formula itself:

$f = Formula("sqrt(x-10)");

$f->{limits} = [10,12];

$f = Formula("log(xy)");

$f->{limits} = [[.1,2],[.1,2]]; # x and y limits

You can also specify the test points explicitly:

$f = Formula("sqrt(x-10)");

$f->{test_points} = [[11],[11.5],[12]];

$f = Formula("log(xy)");

$f->{test_points} = [[.1,.1],[.1,.5],[.1,.75],

[.5,.1],[.5,.5],[.5,.75]];

[There still needs to be a means of handling the tolerances similarly,

and through the ->cmp() call itself.]

The new parser is designed to be used in two ways. First, you can use

it within your perl code when writing problems as a means of making it

easier to handle formulas, and in particular, to genarate be able to

use a single object to produce numeric values, TeX output and answer

strings. This avoids having to type a function three different ways

(which makes maintaining a problem much harder). Since the parser

also included vector and complex arthimatic, it is easier to work with

these types of values as well.

The second reason for the parser is to use it to process student

input. This is accomplished through special answer checkers that are

part of the Parser package (rather than the traditional WeBWorK answer

checkers). Checkers are available for all the types of values that

the parser can produce (numbers, complex numbers, infinities, points,

vectors, intervals, unions, formulas, lists of numbers, lists of

points, lists of intervals, lists of formulas returning numbers, lists

of formulas returning points, and so on).

To use one of these checkers, simply call the ->cmp method of the

object that represents the correct answer. For example:

$n = Real(sqrt(2));

ANS($n->cmp);

will produce an answer checker that matches the square root of two.

Similarly,

ANS(Vector(1,2,3)->cmp);

matches the vector <1,2,3> (or any computation that produces it, e.g.,

i+2j+3k, or <4,4,4>-<3,2,1>), while

ANS(Interval("(-inf,3]")->cmp);

matches the given interval. Other examples include:

ANS(Infinity->cmp);

ANS(String('NONE')->cmp);

ANS(Union("(-inf,$a) U ($a,inf)")->cmp);

and so on.

Formulas are handled in the same way:

ANS(Formula("x+1")->cmp);

$a = random(-5,5,1); $b = random(-5,5,1); $x = random(-5,5,1);

$f = Formula("x^2 + $a x + $b")->reduce;

ANS($f->cmp);

ANS($f->eval(x=>$x)->cmp);

$x = Formula('x');

ANS((1+$a*$x)->cmp);

Context("Vector")->variables->are(t=>'Real');

$v = Formula("<t,t^2,t^3>"); $t = random(-5,5,1);

ANS($v->cmp);

ANS($v->eval(t=>$t)->cmp);

and so on.

Lists of items can be checked as easily:

ANS(List(1,-1,0)->cmp);

ANS(List(Point($a,$b),Point($a,-$b))->cmp);

ANS(List(Vector(1,0,0),Vector(0,1,1))->cmp);

ANS(Compute("(-inf,2),(4,5)")->cmp); # easy way to get list of intervals

ANS(Formula("x, x+1, x^2-1")->cmp);

ANS(Formula("<x,2x>,<x,-2x>,<0,x>")->cmp);

ANS(List('NONE')->cmp);

and so on. The last example may seem strange, as you could have used

ANS(String('NONE')->cmp), but there is a reason for using this type

of construction. You might be asking for one or more numbers (or

points, or whatever) or the word 'NONE' of there are no numbers (or

points). If you used String('NONE')->cmp, the student would get an

error message about a type mismatch if he entered a list of numbers,

but with List('NONE')->cmp, he will get appropriate error messages for

the wrong entries in the list.

It is often appropriate to use the list checker in this way even when

the correct answer is a single value, if the student might type a list

of answers.

On the other hand, using the list checker has its disadvantages. For

example, if you use

ANS(Interval("(-inf,3]")->cmp);

and the student enters (-inf,3), she will get a message indicating

that the type of interval is incorrect, while that would not be the

case if

ANS(List(Interval("(-inf,3]"))->cmp);

were used. (This is because the student doesn't know how many

intervals there are, so saying that the type of interval is wrong

would inform her that there is only one.)

The rule of thumb is: the individual checkers can give more detailed

information about what is wrong with the student's answer; the list

checker allows a wider range of answers to be given without giving

away how many answers there are. If the student knows there's only

one, use the individual checker; if there may or may not be more than

one, use the list checker.

Note that you can form lists of formulas as well. The following all

produce the same answer checker:

ANS(List(Formula("x+1"),Formula("x-1"))->cmp);

ANS(Formula("x+1,x-1")->cmp); # easier

$f = Formula("x+1"); $g = Formula("x-1");

ANS(List($f,$g)->cmp);

$x = Formula('x');

ANS(List($x+1,$x-1)->cmp);

See the files in webwork2/doc/parser/problems for more

examples of using the parser's answer checkers.

CONTROLLING THE DETAILS OF THE ANSWER CHECKERS:

The action of the answer checkers can be modified by passing flags to

the cmp() method. For example:

ANS(Real(pi)->cmp(showTypeWarnings=>0));

will prevent the answer checker from reporting errors due to the

student entering in the wrong type of answer (say a vector rather than

a number).

There are a number of flags common to all the checkers:

showTypeWarnings=>1 or 0 show/don't show messages about student

answers not being of the right type.

(default: 1)

showEqualErrors=>1 or 0 show/don't show messages produced by

trying to compare the professor and

student values for equality, e.g.,

conversion errors between types.

(default: 1)

ignoreStrings=>1 or 0 show/don't show type mismatch errors

produced by strings (so that 'NONE' will

not cause a type mismatch in a checker

looking for a list of numbers, for example).

(default: 1)

In addition to these, the individual types have their own flags:

Real()->cmp:

ignoreInfinity=>1 or 0 Don't report type mismatches if the

student enters an infinity.

(default: 1)

String()->cmp:

typeMatch=>value Specifies the type of object that

the student should be allowed to enter

(in addition the string).

(default: 'Value::Real')

Point()->cmp:

showDimensionHints=>1 or 0 show/don't show messages about the

wrong number of coordinates.

(default: 1)

showCoordinateHints=>1 or 0 show/don't show message about

which coordinates are right.

(default: 1)

Vector()->cmp:

showDimensionHints=>1 or 0 show/don't show messages about the

wrong number of coordinates.

(default: 1)

showCoordinateHints=>1 or 0 show/don't show message about

which coordinates are right.

(default: 1)

promotePoints=>1 or 0 do/don't allow the student to

enter a point rather than a vector.

(default: 1)

parallel=>1 or 0 Mark the answer as correct if it

is parallel to the professor's answer.

Note that a value of 1 forces

showCoordinateHints to be 0.

(default: 0)

sameDirection=>1 or 0 During a parallel check, mark the

answer as correct only if it is in

the same (not the opposite)

direction as the professor's answer.

(default: 0)

Matrix()->cmp:

showDimensionHints=>1 or 0 show/don't show messages about the

wrong number of coordinates.

(default: 1)

The default for showEqualErrors is set to 0 for Matrices, since

these errors usually are dimension errors, and that is handled

separately (and after the equality check).

Interval()->cmp:

showEndpointHints=>1 or 0 do/don't show messages about which

endpoints are correct.

(default: 1)

showEndTypeHints=>1 or 0 do/don't show messages about

whether the open/closed status of

the enpoints are correct (only

shown when the endpoints themselves

are correct).

(default: 1)

Union()->cmp and

List()->cmp:

all the flags from the Real()->cmp, plus:

showHints=>1 or 0 do/don't show messages about which

entries are incorrect.

(default: $showPartialCorrectAnswers)

showLengthHints=>1 or 0 do/don't show messages about having the

correct number of entries (only shown

when all the student answers are

correct but there are more needed, or

all the correct answsers are among the

ones given, but some extras were given).

(default: $showPartialCorrectAnswers)

partialCredit=>1 or 0 do/don't give partial credit for when

some answers are right, but not all.

(default: $showPartialCorrectAnswers)

(currently the default is 0 since WW

can't handle partial credit properly).

ordered=>1 or 0 give credit only if the student answers

are in the same order as the

professor's answers.

(default: 0)

entry_type=>'a (name)' The string to use in error messages

about type mismatches.

(default: dynamically determined from list)

list_type=>'a (name)' The string to use in error messages

about numbers of entries in the list.

(default: dynamically determined from list)

typeMatch=>value Specifies the type of object that

the student should be allowed to enter

in the list (determines what

constitutes a type mismatch error).

(default: dynamically determined from list)

requireParenMatch=>1 or 0

Do/don't require the parentheses in the

student's answer to match those in the

professor's answer exactly.

(default: 1)

removeParens=>1 or 0 Do/don't remove the parentheses from the

professor's list as part of the correct

answer string. This is so that if you

use List() to create the list (which

doesn't allow you to control the parens

directly), you can still get a list

with no parentheses.

(default: 0 for List() and 1 for Formula())

Formula()->cmp:

The flags for formulas are dependent on the type of the result of

the formula. If the result is a list or union, it gets the flags

for that type above, otherwise it gets that flags of the Real

type above.

More flags need to be added in order to allow more control over the

answer checkers to give the full flexibility of the traditional

WeBWorK answer checkers. Note that some things, like whether trig

functions are allowed in the answer, are controlled through the

Context() rather than the answer checker itself. For example,

Context()->functions->undefine('sin','cos','tan');

would remove those three functions from use. (One would need to remove

cot, sec, csc, arcsin, asin, etc., to do this properly; there could be

a function call to do this.)

Similarly, which arithmetic operations are available is controlled

through Context()->operations.

The tolerances used in comparing numbers are part of the Context as

well. You can set these via:

Context()->flags->set(

tolerance => .0001, # the relative or absolute tolerance

tolType => 'relative', # or 'absolute'

zeroLevel => 1E-14, # when to use zeroLevelTol

zeroLevelTol => 1E-12, # smaller than this matches zero

# when one of the two is less

# than zeroLevel

limits => [-2,2], # limits for variables in formulas

num_points => 5, # the number of test points

);

[These need to be handled better.]

Note that for testing formulas, you can override the limits and

num_points settings by setting these fields of the formula itself:

$f = Formula("sqrt(x-10)");

$f->{limits} = [10,12];

$f = Formula("log(xy)");

$f->{limits} = [[.1,2],[.1,2]]; # x and y limits

You can also specify the test points explicitly:

$f = Formula("sqrt(x-10)");

$f->{test_points} = [[11],[11.5],[12]];

$f = Formula("log(xy)");

$f->{test_points} = [[.1,.1],[.1,.5],[.1,.75],

[.5,.1],[.5,.5],[.5,.75]];

[There still needs to be a means of handling the tolerances similarly,

and through the ->cmp() call itself.]

Last modified: Saturday, 22 February 2020, 3:11 PM