WeBWorK Main Forum

Help with answer tolerances

Help with answer tolerances

by Philip Loewen -
Number of replies: 2

Some of my students are having correct answers marked wrong.

For the problem in question, my latest attempt to fix this is to include this block in the setup:

Context()->flags->set(
  tolerance => 0.001,
  tolType => "relative",
  zeroLevel    => 1E-7,
  zeroLevelTol => 1E-7
);

I'm mostly comfortable using the relative error to compare answers, but being nudged to increase the zeroLevel repeatedly makes me uneasy.

I understand that WW evaluates the student expression and the official one at several random points and compares the results. Usually the test points produce values far enough from 0 that the relative error comes out small and all is well. But some unlucky combinations produce values around 1E-8, and for some of these the relative error exceeds any tolerance I would be comfortable allowing

Specific Questions:

  • What do the flags "zeroLevel" and "zeroLevelTol" really mean and do?
  • Is there a simple standard way to accept an answer that has either a small relative error or a small absolute error?
  • Do you know a better question I should be asking? If so, please share it ... and the answer, too, if known.
Thanks!
In reply to Philip Loewen

Re: Help with answer tolerances

by Danny Glin -

The documentation of these settings can be found at https://webwork.maa.org/wiki/Answer_Checkers_and_the_Context#Tolerances_and_Limits:

zeroLevel    => 1E-14,       # when to use zeroLevelTol
zeroLevelTol => 1E-12,       # smaller than this matches zero
                                  #  when one of the two is less
                                  #  than zeroLevel
I may be wrong on this, but my understanding is that zeroLevel determines the threshold for when to stop using relative tolerance and start using zeroLevelTol.  If we are below that threshold, then treat any number smaller than zeroLevelTol as 0 for the sake of comparisons.

I don't think that there is a way to use a combination of relative and absolute error within an answer checker.

When it comes to checking formulas, I do my best to set the domain of evaluation so that the formula does not evaluate to something very close to 0 (or very large, as this also causes issues with relative tolerance). See https://webwork.maa.org/wiki/FormulaTestPoints for details on this.  Note that you can set the domain based on randomized parameters from your question, so you can do things like

$a = random(1,4,1);
$func = Compute("(x-$a)^5");
$func->{limits} = [$a+1,$a+2];
In many cases this allows you to dodge the zero level issues.
In reply to Danny Glin

Re: Help with answer tolerances

by Philip Loewen -
Thank you for taking the time to prepare this super helpful and informative reply.

(My answers of interest concern a sequence of functions, so they depend on both an integer n and a real variable x in somewhat intricate combinations. Nonetheless, I'll play around with FormulaTestPoints on the weekend.)