WeBWorK Main Forum

num_cmp Mystery

num_cmp Mystery

by Andrew Dabrowski -
Number of replies: 8

I'm puzzled by the behavior of num_cmp in a certain context.  Here's a MWE:


DOCUMENT();        # This should be the first executable line in the problem.

loadMacros(
  "PGbasicmacros.pl",
  "PGanswermacros.pl",
  "PGcombinatorics.pl",
  "MathObjects.pl",
    );

Context() -> variables -> are(t => 'Real');

TEXT(beginproblem());
$df= Formula("500 *pi/6*cos((pi/6)*t)") -> reduce;

BEGIN_TEXT
    When \(t=3\),
    $BR$BR
    
    \(\{$df->TeX\}=\) \{ans_rule(20)\}
END_TEXT

ANS(num_cmp($df->eval(t=> 3)));

ENDDOCUMENT();

The correct answer is pi/6*500*cos(pi/2) = 0, and webwork accepts either as correct.

However 500*cos(pi/2) is marked incorrect, although it is also equal to 0.

I thought at first this might be due to WW evaluating cos(pi/2) numerically and not getting exactly 0.

However that doesn't fly because because 500*cos(pi/2) would be closer to pi/6*500*cos(pi/2) than is 0, but only 0 is accepted as correct.

What's going on here?
In reply to Andrew Dabrowski

Re: num_cmp Mystery

by Alex Jordan -
It could be due to rounding error. Neither thing is really 0, since pi is not really pi.

If I do:

TEXT(
  Real("pi/6*500*cos(pi/2)")->value,
  $BR,
  Real("500*cos(pi/2)")->value,
);

Then I see:
1.60305891143483e-14
3.06161699786838e-14

It does seem strange that the zero level would be in between those numbers, with the first counting as equivalent to 0 and the second one not. But I'm not familiar with how num_cmp works.

In reply to Alex Jordan

Re: num_cmp Mystery

by Andrew Dabrowski -
That's what I meant by "evaluating numerically", rather than symbolically.

I don't see why it is, if the "correct" answer is 1.60305891143483e-14, that 0 is considered within the tolerance, but 3.06161699786838e-14 not.

Does anyone know exactly how num_cmp works?
In reply to Andrew Dabrowski

Re: num_cmp Mystery

by Andras Balogh -

I know I should not respond because I don't know how num_cmp works, but I just can't resist.

  • 100*cos(pi/2) and -100*cos(pi/2) are both marked correct.
  • 200*cos(pi/2) and -200*cos(pi/2) are both marked incorrect.
  • (500*pi/6)*cos(pi/2) is marked correct, it is approximately 260*cos(pi/2), which is marked incorrect along with the smaller 200*cos(pi/2).
It seems to me that it either takes the correct formula  (500*pi/6)*cos(pi/2) or a numerical approximation of 0.



In reply to Andrew Dabrowski

Re: num_cmp Mystery

by Alex Jordan -
If the answer checker was the checker from MathObjects for a Real (so, not num_cmp), then I believe 1e-14 is the "zero level". Anything smaller than that in absolute value is considered equivalent to 0. (Well, really if a value is smaller than that, then comparisons are made absolutely instead of relatively, with a tolerance of 1e-12. So that includes being equivalent to 0.)

Both of these numbers are more than 1e-14. The "zero level" would have to be something like 2e-14 for this to be the explanation, so it could be something else. And I see now Andras's post that suggest it is something else, not a zero level issue.

In reply to Alex Jordan

Re: num_cmp Mystery

by Danny Glin -
It's worth noting that num_cmp is not well-maintained. It is kept around for backwards-compatibility, but for newly authored problems you should be using the MathObjects checkers.
Since your answer is already a MathObject Real, you should be able to do
ANS($df->eval(t=> 3)->cmp());

I don't know if this will solve the issue you describe, but based on Alex's comments it should at least have predictable results when dealing with very small numbers.
In reply to Danny Glin

Re: num_cmp Mystery

by Glenn Rice -

Although you are correct about num_cmp not being maintained and using the cmp method is the correct way to do this now, the cmp method also has the same issue for this. If I am not mistaken num_cmp actually uses the MathObject methods in it's implementation.

I think Davide could shed some light on this situation.

In reply to Glenn Rice

Re: num_cmp Mystery

by Danny Glin -
You are correct that num_cmp uses the MathObject methods in the background, but I don't know how it handles being passed a MathObject directly.
Originally num_cmp expected either a perl number or a perl string as an argument. If you pass it a MathObject as in this example, I don't know if all of the attributes of the object and the context are passed to the checker.
That being said, this does not appear to be the root of the issue being discussed.
In reply to Glenn Rice

Re: num_cmp Mystery

by Davide Cervone -

I think Davide could shed some light on this situation.

Here's how the zero-level values work: when $a is compared to $b, if either value is below the $zeroLevel value, which is 1E-14 by default, then if the difference between the two is less than $zeroLevelTol, which is 1E-12 by default, then they are equal. That is, if abs($a) < $zeroLevel || abs($b) < $zeroLevel, then the two are considered equal if abs($a - $b) < $zeroLevelTol, and are not equal otherwise.

In the case discussed here, the correct value is given by pi/6*500*cos(pi/2), which evaluates to 1.60305891143483e-14, and the test value is 500*cos(pi/2), which is 3.06161699786838e-14, neither of these is less than $zeroLevel, so the usual relative tolerance rules are used, and they are not equal by that measure (since they would have to agree on their first 3 or 4 significant digits).

On the other hand, when testing against 100*cos(pi/2), which is 6.12323399573677e-15, this value is less than $zeroLevel, and so the rules above come into play, and we must compare the difference, which is -9.90735511861148e-15, to the $zeroLevelTol of 1E-14 in absolute value, and since it is less than the tolerance, these are considered equal.

When testing against the answer of 0 itself, since that is less than $zeroLevel, the zero-level rules are used, and since the difference between pi/6*500*cos(pi/2) and 0 is 1.60305891143483e-14, and that is less than the $zeroLevelTol of 1E-12, these are also equal.

Finally, for 260*cos(pi/2), which is 1.59204083889156e-14, neither is smaller than $zeroLevel, and so the usual relative tolerance is used, and this don't match 1.60305891143483e-14 to enough digits, so they aren't equal. On the other hand, 262*cos(pi/2) does match to enough digits, so that does match pi/6*500*cos(pi/2).

So that is how it works, and why these examples do what they do.