WeBWorK Main Forum

relative tolerance cmp vs num_cmp

Re: relative tolerance cmp vs num_cmp

by Davide Cervone -
Number of replies: 0
Yes, this is intentional. The legacy answer checkers like num_cmp used relative tolerance as a percentage value, while absolute tolerance was not. This was confusing, so MathObjects used non-percentage values for both. That way, a tolerance of .01 means roughly 2 significant figures, whereas .01 in num_cmp would mean roughly 4 significant figures. Because this seems non-intuitive, MathObjects went with non-percentage values. This does lead to an inconsistency with the older routines, but for backward compatibility, that could not be helped. (The num_cmp tolerances used to be set via tol and relTol, not tolerance, so when MathObjects were original developed, the inconsistency was less obvious).

In any case, the is the way it is, and there is little to do about it now without breaking existing problems.

Note that num_cmp is deprecated in favor of the MathObject checkers.