WeBWorK Main Forum

Relative Tolerance in Number with Units

by Clinton Ferreira -
Number of replies: 20
Hi,

I'm trying to get an answer with units to have a relative tolerance for the numerical part (this is for a physics question giving a force in Newtons where the number and range of the decimals needs to be flexible. So far, i'm using

ANS(NumberWithUnits(8.23E-8,"N")->cmp);

which works fine on its own but I'm not sure of how to include relTol=>0.3 in this as it can be with std_num_cmp();

Thanks.

Re: Relative Tolerance in Number with Units

by Michael Gage -
The syntax changes slightly but you can do the same thing

ANS(NumberWithUnits(8.23E-8,"N")->with(tolType=>'relative', tolerance=>'.01')->cmp )

I believe this will give you 1 per cent tolerance, but you might double check that.

-- Mike

Re: Relative Tolerance in Number with Units

by Clinton Ferreira -
I tried what you suggested but got the following error messages:

Warning messages

• No answer provided by instructor for answer AnSwEr2 at line 471 of (eval 214)
• There is no answer evaluator for the question labeled AnSwEr2 at /home/pg/lib/WeBWorK/PG/Translator.pm line 1129
• Error in Translator.pm::process_answers: Answer AnSwEr2:
•  Unrecognized evaluator type || at /home/pg/lib/WeBWorK/PG/Translator.pm line 1153
• Error in Translator.pm::process_answers: Answer AnSwEr2:
•  Answer evaluators must return a hash or an AnswerHash type, not type || at /home/pg/lib/WeBWorK/PG/Translator.pm line 1158

Re: Relative Tolerance in Number with Units

by Michael Gage -
Hi Clinton,

Check for typos in your input. I actually checked my suggestion this time on our local system :-) Here is the snippet I used:

DOCUMENT();

"PGstandard.pl", # Standard macros for PG language
"MathObjects.pl",
"parserNumberWithUnits.pl",
);

BEGIN_TEXT

Enter the answer of 1 Newton \{ans_rule()\}

END_TEXT

ANS( NumberWithUnits(1, 'N')->with(tolType=>'relative', tolerance=>'.01')->cmp );

ENDDOCUMENT();


This accepts .9901 N but not .99 N

In your example the answer is small enough that the answer evaluator might default to absolute error checking in any case.

Hope this helps.

Re: Relative Tolerance in Number with Units

by Clinton Ferreira -
I'm using Parser.pl not MathObjects.pl - I don't know how to add that module.

Would that make a difference?

Re: Relative Tolerance in Number with Units

by Davide Cervone -
No, MathObjects.pl is simply a renaming of Parser.pl (the older name for MathObjects was Parser objects). The name change was made during the summer, so you may not have the new .pl file unless you updated your PG directory since then. You can continue to use Parser.pl for now, or could simply copy pg/macros/Parser.pl to pg/macros/MathObject.pl if you want to begin using the new name.

Davide

Re: Relative Tolerance in Number with Units

by Clinton Ferreira -
But that isn't working, is it...?

if tolerance=>.01 is 1% tolerance then surely 0.99 N should be accepted. 0.9901 is 0.01% of 1 not 1% of 1.

Re: Relative Tolerance in Number with Units

by Davide Cervone -
Values that are right at the edge of the tolerance are always very touchy. For example, .1 is a repeating decimal in binary notation, so can't be represented exactly, and so 1 - .99 is not exactly .1 when handled in finite-precision binary notation. The numeric issues involved can be quite subtle and counter-intuitive in general. You can't rely on the behavior at the very boundaries of the tolerance setting.

In any case, the tolerance uses a strict inequality, not less-than-or-equal, so even with exact arithmetic, .99 would not match a .1 tolerance.

Finally, the error for .9901 (compared to a correct answer of 1) is .0099, or just under 1% (not the .01% you claim), so this answer is correctly marked, and, together with the answer of .99, does show that the tolerance is correctly set at .01 (or 1%).

Davide

Re: Relative Tolerance in Number with Units

by Davide Cervone -
Note that the error message is about the SECOND answer blank in your problem. It suggests that you have two (or more) answer rules, and that you have only provided one answer checker. You need to include answer checkers for the other blanks before the problem will be processed.

If the NumberWithUnits already IS the second answer checker, then something is going wrong. If that is the case, please post a complete code sample, since it is difficult to debug this without actually seeing the code you are using.

Davide

Re: Relative Tolerance in Number with Units

by Clinton Ferreira -
I also tried

ANS(Real(8.23E-8)->with(tolType=>'relative',tolerance=>'.01')->cmp);

which works fine for the relative tolerance but has no units, so I tried

ANS(Real(8.23E-8)->with(tolType=>'relative',tolerance=>'.1',units=>'N')->cmp);

but that made absolutely no difference to the answers required by the first line when N wasn't included in the answer and when it was included Webwork said it was an undefined variable.

Does this help?

Re: Relative Tolerance in Number with Units

by Davide Cervone -
You need to use NumberWithUnits, as Mike suggested above, because the Real() object does not understand units=>'N', as you have entered here. (Note, however, that you don't need quotes around the tolerance itself, so tolerance=>.01 is fine. Also, the tolerance is relative by default, unless you have changed the Context() settings, so there is no need to set that explicitly, either.).

The MathObject library needs a better units feature; NumberWithUnits and FormulaWithUnits are intermediate measures until the a more complete Units class is added to MathObjects.

Davide

Re: Relative Tolerance in Number with Units

by Clinton Ferreira -
Ok, the code seems to be working now, just not for the tolerances I specify. I used

ANS(NumberWithUnits(8.23E-8,"N")->with(tolerance=>0.03)->cmp);

(i.e. 3% tolerance) then tried different values for the answer but was only allowed to input

8.222E-8 to 8.238E-8 which is a tolerance of 0.097% and not the 3% I want. Then I tried

ANS(NumberWithUnits(8.23E-8,"N")->with(tolerance=>3)->cmp);

(i.e. 300% tolerance) and got the exact same behaviour.

Re: Relative Tolerance in Number with Units

by Michael Gage -
Your answer is below the default limit at which relative tolerance comparisons are automatically switched over to absolute tolerance comparision.

This is usually the desirable behavior, if the answer results in zero for some particular value of the seed, then it is quite possible that round off error will make the instructor's answer a very small number. A student, using a mathematically equivalent but operationally different method of calculating the answer, will have a slightly different round off error and their answer will be marked incorrect -- not what is desired or expected. The errors are small in absolute terms but large relative to the correct answer.

For your problem you want to work with answers that are very small, so the defaults at which the comparison automatically switches from relative to absolute (zeroLevel) needs to be lowered and the absolute comparison with zero (zeroLevelTol) needs to be lowered as well. You can experiment with various settings in the snippet below:

########################################################################
DOCUMENT();
"PGstandard.pl", # Standard macros for PG language
"MathObjects.pl",
"parserNumberWithUnits.pl",
);

Context()->flags->set(zeroLevel => 1E-50);
Context()->flags->set(zeroLevelTol => 1E-55);

BEGIN_TEXT
zeroLevel \{ Context()->flags->get('zeroLevel')\}
$PAR zeroLevelTol \{ Context()->flags->get('zeroLevelTol')\}$PAR
Enter the answer of 1E-23 Newton \{ans_rule()\}
END_TEXT

ANS( NumberWithUnits(1E-23, 'N')->with(tolType=>'relative', tolerance=>.01)->cmp );

ENDDOCUMENT();


If desired the default limits can be reset in global.conf (for the site) or course.conf (for the course), but you should be cautious about setting extreme values for these.

(Edited by Davide Cervone - original submission Tuesday, 18 December 2007, 12:40 PM)

Re: Relative Tolerance in Number with Units

by Davide Cervone -
Mike:

While your code is a nice example, it might also be valuable to print out the ORIGINAL values of zeroLevel and zeroLevelTol. I think the default zeroLevel is 1E-14, and so Clinton's value of 8.23E-8 is above that. The difficulty he is having is, I think, due to having a older version of PG, not to zeroLevel issues. One of the reasons for many of the changes this summer was to improve the reliability of the the flags when they are attached to individual objects (they were not always passed down to sub-objects properly). This has been improved, though there is still some work that could be done on it.

In any case, I think an update of the PG tree will take care of the issue he is having.

Davide

Re: Relative Tolerance in Number with Units

by Michael Gage -
You're right Davide. I managed to transpose the 8.23 into 8E-23 at some point. Sorry about that. :-) On the other hand it was interesting to see how to handle the problem if it does arise (probably in some physics context where 1E-23 is a sizable number and the default values of WeBWorK wouldn't handle the answer checking properly).

One other thing -- for future reference. I originally tried to change the values of $main::zeroLevelDefault and$main::zeroLevelTolDefault in the problem but was unable to affect the way the answer checker behaved because the answer checker had already read those values and replaced them in its interior database.

You have to change the values in the current Context in order for them to affect the answer checkers.

Re: Relative Tolerance in Number with Units

by Davide Cervone -
One other thing -- for future reference. I originally tried to change the values of $main::zeroLevelDefault and$main::zeroLevelTolDefault in the problem but was unable to affect the way the answer checker behaved because the answer checker had already read those values and replaced them in its interior database.

The values are loaded into the context at the time it is created or copied, so if you set these values before the

    Context("Numeric");

(or whatever context you use), then that should do it.

Davide

Re: Relative Tolerance in Number with Units

by Davide Cervone -
Note that you can set both flags in one command as:
    Context()->flags->set(
zeroLevel => 1E-50,
zeroLevelTol => 1E-55,
);

so there is no need for two separate Context() calls.

Davide

Re: Relative Tolerance in Number with Units

by Davide Cervone -
My tests show that
    ANS(NumberWithUnits(8.23E-8,"N")->with(tolerance=>0.03)->cmp);

accepts answers up to 8.476E-8, which is the requires 3%. I suspect that the reason is that your copy of PG needs to be updated. There were a number of important changes this past summer, and one was to fix some problems with passing tolerance values (and other flags) from one object to another. In this case, the NumberWithUnits uses a Real object internally and wasn't passing the flags downward properly. This is working in the current version, so you may want to upgrade.

You can update the PG directory without updating the whole WeBWorK installation, and I would recommend that you do that (after first backing up the PG directory tree in case you decide you need to go back). The current version of PG can be used with WW2.3.2 for sure, and perhaps even earlier.

If you can't update to this version, then you can try using

    ANS(NumberWithUnits(8.23E-8,"N")->cmp(tolerance=>0.03));

instead, and if that doesn't work either, try setting the tolerance in the context itself:
    Context()->flags->set(tolerance=>.03);
ANS(NumberWithUnits(8.23E-8,"N")->cmp);

though that will affect all numeric comparisons, not just this one.

Davide

Re: Relative Tolerance in Number with Units

by Clinton Ferreira -
Finally! Using
 ANS(NumberWithUnits(8.23E-8,"N")->cmp(tolerance=>0.03));
doesn't work, BUT, using
 Context()->flags->set(tolerance=>.03);
ANS(NumberWithUnits(8.23E-8,"N")->cmp);
works perfectly.

Thanks for all your help, I appreciate it.

Re: Relative Tolerance in Number with Units

by Clinton Ferreira -
Sorry, just one more question..

Where can I get a list of all the possible flags [Context()->flags->...] and the options that can go with ->cmp?

Re: Relative Tolerance in Number with Units

by Davide Cervone -
Sorry to have left this unanswered for so long. The holidays slowed me down, I'm afraid.

The context flags depend on which context you are using, but you can get the ones for the current context by using

    warn join('<BR>',lex_sort(Context()->flags->names));

in your problem file after you select the context. There are also some tools in pg/macros/PGinfo.pl for listing the flags and their values, or the entire Context() structure.

As for the cmp() flags, that is harder, since each MathObject class has its own set, and there is no one place where they are all listed, even in the code itself. Many of the most common ones are in the documentation at

    http://devel.webwork.rochester.edu/doc/cvs/pg_HEAD/doc/MathObjects/MathObjectsAnswerCheckers.html

so that is the first place to start. As you have found out, the documentation on MathObjects is something that needs improvement.

Davide