## Forum archive 2000-2006

### Michael Gage - weird WeBWorK bug -- acceptance of wrong answers ### Michael Gage - weird WeBWorK bug -- acceptance of wrong answers

by Arnold Pizer -
Number of replies: 0 weird WeBWorK bug -- acceptance of wrong answers topic started 1/23/2001; 7:56:26 PMlast post 1/25/2001; 3:30:37 PM Michael Gage - weird WeBWorK bug -- acceptance of wrong answers 1/23/2001; 7:56:26 PM (reads: 1502, responses: 11) From: Zbigniew Fiedorowicz Michael Gage - Re: weird WeBWorK bug -- acceptance of wrong answers 1/23/2001; 7:58:01 PM (reads: 1712, responses: 0) Text for the problem mentioned above: ##Authored by Zig Fiedorowicz 1/17/2000DOCUMENT();install_problem_grader(~~&full_partial_grader);$showPartialCorrectAnswers = 1;if (!($studentName =~ /PRACTICE/)) {$seed = random(1,4,1);if ($studentName =~ /VINCE VERSION1/) {$seed = 1;}if ($studentName =~ /VINCE VERSION2/) {$seed = 2;}if ($studentName =~ /VINCE VERSION3/) {$seed = 3;}if ($studentName =~ /VINCE VERSION4/) {$seed = 4;}SRAND($seed);}$a = random(3,7,1);$b = random(4,15,1);$m = random(3,7,1);$m1 = $m+1;$n = random(8,12,1);TEXT(&beginproblem);BEGIN_TEXT{bf Note:} You can get full credit for this problem by just answering the lastquestion correctly. The initial questions are meant as hints towards the finalanswer and also allow you the opportunity to get partial credit.$PARConsider the indefinite integral $$int x^{m}\left(a+b x^{m1}\right)^{n}\, dx$$$BRThen the most appropriate substitution to simplify this integral is$BR$$u$$ = \{ ans_rule()\}$PARThen $$dx = f(x)\,du$$ where$BR(f(x)) = \{ ans_rule()}$PARAfter making the substitution we obtain the integral $$\int g(u)\,du$$ where$BR(g(u)) = \{ ans_rule()\}$PARThis last integral is: = \{ ans_rule()\} (+ C)$BR(Leave out constant of integration from your answer.)$PARAfter substituting back for $$u$$ we obtain the following final form of the answer:$BR = \{ ans_rule()\} $$+ C$$$BR(Leave out constant of integration from your answer.)$PAREND_TEXT&ANS(function_cmp("$a+$b*x^($m+1)","x"));&ANS(function_cmp("1/($b*($m+1)*x^$m)","x"));&ANS(function_cmp("u^$n/($b*($m+1))","u"));&ANS(function_cmp_up_to_constant("u^($n+1)/($b*($m+1)*($n+1))","u"));&ANS(function_cmp_up_to_constant("($a+$b*x^($m+1))^($n+1)/($b*($m+1)*($n+1))","x"));BEGIN_TEXTThis is similar to Problems 19 and 20 in Section 5.6 of the text.END_TEXTENDDOCUMENT(); <| Post or View Comments |> Michael Gage - Re: weird WeBWorK bug -- acceptance of wrong answers 1/24/2001; 12:08:44 AM (reads: 1696, responses: 0) Hi, On our machine we get a floating point error -- as expected since webwork will be trying to take the square root of a negative number. It's possible that your machine signals floating point exceptions differently than ours and we're not catching it. Try entering (-1)^(1/2) into a problem requiring a numerical answer. What happens with that? (I get a floating point error.) Here is another trick: Replace the 2nd and 3rd answers by #&ANS(function_cmp("1/($b*($m+1)*x^$m)","x")); #&ANS(function_cmp("u^$n/($b*($m+1))","u")); &ANS(fun_cmp("1/($b*($m+1)*x^$m)",var=>["x"], debug =>1 )); &ANS(fun_cmp("u^$n/($b*($m+1))",var=>["u"] , debug =>1 )); (For more details on fun_cmp , the newest of the function evalautor makers see http://webhost.math.rochester.edu/webworkdocs/manpages ) The debug mode prints out some intermediate "error" messages showing the progress in evaluating the functions. It will show the evaluation points and the differences between the student and instructor's values, not the student's values alone. Still it may give you some ideas about what is happening. Let me know what happens with these two experiments. Take care, Mike <| Post or View Comments |> Zbigniew Fiedorowicz - Re: weird WeBWorK bug -- acceptance of wrong answers 1/24/2001; 10:28:07 AM (reads: 1695, responses: 0) I was wondering whether (-1)^.5 was the Open Sesame for our WeBWorK installation. Apparently not quite. It always seems to work for answers evaluated by function_cmp(). But it doesn't seem to work for function_cmp_up_to_constant() Here are the results I get for the problem we were discussing: Answer 1 entered:--> (-1)^.5 <-- Correct. Answer 2 entered:--> (-1)^.5 <-- Correct. Answer 3 entered:--> (-1)^.5 <-- Correct. Answer 4 entered:--> (-1)^.5 <-- Incorrect. There is an error in WeBWorK's answer to this problem, please alert your instructor. error: "nan" does not make sense here Answer 5 entered:--> (-1)^.5 <-- Incorrect. There is an error in WeBWorK's answer to this problem, please alert your instructor. error: "nan" does not make sense here At least one of the above answers is NOT correct. Your score on this attempt is 60%. Zig <| Post or View Comments |> Zbigniew Fiedorowicz - Re: weird WeBWorK bug -- acceptance of wrong answers 1/24/2001; 2:38:47 PM (reads: 1688, responses: 0) The weird WeBWorK bug (function_cmp() accepting (-1)^.5 as the correct answer to any problem) is not specific to Ohio State. It also affects Indiana. I just logged in as a practice user into one of Indiana's courses and got the same result. See the HTML source below. An unrelated question: the Indiana motd mentions there is a problem printing WeBWorK hard copy with Netscape 6. What's the problem? Zig <| Post or View Comments |> Michael Gage - Re: weird WeBWorK bug -- acceptance of wrong answers 1/24/2001; 10:52:10 PM (reads: 1693, responses: 0) This is great. I go teach my classe and someone else solves the problem for me. (see below) We'll get this patch into PGanswermacros.pl, test it as fast as we can, and put a new version in the updates directory on "Download Webwork". I think I'll apply this test to both the student's answer and the instructors code as well, Bill. Does that seem right to you? Thanks very much, Bill. Take care, Mike From Bill Wheeler at Indiana University: Dear Zig, Thanks for bringing this "weird bug" to our attention. The cause is that, on some Unix implementations, including Solaris, the C math library apparently does not throw a floating point exception signal when it attempts to compute pow(-1, 0.5) , i.e., in WebWork notation (-1)^(0.5). Instead, the library just returns the IEEE NaN. Consequently, WebWork's attempt to catch an SIGFPE in order to detect the error fails. The patch I give below is based on the fact that NaN is a number but that (BY DEFINITION) NaN != NaN . In other words, in Perl, if you set$x = (-1)**(0.5), then the value of $x is NaN but$x != $x . The patch below seems to work. I've installed it on our test system, but I have not yet installed it on our production system. Sincerely, Bill Wheeler ----------------------------------------------------------------------------- The patch goes in PGanswermacros.pl in the calculate_difference_vector subroutine. Insert the following lines between lines 2373 and 2374 (at least those are the line numbers in our current copy of PGanswermacros.pl): use it to test that the student's function has evaluated to a real number. ########################## Patch for Wierd Bug (=NaN) phenomenon ########  if ($inVal ne "" && $inVal !=$inVal) { $err1 .= " For some values of the independent variable, your function does not produce real numbers. ";$errors .= " \$err1 "; }  ########################### <| Post or View Comments |> Zbigniew Fiedorowicz - Re: weird WeBWorK bug -- acceptance of wrong answers 1/25/2001; 11:09:53 AM (reads: 1647, responses: 0) Dear Bill, Mike and Arnie, Thanks for the patch. While I have not tested it, I am sure it will fix the immediate problem. However I am concerned with the idea of ad hoc patching of bugs in a complex software system like WeBWorK. This bug was introduced in version 1.6.xxx. It did not occur in version 1.5, where the student was just told their answer is incorrect (which I think is best). [I just checked by replacing v 1.6.01 of PGanswermacros.pl with the 1.5 version.] Moreover (if I am not mistaken) this patch would not affect function_cmp_up_to_constant() which in the 1.6 version would respond to (-1)^.5 by telling the student to inform their instructor that there is a bug in the problem. Zig <| Post or View Comments |> Zbigniew Fiedorowicz - Re: weird WeBWorK bug -- acceptance of wrong answers 1/25/2001; 11:40:15 AM (reads: 1671, responses: 0) Just a followup to my previous message. With Bill's patch installed, function_cmp_up_to_constant() produces the following error message in response to (-1)^.5: For some values of the independent variable, your function does not produce real numbers. There is an error in WeBWorK's answer to this problem, please alert your instructor. error: "nan" does not make sense here I am sure you will agree that this is suboptimal. Zig <| Post or View Comments |> Michael Gage - Re: weird WeBWorK bug -- acceptance of wrong answers 1/25/2001; 2:26:16 PM (reads: 1667, responses: 0) I'm still working on the best fix for this problem. Would an error message such as:  WeBWorK was unable evaluate your function. Please check that your equation doesn't take roots of negative numbers, or divide by zero.  be appropriate?As to the cause: While there are a good number of changes in the answer macros in WeBWorK 1.6, I don't think that they caused this problem. The basic problem is that WeBWorK has long depended on catching a floating point error to determine whether an illegal operation such as (-1)^(1/2) has been evaluated. As Bill points out in some versions perl does not signal a floating point error, but instead returns a string value of "NaN" (not a number). At Rochester, we are using freeBSD operating system and WeBWorK 1.6 on both webwork and webwork2, but webwork2 is running version 5.6 of perl, while webwork is running an earlier version. webwork signals floating point errors for the formula above, webwork2 does not, but returns a value of "NaN", just as Bill describes above. It may be that the crucial change is whether one uses perl 5.6 or an earlier version. I'm experimenting with various fixes along the lines suggested by Bill above. There will be an updated version of PGanswermacros.pl once a good fix is ready. The problem of testing for a result of "NaN" is pretty localized and seems unlikely to introduce new bugs. ( I hope :-) ). Take care, -- Mike <| Post or View Comments |> Zbigniew Fiedorowicz - Re: weird WeBWorK bug -- acceptance of wrong answers 1/25/2001; 2:52:51 PM (reads: 1652, responses: 0) Mike, >While there are a good number of changes in the answer macros in WeBWorK 1.6, I don't think that they caused this problem. I disagree. The version 1.5 PGanswermacros.pl function_cmp() handles (-1)^.5 just fine under exactly the same OS/Perl setup as we are using for version 1.6. Moreover it is just 1.6 function_cmp() which has this problem. The v1.6 std_num_cmp() also deals with (-1)^.5 properly. As my preference for the error message: if a student's answer produces NaN the error message should just say their answer is incorrect (which is what happens with v. 1.5). Zig <| Post or View Comments |> Zbigniew Fiedorowicz - Re: weird WeBWorK bug -- acceptance of wrong answers 1/25/2001; 3:07:06 PM (reads: 1647, responses: 0) Mike, Let me just clarify one point. My main concern is not that Bill's fix is buggy. My primary concern is that there might be other bugs introduced in PGanswermacros.pl in going from version 1.5 to 1.6. It is for this reason that I would like you to investigate the difference in behavior between v. 1.5 and v. 1.6 in dealing with (-1)^.5 Zig <| Post or View Comments |> Michael Gage - Re: weird WeBWorK bug -- acceptance of wrong answers 1/25/2001; 3:30:37 PM (reads: 1678, responses: 0) I agree about the need to do these bug fixes carefully, which is why it's taking a little time to finalize the fix. You are right about function_cmp -- the underlying engine for that checking was changed and that contributed to the problem. I'm still pretty sure that the underlying trigger is the switch in perl versions, however. For example std_num_cmp() gives a floating point error on old versions of perl and catches "NaN" on version 5.6. The latter behavior is slightly fortuitous, since it was not active for something like (-1)^(1/2) on the version of perl we tested it on. Basically what I'm planning is to copy that extra check from std_num_cmp() over to the new version of function_cmp and its siblings. Please keep the observations coming. It helps to insure that WeBWorK will work in a wider variety of environments. Not just our particular set up at Rochester. Thanks. Take care, -- Mike <| Post or View Comments |>