** Using WeBWorK **

This is definitely a bug in function_cmp_up_to_constant()

&ANS(function_cmp_up_to_constant($answer,"u"));

will accept any answer at all as correct. This seems to be particular to using u as the variable.

Zig

<| Post or View Comments |>

&ANS(function_cmp_up_to_constant("-u**($m+1)/$b","u"));

where $m = 8 and $b = $a*($m+1) = 36. I figured the problem was that the difference
between -u^9/36 and -u^9/32 was below the default floating point tolerance,
so I changed it to:

&ANS(function_cmp_up_to_constant("-u**($m+1)/$b","u",.999999,1.000001,0.001));

No luck: WeBWorK still thinks -u^9/32 is OK.

Hi Zig,

I have tried these variations using a correct answer of u^9/36 and an entered answer of u^9/32:

ANS( fun_cmp("-(u**(9))/36+c",var=>'u',params=>'c',domain=>[.9999,1.0001],reltol=>.001) ); rejects u^9/32
ANS( function_cmp_up_to_constant("-(u**(9))/36","u",.9999999,1.000001, 0.001) );accepts u^9/32
ANS( function_cmp_up_to_constant("-(u**(9))/36","u",.9999999,1.000001, 0.001) ); accepts u^9/32
ANS( fun_cmp("-(u**(9))/36+c",var=>'u',params=>'c',domain=>[.9999,1.0001],reltol=>.001) ); rejects u^9/32

better results:
ANS( fun_cmp("-(u**(9))/36+c",var=>'u',params=>'c',domain=>[0,5.0001],reltol=>.001) ); rejects u^9/32
ANS( function_cmp_up_to_constant("-(u**(9))/36","u", 0, 5, 0.001) ); rejects u^9/32

and get these answers:
Answer 6 entered:--> u^9/32 <-- Incorrect.
Correct answer: -(u**(9))/36+c
Answer 7 entered:--> u^9/32 <-- Correct.
Correct answer: ( -(u**(9))/36 ) + Q
Answer 8 entered:--> u^9/32 <-- Correct.
Correct answer: ( -(u**(9))/36 ) + Q
Answer 9 entered:--> u^9/32 <-- Incorrect.
Correct answer: -(u**(9))/36+c
Answer 10 entered:--> u^9/32 <-- Incorrect.
Correct answer: -(u**(9))/36+c
Answer 11 entered:--> u^9/32 <-- Incorrect.
Correct answer: ( -(u**(9))/36 ) + Q

I haven't yet been able to reproduce any difference in using u or t as the variable.

I haven't figured out what the difference in the preparation of the function is between the two versions fun_cmp and function_cmp_up_to_constant. Both are using the same "back end" and are just handling the initializations differently. I thought that perhaps function_cmp_up... wasn't initializing the relative tolerance mode, but that doesn't seem to be the problem from looking at the code.

For new problems at least you may find it more convenient to use fun_cmp which is documented at http://webwork.math.rochester.edu/docs/manpages/fun_cmp.html since it gives more explicit control. The debug option is also handy to see what is going on numerically, although it's output is a bit confusing at first.

In any case, I'm pretty sure that this is a tolerance issue. Among other things, relative tolerance is a little problematic on a function whose answer is only determined up to a constant.

The most serious problem, however, is that restricting the range of choices to be very close to 1 is self-defeating since the evaluator first samples the student's function and calculates a constant for the professor's function which makes it a close fit for the student's function. Then more points are sampled to see how close this fit is. If all the points are close together the fit is likely to be quite good.

A better strategy is to widen the domain on which the points are sampled say to [0,5] that seems to solve the problem with both.

Take care,

Mike

<| Post or View Comments |>

You are of course right that my restricting the comparison domain to a small interval made matters worse.

Howevever I still maintain that function_cmp_up_to_constant()'s discrimination of incorrect answers is so poor, that it can only be described as buggy.

Here are some examples:

function_cmp_up_to_constant("-x^9/36")

accepts the following answers as correct:

-x^9/20, -x^8/30, -x^7/36, -x^6/36, 1234567

function_cmp_up_to_constant("-x^2/36") accepts -x^2/34 as correct.

Calculus students are generally not prone to complain about incorrect answers being accepted as correct, so it is hard to tell how often this occurs in practice.

Zig

<| Post or View Comments |>

I just logged into your current Math 142 Calculus 2 course at Rochester as practice3 and started doing Problem Set 6. I entered 123456 as the answer to Problem 2, 3, 4, 5, 6, 7, 9 and 11 and this was accepted as correct. This was 8 out of 12 problems and 8 out of 9 indefinite integral problems.

Surely this is unacceptable. At some point some student is bound to stumble onto this and it will spread over the grapevine that 123456 is Open Sesame for indefinite integration problems in WeBWorK.

The algorithm behind function_cmp_up_to_constant() has to be rethought. Perhaps there is no good way of doing function_cmp_up_to_constant() or perhaps indefinite integration problems should be written by expert numerical analysts?

Zig

<| Post or View Comments |>

Just a followup to my previous message. I tried Math 142 Set 6 also as practice1, practice2, and practice4 and got the same results as with practice3, with one exception. WeBWorK would not accept 123456 as the correct answer to problem 11 for practice4. After a little bit of experimentation though, I found that 9876543 would work.

Zig

<| Post or View Comments |>