WeBWorK Problems

Custom List Checker that ignores list-length

Custom List Checker that ignores list-length

by Andrew Parker -
Number of replies: 4
I've been trying to create a customized answer checker for a problem requiring all solutions to a trig equation. I saw an old posting about this that recommended using adaptive parameters, but it was not robust enough to handle the possibility of students using "-2 pi n" instead of "+ 2 pi n". I worked on a version that used "->D('n')" but ultimately switched over to a different approach because of tangent equations and the potential for student answers to use pi as their period.

The answer checker generates a set of solutions from the provided formula ("+2pi n" by evaluating at n = -3..3), and generates a set of student "solutions" by evaluating at n = -10..10 (may be overkill, but needs to be at least twice the range because of the potential for "+ pi n" in the case of tangent). Each student value is checked as a solution to the given equation, and is also tested for overlap with the values from earlier expressions. Then, so long as the set of student values contains the set of actual solutions (and there are no incorrect values) the student should receive full-credit.

The issue that I'm still having is that even though my answer checker cleanly evaluates student responses for tangent equations, WeBWorK will complain if a student uses 1 formula (say pi/3 + pi n) instead of 2 (pi/3 + 2 pi n, 4 pi/3 + 2 pi n), or vice versa. Due to the inclusion of sine and cosine in the randomization of the problem, two formulas is the default for the list of correct answers.

TL;DR - can I override the requirement that the length of the student list match the length of the list provided as the answer?

see attached. (randomization to sine and cosine is disabled in the attached code for testing purposes)
In reply to Andrew Parker

Re: Custom List Checker that ignores list-length

by Davide Cervone -
I suspect the way to think about this is that the score returned by the list_checker is the number of answers out of the correct answer list that are matched by the student answers, not the number of students answers that are correct. In most cases these are the same thing, but in your case, where one equation can match two or more correct ones, they are not.

In your problem, the number of entries in the correct list is $m, so when the student's answer covers all the correct ones (and nothing else, with no repeats), return $m as the score, otherwise return 0. The actual number of student answers used to do the covering of the correct answers is immaterial.

If you want to offer partial credit, then things get a little more complicated. You have to decide what percentage the student answer represents. One approach might be to consider the correct list to be the full list of $m * 7 (7 is the number of integers from -3 to 3), and give credit for the number of those that are actually covered by the student's answer. The problem here is that if the student has equations that overlap, and you don't want to count those as correct, you need to be more sophisticated about it. Perhaps in that case, use the total number of correct points over the total number of correct points including repeats.

In any case, you could consider computing $score as the number between 0 and 1 that represents the amount of the problem the student got right, and then returning $score * $m so that when the list checker divides by $m you will get the correct score.

Hope that helps.
In reply to Davide Cervone

Re: Custom List Checker that ignores list-length

by Andrew Parker -
Thanks Davide - this was exactly it. So the list_checker compares the returned score to max( scalar(@$student), scalar(@$correct) )?

I had assumed that returning `n` for an `n`-item list would be sufficient, so now I'm capturing any student answer that meets the criteria, has maximum score, but that maximum score is less than the "expected" maximum (based on the "correct" answer).

if ( $studentCumulative->contains($correctSet) && $score == scalar(@$student) && $score < scalar(@$correct) ) { $score = scalar(@$correct) };

In reply to Andrew Parker

Re: Custom List Checker that ignores list-length

by Davide Cervone -
> So the list_checker compares the returned score to max( scalar(@$student), scalar(@$correct) )?

Yes.

> I had assumed that returning `n` for an `n`-item list would be sufficient.

In general, that would not work. For example, if you were asking for a list of x values and there were 3 in the correct list, and the student entered two of them correctly but was missing the third, you don't want to give full credit for it, even though all the student answers are correct (since the answer isn't complete). So you want to go by the length of the correct answer list most of the time.

But if the student gives the three correct answers plus an additional one, you also don't want to give full credit, as they have some wrong answers, so that is why the length of the student list also has to be taken into account.

Your formula looks like it should give full credit correctly. I'm not sure if it will be the correct partial credit if not all the student answers are correct, however. For example, if there are three formulas in the correct list, and one student answer covers two of them but not the third, they will get 33% rather than 67%. I'm not sure this situation will occur in your problem (I haven't looked at it that carefully), but just include it to be complete in the discussion of how to handle variable-length correct answers.

One possible alternative for the partial credit would be to use

$score = ($correctSet->intersect($studentCumulative))->length / $correctSet->length

as the desired score and multiply that by max($n,$m) (the max of the lengths of the correct and student lists). Something like that might be a better partial credit value.

In reply to Davide Cervone

Re: Custom List Checker that ignores list-length

by Andrew Parker -
Ha! You predicted my followup question - I needed a method to determine the cardinality of the intersection of the student set and the correct set. This is perfect, thanks!