Forum archive 2000-2006

Mark Schmitt - Conditional set definitions

Mark Schmitt - Conditional set definitions

by Arnold Pizer -
Number of replies: 0
inactiveTopicConditional set definitions topic started 9/2/2002; 11:20:56 AM
last post 9/7/2002; 4:27:19 AM
userMark Schmitt - Conditional set definitions  blueArrow
9/2/2002; 11:20:56 AM (reads: 1728, responses: 4)
I have been working to install WeBWorK as a quiz system, and one of the course instructors asked me if it's possible to have the set definition file randomly choose different problems for different students.

This is a little different then changing numbers in a problem. Most of the problems are multiple choice, and cover distinct material. All the problems have been rated by difficulty (1-5). The instructor wants to pick all the Level 2-5 problems and then enough Level 1 problems to reach a total of 20. (There are more than enough questions.)

I suppose I could put all the questions into the same problem and give a 1 problem quiz, but that would make it difficult for students to notice if they have skipped a part.

Any ideas?

Thanks in advance.

Mark

<| Post or View Comments |>


userMichael Gage - Re: Conditional set definitions  blueArrow
9/2/2002; 1:49:26 PM (reads: 2120, responses: 0)
Hi Mark,

I don't believe this is currently possible without modifying the build problem set procedures.

Here is something to get you started, I haven't been able to check it. If you can develop it into a nice feature which is still backward compatible we'll be happy to add it to our CVS code.

The key lines are 278--296 in /scripts/proceduresForBuildProbSetDB.pl.

Assuming that there are 5 problems and @scramble = (3,4,2,0,1); then replacing trhe second line

$probNumber=$i+1;

by $probNumber=$scramble[$i]+1

should give you the scrambled order. Look at NchooseK in PGchoicemacros.pl to see how to write a subroutine that permutes integers. The +1 is to change everything from a run of integers starting at 0 which is used internally by WeBWorK to a run starting at 1, which is how the problems are labeled and stored.

Let us know how it goes.

--Mike

 

    for($i=0; $i<@problemList; ++$i) {
$probNumber=$i+1;
&putProblemFileName ($problemList[$i],$probNumber, $PIN);



if ($method eq 'readFromLogFile') {$probSeed = shift @seedsArrayFromLogFile;}
else {
# note continuationflag is zero for first problem & if unspecified
unless( $problemContinuationFlagList[$i] ) { $probSeed = int(rand(5000)); }
}



push (@seedList,$probSeed); ## put seed in array to be saved later in log file
&putProblemSeed ($probSeed,$probNumber, $PIN);
&putProblemValue ($problemValueList[$i],$probNumber, $PIN);
&putProblemMaxNumOfIncorrectAttemps ($problemAttemptLimitList[$i],$probNumber, $PIN);
&putProblemAttempted(0,$probNumber, $PIN);
&putProblemStatus(0,$probNumber, $PIN);
&putProblemNumOfCorrectAns (0,$probNumber, $PIN);
&putProblemNumOfIncorrectAns (0,$probNumber, $PIN);
}



<| Post or View Comments |>


userMark Schmitt - Re: Conditional set definitions  blueArrow
9/6/2002; 4:17:20 AM (reads: 2002, responses: 0)
I have a working version of my hack to randomize problem lists. It is backward compatable, so any one who would like to try it and give some feedback would be most welcome.

The major changes are in FILE.pl and proceduresforBuildProbSetDB.pl

In FILE.pl, I added one new protected line for the setDef file, of the form:

Group = n of k

This command tells the scripts to treat the first k problems as a group and only select n of them for each student. Repeated use allows for multiple groups. Any problems that aren't in groups are ignored. The absence of the Group command will make the scripts behave as normal.

After the problems from each group are picked, the list of included problems is scrambled to create more randomness.

This feature was requested by the Geology Dept at Wayne State University, which is using WeBWorK for controlled environment testing in lieu of scantrons.

As usual, all feedback greatly appreciated.

Mark

<| Post or View Comments |>


userArnold K. Pizer - Re: Conditional set definitions  blueArrow
9/6/2002; 5:33:29 AM (reads: 2008, responses: 0)
Hi Mark,

This raises a few issues related to automatic scoring and the reporting of student progress (items 1-3 on the Prof page).

For automatic scoring (e.g. totalling the scores in a set and appending the total to a totals file) it is required that all students have the same number of problems and that problem n is worth the same number of points for each student, n = 1,2, ... When thinking about allowing random choices among groups, I was thinking of putting the restriction on that if problems A and B could be substituted for each other, they must be worth the same number of points. Also in the data reported for problem 1, with random problems, problem 1 be represent different problems for different students.

The other thing to think about is how to report statistical data. Currently we report statistical data for Prob. 1, Prob 2, etc. But with a random selection, Prob 1 is not well defined. You could replace Prob. 1 by the actual problem name but if the assignment chooses e.g. 20 problems from a group of 1000 problems, should you display a sparse table of 1000 columns? Also since the problem names are long, this would be a very wide table. On the other hand, if every students prob 1 is very similar, people might want statistical data for prob1.

These are just some things to think about.

<| Post or View Comments |>


userMark Schmitt - Re: Conditional set definitions  blueArrow
9/7/2002; 4:27:19 AM (reads: 2008, responses: 0)
Arnie,

Thanks for the topics to think about. For the automatic scoring, I think you are right. We should keep the groups homogenous wrt point value, and that means randomizing in groups, rather than on the whole. Since we're currently grouping by difficulty rating, that means we hold to the format of increasing difficulty as you progress through the set.

The statistical reporting opens up a much more interesting set of problems. I like the idea of working with the actual problem names, because then we can analyze the effectiveness of particular questions. In the situation I'm dealing with, we might be looking at 20 out of 30, but with 500 students, that's still a lot of students seeing each question. The sparse table problem won't be bad for us, but certainly could be in other situations. I'm not sure that there is a good solution to that problem in general, but I'll think some, and ask my colleagues to do the same. As for making the table super wide, what if we made a simple assignment of a number to each problem name, used the number in the table, and printed a key at the bottom of the page? It would cut down on the table width, but keep all the useful data.

Now my big question is, how hard would it be to switch to problem name rather than number. Certainly, save_problem_state in processProblem8.pl would need modifying, and the putProblem* subroutines in DBglue8.pl would need massaging, but what else would need work? Are we talking a major overhaul, or just a bunch of minor edits? If it's just editing, I'd love to get to work on it, provided I know where I'm working. Please advise.

 

Again, thanks for the feedback.

 

Mark

<| Post or View Comments |>