Forum archive 2000-2006

Jeff Denny - placement tests in webwork

Jeff Denny - placement tests in webwork

by Arnold Pizer -
Number of replies: 0
inactiveTopicplacement tests in webwork topic started 10/21/2004; 9:24:09 AM
last post 10/21/2004; 3:13:04 PM
userJeff Denny - placement tests in webwork  blueArrow
10/21/2004; 9:24:09 AM (reads: 1177, responses: 3)
My chair wants to consider using webwork as a vehicle for delivering our placement exams. The intention is to allow students to take the placement exam from home in the summer before their freshman year. This was tried with webct a few years ago, and then quickly the idea was dumped. Cheating is an obvious problem, but the chair hopes that the honest students would get a more accurate placement. (Currently our students are not required and hardly encouraged to take the placement test.)

Does anyone have experience with this? I'm concerned about having our office on first year students running the test. Has anyone encountered success or failure in their interactions? I'm looking for any pitfalls to avoid or successes that we should hope for.

Thank you!

<| Post or View Comments |>


userJohn Jones - Re: placement tests in webwork  blueArrow
10/21/2004; 12:40:00 PM (reads: 1415, responses: 0)
We have a web-based placement test, but we don't use webwork for it. Actually, webwork comes into the picture, but in a limited way.

Some questions you should consider are:

  • Are you going to require students to take it?
  • Are the results binding in any way, or just advisory?
  • Who, if anyone, will need to see the scores?
  • Do you need scores to interface to some other system?
If it is only advisory and you don't need to keep records, then setting it up with webwork via guest logins should not be very hard. If you want to track who has taken it and their scores (probably a good idea), you can probably hack webwork to handle that too, but it will start to be a hassle.

For a while, our test was advisory but we wanted to keep records. We coded the test as a long webwork problem (all multiple choice), generated lots of versions, and stored them statically (the html page, the images, and the answer keys). We then wrote other software to deliver, score, record, and report scores. That duplicates part of what webwork can do, but for multiple choice tests it is easy to code and it was worth having it taylored to be exactly how we wanted it. Students could cheat, but who cheats on a self-advisory placement test? Well, probably lots of people, but talk about someone cheating themself.

The test turned out to be a good predictor of failure in our biggest course, so taking it was made mandatory. I think at the moment, students don't have to acheive a passing score to enroll in a particular course, but we require that they have some score. This again cuts down on the incentive to cheat. It puts the department in the position of trying to be helpful in placing students rather than trying to obstruct their getting into classes they need.

Students could answer randomly, but the hope was that most students would actually do the placement test and heed our warnings if they did poorly. The scores are uploaded to a university system so that academic advisors across campus can see them (and they have access to department recommendations on what scores should be good enough for each course).

John

<| Post or View Comments |>


userSam Hathaway - Re: placement tests in webwork  blueArrow
10/21/2004; 2:13:45 PM (reads: 1398, responses: 0)
Gavin LaRose <glarose@umich.edu> from University of Michigan has written modifications to WeBWorK to support Gateway Testing that may be useful to you. Features include:

 

  • multiple "versions" (different seeds) per UserSet/UserProblem
  • ability to verify the user's identity by the attestation of a proctor
  • limiting of number of attempts per version
  • time limit on submitting an attempt (per version)
  • rate limit on getting new versions (like one version per day)

He's also developing a mechanism for pulling problems from a pool on a per-version basis, to allow greater variation between versions.

Because these features require fairly extensive modifications to the database layer, we are postponing integration into the official WeBWorK source until version 2.2, which will include a more flexible database layer.

A version of Gavin's modifications that works by modifying the current database layer is currently in the rel-2-1-a1 branch of the WeBWorK CVS repository. I'm not sure what the current status of the code is, but you might be able to get it set up now if you are comfortable with CVS, Perl, Apache, etc. (or can hoodwink someone who is into helping). You'd have to install it under a separate Apache server process from your production WeBWorK system and not share courses between the two.

If you're interested in pursuing this, I suggest you get in touch with Gavin directly for more information. Otherwise, wait for WeBWorK 2.2.

<| Post or View Comments |>


userGavin LaRose - Re: placement tests in webwork  blueArrow
10/21/2004; 3:13:04 PM (reads: 1411, responses: 0)
Hi all,

 

I'm not sure what the current status of the code is...

I'm just to the point of having all of the indicated features appearing to work. Remaining to be done is some aesthetic changes to how grades and so forth are displayed, and checking on how the instructor's interface displays results for the versioned sets (tests). I'll be doing more testing in the next day or so, and hopefully will be using this for a pilot test in our calculus I class starting next week. If that works, we'll be testing it more thoroughly next term with the intent of considering it as an alternative to MapleTA for next Fall.

More generally, we are also in the process of moving our placement testing from a pencil-an-paper version to an on-line version (currently using MapleTA). We ran a pilot test of this, involving some 200 or so students, this summer, and are analyzing how well it worked this semester with the intent of revising it somewhat and putting it into full production next summer.

Gavin

<| Post or View Comments |>