Is anyone doing item analysis and other assessment in Webwork? Assessment has become increasingly important at our institution, we are now required to regularly assess whether we reach set outcomes for our students. It seems like webwork would be an be ideal tool to do assessment with, but I really haven't found an ideal way to do it. I was wondering whether someone else might have?
The problem with doing assessment is that the tagging of items in the NPL is rather arbitrary. It would be great if the math community had agreed on tagging labels, but that not being the case, labels in NPL are all over the place. One can't just look up problems on "solving rational equations" and get a meaningful list of problems from webwork. The fact is that each institution uses its own labels, names and categories for assessment.
At our college, we give all intermediate algebra students the same departmental final exam with 18 questions. Each question assesses a certain outcome our department has defined. Problem 3, for example is on "Addition of rational expressions," and problem 10 is on "Simplifying rational exponents." It would be great to use webwork, with its vast NPL problem library, to generate these departmental tests. It would be especially helpful to be able to do some form of "local tagging" of NPL problems. In other words, when I find a problem in NPL I like, and that fits "Addition of rational expressions" I would like to be able to apply this label, within my course, to the problem, and in this manner tag the problem for my particular assessment outcome. Then when I make my final, I could easily pull up my group of problems that are tagged for this item with a webwork search, using my label as a keyword. Afterwards the webwork stats would allow me to easily analyse results based on each outcome.
Assessment outcomes are typically somewhat broad, like the examples I gave above, so another feature that would be great for assessment would be the ability to pool different problems that measure the same assessment outcome, and have webwork select one at random from the pool for a problem set.
Obviously, right now webwork can't do this, but my question is whether this is something that is possible, or something that others are interested in?