Is it possible to see the distribution of answers to a specific question across all students? Having this capability would help me identify the common mistakes
and address those in class.
The "Answer log" page gives this information, but only to a single student at a time. When you have 150 students, checking each one separately would take a while.
Yoav
I've thought about this myself. However, most problems in WeBWorK (at least what I consider good problems) involve a random element in that coefficients or some other aspect of the problem is randomly chosen.
I have never figured out how to parse such a table so that it makes any sense in terms of finding patterns of mistakes.
On the other hand, I have thought about using something similar to "Answer Hints" and code what I would expect students to answer based on known common mistakes. It would then be possible to count how many students' answers matched the corresponding predicted mistake answers.
However, this would require significant work, both to the problem library and to the supporting WeBWorK system. The "pg" file defining problems would have to include some type of coded method (like ANS) that would define expected answers based on common errors. Then the student evaluation routine of WeBWorK would parse the same PG files but would interpret them to compute predicted erroneous answers and then check the student answers in the database for matches to these answers.
I have settled on giving students feedback for common mistakes within the problem using the AnswerHints macros. But this does not let me, as instructor, know which/how many students are encountering these issues.
Best wishes,
D. Brian Walton
I have never figured out how to parse such a table so that it makes any sense in terms of finding patterns of mistakes.
On the other hand, I have thought about using something similar to "Answer Hints" and code what I would expect students to answer based on known common mistakes. It would then be possible to count how many students' answers matched the corresponding predicted mistake answers.
However, this would require significant work, both to the problem library and to the supporting WeBWorK system. The "pg" file defining problems would have to include some type of coded method (like ANS) that would define expected answers based on common errors. Then the student evaluation routine of WeBWorK would parse the same PG files but would interpret them to compute predicted erroneous answers and then check the student answers in the database for matches to these answers.
I have settled on giving students feedback for common mistakes within the problem using the AnswerHints macros. But this does not let me, as instructor, know which/how many students are encountering these issues.
Best wishes,
D. Brian Walton
You can actually enter wildcards in the fields at the top of the Show Past Answers form to see more than one student or more than one problem at a time.
For example, entering "*" in the student field will get you the answers for all the student for that problem, or entering "*" in the problem number field will get you all the problems for a given student. I don't remember if they use regex syntax (with * handled as a special case), or file glob wildcards, but expect the former.
Davide
For example, entering "*" in the student field will get you the answers for all the student for that problem, or entering "*" in the problem number field will get you all the problems for a given student. I don't remember if they use regex syntax (with * handled as a special case), or file glob wildcards, but expect the former.
Davide
Thanks guys,
I have found a slightly different way to analyze the answers - download the
file "answer_log" and use my own script to do it.
Here are two lines from the file:
[Sun Aug 19 00:40:19 2012] |yfreund|multi-Step_questions|1|0000000 1345351219 1 1 0
[Sun Aug 19 00:41:01 2012] |yfreund|multi-Step_questions|1|1000000 1345351261 {0,2,3,4,5,6,7,8} 1 0
Most of the fields in each line are self explanatory. But what is 1345351261 ?
Is it maybe the random seed for this procedure?
Is it possible to add to the log file the correct answers?
Actually, now that I think of it. There is a simple solution, I can
ask the students to type in the problem parameters, this is not much work for the student, and would provide me with the input they saw as part of the log file. From this input I can compute the correct answer.
Yoav
I have found a slightly different way to analyze the answers - download the
file "answer_log" and use my own script to do it.
Here are two lines from the file:
[Sun Aug 19 00:40:19 2012] |yfreund|multi-Step_questions|1|0000000 1345351219 1 1 0
[Sun Aug 19 00:41:01 2012] |yfreund|multi-Step_questions|1|1000000 1345351261 {0,2,3,4,5,6,7,8} 1 0
Most of the fields in each line are self explanatory. But what is 1345351261 ?
Is it maybe the random seed for this procedure?
Is it possible to add to the log file the correct answers?
Actually, now that I think of it. There is a simple solution, I can
ask the students to type in the problem parameters, this is not much work for the student, and would provide me with the input they saw as part of the log file. From this input I can compute the correct answer.
Yoav
I'm guessing that that mystery field is some kind of universal time stamp. I notice that its ones digit always agrees with the ones digit of the seconds field of the time that the answer was submitted.
The time stamp field is a Unix timestamp, measuring "the number of non-leap seconds since whatever time the system considers to be the epoch," which on most systems is 00:00:00 UTC, January 1, 1970.
To wit:
$ perl -e 'print scalar(localtime("1345351219")),"\n";'
Sun Aug 19 00:40:19 2012
Gavin
What are the fields?
I think I have determined:
timestamp | user | assignment | question_number | ? | UTC_seconds | answer
What is the field where the question mark is? I've looked at the values in that field from the answer log that I have downloaded, and it appears to be something coded in binary, but I'm not sure what.
1s and 0s for if each answer within the question was scored as fully correct or not.