PREP 2011 Web Conference II

Prep Main Page > Web Conference 2

Web-Conference 2:

Date: June 2, 3-5pm EDT

Presenters: Jason Aubrey, Dan Flath, Gavin LaRose

Session Archive

Resources

• Good Problems Wiki
• Sample Problems

Agenda

1. Discussion of the problems that were written for the assignment for this conference
   1. Problem authoring issues and questions
2. Discuss papers on assessment/usability
   1. Problem intent, and the degree to which this will be obvious or work for the student
   2. Meta-context may also be important
3. Discussion of assessment and what makes a good problem, what factors are not conducive to problems usefulness; consider heuristics and rules.
   1. Look at sample problem list.
   2. Develop rubric page.
   3. Develop ideas that may not be directly connected with the rubric, but which will be useful.
   4. Distinction between a good problem and a good problem set

Conference material

• Discussion of the problems that were written for the assignment for this conference
  • Clarify any questions about problem structure
  • Discuss problems that are specifically "good" or "bad"
  • Go through and critique/comment on problems developed from assignment from first workshop---check code and usability (some error checking, mostly style and quality of problems)
• Discuss papers on assessment/usability
• Discussion of assessment and what makes a good problem, what factors are not conducive to problems usefulness. some heuristics:
  • Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
  • Problems follow Best Practices
  • Problems have "nice enough" numbers
  • The problems are clean and clear (and well-written)
  • The concepts that are being communicated and evaluated are clear
  • They have hints and solutions: support for students who are stuck or who lack other support structures
  • They are stable and well tested
• From this discussion, develop a rubric for assessing quality (technical and pedagogical) of WeBWorK problems; this is a Wiki page
• Possibly include: discussion of different types of problems---fill in the blank, numerical, etc.

Follow-up

• Revise wiki on good problems to reflect discussion

Assignment for web conference 3

• Continue work on rubric (feel free to edit the wiki, or to e-mail comments to Gavin or Jason)
  • What issues are not captured by the list of heuristics?
  • Can we shape the list of heuristics into an actual rubric to "score" a WeBWorK problem with some general assessment of how "good" it is? Is one or the other of the rubric suggestions provided better for this?
• Explore NPL and evaluate some number of problems for the model course on which each person is working with the established rubric
  • Identify 3-5 good problems, or sub-optimal problems with suggestions to improve them, based on the rubric, or just with comments about what is good about the problem (or what can be improved). This should also inform the rubric.
  • As part of the NPL exploration, look for the following information:
    • How many problems are available for the Hughes-Hallett calculus text, section 4.3?
    • Are these specifically good? Why or why not?
    • What non-calculus courses have NPL problems?
    • Can we tell which textbook problems we're finding?