PREP 2011 Web Conference II

Prep Main Page > Web Conference 2

Web-Conference 2:

Date: June 2, 3-5pm EDT

Presenters: Jason Aubrey, Dan Flath, Gavin LaRose

Discussion of the problems that were written for the assignment for this conference
1. Problem authoring issues and questions
Discuss papers on assessment/usability
1. Problem intent, and the degree to which this will be obvious or work for the student
2. Meta-context may also be important
Discussion of assessment and what makes a good problem, what factors are not conducive to problems usefulness; consider heuristics and rules.
1. Look at sample problem list.
2. Develop rubric page.
3. Develop ideas that may not be directly connected with the rubric, but which will be useful.
4. Distinction between a good problem and a good problem set

Discussion of the problems that were written for the assignment for this conference
- Clarify any questions about problem structure
- Discuss problems that are specifically "good" or "bad"
- Go through and critique/comment on problems developed from assignment from first workshop---check code and usability (some error checking, mostly style and quality of problems)
Discuss papers on assessment/usability
Discussion of assessment and what makes a good problem, what factors are not conducive to problems usefulness. some heuristics:
- Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
- Problems follow Best Practices
- Problems have "nice enough" numbers
- The problems are clean and clear (and well-written)
- The concepts that are being communicated and evaluated are clear
- They have hints and solutions: support for students who are stuck or who lack other support structures
- They are stable and well tested
From this discussion, develop a rubric for assessing quality (technical and pedagogical) of WeBWorK problems; this is a Wiki page
Possibly include: discussion of different types of problems---fill in the blank, numerical, etc.

Continue work on rubric (feel free to edit the wiki, or to e-mail comments to Gavin or Jason)
- What issues are not captured by the list of heuristics?
- Can we shape the list of heuristics into an actual rubric to "score" a WeBWorK problem with some general assessment of how "good" it is? Is one or the other of the rubric suggestions provided better for this?
Explore NPL and evaluate some number of problems for the model course on which each person is working with the established rubric
- Identify 3-5 good problems, or sub-optimal problems with suggestions to improve them, based on the rubric, or just with comments about what is good about the problem (or what can be improved). This should also inform the rubric.
- As part of the NPL exploration, look for the following information:
  - How many problems are available for the Hughes-Hallett calculus text, section 4.3?
  - Are these specifically good? Why or why not?
  - What non-calculus courses have NPL problems?
  - Can we tell which textbook problems we're finding?