GoodProblems

Prep Main Page > Web Conference 2 > Good Problems

Heuristics

• Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
• Problems follow Best Practices
• Problems have "nice enough" numbers
• The problems are clean and clear (and well-written)
  • It is clear from the problem what answer is expected of the student
• The concepts that are being communicated and evaluated are clear
• They have hints and solutions: support for students who are stuck or who lack other support structures
• The written solutions inform the manner in which the problem is framed
• They are stable and well tested
• The problem has a clear learning objective, e.g., as a COMMENT()
  • The problem is written to promote students' accomplishment of the learning objective
• Problems are accessible to screen readers and other accessibility tools
  • The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)

Related Ideas

• Add some sort of ranking system in the NPL
• NPL branching: should/could there be a curated version of the NPL that would try to eliminate duplication? What standards could we use? (This list of heuristics, or a rubric score?)

Heuristics, Reorganized

Category	Considerations
WeBWorK Coding	Problems follow Best Practices They are consistent over possible randomizations, and well tested Problems are accessible to screen readers and other accessibility tools The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)
Technical Issues	Problems have "nice enough" numbers The problems are clear and well-written, and it is clear from the problem what answer is expected of the student They have hints and solutions: support for students who are stuck or who lack other support structures
Learning	Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.) The problem has a clear learning objective, e.g., as a COMMENT() The problem is written to promote students' accomplishment of the learning objective The written solutions inform the manner in which the problem is framed

Rubric

Category	Considerations	Included?
WeBWorK Coding	1. Problems follow Best Practices
	2. They are consistent over possible randomizations, and well tested
	3. Problems are accessible to screen readers and other accessibility tools
	4. The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)
Technical Issues	1. Problems have "nice enough" numbers
	2. The problems are clear and well-written, and it is clear from the problem what answer is expected of the student
	3. They have hints and solutions: support for students who are stuck or who lack other support structures
Learning	1. Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
	2. The problem has a clear learning objective, e.g., as a COMMENT()
	3. The problem is written to promote students' accomplishment of the learning objective
	4. The written solutions inform the manner in which the problem is framed

Issues: This includes aspects of the problem that are most usefully considered as it is written as well as those which are evaluated after it is written.