Difference between revisions of "GoodProblems"

From WeBWorK_wiki
Jump to navigation Jump to search
m
m (added tag)
 
(4 intermediate revisions by one other user not shown)
Line 12: Line 12:
 
* The written solutions inform the manner in which the problem is framed
  
* The written solutions inform the manner in which the problem is framed
 
* They are stable and well tested
  
* They are stable and well tested
−
* The problem has a clear learning objective
 +
* The problem has a clear learning objective, e.g., as a COMMENT()
 
** The problem is written to promote students' accomplishment of the learning objective
  
** The problem is written to promote students' accomplishment of the learning objective
 
* Problems are accessible to screen readers and other accessibility tools
  
* Problems are accessible to screen readers and other accessibility tools
Line 18: Line 18:
   
   
  +
'''Related Ideas'''
 
* Add some sort of ranking system in the NPL
  
* Add some sort of ranking system in the NPL
 
* NPL branching: should/could there be a curated version of the NPL that would try to eliminate duplication? What standards could we use? (This list of heuristics, or a rubric score?)
  
* NPL branching: should/could there be a curated version of the NPL that would try to eliminate duplication? What standards could we use? (This list of heuristics, or a rubric score?)
   
−
Observation: it may be possible to group these into categories:
  
−
* Technical issues (numerical values, available solutions, etc.)
  
−
* Functional issues (clear learning objective, etc.)
  
−  
−
==Rubric==
  
   
  +
'''Heuristics, Reorganized'''
 
{| style="border-collapse: collapse; padding:3px; border:1px black solid;"
  
{| style="border-collapse: collapse; padding:3px; border:1px black solid;"
−
!style="padding:3px; border:1px black solid;"|Problem Type
 +
!style="padding:3px; border:1px black solid;"|''Category''
−
!style="padding:3px; border:1px black solid;"|Problem Should Have
 +
!style="padding:3px; border:1px black solid;"|''Considerations''
−
!style="padding:3px; border:1px black solid;"|Problem Should Not Have
  
 
|-
  
|-
−
|style="padding:3px; border:1px black solid;"|Skill/practice (e.g., find derivative)
 +
|style="padding:3px; border:1px black solid;"|WeBWorK Coding
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
  +
# Problems follow [http://webwork.maa.org/wiki/Problem_Authoring_Best_Practices Best Practices]
−
#
  
  +
# They are consistent over possible randomizations, and well tested
−
#
  
  +
# Problems are accessible to screen readers and other accessibility tools
  +
# The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)
  +
|-
  +
|style="padding:3px; border:1px black solid;"|Technical Issues
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
  +
# Problems have "nice enough" numbers
−
#
  
  +
# The problems are clear and well-written, and it is clear from the problem what answer is expected of the student
−
#
  
  +
# They have hints and solutions: support for students who are stuck or who lack other support structures
 
|-
  
|-
−
|style="padding:3px; border:1px black solid;"|Conceptual/multipart
 +
|style="padding:3px; border:1px black solid;"|Learning
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
  +
# Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
−
#
  
  +
# The problem has a clear learning objective, e.g., as a COMMENT()
−
#
  
  +
# The problem is written to promote students' accomplishment of the learning objective
−
|style="padding:3px; border:1px black solid;"|
  
  +
# The written solutions inform the manner in which the problem is framed
−
#
  
−
#
  
−
#
  
 
|}
  
|}
   
−
''Alternate formulation''
  
  +
==Rubric==
   
 
{| style="border-collapse: collapse; padding:3px; border:1px black solid;"
  
{| style="border-collapse: collapse; padding:3px; border:1px black solid;"
−
!style="padding:3px; border:1px black solid;"|Problem Type
 +
!style="padding:3px; border:1px black solid;"|''Category''
−
!style="padding:3px; border:1px black solid;"|Problem Should Have
 +
!style="padding:3px; border:1px black solid;"|''Considerations''
−
!style="padding:3px; border:1px black solid;"|Problem Should Not Have
 +
!style="padding:3px; border:1px black solid;"|''Included?''
 
|-
  
|-
−
|style="padding:3px; border:1px black solid;"|Fill in the blank/Select an answer
 +
|style="padding:3px; border:1px black solid;" rowspan=4|''WeBWorK Coding''
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
  +
1. Problems follow [http://webwork.maa.org/wiki/Problem_Authoring_Best_Practices Best Practices]
−
#
  
−
#
  
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
−
#
  
−
#
  
 
|-
  
|-
−
|style="padding:3px; border:1px black solid;"|Numerical
  
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
  +
2. They are consistent over possible randomizations, and well tested
−
#
  
−
#
  
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
−
#
  
−
#
  
 
|-
  
|-
−
|style="padding:3px; border:1px black solid;"|Formula
  
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
  +
3. Problems are accessible to screen readers and other accessibility tools
−
#
  
  +
|style="padding:3px; border:1px black solid;"|
−
#
  
  +
|-
  +
|style="padding:3px; border:1px black solid;"|
  +
4. The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)
  +
|style="padding:3px; border:1px black solid;"|
  +
|-
  +
|style="padding:3px; border:1px black solid;" rowspan=3|''Technical Issues''
  +
|style="padding:3px; border:1px black solid;"|
  +
1. Problems have "nice enough" numbers
  +
|style="padding:3px; border:1px black solid;"|
  +
|-
  +
|style="padding:3px; border:1px black solid;"|
  +
2. The problems are clear and well-written, and it is clear from the problem what answer is expected of the student
  +
|style="padding:3px; border:1px black solid;"|
  +
|-
  +
|style="padding:3px; border:1px black solid;"|
  +
3. They have hints and solutions: support for students who are stuck or who lack other support structures
  +
|style="padding:3px; border:1px black solid;"|
  +
|-
  +
|style="padding:3px; border:1px black solid;" rowspan=4|''Learning ''
  +
|style="padding:3px; border:1px black solid;"|
  +
1. Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
  +
|style="padding:3px; border:1px black solid;"|
  +
|-
  +
|style="padding:3px; border:1px black solid;"|
  +
2. The problem has a clear learning objective, e.g., as a COMMENT()
  +
|style="padding:3px; border:1px black solid;"|
  +
|-
  +
|style="padding:3px; border:1px black solid;"|
  +
3. The problem is written to promote students' accomplishment of the learning objective
  +
|style="padding:3px; border:1px black solid;"|
  +
|-
  +
|style="padding:3px; border:1px black solid;"|
  +
4. The written solutions inform the manner in which the problem is framed
 
|style="padding:3px; border:1px black solid;"|
  
|style="padding:3px; border:1px black solid;"|
−
#
  
−
#
  
−
#
  
 
|}
  
|}
  +
  +
''Issues'': This includes aspects of the problem that are most usefully considered as it is written as well as those which are evaluated after it is written.
  +
  +
[[Category:PREP 2011]]

Latest revision as of 12:11, 16 June 2021

Prep Main Page > Web Conference 2 > Good Problems

Heuristics

  • Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
  • Problems follow Best Practices
  • Problems have "nice enough" numbers
  • The problems are clean and clear (and well-written)
    • It is clear from the problem what answer is expected of the student
  • The concepts that are being communicated and evaluated are clear
  • They have hints and solutions: support for students who are stuck or who lack other support structures
  • The written solutions inform the manner in which the problem is framed
  • They are stable and well tested
  • The problem has a clear learning objective, e.g., as a COMMENT()
    • The problem is written to promote students' accomplishment of the learning objective
  • Problems are accessible to screen readers and other accessibility tools
    • The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)


Related Ideas

  • Add some sort of ranking system in the NPL
  • NPL branching: should/could there be a curated version of the NPL that would try to eliminate duplication? What standards could we use? (This list of heuristics, or a rubric score?)


Heuristics, Reorganized

Category Considerations
WeBWorK Coding
  1. Problems follow Best Practices
  2. They are consistent over possible randomizations, and well tested
  3. Problems are accessible to screen readers and other accessibility tools
  4. The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)
Technical Issues
  1. Problems have "nice enough" numbers
  2. The problems are clear and well-written, and it is clear from the problem what answer is expected of the student
  3. They have hints and solutions: support for students who are stuck or who lack other support structures
Learning
  1. Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)
  2. The problem has a clear learning objective, e.g., as a COMMENT()
  3. The problem is written to promote students' accomplishment of the learning objective
  4. The written solutions inform the manner in which the problem is framed

Rubric

Category Considerations Included?
WeBWorK Coding

1. Problems follow Best Practices

2. They are consistent over possible randomizations, and well tested

3. Problems are accessible to screen readers and other accessibility tools

4. The problems provide a good idea of what is being asked when a hardcopy is generated (drop down messages, colors on graphs, graph scaling, and table size)

Technical Issues

1. Problems have "nice enough" numbers

2. The problems are clear and well-written, and it is clear from the problem what answer is expected of the student

3. They have hints and solutions: support for students who are stuck or who lack other support structures

Learning

1. Problems have a clear sense of what they are trying to do (e.g., develop skills, develop understanding, evaluate student understanding, etc.)

2. The problem has a clear learning objective, e.g., as a COMMENT()

3. The problem is written to promote students' accomplishment of the learning objective

4. The written solutions inform the manner in which the problem is framed

Issues: This includes aspects of the problem that are most usefully considered as it is written as well as those which are evaluated after it is written.

Retrieved from "https://webwork.maa.org/mediawiki_new/index.php?title=GoodProblems&oldid=21277"