Difference between revisions of "Testing Checklist"

From WeBWorK_wiki
Jump to navigation Jump to search
Line 10: Line 10:
 
* Assign homework set to students.
 
* Assign homework set to students.
 
* Answer a problem correctly.
 
* Answer a problem correctly.
  +
* Generate a Hardcopy
   
 
== Intermediate Checklist ==
 
== Intermediate Checklist ==
Line 21: Line 22:
 
** Check that assigning, due dates, answer dates work as expected
 
** Check that assigning, due dates, answer dates work as expected
 
** Check that overrides work as expected
 
** Check that overrides work as expected
  +
** Check that gateways work as expected
 
* Student Pages
 
* Student Pages
 
** Check rendering of Homework Sets page and Set List page
 
** Check rendering of Homework Sets page and Set List page
Line 32: Line 34:
 
** Add user manually
 
** Add user manually
 
** Delete a user
 
** Delete a user
** Check the "SetDetail" page. Override a date and check assign/unassign.
+
** Check the "UserDetail" page. Override a date and check assign/unassign.
 
* Homework Sets Editor
 
* Homework Sets Editor
 
** Check Filter/Sort
 
** Check Filter/Sort
Line 48: Line 50:
 
** Check that various settings can be overridden for students
 
** Check that various settings can be overridden for students
 
* Problem Page
 
* Problem Page
** Check problem rendering
 
  +
** Check that hide hints option works.
*** Check rendering of functions
 
*** Check rendering of graphs
 
*** Check rendering of matricies
 
 
** Check that hints and solutions work as advertised
 
** Check that hints and solutions work as advertised
 
** Check that the mathview editor functions (including typeahead)
 
** Check that the mathview editor functions (including typeahead)
Line 101: Line 100:
 
** Change a permission
 
** Change a permission
 
** Check mathview option
 
** Check mathview option
  +
  +
== PG Checlist ==
  +
* For each of the following check rendering in both HTML and hardcopy
  +
** Check problem rendering
  +
** Check rendering of functions
  +
** Check rendering of graphs
  +
** Check rendering of matricies
  +
** Check Scaffolding problem functionality
  +
** Check popup functionality
  +
** Check radio button functionality
  +
** Check multianswer functionality
   
 
== Thorough Checklist ==
 
== Thorough Checklist ==
Line 107: Line 117:
 
* Recheck rendering of all pages on both Firefox, IE 8 & 9 & 10 (IE8 will be a little janky)
 
* Recheck rendering of all pages on both Firefox, IE 8 & 9 & 10 (IE8 will be a little janky)
 
* Recheck rendering of all pages in smaller windows
 
* Recheck rendering of all pages in smaller windows
* Recheck rendering of student pages using a phone/tablet.
+
* Recheck rendering of student pages using a phone/tablet (you can use phone emulation in Chrome)
 
* Check functionality of permissions
 
* Check functionality of permissions
 
* Check functionality of reduced credit
 
* Check functionality of reduced credit
Line 115: Line 125:
 
*** Number of different show me another uses
 
*** Number of different show me another uses
 
*** Show hints/solutions/check answers
 
*** Show hints/solutions/check answers
* Check Scaffolding problem functionality
 
 
* Achievements
 
* Achievements
 
** Enable course achievements
 
** Enable course achievements

Revision as of 10:14, 19 December 2014

This is a checklist of things to test when verifying that a new version/pull/merge/whatever is working.

Basic Checklist

  • Create a course.
  • Create a user.
  • Create a homework set.
  • View library problems.
  • Add library problems to homework set.
  • Assign homework set to students.
  • Answer a problem correctly.
  • Generate a Hardcopy

Intermediate Checklist

  • Course Admin
    • Delete Course
    • Archive Course
    • Unarchive course
    • Upgrade courses
    • Rename Course
  • Problem Sets
    • Check that assigning, due dates, answer dates work as expected
    • Check that overrides work as expected
    • Check that gateways work as expected
  • Student Pages
    • Check rendering of Homework Sets page and Set List page
    • Change password and change email and check that options work as intended.
    • Check grades page
  • Classlist Editor 2
    • Check filter/sort functionality
    • Edit a user and change permissions
    • Change the password for a user
    • Import and export Users
    • Add user manually
    • Delete a user
    • Check the "UserDetail" page. Override a date and check assign/unassign.
  • Homework Sets Editor
    • Check Filter/Sort
    • Edit multiple sets
    • Change options and dates (using datepicker)
    • Check Publish functionality
    • Check Import/Export
    • Check Create/Delete
    • Manually assign, unassign homework to users
  • Homework Set Detail Page
    • View and rearrange problems
    • Check header functionality (edit, change header, view, etc...)
    • Check that options can be changed for default sets
    • Check options for gateway sets
    • Check that various settings can be overridden for students
  • Problem Page
    • Check that hide hints option works.
    • Check that hints and solutions work as advertised
    • Check that the mathview editor functions (including typeahead)
    • Check different display options
  • Hardcopy Page
    • Check that you can generate a hardcopy
    • Check that the solutions/hints functionality works
    • Check that you can enable/disable student answers
    • Check that instructors can generate other students things.
  • Gateway Page
    • Check that you can generate new versions of gateways
    • Check that the versions are generated according to the gateway options
    • Check that proctoring works correctly
    • Check that locations filter works correctly
  • Library Browser
    • Check to see that the library has been fully loaded and things are accessable
    • Check Local Problems, From This Course, etc...
    • Check Advanced Search options
    • View some Problems
    • Create a set and edit a set using the browser
    • Check reseed, edit, view and hide buttons
  • PG Editor
    • Edit a problem
    • Create a new version
    • append
    • View and then Update
    • Check links at top
  • Past Answer
    • Check past answer page formats correctly (including typsetting formulas)
    • Check that you can view other users past answers as the instructor
    • Check that wildcards work correctly
    • Check that "allowed to view past answers" permission works correctly
    • Check that when students view past answers they cannot see which answers are correct
  • Statistics, Progress and Grades
    • Check that statistics page still function and render correctly
    • Check that the student progress page still works correctly
    • Score some sets and check that the scoring is sane
  • Email
    • Send email
    • Check macros
  • File Manger
    • Check that file list is sane
    • Check every function of side buttons
    • Upload a file
  • Course Configuration
    • Change theme
    • Change language, inactivity time, timezone
    • Test Datepicker option
    • Change a permission
    • Check mathview option

PG Checlist

  • For each of the following check rendering in both HTML and hardcopy
    • Check problem rendering
    • Check rendering of functions
    • Check rendering of graphs
    • Check rendering of matricies
    • Check Scaffolding problem functionality
    • Check popup functionality
    • Check radio button functionality
    • Check multianswer functionality

Thorough Checklist

  • Recheck rendering of all pages using math3
    • Recheck math3 limited bootstrap functionity for set descriptions and achievements.
  • Recheck rendering of all pages on both Firefox, IE 8 & 9 & 10 (IE8 will be a little janky)
  • Recheck rendering of all pages in smaller windows
  • Recheck rendering of student pages using a phone/tablet (you can use phone emulation in Chrome)
  • Check functionality of permissions
  • Check functionality of reduced credit
  • Check functionality of restrict to ip
  • Check functionality of Show Me Another
    • Check Show Me Another settings including, but not limited to
      • Number of different show me another uses
      • Show hints/solutions/check answers
  • Achievements
    • Enable course achievements
    • Import/export achievements (import using fresh copy of the achievement folder from courses.dist/modelCourse)
    • Score achievements
    • Manually assign and award achievements
    • Check evaluator
    • Edit an achievement
    • Create/Delete achievements
    • Earn an achievement
    • Check formatting of achievement page
    • Enable achievement items
    • Use an achievement item
  • Do PG editor checks with PG editor 3
  • Essay Questions
    • Assign an essay question and answer it
    • Check that the grading link appears in the appropriate places (statistics page, problem set detail, and the problem set page)
    • Check the problem grader
    • Assign a grade
    • Give a comment and check the comment is displayed to a student
    • Check that formatted math can be produced using both latex (\( and \) ) and asciimath ( ` and ` )
    • Check formatting of submitted answers on both the grader page and the problem page
  • OPL
    • Download the latest OPL using git
    • Run OPL update and look for errors

Code Checklist

Check for console.log() commands in the javaScript code.

  • These can cause trouble with IE browsers because the function console.log() is undefined unless the Developer frame is open in the browser. The code will run fine while you are checking it (with the developer frame open) and then crash silently when someone else uses it.
  • For production code these should be commented out, or turned on only by an if (debug) {} type statement.

Accessibility Checklist

  • Check pages with an accessibility validator (like WAVE).
    • It should mostly validate on student pages with only a few issues.
    • There will be some errors, in particular PG problems and instructor pages.
  • Visit pages and check contrast. (Use something like Juicy Studio Colour Contrast Analyzer with the Luminosity Contrast Ratio option.) Contrasts should all pass AA or better.
  • Visit pages and use tab to navigate. In particular you should be able to navigate easily using only tab and the enter button.