Testing Checklist

This is a checklist of things to test when verifying that a new version/pull/merge/whatever is working.

Basic Checklist

• Create a course.
• Create a user.
• Create a homework set.
• View library problems.
• Add library problems to homework set.
• Edit Set in Problem Set Detail 2 (including viewing problems).
• Assign homework set to students.
• Answer a problem correctly.
• Generate a Hardcopy

Intermediate Checklist

• Course Admin
  • Delete Course
  • Archive Course
  • Unarchive course
  • Upgrade courses
  • Rename Course
  • Manage Locations
  • Hide Courses
  • gitUpgradeNotifier
• Problem Sets
  • Check that assigning, due dates, answer dates work as expected
  • Check that overrides work as expected
  • Check that gateways work as expected
• Student Pages
  • Check rendering of Homework Sets page and Set List page
  • Change password and change email and check that options work as intended.
  • Check grades page
• Classlist Editor 2
  • Check filter/sort functionality
  • Edit a user and change permissions
  • Change the password for a user
  • Import and export Users
  • Add user manually
  • Delete a user
  • Check the "UserDetail" page. Override a date and check assign/unassign.
• Homework Sets Editor
  • Check Filter/Sort
  • Edit multiple sets
  • Change options and dates (using datepicker)
  • Check Publish functionality
  • Check Import/Export
  • Check Create/Delete
  • Manually assign, unassign homework to users
• Homework Set Detail Page
  • View and rearrange problems
  • Check header functionality (edit, change header, view, etc...)
  • Check that options can be changed for default sets
  • Check options for gateway sets
  • Check that various settings can be overridden for students
• Problem Page
  • Check that hide hints option works.
  • Check that hints and solutions work as advertised
  • Check that the mathview editor functions (including typeahead)
  • Check different display options
• Hardcopy Page
  • Check that you can generate a hardcopy
  • Check that the solutions/hints functionality works
  • Check that you can enable/disable student answers
  • Check that instructors can generate other students things.
• Gateway Page
  • Check that you can generate new versions of gateways
  • Check that the versions are generated according to the gateway options
  • Check that proctoring works correctly
  • Check that locations filter works correctly
  • Check that grades are generated correctly in scoring tools.
  • Check that statistics and student progress work correctly.
• Just In Time Sets
  • Check that you can create Just In Time sets, including checking that expand/collapse functionality works.
  • Check that the "Att. to Open Children" field works, including the "max" option.
  • Check that the "Counts Parent Grade" field works.
  • Check that restrict problem progression works
  • Check that Email Instructor on Failed Attempt works.
  • Check messages on Problem page for when there are open subproblems, for when problems do/do not count for a grade/for when you can/cannot proceed.
  • Check formatting of Problem Set, and Statistics page.
  • Check that grading works on Scoring Tools, Student Progress and Grades page.
• Library Browser
  • Check to see that the library has been fully loaded and things are accessable
  • Check Local Problems, From This Course, etc...
  • Check Advanced Search options
  • View some Problems
  • Create a set and edit a set using the browser
  • Check reseed, edit, view and hide buttons
• PG Editor
  • Edit a problem
  • Create a new version
  • append
  • View and then Update
  • Check links at top
• Past Answer
  • Check past answer page formats correctly
  • Check that you can view other users past answers as the instructor
  • Check that "allowed to view past answers" permission works correctly
  • Check that when students view past answers they cannot see which answers are correct
• Statistics, Progress and Grades
  • Check that statistics page still function and render correctly
  • Check that the student progress page still works correctly
  • Score some sets and check that the scoring is sane
• Email
  • Send email
  • Check macros
• File Manger
  • Check that file list is sane
  • Check every function of side buttons
  • Upload a file
• Course Configuration
  • Change theme
  • Change language, inactivity time, timezone
  • Test Datepicker option
  • Change a permission
  • Check mathview option

PG Checklist

• For each of the following check rendering in both HTML and hardcopy
  • Check problem rendering
  • Check rendering of functions
  • Check rendering of graphs
  • Check rendering of matricies
  • Check Scaffolding problem functionality
  • Check popup functionality
  • Check radio button functionality
  • Check multianswer functionality
  • Check niceTables functionality
  • Check LiveGraphics3D
  • Check flash applet problems
  • Check askSage
  • Check complex number problems.
  • Check alternate contexts.
  • Check unit problems.

Thorough Checklist

• Recheck rendering of all pages using math3
  • Recheck math3 limited bootstrap functionity for set descriptions and achievements.
• Recheck rendering of all pages on both Firefox, IE
• Recheck rendering of all pages in smaller windows
• Recheck rendering of student pages using a phone/tablet (you can use phone emulation in Chrome)
• Check functionality of permissions
• Check functionality of reduced credit
• Check functionality of restrict to ip
• Check functionality of restricted release.
• Check functionality of problem rerandomize.
• Check functionality of Show Me Another
  • Check Show Me Another settings including, but not limited to
    • Number of different show me another uses
    • Show hints/solutions/check answers
• Achievements
  • Enable course achievements
  • Import/export achievements (import using fresh copy of the achievement folder from courses.dist/modelCourse)
  • Score achievements
  • Manually assign and award achievements
  • Check evaluator
  • Edit an achievement
  • Create/Delete achievements
  • Earn an achievement
  • Check formatting of achievement page
  • Enable achievement items
  • Use an achievement item
• Do PG editor checks with PG editor 3
• Essay Questions
  • Assign an essay question and answer it
  • Check that the grading link appears in the appropriate places (statistics page, problem set detail, and the problem set page)
  • Check the problem grader
  • Assign a grade
  • Give a comment and check the comment is displayed to a student
  • Check that formatted math can be produced using both latex (\( and \) ) and asciimath ( ` and ` )
  • Check formatting of submitted answers on both the grader page and the problem page
• OPL
  • Download the latest OPL using git
  • Run OPL update and look for errors
  • Mass test problems using sendXMLRPC

Code Checklist

Check for console.log() commands in the javaScript code.

• These can cause trouble with IE browsers because the function console.log() is undefined unless the Developer frame is open in the browser. The code will run fine while you are checking it (with the developer frame open) and then crash silently when someone else uses it.
• For production code these should be commented out, or turned on only by an if (debug) {} type statement.

Accessibility Checklist

• Check pages with an accessibility validator (like WAVE).
  • It should mostly validate on student pages with only a few issues.
  • There will be some errors, in particular PG problems and instructor pages.
• Visit pages and check contrast. (Use something like Juicy Studio Colour Contrast Analyzer with the Luminosity Contrast Ratio option.) Contrasts should all pass AA or better.
• Visit pages and use tab to navigate. In particular you should be able to navigate easily using only tab and the enter button.