Planet WeBWorK

November 29, 2018

Michael Gage

Embedding single WeBWorK problems in HTML pages

Whether writing a full text book or just an explanation and worksheet for a class it is often useful to include an active (even better an interactive example) within the text itself. You can always use a link to refer to a WeBWorK homework set but that doesn’t have the immediacy of a problem embedded in the page itself.  

Now you can use WeBWorK to do that -- there is a new mechanism that allows you to export the content of an individual problem without all of the buttons and navigation items that usually appear for a standard homework set. Here is an example.

Problem 1 -- interval notation

In many websites this problem is interactive, try it.   After the jump I have attached a link to an HTML page with many more examples which you can try out. You can also download the page and modify it to display other questions.  On the HTML page I have wrapped each example with a "knowl" slide down javaScript that allows you to slide open or to close each example question individually.

Read more »

by Michael Gage ( at November 29, 2018 05:55 PM

June 13, 2017

Michael Gage

Installing the WeBWorK opaque server and the Moodle opaque client

Installing the WeBWorK opaque server and the Moodle opaque client


The Opaque question type and the Moodle Opaque client were created by Tim Hunt to allow webservices to power questions used in the Moodle quizzes. They were specifically designed to work with the OpenMark question engine at the Open University in England with the expectation that they could interoperate with other question engines as well. 
The WeBWorK Opaque server described in this document accepts requests from the Moodle Opaque client and returns a rendered PG question. 
The combination of the Moodle Opaque client and the WeBWorK Opaque server allows any WeBWorK question to be included in a Moodle quiz giving Moodle quizzes access to the entire range of mathematics questions commonly used in WeBWorK homework installations, in particular all of the questions in the OpenProblemLibrary. The WeBWorK/Opaque question type behaves much like STACK questions (created by Chris Sangwin) and supplements the mathematics questions available from STACK (System for Teaching and Assessment using a Computer Algebra Kernel).
This is a first attempt to interface WeBWorK (or to my knowledge any question engine besides OpenMark) with the Opaque client. Suggestions for improvements particularly from experienced Moodle users familiar with question “behaviours” and how to configure them to best advantage are welcome.
The main steps are as follows:
  1. Install the opaque question type client in Moodle
  2. Configure the Moodle question engine (and test)
  3. Create a Moodle quiz with opaque questions (and test)

The steps above do not require you to set up your own opaque server. You can use the publically available opaque servers listed to test that the Opaque client and Moodle quiz have been set up correctly. 
If you already have a WeBWorK installation then adding the WeBWorK Opaque server is straight forward. “Install the WeBWorK Opaque Server”. 
To see the tools available for duplicating an existing WeBWorK course asa course using Moodle quizzes read the section “How to transfer a WeBWorK course with homework sets to a Moodle course with quizzes”.

Read more »

by Michael Gage ( at June 13, 2017 11:03 AM

August 08, 2016

Geoff Goehle

CodeMirror and PG

Do you edit PG problems using the WeBWorK interface? Do you have trouble finding missing semi-colons because the error line numbers are wrong? Do you grind your teeth every time you try to tab in the editor and you lose focus on the text box? Then this post is for you!

The classic editing environment in all of the WeBWorK PG editors is nothing more than a HTML <textarea>. This leaves a lot to be desired. It doesn't have line numbers, it doesn't have any code coloring, and if you tab you leave the textarea. In an upcoming update we will replace the textarea editor with a CodeMirror based one. CodeMirror is a Javascript based editor that supports coloring, line numbers, regex based search and replace, and many other features. Rather than just posting a screen shot, I've included a functional version of the editor below.

Lets go through some of the new features. Feel free to experiment and explore with the demo above as you follow along.


First, notice that the editor supports code coloring. Most of the code coloring is perl based, however there are a couple of twists. You may notice that both the BEGIN_TEXT/END_TEXT block and the BEGIN_PGML_SOLUTION/END_PGML_SOLUTIION blocks are colored. There have been PG specific upgrades to the coloring code which highlight variables, latex, ascii math, as well as command subsitution in both the classical EV3 blocks as well as newer PGML blocks. The PGML coloring in particular is fancy since it correctly colors nested blocks.

Line Numbers

One nice addition is line numbers. However, another behind the scenes upgrade is a fix to a long standing bug which caused the line number errors in PG error messages to not match the line numbers in the original code. So now if you miss a semicolin somewhere in your code, the perl error line number should be closer to the actual error.

Search and Replace

Changing over to a Javascript editor means that the browser based find function no longer interacts with the editor. Luckily the Codemirror editor has its own search and replace functionality which is even more powerful than the browser search. For example, use Ctl-F and search for Text. You should see the words heighlighted as well as indications on the scroll bar. Next do a search replace with Shift-Ctl-F and search using the Regular Expression /\[`([^`]+)`\]/ and replace with [``$1``]. This will replace all regular PGML Latex with display style PGML Latex. These sorts of Regular Expression manipulations can be really powerful once you get used to them. You can read more here if you haven't seen them before.

Trailing Spaces

In PGML spaces matter. You leave two trailing spaces on a line to create a newline and you leave three to have a new paragraph. Since you can't usually "see" trailing spaces when in a PGML block the editor will highlight trailing whitespace with a red line.

Tabbing and Brackets

A couple of other common code editor features that are included are bracket matching and tabbing. When you type or move past brackets the matching bracket will be highlighted. When you press tab a soft tab consisting of four spaces will be generated. Finally when you are on a tabbed line and create a new line it will automatically start with the same indentation as the previous one.

There are other features that could be added, if there ends up being enough interest. The most interesting, but also the most complicated, would be a autocompletion. The main issue with this feature is it would have to be aware of what macros were loaded, and if the macros ever changed then the code completion might be wrong. Another potential edition would be true smart tabbing which would try to parse the code and only insert tabs when appropriate.

by goehle ( at August 08, 2016 01:51 PM

LiveGraphics and Javascript

Its been a while since I've posted, but that does not mean nothing is happening.  The just-in-type homework system is now in the pipeline to be merged into develop and should be available in the regular release in Fall 2015.  I've also been spending time rebuilding WeBWorK's default 3d grapher using javascript, webgl, and canvas.  This will eliminate the need for the current java applet and is much more portable.  Read more about it after the jump.

The best WeBWorK problems are those which have a feature or capability that would be impossible using traditional pen and paper homework.  Some of my favorites are the multivariable calculus WeBWorK problems which include an interactive 3d graph.   Students generally have a hard time visualizing 3d objects and including a manipulable 3d view of a surface is something that would be impossible without web based homework.  WeBWorK has a number of this type of problem and they were all written using a java applet called LiveJar.  This applet takes raw Mathematica code describing a graphical object, either generated by the problem pg code or included via a file, and renders a graph similar to what you would find in Mathematica.  Unfortunately java applets are increasingly difficult to get running and too many of my students ended up with problems which looked like:
Even using a properly signed applet wasn't good enough and after several student complaints I had to remove problems with manipulable 3d graphs from my multivariable calculus class last semester.

In an effort to save those problems I've put together a javascript program based on the webgl wrapper x3dom which mostly replicates the original LiveJar applet.  The system is has only just been completed, and has some rough edges.  However it does a passable job of plotting surfaces.  Instead of the previous image students will see the following, which is much better:
They can click on the graph and drag it to rotate, or use the mouse wheel to zoom.  Since javascript is so portable, I can even include an example of a manipulable graph in this blog.  Use the controls mentioned previously or those listed here to try it out.

(Note: If you are using https you will probably need to "allow insecure content" to see the graph.) For those of you who are interested, x3dom is a webgl interpreter that uses a regular html structure to define and manipulate the graphics objects in a canvas region.  For example, to render a blue sphere you just include the following html in a <x3d> html element:
<Shape render="true">
       <Material diffuseColor="0 0 1" shininess="0.2"/>
       <Sphere solid="true" radius="1"/>
Then the x3dom javascript does the rest using canvas, html5 and webgl.  (For those devices not webgl compatible it has a flash based fallback.)  The javascript replacement for LiveJar basically just parses the provided mathematica code and generates the correct x3dom html using jquery.  Some things to keep in mind about this js replacement.
  • It is specifically a replacement for LiveJar and built to correctly render the ~50 problems in the OPL which use it.  New problems could certainly be made with this script, but there are other more fully featured javascript 3d graphers out there which allow you to skip the Mathematica entirely. 
  • There is a hidden configuration option in defaults.config which can be used to turn the java applet back on. 
  • These graphs will render in most phones and tablets, and can even be manipulated using a touch screen, but the performance can vary. 

by goehle ( at August 08, 2016 12:42 PM

July 28, 2016

Geoff Goehle

WeBWorK LTI - Authentication

Do you use Blackboard (or Moodle or Canvas etc...) and wish that you could have Blackboard automatically create student WeBWorK accounts, let students go from Moodle to WeBWorK without having to log in, and have students' WeBWorK grade automatically transferred back to your Canvas gradebook?   Well then the newly revamped WeBWorK LTI interface is right up your ally.  In this post I'll talk about how to set up the LTI interface to allow students to log in to WeBWorK via Blackboard (or Moodle or Canvas, etc...) as well as automatic account creation.  In the next post I'll talk about how to set up the grade passback feature.  (This is a preview of a feature that will be available in WeBWorK 2.12.) 

Learning Management Systems (LMS) like Blackboard, Moodle and Canvas all support a protocol called Learning Tools Interoperability (LTI) which can be used to connect learning tools like WeBWorK to the LMS in a standardized fashion.  Unfortunately the LTI scene is somewhat turbulent.  The most common version, 1.2, is widespread but has limited functionality.  In particular learning tools can only pass a single grade back to the LMS in version 1.2.  This is problematic for something like WeBWorK which has many grades ranging from problem grades to set grades to course grades.   Unfortunately the  more fully featured LTI version 2.0, which does allow you to transfer full gradebooks, has not seen widespread adoption and instead some LMS are implementing their own custom solutions.  For example, there is already a great deal of integration between WeBWorK and Moodle (see here for example) that doesn't make use of LTI.  Unfortunately, instructors and  departments often don't get to choose which LMS they use.  So we have upgraded WeBWorK's existing LTI module to make the most out of the more limited version 1.2 functionality.  The result is not perfect, but it does work with a wide set of LMS including Blackboard, Canvas and Moodle.  In this post I will talk about how to set up the LTI authentication interface.  I will talk about how to enable and use the grade passback feature in the next post. 

First Time Setup - WeBWorK

Before you can use WeBWorK's LTI features you need to do some initial setup.  This will need to be done by a system administrator and in particular often requires administrator access to both WeBWorK and your Learning Management System.  All of the configuration for WeBWorK is done in the authen_LTI.conf file.  First you should copy authen_LTI.conf.dist to authen_LTI.conf.  Then you need to uncomment the line
in localOverrides.conf.  Now there are some things you will need to do in authen_LTI.conf.  The only thing you must set is the LTI Consumer Secret.  This acts like a global password and is used to authenticate all communication between WeBWorK and the LMS.  So pick something reasonably complicated.
$LTIBasicConsumerSecret = ""; #This must be set
There are some other options that can be set as well.  I'll go through them in order of most things you will most likely need to think about to less likely.

$NonceLifeTime  - When the LMS sends a user to WeBWorK it makes a request identified by a "Nonce".  This variable controls how long the Nonce is valid from the time it is created by the LMS.  It should be short enough to prevent "casual" man-in-the-middle attacks.  However, you may need to set it to something longer if your servers/network are slow or if the clocks on your servers are not synced.  

$preferred_source_of_username - When a user logs into WeBWorK from the LMS WeBWorK needs to determine the username of the person logging in.  The default is to use the email address of the student logging in as their user name.  This is because many/most LMS do not actually provide a traditional username (also called a "sourcedid").  This variable controls if the email or the sourcedid is used as the username for the student in WeBWorK.  If your LMS doesn't provide a sourcedid but you don't want to use the entire email address you can also set $strip_address_from_email and WeBWorK will strip off the server portion of the email address when creating the username. 

$debug_lti_parameters - Getting things set up for the first time can be tricky and the error messages are kept vague by default (for security).  Set this flag to have WeBWorK split out a lot more information about the LTI authentication process.  This can be a big help with troubleshooting.

$LMSManageUserData -   WeBWorK will try to keep your student's data up to date with the LMS if this flag is on.  Student data doesn't change much so you could turn it off with relatively few side effects.  You can also set custom "local functions" in authen_LTI.conf which customize users before they are created (and sets before they are assigned to users).  If you do that then you should definitely turn this variable off.

$LTIAccountCreationCutoff - For security reasons users with permission levels of ta or higher are not automatically created.  In other words you will need to manually add professor users to a course before they can log in via the LMS.  If you want to allow professors to be created automatically set this variable to 'professor'. 

%LMSrolesToWeBWorKroles - If your LMS uses roles not listed in this hash you will need to add those roles so that users get the right WeBWorK permission level. 

There are some more variables that can be set which relate to the automatic grading features, but I'll talk about those in the next post.  

First Time Setup - LMS

Next you need to set up your LMS.  In general you will need to provide the LMS  the following things: 
  1. The address of your WeBWorK server.  This has to be the address of the server as specified in site.conf, e.g. the combination of $server_root_url and $webwork_url
  2. The "Tool Provider Key" or "Consumer Key"; this is just some made up word, like "webwork".
  3. The "Tool Provider Secret" or "Consumer Secret"; this is the password used to identify the LMS to your WeBWorK site (and vice versa) and should be the same as  $LTIBasicConsumerSecret  
  4. What kind of user information is provided to WeBWorK.  
The specifics in setting up the LMS vary depending on what LMS you use.  I've posted links to setup instructions for Blackboard, Moodle and Canvas below.  Note, when the instructions refer to the "LTI" or "Tool", that is WeBWorK.
  • Blackboard:  The basic instructions are here.
    • In the Administrator Panel (under Building Blocks) click LTI Tool Providers and then Register Provider Domain
    • Input the address of the WeBWorK server in Provider Domain
    • Use Set Globally for Default configuration and input the Tool Provider Key (any word, e.g. webwork) and the Tool Provider Secret (e.g.$LTIBasicConsumerSecret)
    • Enable Send User Data. (Restricting to ssl is safer if you have https enabled.) Then enable all three user fields.  
  • Moodle:  The basic instructions are here
    • Go to Site administration > Plugins > Activity modules > LTI > Manage external tool types and click "Add External Tool Configuration"
    • Input the Tool Name and the WeBWorK server address as the Tool Base URL
    • Input the Consumer Key (any word e.g. webwork) and the Shared Secret (e.g. $LTIBasicConsumerSecret). 
    • In Privacy set both "Share" configurations to "Always".  If you have https available its safer to also enable "Force SSL".  
    • Canvas:  The basic instructions are here
      • Go to Settings > Apps > View App Configurations > Add App
      • Choose "Manual Entry" for configuration type. 
      • The "Launch URL" is the url of the course and can be left blank.  The "Domain" is just the server address
      • Input the Consumer Key (any word e.g. webwork) and the Shared Secret (e.g. $LTIBasicConsumerSecret). 
      • You should set "Privacy" to "Public". 
    A couple of other things to think about: 
    • Its usually best to have WeBWorK open in a new window.  There are usually settings you can set which will control this.  However feel free to experiment with other settings until you find a mode you like. 
    • If you want to get fancy you can also try to have the LMS provide section or recitation information by setting a custom parameter.  In all of the configurations there is a "Custom Fields" section.  You would specify "section = #" or "recitation = #" to set the section or recitation.

    Adding Links To WeBWorK

    Once you have set up your WeBWorK configuration files and added WeBWorK as an LTI Tool to your LMS you will need to make a link/assignment in the LMS that students can use to get to WeBWorK.  Again, the process for this varies depending on what LMS you are using.  I'll describe the process for Moodle, Blackboard and Canvas.
    • Blackboard:  Some basic instructions are here
      • Go to Conent > Build Content > (Create) Web Link
      • Add a Name for your link and the URL of the course you are linking to.  You can get the url using "copy link location" on the link to the course in the main WeBWorK page.  Alternatively the url structure is:
      • Check "This link is a Tool Provider"
    • Moodle: Some basic instructions are here
      • Go to a Course, Turn "Editing" on, and click "Add an activity or resource".
      • Choose "External Tool" and click "Add".
      • Pick an activity name and add the URL of the course you are linking to.  You can get the url using "copy link location" on the link to the course in the main WeBWorK page.  Alternatively the url structure is:
    • Canvas:  Some basic instructions are here
      • Go to your Course, then Assignments, then add an assignment. 
      • Select "External Tool" for Submission type. 
      • Input the URL of the course you are linking to.  You can get the url using "copy link location" on the link to the course in the main WeBWorK page.  Alternatively the url structure is:
    There are other places where you can add the link to WeBWorK.  Depending on your LMS you can add also add the link to the available "modules" or in the "navigation bar".  Consult the documentation for your LMS for more details.  

    Logging In and Creating Students

    Now you should be all set up.  You can test it out by creating a test student in your LMS and using the link to log into the LMS from WeBWorK.  The log into WeBWorK should not require a password, the WeBWorK user for the student should be automatically created, and the user should also automatically be assigned all "visible" homework sets.  
    (The warnings in the above image are the output created by the $debug_lti_parameters flag.)
    When a user logs into WeBWorK for the first time a WeBWorK user is automatically created for them.  This can save a lot of work in setting up your WeBWorK class. There are a few features/quirks of this process that should be mentioned.

    • Students created via the LMS will not have neither a student id nor an initial password.  In particular they will not be able to log into WeBWorK directly.  The $external_authentication flag is set to one by default and actually prevents them from seeing the WeBWorK login page.  They will need to log in via the LMS unless you get fancy. 
    • Accounts with a permission level of TA or higher are not automatically created by default for security reasons.  So you will have to add your TA's and other professors manually. 
    • Students created via the LMS are assigned all visible sets.  However, the due dates of these sets are not adjusted to take late registrations into account unless you get fancy. 
    • You can define the local routines which can be used to modify newly created users and newly assigned sets.  For example, you can give newly created users a random student_id/password or you can alter the due dates of a student depending on how late they register.   

    by goehle ( at July 28, 2016 01:24 PM

    May 17, 2016

    Geoff Goehle

    Release 2.12

    Hello everyone. I'm happy to announce the release of WeBWorK version 2.12. This version marks a shift to a new release schedule. Starting now we will put out new versions of WeBWorK in May and in August. In particular we won't be doing a December release since its too difficult test and release a new version in time for people to upgrade their servers over the winter break. The big feature for this release is a completely revamped LTI interface that supports grade passback. Now you can have WeBWorK grades automatically sent back to your LMS of choice (e.g. Blackboard, Moodle, Canvas). Disclaimer: Some administrative access required. Checkout the release notes and upgrade instructions after the jump or a slightly more technical version on the WeBWorK wiki at 

    Major Changes

    LTI Grade Passback 

    This is a complete rewrite of the LTI authentication module including upgrades to allow grades to be passed back from WeBWorK to the LMS (e.g. Blackboard, Moodle, etc...). The authentication feature includes the ability to automatically create students logging into WeBWorK for the first time. The grading feature has two modes. You can either have WeBWorK pass back a single "homework" grade for the entire course, or you can set it up on a set by set basis. You can read more about this module at the following links. Note: The new LTIAdvanced module not a strict upgrade from the older LTI authentication module LTIBasic. The older module still works, but should be considered legacy code.

    Hardcopy Themes 

    This adds support for different hardcopy themes. Instructors will set the theme that students have access to on the Course Configuration page, and can also select the different themes on the instructor download handouts page. Right now there are two themes: the traditional two column WeBWorK theme and a new "one column" theme which the full page width. The system is also set up so that universities can add their own themes for tests or with branding. Note: You will need to make sure the fullpage latex package is available on your server to use this feature.

    Past Answer Refactor

    This is a rewrite of the existing Past Answer's page. First, it replaces the old "wildcard/regex" feature in Past Answers with multiselect lists. Now instructors will explicitly select which sets, users and problems they would like to search for past answers. It also adds the ability to download answers from the Past Answer page as a CSV. 

    OPL Statistics 

    This adds problem usage statistics to the Library Browser. When enabled and configured you should see both local and 'global' statistics on how many times problems have been assigned, students average scores and, average attempt numbers. Here local statistics are generated from the data on your WeBWorK server and reflects how your courses use problems. Global statistics are generated from over a dozen different universities and includes information on 23000 problems. You can donate data from your school by running update-OPL-statistics and then upload-OPL-statistics. 

    Translator Refactor and Advanced Problem Debugging

    This is an extensive under the hood revamping of how PG tracks problem resources, aliases, answer blanks and evaluators. In general the new approach uses less "dead reckoning" and is smarter about treating answer blanks and evaluators as different components of the same "object". You should not see any changes in functionality.   The one exception is that users with admin privileges will have the ability to see detailed debugging information about problems. There are four options available. AnswerGroupInfo shows information about answer "groups". This is most helpful for multianswer style problems. Show Auxiliary Resources shows information about auxiliary files used by the problem. AnswerHashInfo shows information about the answer hash, including answers, filters and evaluators. PGInfo prints the PG environment, including all available variables. 

    Periodic Randomization

    This feature allows you to force problems to generate a new seed after a fixed number of attempts. It is sort of "max attempts lite" in that students can keep trying, but they have to start over. The feature is enabled via the Course Configuration page (in the Optional Modules section). Instructors can set the default number of attempts before re-randomization. The number of attempts before re-randomization can also be set on a per problem basis.

      Minor Changes

      • Updated the orientation and demo sets to more accurately reflect how things currently work.
      • Refactored the attempts table code out into its own module. 
      • Refactored the show me another feature out into its own module.
      • Changed how options are grouped on the config page. Now there is a "Optional Modules" tab which has all of the different module options in one place. 
      • When generating individual set scores from the Scoring Tools page just-in-time sets now provide information about sub problems as well as top level problems 
      • Fixed a bug where apache child processes had the same random seed. 
      • Polished two existing features to make them more obvious. The first is the ability to display extra grade information to students on the Grades page by using email merge and the second is to restrict which students instructors can see to a particular section or group sections. 
      • Started making changes about how reduced scoring dates are reported to make it clearer that the reduced scoring date occurs before the due/close date. 
      • Fixed a bug where students may be unable to open a problem they previously attempted. 
      • Added localization files for Russian 
      • Add differentiation to parserRoot 
      • Added video macros to pg so that you can use the video command to deliver video hints via html5 
      • Fixed bugs with correct answers
      • Changed the ? popups to click instead of hover since the hover detection was spotty 
      • Added better "more like this" messages to the library browser 
      • Refactored all of the server rendering scripts to one big script In particular this enables automated sanity checking of the OPL 
      • Fixed a bug with gateways with multiple pages and checkboxes 
      • Fixed a bug where the reduced credit date wasn't set on newly created sets
      • Polished the hide scores feature on Gateways. Now if hide scores is selected scores aren't shown in grades and you cannot check answers or see correct answers. 
      • Changed the default text in the welcome message for a course and made it so that students do not see the info box if the welcome message is the default text. 
      • Fixed a bug where on very new installations the Library Browser would fail with MySQL errors 
      • Added ability to reorder problems on tables using the Problem Set Detail 2 using touch-punch 
      • Made scaffold problems more accessible by allowing the closed tab headings to be navigated to and opened using the keyboard.
      • Added a parserWordCompletion macro.  This new macro allows authors to ask free response fill in the blank questions with interactive help. As a student types their answer into the answer blank, jQuery's autocomplete feature generates a drop-down list of allowable answers that match what has already been typed. 
      • The Grades page was overhauled with an number of small modifications. The total percentage score for homeworks is now shown and the "C" has been replaced by a 100. There is also a course homework grade at the bottom of the Grades table.  (This can be turned off on the Course Configuration page if WeBWorK's grading scheme doesn't match with what you use.) 

      Upgrade Instructions

      This is the short version of the upgrade instructions.  For the longer version, with commands, visit the release notes on the wiki page at 

      1. Go to /opt/webwork/webwork2, as well as /opt/webwork/pg and run git pull origin master on each directory (assuming that origin is your remote name for the openwebwork repo).
      2. Depending on your system there may be new perl modules added for 2.12. Run and look for missing modules. They may include:
        • Digest::SHA
      3. You should also run pdflatex /opt/webwork/webwork2/bin/check_latex.tex and look for missing latex packages. They may include
        • fullpage (If you are missing fullpage it is in the texlive-latex-extra package in apt based distributions and texlive-preprint in yum/dnf distributions.)
      4. Check localOverrides.conf.dist and look for changes to add to the existing localOverrides.conf
        • Library Statistics - This is a new feature (see below). There are variables which control whether or not global and local problem statistics are shown in the library browser.
        • Problem Re-Randomization - The problem rerandomization feature (see below) has default options which need to be set.
      5. Check your apache configuration file and look for changes to add to your existing configuration file. In particular you may need to add the following
        • Code to set the location of the pg directory as an environment variable
        • A line to set the server admin email
      6. Check your courses.dist/modelCourse/course.conf and look for changes against your course.conf file. They may include:
        • Removing a number of outdated settings.
        • Commented out settings for controlling which instructors can see which students.
      7. Restart the webserver.
      8.  Run the upgrade admin database script. You will not be able to access your admin course until this is done, and you will need to do that to upgrade the course databases.  Next, log into the admin course and upgrade the databases for all of your courses. Your server will not be functional until you perform this step.
      9. Download the latest version of the OPL. Note: If you have library statistics enabled this will also update your statistics tables.
      10. Download the latest version of MathJax.


      • I cannot access the admin course. This could be because you have not run yet.
      • I cannot generate a "one column" hardcopy. You may be missing the fullpage latex package. In apt based systems its part of the package texlive-latex-extra and in yum/dnf based systems its part of texlive-preprint
      • I see little vertical lines next to MathJax rendered formulas. This is caused by an incompatibility between older versions of MathJax and Chrome. Update MathJax with cd /opt/webwork/MathJax and git pull to fix the problem.
      • I do not see links to some of the editors in the Main Menu, including the links to Problem Set Detail 2, or I get an error about "SimplePGEditor". To fix this remove the definition of the %showeditors from localOverrides.conf. This hash should be defined in defaults.config and if you want to change things in localOverrides.conf you should set hash elements individually.
      • I can't get LTI to work. First make sure that you are using the new version of LTI. (Make a fresh copy of authen_LTI.conf from the distribution file.) Also enable the $debug_lti_parameters flag. For security reasons the default error messages from the LTI module are very vague.
      • I dont see the Library Statistics. Make sure that you have enable the appropriate flags in localOverrides.conf (see the first section of this page) and run OPL-update

      by goehle ( at May 17, 2016 06:31 PM

      May 01, 2016

      Geoff Goehle

      Python in WeBWorK

      Do you dream of using WeBWorK to teach students Python? If so then you have very specific dreams. After the jump I will talk about a new set of macros recently created for use in CS courses at WCU. Rather than creating an automated program tester, of which there are many, these macros are really aimed at filling the gap between labs and projects that we have in our introductory computer science courses.


      In our introductory computer science courses there seems to be something of a gap between lecture and labs, where students are introduced to new features and concepts in Python, and class projects, which are larger relatively involved programs meant to test and expand students knowledge of the concepts. Outside of weekly labs there isn't any opportunity for students to practice basic syntax and concepts, however. Our goal is to create weekly homework assignments which will help students improve their fundamentals. Of course we want to bring along the instantaneous grading and feedback that are a hallmark of WeBWorK. To accomplish this we have created a collection of PG macros. The macros provide access to two main types objects, the first is the PythonOutput object. The pg file will include a short Python script that will be used to create the object. The code is then run and the standard output of the script will be the correct answer for the problem. The basic idea is that you will present the code to students and ask them what the output will be. For example:

      This problem is testing if the student understands the formatting syntax for Python strings. The students answer will be compared exactly, using none of the smart comparisons you might usually expect from WeBWorK. (Of course students could just run the code and check the output, but that is not the absolute worst learning outcome.)

      The second type of object is a PythonCode. The pg file will include a short Python script that is used to create the object, just as before. However, in this case the student's will provide their own python script. The output of the student's Python script will be compared to the output of the "correct" Python script. The basic template for this kind of problem is that students will be provided with a description of what their code should do and will need to write a script to fulfil that description.

      This problem asks students to create a script which reads the first argument provided to the script, and if it is equal to 1 then print out the contents of the standard input, and print "hello" otherwise. (You can specify the arguments and contents of stdin in the pg file; more on that later.) These types of problems will probably work better if students are asked to write relatively short scripts, however there is support for running fairly complicated scripts including running multiple test cases.

      Of course, since you are running code provided by students via a web browser, security is going to be a concern. All Python code evaluated using this system is run in a code jail based on the EDX code jail. In particular the code is run as a separate user using a specially cordoned off python executable. Using AppArmor the python executable (python3 in our case) is only allowed to access the python libraries in the sand box and temporary files created by the jailing code. Because the enforcement happens at the kernel level via AppArmor, the system is reasonably secure

      Object Methods

      Lets take a closer look at the methods available to these objects and then we will do a deeper dive in actually coding problems.

      PythonCode() and PythonObject()

      For both objects these constructor methods takes the Python code as a string input and returns the object. Generally you would generally do something like:

      $python = PythonCode(<<EOP);
      for i in range(0,10):
      print("The number is {}".format(i))


      This either sets or returns the code used to run the object.


      This method takes in various options which can be used to provide the code things like standard input, command line arguments, and even files with text. The possibilities are:

      • files=>[['file_name','content'],['file_name_2','other content']] - This is a list (reference) of (references to) pairs, each pair is a filename and a bytestring of contents to write into that file. These files will be created in a jailed tmp directory and cleaned up automatically. No subdirectories are supported in the filename. The files will be available to the jailed code in its current directory.
      • argv=>['arg1','arg2'] - This is an array ref of command-line arguments which will be provided to the code.
      • stdin=>"This string \n will be \n in stdin." - This is a string and will be provided to the code through the stdin channel.

      tests() (For PythonCode objects only)

      This is used to provide the code with a (ref) list of hash references each containing one or more entries for "argv", "files", or "stdin" with the format described above. The correct code and the student code will be run once for each set of inputs and the output of the correct code and the student code will be compared. A correct answer is when the student code output matches the correct output in all of the test cases.

      error() (For PythonObject objects only)

      This will return the error type (e.g. "TypeError" or "SyntaxError") for the Python code, if there is one. This overrides the standard output as the correct answer, if there is an error.


      This runs the jailed code. It returns the status of the jailed code. The stdout and stderr of the code are stored in the stdout and stderr attributes.


      This returns the status of evaluated code.


      This returns the stdout output of evaluated code.


      This returns the stderr output of evaluated code.


      This returns a comparator for the object. For PythonOutput the students answer is compared to the stdout of the code. If there is a runtime error then the correct answer is the class of the error (i.e SyntaxError). For PythonCode the students answer is run as python code and the two outputs are compared for equality. If there is data for multiple collections of test input, then the outputs will be compared for all of the collections.


      Lets take a look at a couple of examples in greater depth. The following code is for a problem which tests if students can parse the logic of an if statement.

      # We use PGML for these problems because it provides code and
      # preformatted environments that are useful for presenting code and
      # output. We also include the WCUCSmacros file.


      $val = random(1,50);
      $val2 = random(1,50);

      while($val == $val2) { $val2 = random(1,255); }

      # There are an assortment of helper functions in WCUMacros
      # which can provide random text strings and variable names.

      $stringt = random_phrase();
      $stringf = random_phrase();

      while($stringt eq $stringf) {$stringf = random_phrase(); }

      # We define the actual python object, with the code here.
      # Notice the correct Python formatting and the Perl
      # interpolated values.

      $code = PythonOutput(<<EOS);
      finished = True
      score = $val
      num = $val2

      if not finished and score > 20:

      # Here we actually evaluate the code. You if you skip this step
      # the stdout attribute won't be populated. It is done manually so
      # that you can set options before it is run.


      # Here we have the text of the problem. Notice the code wrapped in
      # ``` and the answer blank definition with the comparator.

      Determine the printed output of the following section of code and
      write it in the answer box below. If the code will not run because
      of a syntax error enter "SyntaxError". If the code produces a runtime
      error enter the type of the error.

      ```[@ $code->code @]```

      Output: [____]{$code->cmp}{50}

      # Here we have the solution. The ": " makes preformatted text.
      The correct output is:
      : [@ $code->stdout @]


      When the code is rendered it looks like the following. Here we have provided an incorrect answer and asked the correct answer to be displayed.

      It is useful to note that the numerical value for score can end up being bigger or smaller than 20. So, assuming we had set finished = True, we could have had the result of the conditional depend on the problem seed, which is a good thing. What is more, because the answer is determined by the Python code itself, all of the quirks of the language will be faithfully recreated.

      Next lets look at a more complicated example where students are asked to write a program. In particular we are going to ask students to read in a file name from the first argument, open the file and print its contents to stdout. The pg code looks like this:


      # Here is our "correct" Python code. (We do not actually have to
      # evaluate the code because we are using test cases.)
      $code = PythonCode(<<EOS);
      import sys
      if (len(sys.argv) != 2):
      print('No filename provided.')
      filename = sys.argv[1]
      fh = open(filename);

      # We can use the random_word macro to come up with random words
      # for variable and file names.
      $filename = random_word().'.txt';

      # This is where we populate the input data for our tests. We have
      # two tests. One has an argument with the file name, an a file
      # with that name and a random phrase.
      $code->tests({argv => [$filename], files=>[[$filename,random_phrase()]]},
      # The second test has a file, but no argument. In this case the script will
      # exit silently.
      {files=>[[random_word().'.txt','This should not be read.']]});

      # Here is the PGML for the problem text. Notice the markdown style text
      # text formatting.
      Write a Python script which will do the following:
      * It should accept the name of a file as a command line argument.
      * If there is not exactly one command line argument it should print the string
      _No flename provided._ and exit.
      * If there is a command line argument it should open that file and print the contents.


      # We can also provide our code as a solution.
      Solution Code:
      ```[@ $code->code @]```

      Now lets take a look at what this looks like when run. In the following we have an example of a student submitting a script that opens the file and prints it, but does not check the number of arguments first.

      Notice that students are shown the results of each of their tests, as well as any errors that were reported during any tests. The "show correct answer" output shows the correct output, and the correct code is contained in the popover for the correct answer. In addition the system will use pylint to comment on their syntax and formatting. When it finds errors it will post them in a comment box like the one below.

      These features represent the basic foundation of our python macros. As we actually write problems and use them in class I'm sure we will come up with new features and best practices. Check back in a couple of semesters to see how things have progressed.

      by goehle ( at May 01, 2016 11:32 AM

      March 05, 2016

      Geoff Goehle

      OPL Statistics

      A feature recently added to the development branch of WeBWorK (to be released in 2.12) is the option to display problem statistics in the Library Browser.  You can display local statistics, which are local to your university, or "global" statistics which were generated using data contributed by a number of different institutions.  Find out more below, including how you can contribute your data to the global repository.

      If you use the Library Browser in the development version of WeBWorK you may notice that the problems have some extra data in the "header" portion.
      The three quantities displayed are:

      • Usage - The total number of students who have had the problem assigned to them. 
      • Attempts - The average number of attempts students have made on this problem. 
      • Status - The average status that students ended up with. 
      In general problems with high usage numbers don't have bugs because they have been so thoughoutly tested.  Problems with a large number of attempts are harder for students to do, and if the average status is well below 85% or 90% then it means a significant number of students are never able to do the problem correctly.  (Especially considering most people allow students an unlimited number of attempts.) All three of these quantities are useful when you are deciding to add a problem to your homework set.  Do you intend for all students to be able to do the problem?  How much effort will they need to put in to it? Etc...

      The difference between GLOBAL and LOCAL statistics is that Local statistics are generated only from the student data on your WeBWorK server, while Global statistics are generated from a central database of contributed data.  Currently the Global statistics database has data on 23000 problems with an average of 367 assigned students per problem. The data was donated by over 15 different institutions.

      How can you use this feature yourself?  It is pretty straightforward.  First you need to set the following variables in your localOverrides.conf file.
      # This flag controls if local statistics are shown in the library
      $problemLibrary{showLibraryLocalStats} = 1;
      # This flag controls whether global statistics will be displayed
      $problemLibrary{showLibraryGlobalStats} = 1;
      You also need to generate the statistics (or import it in the case of the global statistics table).  To do this just run 
       It is recommended that you run this once at the beginning of each semester to keep your local data tables up to date.  And that's it!  You need to make sure the feature is enabled and that you run the data generating script on a regular basis.  If you want to go a step further you can donate your local data to the central database.  Only aggregate data (e.g. total number of times a problem was assigned, average status on a problem, average number of attempts) is transmitted.  In particular no student data ever leaves your server.  If you decide you want to donate your data just run
      The script will ask you some questions, generate a .tar.gz. file, and upload it to the central server.  It leaves behind a copy of the .tar.gz file in case you want to see what exactly is transmitted.

      by goehle ( at March 05, 2016 05:28 PM

      WeBWorK LTI - Grading

      Do you use Blackboard (or Moodle or Canvas etc...) and wish that you could have Blackboard automatically create student WeBWorK accounts, let students go from Moodle to WeBWorK without having to log in, and have students' WeBWorK grade automatically transferred back to your Canvas gradebook?   Well then the newly revamped WeBWorK LTI interface is right up your ally.  In the last post I talked about how to set up the LTI interface to allow students to log in to WeBWorK via Blackboard (or Moodle or Canvas, etc...) as well as automatic account creation.  In this post I'll talk about how to set up the grade passback feature. (This is a preview of a feature that will be available in WeBWorK 2.12.) 

      With the newly revamped LTI interface it is possible for WeBWorK to automatically pass grades back to your Learning Management System (LMS), e.g. Blackboard, Moodle, Canvas, etc...  However because we are using the more widespread, but also more restrictive, version 1.2 of LTI there are some quirks.  The way LTI grade passback works is when you create the "Assignment" in your LMS that points to WeBWorK that "Assignment" gets a corresponding column  in your LMS gradebook.  WeBWorK then returns a percentage grade which is used to assign the student grade in the LMS.  In particular there is exactly one column in the LMS grade book for each link to WeBWorK in the LMS.  Given this restriction there are two modes for the WeBWorK LTI grading interface.

      • Course Grade Mode: In this mode you create a single link/assignment in the LMS which points to WeBWorK.  WeBWorK will then return the "cumulative" homework grade as a percentage to the LMS.  In other words the grade in the LMS column is the course homework grade. 
      • Homework Grade Mode:  In this mode you create a single link/assignment in the LMS for each problem set in WeBWorK.  WeBWorK will then return the total score for the problem set to the LMS.  In other words you will have multiple columns in the LMS gradebook, one for each link, and they will be filled out with the student scores on individual WeBWorK problem sets.  
      The first mode is the easiest to set up, but the second mode is more granular and ends up transferring more data to the LMS gradebook.

      First Time Setup

      There are several configuration variables in authen_LTI.conf which need to be set to enable grade passback.  

      $LTIGradeMode -  If this variable is not set then the grade passback feature is turned off.  If it is set to "course" then the Course Grade Mode described above is used.  If it is set to "homework" then the Homework Grade Mode described above is used. 

      $LTIGradeOnSubmit - If this flag is enabled then WeBWorK will attempt to update student grades every time they submit an answer to a problem.  This keeps the student grades current, but can be a drain on your server.  If either the WeBWorK server or the LMS server bogs down it might be a good idea to turn this off. 

      $LTIMassUpdateInterval - Because instructor activities like adding new sets, deleting problems, or marking problems correct can change student grades we periodically update all student grades just to make sure things are current.  This interval controls how often that mass update happens.  It is set for once every 24 hours by default.  

      In addition to configuring WeBWorK you may also need to enable the grading feature on your LMS.  How this works varies among different LMS.  I will describe how to configure Blackboard, Moodle, and Canvas. 
      • Blackboard - There is a global flag which controls if Tools are allowed to submit grades.  You will need administrator access to change this.
        • In the Administrator Panel go to Building Blocks > LTI Tool Providers > Global Properties
        • Enable the setting to allow any Tool Provider to post grades to the Grade Center.
      • Moodle - There is a flag which controls if Tools are allowed to submit grades.   This can be set for individual courses or site wide. 
        • Go to Site administration > Plugins > Activity modules > LTI > Manage external tool types.
        • Edit the WeBWorK tool and under Privacy enable "Accept grades from the tool".
      • Canvas - Grade passback is always enabled in Canvas and requires no global configuration. 

      Adding Graded Links

      You add links to WeBWorK the same way you do if you are just using the authentication feature.  In particular you create an "Assignment/Link" and provide the url of the WeBWorK course.  The only additional thing you have to worry about is setting up the grade.  In general you need to make sure grading is enabled for the link and set the total number of points the assignment is work.  It is recommended that you use 100 points so that the grade looks like a percentage.  However, the grade given to the student will always be a percentage of the total available points so  you can use however many points you want.  I'll describe the specifics for Blackboard, Moodle and Canvas.  
      • Blackboard: You need to make sure "Enable Evaluation" is set to yes.  You can also set the number of points available and the due date of the assignment. 
      • Moodle: All you need to do is set the points possible as well as any of the other Moodle specific options. 

      • Canvas:  All you need to do is set the number of points possible. 
      Its important to remember that the only information transferred from WeBWorK to the LMS is the percentage grade of the assignment.  In particular things like visibility and due dates are not automatically synchronized.  It is recommended that you do not set due dates in the LMS and allow WeBWorK to control the availability of the problem sets.  

      Course Grade Mode

      Lets look at an example of Blackboard set up with Course Grade mode.  Here we have a single link to our WeBWorK course in the Content area.  Students will use this link to access WeBWorK. 

      Once in WeBWorK they navigate and finish problem sets in exactly the same way they normally would.  WeBWorK behaves just as it would if there were no LMS at all.  The only difference is when students submit a grade they should get a message like the one below.  
      Note:  In many LMS instructors do not have grades.  In this case you will see the message "Your score was not successfully sent to the LMS".  This is fine, unless its not.  If you suspect that there is a problem enable $debug_lti_parameters to see more verbose error messages.  The goal is to see the following in WeBWorK (if you don't have Homework Totals in your version of WeBWorK don't worry, its not out until 2.12)
      and the following in Blackboard

      And that is pretty much it.  The Course Grade Mode is not particularly complex.  Some things to remember: 
      • How the WeBWorK grade is used in the "Total Grade" on the LMS is entirely up to you and how you have set the weights in the LMS.  See the documentation for the LMS for how to set weights on grades.  
      • Only make one (graded) link to WeBWorK.  If you make more than one then your gradebook will have multiple columns and WeBWorK will get confused about which one to send the grade back to.  
      • Users have to log in to WeBWorK via the LMS at least once for this to work.  The easiest way to guarantee this is to make the LMS create the users.  However, once they have logged in via the LMS once they can log in directly via WeBWorK after that (assuming your authentication system is set up that way).  

      Homework Grade Mode

      This grading mode is more granular in that students get all of their homework grades, as well as any other LMS grades, all in one place.  The cost is that you need to set up a separate link to each WeBWorK problem set.  For example in this Canvas course we have links to three different WeBWorK assignments
      The links are set up exactly the same as described in the previous post with the exception that they should point to the homework set that they correspond to.  For example a link to the Orientation set would have the url 
       In general the link to set <set name> should have the following format:

      • Default or Just-In-Time Set: http://<server address>/webwork2/<course name>/<set name>
      • Gateway: http://<server address>/webwork2/<course name>/quiz_mode/<set name>
      • Proctored Gateway: http://<server address>/webwork2/<course name>/proctored_quiz_mode/<set name>
      If the set name is not somewhere in the url for that set on the LMS side then it is incorrect.  Once things are set up students then click on the links to access their homework and complete their assignments as normal.  If things are working then they should see their grades appear in the LMS.
      There are a couple of caveats.

      • Students have to click on the link corresponding to the set at least once for grading to work.  Once in WeBWorK they can navigate like normal.  However, if they try to navigate to sets which are open, but which they haven't visited via the LMS get get messages like

      • Homework due dates and problem totals are not synced between WeBWorK and the LMS.  The easiest thing to do is to set all of the homework sets to be open forever in the LMS and let WeBWorK manage access to the homework sets.  You can also set the number of points for each set to 100 in the LMS and the score will be a percentage.  You can also keep the due dates and number of points synchronized, but you will have to do so by hand.  
      • Also note that depending on how you weight things in the LMS the contributions of the homework to the total grade in the LMS will be different than the Homework Total in WeBWorK.  (In particular the WeBWorK grade is the total number of points earned over the points available, while, assuming you set the points to 100 for every assignment in the LMS, the LMS grade is the average of all the homework percentages.)  You can turn off displaying the Homework Total in WeBWorK using the Course Configuration menu if this bothers you or your students.  
      In general using Homework Grade mode is more work, but if you make heavy use of an LMS it is probably worth it.  You can use different grade modes in different courses by setting the $LTIGradeMode variable in the course.conf file.   So you are not locked into using one type of grading for all of your courses.  

      by goehle ( at March 05, 2016 05:05 PM

      Hardcopy Themes

      Are you tired of the standard "two column" hardcopy format that has been used by WeBWorK since the project was started?  Well starting in version 2.12 there is the option to choose a "hardcopy theme".  Currently there is are two themes, a one column mode and a two column mode.  However, the system is build so that you could add your very own.

      Choosing a hardcopy theme is simplicity itself.  When generating a hardcopy simply choose "One Column" or "Two Columns" for the Hardcopy Theme option.

      You will get either this:

      or this: 

      You can also set the default theme on the Course Configuration page.  In particular the theme chosen on the Course Configuration page is the only theme that students are allowed to use.

      Unless you are willing to get your hands dirty, then that is the extent of the feature.  However, there is also the possibility of adding new Hardcopy themes to the mix.  To create a new theme you need to add a folder to the webwork2/conf/snippets/hardcopyThemes folder.  This new folder should contain the following files:

      • hardcopyPreamble.tex - This tex file contains the LaTeX code that will be at the beginning of the hardcopy.  This is where all of the preamble code goes, including packages, macros, page formatting, etc...
      • hardcopyPostamble.tex -  This tex file contains the LaTeX code that will be at the end of the hardcopy.  It ends the document.  
      • hardcopyProblemDivider.tex - This tex file contains the LaTeX code that will be inserted between each problem.  
      • hardcopySetDivider.tex - This tex file contains the LaTeX code that will be inserted between each set. 
      • hardcopyUserDivider.tex - This tex file contains the LaTeX code that will be inserted between each user.  
      • - Normally this pg file would be rendered and placed at the top of each set.  However, most sets now have their own Hardcopy Header file specified on the Set Detail page which takes precedence over this one.  
      • -  This pg file is rendered and the resulting tex code is put at the bottom of each set.  
      The changes you make to these files, in particular the hardcopyPreamble.tex file, will control what the resulting hardcopy looks like.  You could put more space between problems by adding a vspace to the problem divider.  You can add university specific detailing to the footers.  You could change the font or margins.  Once you have gotten the hardcopy format you desire its time to add it to WeBWorK.  In order to do that you should add overiddes to the following variables in localOverrides.conf.
      # Available Hardcopy themes (located in snippets)
      $hardcopyThemes = [qw(oneColumn twoColumn)];
      # Hardcopy theme names (pretty names for themes)
      $hardcopyThemeNames = {oneColumn => 'One Column',
            twoColumn => 'Two Columns'};
      # Default Hardcopy them
      $hardcopyTheme = "twoColumn";
      The theme name (e.g. oneColumn, twoColumn) should be the name of the folder you made in hardcopyThemes.   The pretty theme name (e.g. One Column, Two Columns) is the name that is actually displayed to users.  Once you add your theme to $hardcopyThemes and $hardcopyThemeNames you are done.  The theme will automatically appear on the Hardcopy page and the Course Configuration page as an option.  If you make a particularly good theme, please consider contributing it to the main repository so other people can use it.

      by goehle ( at March 05, 2016 05:05 PM

      Grades, External Grades, and Email

      The grades page in WeBWorK has seen some changes for the next version (2.12).  In particular homework percentages are now displayed for each homework and a "Homework Totals" score is displayed as well.  This post also presents an existing feature which most people may not know about; how to display external grades in WeBWorK.

      Grade Page

      There have been a number of tweaks made to the grades page, as you can see below. 
      The two main changes are the addition of a percent score for individual homework sets and a Homework Totals score for all sets.  The Homework Totals score is the total number of points earned out of the total available points.  In particular, it matches the score you would get using the Scoring Tools page if you selected all homework sets.  If you want to change the weight of a homework set simply change the value of all of the problems in the set.  (For example if you set the value of all the problems in the set to zero then the sets grade doesn't count toward the Homework Total.)  If you do something entirely different with your homework scores and don't want students to get the wrong impression from the Homework Total, you can turn the feature off in the Course Configuration page.  
      Another feature, which isn't new but most people don't know about, is the ability to display external grades in WeBWorK.  The pertinent message on the grades page is
      There is no additional grade information. A message about additional grades can go in in [TMPL]/email/report_grades.msg. It is merged with the file [Scoring]/report_grades_data.csv. These files can be edited using the "Email" link and the "File Manager" link in the left margin.
      In particular lets use the Email link to create the following message and save it as report_grades.msg 

      Hello $FN

      Your grade for midterm 2 is $COL[7]

      Your 6th week grade is $COL[-1]
      Recall that when you are emailing students you can choose a Merge File to include data from a csv in your email.  The exact same macros are used in the report_grades.msg.  The list of available macros is: 
      • $FN - first name
      • $LN - last name
      • $SID - student ID number
      • $SECTION - student's section
      • $RECITATION - student's recitation
      • $STATUS - C, drop, withdrawl, audit, etc.
      • $EMAIL - student's email address
      • $LOGIN - student's login name
      • $COL[3] - third column in merge file
      • $COL[-1] - last column in merge file
      As an aside, emailing students their WeBWorK grades periodically is a great way to make sure that they are paying attention to doing their homework.  

      The merge file we use for displaying external grades, report_grades_data.csv,  is a csv with the same format that is produced by the Scoring Tools page.  In particular all of the csv's from the scoring tools page go in the "scoring" folder, which is located one directory up fro the "templates" folder.  The easiest way to add external grades is to download one of these scoring csv files, and edit it.  Delete all but the first six columns and then add new grade columns as you see fit.  Finally re-upload it to the scoring folder with the name "report_grades_data.csv".  The import part is that the first six columns are, in order,
      1. STUDENT ID
      2. LOGIN ID
      3. LAST NAME
      4. FIRST NAME
      5. SECTION
      You can put whatever data you want in the remaining columns. For example we might have a report_grades_data.csv file in the "scoring" directory with the following data.
      admin admin Admin Prof 9 10 67 89
      teststud teststud Funky Spoon 3 5 78 56
      practice1 practice1 PRACTICE1 JANE 2 4 56 48
      practice3 practice3 PRACTICE3 JANE 2 4 48 43
      practice4 practice4 PRACTICE4 JANE 2 4 78 16
      practice5 practice5 PRACTICE5 JANE 2 4 99 56
      practice6 practice6 PRACTICE6 JANE 2 4 26 78
      practice7 practice7 PRACTICE7 JANE 2 4 89 98
      practice8 practice8 PRACTICE8 JANE 2 4 92 82
      practice9 practice9 PRACTICE9 JANE 2 4 56 81

      With the report_grades.msg above and this report_grades_data.csv file if we visit the Grades page we see the following at the bottom:
      As a bonus, we can also send this message to all of the students using the Email page with no additional work. There is not currently a way to integrate external grades into WeBWorK so that they are included in course calculations, but this, at least, allows you to keep students up to date.

      by goehle ( at March 05, 2016 04:34 PM

      February 25, 2016

      Geoff Goehle

      Release 2.11

      I wrote this when version 2.11 was released (almost two months ago at this point) but never got around to actually proofing and posting it.  So here is the late version...

      WeBWorK version 2.11 has been released.  The big addition for this release is a the new adaptive homework type, just-in-time.  I wrote about just-in-time a while ago on this blog and it has finally made its way into the master version of WeBWorK.  There are also some other big new features which I've described below. (The official release notes can be found at

      Major Additions

      Just-In-Time Assignments

      This is a new adaptive homework assignment type. The basic idea is that instead of having a linear list of problems, the problem set has a "tree" of problems. Each problem has a number of "child" problems which are only shown to the student if they have trouble with the parent problem. The basic idea is that these child problems will have review content to help the student solve the original problem. There are a number of other features including restricting the progress of students through the homework and emailing the professor if the student fails to complete the assignment.

      Editing Course.conf

      The ability to edit course.conf from the web has long been a security and stability issue. (One missing semicolon can lock a professor out of their course and require command line access by a super user to recover.) Because of this users with "professor" privilege will not be able to edit either course.conf or simple.conf from the File Manager. Instead only users with "admin" privilege will be able to edit those files. This is likely to feel restrictive to users who are used to making lots of modifications via course.conf. There are really three main ways to deal with this:

      • Common configuration changes can easily be added to the Course Configuration page. If you have a configuration page that you make a lot submit it as a github issue.
      • Administrators in the admin course are always given admin privileges if they are added to a course. Any admin user can grant any other user admin privileges using the Classlist Editor. You can use this to give trusted professors permission to edit course.conf. Courses that don't have any "admin" privileged users can have one added using wwsh.
      • You can disable this feature by editing the $uneditableCourseFiles variable. Do this at your own risk.

      Def File Changes 

      Because of all the new features added to WeBWorK the way problems are defined in def files needed to be refactored. Now problems in def files are defined using the same "property = value" format as the rest of the problem set. You can read more about the new def format at (specifically the Problem List Version 2 portion). Older def files using the "version 1" problem list can still be imported, but all exports will use version 2.

      XMLRPC Rendering 

      Driven by the Mathbook XML project and the Moodle opaqueserver linkage the ability to render problems via the XMLRPC pathway has been greatly improved. In particular problems can be embedded in pages using iframes. The embedded problems can be interacted with, answers can be checked, and there is even the option to store answers and scores locally in the browser.

      Live Graph 3D 

      The java based Live Graph 3D grapher has been replaced by a drop in javascript alternative. This javascript alternative is webGL based and should work on any device which has webGL support. It should work as a replacement without any additional configuration.  This grapher has been tested with the existing problems in the OPL which used the old java grapher. If there are issues with local problems please report them.

      Minor Additions

      • wwsh: Due to recent changes in Perl wwsh no longer gives you an interactive perl interface. Instead you call wwsh with a script as follows wwsh <course name> <script name>. Several scripts have been provided to help recover courses. 
      • Set Defs Search: You can now set configuration variables which allow the Homework Sets Editor page to search for set def files in folders other than templates. In particular it can look for def files in subdirectories of templates and optionally in the OPL. 
      • No More Timezones: Timezones in date entry have been removed, as have most time zones in displayed dates. (Due dates displayed to students still have time zones.) Now all times are interpreted to be in the time zone defined in the Course Configuration page. This should eliminate the longstanding problem of entering a date with the wrong daylight savings timezone and having your time unexpectedly shifted by an hour.
      • Library Browser Icons: There are now icons in the library browser to help signify when problems use math objects and when problems don’t have any randomization. 
      • Course Renaming: In addition to renaming courses you can now give courses new Course Titles and new Institutions. This is done using the Rename function on the course admin page.
      • Numerous Bug Fixes:  Many bug fixes, most of which can be found listed in the release notes. 

      Upgrade Instructions

      1)  Go to /opt/webwork/webwork2, /opt/webwork/pg and run
      git pull origin master
      on each directory (assuming that origin is your remote name for the openwebwork repo).

      2) Depending on your system there may be new perl modules added for 2.11. Run and look for missing modules. They may include:
      LWP::Protocol::https (version > 6.06)
      An outdated LWP::Protocol::https module can keep the Library Browser from running correctly if you are using https.

      3) Check localOverrides.conf.dist and look for changes to add to the existing localOverrides.conf. There are a couple of important things here that will negatively affect your system if you don't deal with them.

      The show editors hash should be removed. If it is not removed then there will be minor annoyances (e.g. the link to the problem set detail page that appears when you are looking at a homework set will not appear). Instead of redefining this hash, overrides should be made to the hash elements individually. For example
      # These will enable the versions of the problem set detail page and library browser page that do not use the xmlrpc pathway
      #$showeditors{problemsetdetail1} = 1;
      #$showeditors{librarybrowsernojs} = 1;

      # This will enable the slightly fancier PG problem editor 3
      #$showeditors{pgproblemeditor3} = 1;

      # This will turn off ww3 (if you are on the ww3 branch)
      #$showeditors{ww3} = 0;

      While this does not appear by default in localOverrides.conf.dist, something you may want to modify using localOverrides.conf is the variable controlling which files users with professor privilege can access. For security reasons default only users with admin privilege will be able to access course.conf using the File Manager. The variable you would change is
      # For safety reasons these course specific files cannot be edited from the File Manager except by an administrator.  These are paths relative to the course directory.
      $uneditableCourseFiles = ['simple.conf',

      Options which control if the homework sets editor will look for set .def files in places other than templates. In can search in subfolders of templates or even the OPL. The options are 
      # Searching for set.def files to import 
      # Uncomment below so that when the homework sets editor searches for set def
      # files, it searches beyond templates; it can search deeper subfolders of 
      # templates, and optionally also descend into Library
      #$options{setDefSearchDepth}=4; #search down 4 levels

      4) Check site-conf.dist and look for changes to add to the existing site.conf. Things to look for include:

      The version of your server is determined automatically from your server banner. However, if you have turned off the version in your server banner you can specify it manually with
      $server_apache_version = ''; # e.g. '2.22.1'  

      Specifying if tls is allowed when sending email. This is enabled via
      $tls_allowed = 1;  

      Specifying an email address to display when apache has problems with webwork pages, assuming you don't want to use the default apache admin email. This is done via
      $webwork_server_admin_email ='';

      Specifying a default timeout for the checkurl command. This is done via
      $externalPrograms{checkurl}     = "/usr/bin/lwp-request -d -t 40 -mHEAD "; 

      5) Check your apache configuration dist file and look for changes to add to the existing apache configuration file. Things to look for include

      A line which ensures that each apache child process has a different random number seed. The main impact this has is that without the following line practice users are not shuffled randomly.
      PerlChildInitHandler "sub { srand }"

      The code which overrides the administrator email using the variable mentioned in the previous section. The code looks like

      my $webwork_server_admin_email = $ce->{webwork_server_admin_email};
      if ($webwork_server_admin_email) {
      $webwork_server_admin_email = $ce->{webwork_server_admin_email};
      warn "set email from site.conf";
      } else {
      $webwork_server_admin_email= $ENV{SERVER_ADMIN};

      Optional code to make use of the opaque server interface with Moodle.

      Optional code to improve performance by using mod_expires.

      6) Restart the webserver. (with apachectl graceful or similar command.)

      7) Log into the admin course and upgrade the databases for all of your courses. Your server will not be functional until you perform this step.

      8) Download the latest version of the OPL using
      cd /opt/webwork/libraries/webwork-open-problem-library
      git pull origin


      • You can get a warning when trying to log into the admin course: You will need to run You will also need to upgrade all of your other course databases using the admin course.
      • You can't edit the course.conf file: You need "admin" permissions. See item 2 of Primary Changes.
      • You can't find the "SetName" link in the sidebar that normally brings you to the Problem Set Detail page for a particular set: You need to remove the %showEditors hash from your local overrides file. See item 4 of Upgrade Instructions.
      • The library browser may stop responding: When you select a subject the chapter list may not populate and you may get error popups. The main cause of this is a missing or outdated LWP::Protocol::https (version < 6.06) package.
      • Remember to update the OPL and run OPL-update.

      by goehle ( at February 25, 2016 03:50 PM

      October 09, 2015

      Michael Gage

      Using WeBWorK questions in Moodle quizzes -- the Moodle/WeBWorK question bridge

      This is a copy of a post that I made to the Moodle forum on Quizzes.

      I’m happy to announce a version of WeBWorK that can serve mathematics questions via a Moodle quiz.  This gives a tighter connection between Moodle and WeBWorK, allowing students to work exclusively with the Moodle quiz interface while WeBWorK produces the mathematics questions and processes the student answers in the background.  The result is similar to using STACK to produce and process mathematics questions.

      This is a first attempt at realizing the question level WeBWorK-Moodle bridge which I proposed in this forum in my earlier post last January ( and on my blog ( )

      It makes most of the 25K+ mathematics questions written for WeBWorK’s OpenProblemLibrary ( available for use in Moodle quizzes as well.  The OPL began with questions designed for single and multivariable calculus courses (and that is still the majority of the collection) but has expanded in both up and down to include questions for linear algebra and differential equations, complex variables, operations research, and statistics; as well as precalculus and developmental mathematics problems and many other math subjects.

      To see this Moodle/WeBWorK connection in action,  view the Moodle course for linear algebra and differential equations at (  You will need to login as “visitor”, using the password “visitor” in order to view the quizzes. (Guests are not allowed to view Moodle quizzes.)

      This is  a mockup of the matrix algebra/ODE course which I taught in 2009 using the Moodle/WeBWorK assignment connection (which presents an entire homework assignment using the WeBWorK interface).  In each week  you’ll see the original WeBWorK assignment that I used (the icon is a yellow spider web on blue background) and near it the Moodle Quiz version of the same assignment.  Try each of them to get a comparison between the WeBWorK/Moodle assignment bridge and the new WeBWorK/Moodle question bridge.

      I have set the behavior of the Moodle Quiz so that it closely approximates the original WeBWorK assignment.  These assignments are meant primarily for homework/practice not for normative assessment.  The student can retry each question as often as she wishes and if there are several answer blanks then evaluation is given separately for each blank.

      The Moodle portion of this bridge is built on the Opaque question type module implemented by Tim Hunt for interfacing with webservice test engines.  I am indebted to Tim for help and advice while I was implementing this Moodle/WeBWorK question interface.

      Information for installing and configuring the Moodle Opaque question type is available on github at  Specifically you need the opaque qtype module: and the opaque behaviour module:

      I’ll provide more detailed suggestions for creating your own Moodle/WeBWorK questions later but for those somewhat familiar with both WeBWorK and Moodle here is the short explanation.
      1. You create a new Opaque engine pointing to the url: and set the question bank URL for the engine to .
      2. For each question the questionID should be the same as the path of a WeBWorK problem in the OpenProblemLibrary with the following replacements.
      The initial segment should be “library” instead of “Library”, each slash is replaced by two underscores, and each hyphen is replaced by three underscores. (This is required in order to conform to the Opaque client naming scheme.)
      For example:
      is obtained using the questionID:

      For those who are really impatient the WeBWorK server function is provided by adding the project opaque_server along side a standard WeBWorK course. The Opaque Server can be found at ( along with minimal documentation for installing.  

      This is the first version of the Moodle/WeBWorK question bridge so the behavior may change slightly in the future.  Because STACK and WeBWorK are already very similar I would be interested in creating a question type that was interchangeable between the two background mathematics engines.  Instructors would be able to mix STACK and WeBWorK questions in the same quiz in such a way that it was invisible (or nearly invisible) to students which engine was handling the question.

      Your comments are welcome.

      -- Mike Gage, University of Rochester, Rochester, NY

      by Michael Gage ( at October 09, 2015 10:54 AM

      February 09, 2015

      John Travis

      Upcoming WeBWorK related PREP summer workshops

      Registration has just been opened for two faculty development workshop to be held during Summer 2015 through the MAA's Professional Enhancement Program (PREP).

      The first one is a repeat of a very well received workshop offered the past two summers:

      Davide Cervone, Gavin LaRose,
      Paul Pearson, and John Travis

      June 8, 15, 22, and 29, 2015

      This online workshop introduces participants to the technical skills necessary to identify, edit and create high-quality WeBWorK problems.  Participants new to WeBWorK will also be provided an extra workshop introducing them to WeBWorK as well.  Two sessions (about 2 hours each) will be provided on each date noted above.

      The second offering is a new course aimed for potential WeBWorK system administrators:

      Mike Gage, Jason Aubrey, Geoff Goehle, 
      Danny Glin, and John Travis

      July 6, 13, 14, and 20, 2015

      Participants in this online workshop will cover the technical skills necessary to install and maintain a robust, reliable, and secure WeBWorK server.  In particular, participants will be walked through the steps necessary for installing WeBWorK on their own servers.  Most days will include morning and afternoon sessions.

      Advance registration for these workshops is $175 (before posted deadlines) and is available at .

      We are looking forward to a full house for both workshops.  Previous workshops have required us to close registration so if you are interested in either then please make plans to register soon.

      Questions about either workshop:

      by drjt ( at February 09, 2015 03:04 PM

      January 19, 2015

      Michael Gage

      Connecting WeBWorK to Moodle 2.x's question type for use in Moodle quizzes.

      This note was posted on moodle's quiz forum on January 7, 2015.

      I am working on plugging WeBWorK in as a back end for analyzing mathematics questions in a way similar to STACK. I will be using the opaque question type as a starting place. 

      I am just beginning this project and I expect to have questions for this forum as I proceed so I thought it a good idea to introduce myself.

      I am Mike Gage, a math professor at the University of Rochester, (Rochester, NY, USA) and one of the originators of the open source online homework system WeBWorK. It was originally designed to deal with mathematics at the calculus level but it now has questions from middle school through ordinary differential equations, linear algebra and complex analysis.

      It already interoperates with Moodle as an assignment module -- there is single sign-on between Moodle and WeBWorK and after the student finishes their WeBWorK homework assignment the grades are returned to the google grade book. My goal is to make WeBWorK work as a question type for Moodle2.x. We already have a question type version for Moodle1.9 

      For those interested here is an example of a course using WeBWorK for assignments (search for the spiderweb icon and click on it).

      For those interested in an example of WeBWorK being used to create question types in Moodle 1.9 (Created by Matt Leventi in 2007) 
      (you can log in using the guest button). This version could use improvement but I don't plan to work on this further unless there is a large demand -- it seems better to put the effort into a Moodle2.x version. 

      Finally for those interested in WeBWorK in its native form you can look at this link: (you can login using the guest button)

      The home wiki for WeBWorK is

      Thank you in advance for considering the questions I will have as I start on adapting
      the opaque question type to WeBWorK. smile

      Take care,


      by Michael Gage ( at January 19, 2015 04:48 PM

      January 04, 2015

      Geoff Goehle

      Release 2.10

      We are happy to announce the release of WeBWorK version 2.10, now available from your friendly neighborhood github repository.  This new version features many additions.  For example the Show Me Another feature allows students to request different versions of existing problems.   There are also a number of additions which help bring WeBWorK meet web accessibility standards.  For site administrators we have a new "update notification" system so your server will never be out of date again.  Read more about it in the release notes or after the jump. 

      This version contains a couple of major new features and a collection of smaller features.  Lets go through the larger changes. 


      We made a number of changes to the math4 interface to make it much friendlier to screen readers and other accessibility tools.  Rather than one large change this is a huge number of small tweaks and improvements.   We brought all color contrasts up to AA level and added an easy to see focus box and a skip to main content link.  Tables and fields should mostly meet accessibility standards now. The big exception is that fields in problems do not have proper labels.  To fix this we added aria labels to fields generated by PG. These labels identify each answer field as "Answer 1", or "Answer 1 Part 1", etc...  These are not technical replacements for input labels, but are functional replacements for most screen readers. We also made tweaks to make the student interface of WeBWorK easy to navigate using the tab key, reordered the problem results table to make more sense when read left to right, and added links which change focus from the "correct/incorrect" text to the corresponding field.  The VPAT  for release version 2.10 can be found here.

      Update Notifier:

      When properly set up and enabled using the configuration files this feature will notify you when there are updates, or new versions, available for WeBWorK or PG.  It will also notify you when there are updates to the OPL and if you need to run OPL-update.  The notification appears on the Course Administration page of the admin course.  Caution: This uses git ls-remote to check in with the github repositories.  If the wait is too long the system can be disabled.  You can read more about how to set up and configure this feature here

      Theme Overrides:  

      Using the files math4-overrides.css and math4-overrides.js administrators can include their own js and css in a way which will be compatible with future updates.  Using this administrators can change coloring, the header images, or even basic functionality.  As an example of how this can be applied we added a red and green variant of the math4 theme.   You can read more about it here.

      Show Me Another: 

      When enabled this feature will give students the option of viewing a different version of their current problem. Depending on the configuration this version may or may not come with a solution, hints, the ability to check answers, or the ability to view correct answers.  The problem is guaranteed to be different so the feature will fail on problems with no randomization.  To enable the feature and adjust its settings go to the  PG - Problem Display/Answer Checking portion of the Course Configuration page.  Note that in order to use the feature you will need to enable it for each individual problem.   You can read more here.  

      Reduced Scoring: 

      Changed how reduced scoring works so that instead of a fixed reduced scoring period, each set has an individual "reduced scoring date".  After this reduced scoring date students will receive reduced credit for their answers.  This reduced scoring date can be set differently for individual sets or even individual students.  This can, for example, allow you to extend a due date for a student with reduced credit without changing the due date of the set for the class.  You can read more about it here.

      PG Additions:  

      There were a number of new PG macros added which provide some nice functionality and are listed below.  
      •  This allows problem authors to create tables with a large degree of control, including accessibility support. 
      • This is a context for Percentages.
      •  This context allows ',' instead of '.' as decimal places.
      •  This context allows ']2,3[' as an alternate notation for open intervals.
      •  This allows for inequality based set notation.
      •  This allows for partitions of integers.
      •  This allows for permutation notation.
      •  This allows for more generalized roots.
      •  This new Scaffolding structure is an upgrade to the existing compoundProblem series of macros.

      Smaller Changes:

      There are also a bunch of smaller changes and improvements.  You can find the full(ish) list at the release notes page here.  The most important ones are listed below.  

      • Changed password hashing to use SHA512. This allows passwords to be functionally longer than 6 characters and is much safer. 
      • Added option to use a custom file in place of "site-info.txt".  This will keep git from overwriting your site-info.txt changes. 
      • Changed how display options work so that they are persistent.  Now there will be no "Display Options" sidebar menu.  Instead these are set in the "User Settings" page (formerly the "Password/Email" page) and persist across page calls and logins.
      • Removed the configuration option for Drag Math.  The feature can still be enabled in course.conf
      • Removed the configuration option for pgMatrixMacros since those have been replaced by MathObjects.  The flag can still be modified in course.conf.
      • Fixed a bug where fileFromPath and read_whole_problem didn't work on newer systems.


      Because of the number of additions and upgrades since version 2.9, the upgrade process for version 2.10 is more involved.  In particular a number of new modules have been added since version 2.9, as well as significant database upgrades. There are also changes to the configuration files which will need to be integrated into your existing configurations. The process for upgrading to 2.10 is described below.:
      1) Go to /opt/webwork/webwork2, /opt/webwork/pg and run
      git pull origin master
      on each directory (assuming that origin is your remote name for the openwebwork repository).  
      2) There were a number of new perl modules added for 2.10. Run and look for missing modules. They may include
      • Array::Utils
      • Dancer
      • Dancer::Plugin::Database
      • Iterator
      • Iterator::Util
      • Path::Class
      • XML::Parser::EasyTree 
      Install using your distribution's package manager or via cpan
      cpan Array::Utils Dancer::Plugin::Database Iterator::Util Path::Class XML::Parser::EasyTree
      3) There were a couple of new LaTeX packages added for 2.10.  Change to a writeable directory outside the webwork2 directory (like ~)  Run pdflatex /opt/webwork/webwork2/bin/check_latex.tex and look for missing packages. Added packages include booktabs, tabularx, colortbl, caption, xcolor, path.  In apt based systems you may have to install texlive-latex-recommended and yum based systems may have to install texlive-path. Warning: There currently is no package on CentOS 7 which provides the path package. You can manually download and install the Fedora package and it will work.
      4) Check localOverrides.conf.dist and look for changes to add to the existing localOverrides.conf. They may include
      • Information for the Upgrade Notifier.
      • Information for a local "site_info.txt" file.
      • Default values for the showMeAnother feature.
      5) Check site-conf.dist and look for changes to add to the existing site.conf. They may include paths for the external programs git and curl.  
      6) Run the command from webwork2/bin to upgrade your admin course database. Warning: You will not be able to access the admin course until you perform this step.
      7) Restart the webserver (with apachectl graceful or a similar command).
      8) Log into the admin course and upgrade the databases for all of your courses.
      9) Download the latest version of the OPL using
      cd /opt/webwork/libraries/webwork-open-problem-library
      git pull origin
      Hopefully you dont run into too much trouble or accidentally set your server on fire.  If you do have questions or need help, though, feel free to stop by our friendly neighborhood forums or checkout our irc channel.  Bugs can be reported at bugzilla or github.  Happy upgrading. 

      by goehle ( at January 04, 2015 07:20 PM

      December 22, 2014

      Geoff Goehle

      Upgrade Notifier

      Tired of not knowing if your WeBWorK server is up to date?  Do you wake up at night worrying that an important security fix was released and you didn't hear about it?  Your wishes have been granted by the new Upgrade Notifier feature.  This feature will let know you if there are new updates for your chosen WeBWorK, PG, or OPL branch and can even remind you to update your library index.  Available now in the develop branch (and coming up in version 2.10) and described in more detail below.

      The Upgrade Notifier appears on the Course Administration page of the admin course.  When you log in you will see something like this:
      As you can see this installation has fallen behind!  There are upgrades available from branch develop in remote origin for WeBWorK and PG.  This means that there are commits in the develop branch of the origin remote (usually the openwebwork github repository) which are not in the currently selected git branch on our server.  In addition, the OPL is also out of date.

      Note:  The Upgrade Notifier is very much git based in terms of its setup and configuration. In particular you should know about remotes and branches and be running your WeBWorK server from a git based installation.  You can read more about github and WeBWorK in this article, or learn git here.

      Why origin and develop?  Well that comes from our configuration file.  The upgrade notifier block of localOverrides.conf.dist looks like this:

      # Repository Information
      # This is where you put your remote and branch for your WeBWorK, PG and OPL
      # github repositories.
      # To disable checking for a particular repository, set the remote and branch
      # variables to be empty strings.
      $enableGitUpgradeNotifier = 1;
      $gitWeBWorKRemoteName = "origin";
      $gitWeBWorKBranchName = "master";
      $gitPGRemoteName = "origin";
      $gitPGBranchName = "master";
      $gitLibraryRemoteName = "origin";
      $gitLibraryBranchName = "master";

      As we can see there is an option to turn the Upgrade Notifier on and off, as well as options to set the remote and branch for WeBWorK, PG, and the OPL.  Most people will use this default setup.  For the above server we have changed the branch for PG and WeBWorK to develop but left the remote as origin (because that currently points to openwebwork).  If we wanted to we could even have the update notifier point at our own personal repository or at some feature branch we care about.  We haven't changed the library branch or remote, and usually you won't.  Of course if you want to use this feature you will need to copy these lines over to localOverrides.conf

      Note:  If you are working with a fresh installation or upgrade, you should also make sure that the external program git is defined in the site.conf file by the line $externalPrograms{git} = "/usr/bin/git";

      Once your repository information is configured, whenever you visit the Course Administration page your server will use git and ls-remote to check and see if the configured remote and branch has commits which are not in your currently selected local branch.  If you are using the default setup of pointing to origin and master then you will only see an upgrade message for PG or WeBWorK when there are important bugfixes or a new version.  If you point to a different branch, like develop, you will see upgrade messages more often.  (Usually you will want to select a local branch which tracks the configured remote and branch.  However, you can have additional commits and customization in your local branch without affecting the upgrade notifier.)   The OPL gets updated frequently so you are likely to see upgrade messages for that component frequently, even on the master branch.  If you would like to disable notifications for any portion of the system, just set the remote name and branch name for that component's variables to be empty strings.

      Note: The upgrade notifier will also let you know when there is a new version of WeBWorK available.  This is mostly important for people using the default setup.  Upgrades to the master branch of WeBWorK or PG will always be pretty safe.  However if you see the new version message then the upgrade process will likely be a lot more involved.  However, it will be worth it because you will also get all of the new features that come with a new release.  Our releases are biannual so you should only see this message twice a year.

      When there are upgrades available you can download them by using the following command as necessary in /opt/webwork/webwork2 or /opt/webwork/pg or /opt/webwork/libraries/webwork-open-problem-library 
      git pull <remote name> <branch name>
      If you are using  the default setup then this command should be pretty safe, unless there is a new WeBWorK version.  Since the above server is tracking develop there could be any number of changes waiting upstream and we may have additional configuration or conflict fixing to do.  In any case, after updating all three components on our server we get a message which looks like this
      Everything is up to date, but we forgot to run OPL-update.  After doing so we refresh the page and get the message.

      Finally even system administrators get to experience the thrill of seeing the green bar.

      by goehle ( at December 22, 2014 01:11 PM

      July 31, 2014

      Geoff Goehle

      Customizing Colors

      This post will address a relatively superficial, but often requested, feature which has been recently added to the develop version of WeBWorK - customizing theme colors and logos.  This latest change adds a couple of color variations for math4 right out of the box, as well as a built in way for system administrators to customize math4 themselves.

      The first thing you will notice if you download the latest version of develop is that there are a couple of new themes available in the Course Config page - math4-green and math4-red.  Selecting either of these new themes will give you this:

      You should notice that these variants are basically the same as math4 with different colors.  This is because they are implemented in such a way that they are just additional color css layered on top of math4.  So in theory they will always be up to date with the latest math4.  Of course, if you give a man three colors he is going to want 256.  That is why this update also allows sysadmins to customize math4.

      The key to this customization are the files
      These files are not shipped by default (although they have .dist versions) and are not tracked by git, so they won't be overwritten when you update.  However, if you put either of these files into the math4 theme folder they will be included into all WeBWorK pages after the normal math4 css and javascript.  This means that you can use them to make changes to math4 without potentially breaking your upgrade process. 

      Lets go through an example where we will change math4 to use the Western Carolina University colors and logo.  First we copy over the distribution versions of the override files to give us something to start with
      cd /opt/webwork/webwork2/htdocs/themes/math4/
      cp math4-overrides.css.dist math4-overrides.css
      cp math4-overrides.js.dist math4-overrides.js
      We start with the coloring.  The math4-overrides.css file automatically simplifies the coloring of math4 so that there are only a handful of colors that are used over and over.   They are:
      • Dark Blue: #003388
      • Light Blue: #0088CC
      • Dark Green: #519951
      • Light Green: #88FF88
      • Dark Red: #FF9494
      • Light Red: #CC7676
      • Dark Yellow: #BDB45E
      • Light Yellow: #EDE275
      Because we are using css (and not less) we have to change these by using a global find replace on the hex value, but that isn't so difficult.  The Dark Blue and Light Blue are the primary theme colors and are used almost everywhere.  The Green, Red, and Yellow colors are mostly used for alerts, such as marking things correct or incorrect.  We will be conservative and just change the main colors.  We would like to use WCU's school colors
      • Purple: #492C88
      • Gold: #CD7F32
      I'm going to replace the Dark Blue by Purple and the Light Blue by Gold.  We could do this using any text editor and a global find/replace but lets get fancy
      sed -i 's/003388/492C88/g' math4-overrides.css
      sed -i 's/0088CC/CD7F32/g' math4-overrides.css
      This by itself gets us much of the way there
      We could spend a lot more time tweaking the colors.  For example, something that is missing by default (for ease of initial configuration) are hover colors for the buttons.  Adding a slightly different hover color will make the buttons seem much more dynamic. 

      However, we will leave the colors as they are and address another often requested feature: changing the logos.  This is not done via css, rather it is done by javascript.  If we open math4-overrides.js we see the line
      /* This changes the MAA Logo on the top to a new image */
      //    $('.maa_logo a img').attr('src','new-path-here');
      There is a similar line for the WeBWorK logo.  So if we want to put our school logo in place of the MAA logo we could put the appropriate image file (wcu.jpg) into webwork2/htdocs/images and change our js line to
      $('.maa_logo a img').attr('src','/webwork2_files/images/wcu.jpg').css('height','40px');
      Remember that things in htdocs will be available on the web under the webwork2_files directory.  This gives us the following
      And that is it.  If we wanted to we could also replace the WeBWorK logo in much the same way.

      So after only a few commands we have been able to give WeBWorK a whole new look.  We could go farther, too; much farther.  The thing to remember is that the override files allow system administrators to execute arbitrary css and (jQuery) javascript code that will be run after the usual math4 code.  The javascript/css combination is extremely powerful and allows you to change pretty much anything,  including html content.  So for example if I wanted to change "Hmwk Sets Editor2"  to "HW Sets Editor" in the Instructor Tools navigation menu I could just add the following to math4-overrides.js
      $('#site-links a:contains("Hmwk Sets Editor2")').html('HW Sets Editor');
      Of course, you have to come up with the javascript yourself, but the sky really is the limit here.  You can add whole features this way, all without touching core WeBWorK code.  (Although if you do that you might as well just write it into the core code and submit it as a pull request!)  There are a few gotchas to keep in mind, but in general this should provide power administrators with a great deal of flexibility.

      • Remember that unless you delete math4-red and math4-green your instructors will still be able to manually change to those themes.  Any overrides that you put into htdocs/themes/math4/math4-overrides.js or the corresponding css file will not automatically make it into math4-red or math4-green.  You will need to link to the math4-override files in the math4-red/math4-green subdirectories in order for your changes to carry over.  Of course if you copy over math4-overrides.css then math4-red and math4-green won't be red/green anymore. 
      • While your modifications in the override files will not be overwritten by an upgrade, and should generally just work, it is possible for WeBWorK updates to change math4 in such a way as to be incompatible with your override files.  If this happens you will have to merge your changes with the new math4-overrides distribution files.  
      • By default using this file removes some of the "dynamic" elements of bootstrap, like grading on the buttons, or a slightly different highlight color on the buttons. This is done for ease of configuration.  If you look at  htdocs/js/vendor/bootstrap/css/bootstrap.css you can copy/modify the relevant css code to get fancier buttons working again.  However this all has to be done "manually" and can't be automated with a global find/replace.
      • Some of the red/green coloring is done using inline styling via pg.  This means that it can't be overridden using the math4-overrides.css file.  If you really want to alter all of the correct/incorrect coloring you will need to use javascript and math4-overrides.js to do it. 

      by goehle ( at July 31, 2014 07:56 AM

      June 20, 2014

      Geoff Goehle

      Release 2.9 Changes

      We are happy to announce that the latest version of WeBWorK, 2.9, has been released.  This is a smaller release than 2.8, as evidenced by the fact that there are no database or module additions. The primary changes to WeBWorK for this release are: support for Apache version 2.4 (and hence the latest versions of Ubuntu and Fedora), polishing the math4 interface, adding a couple of new PG modules, and general bug fixing.  A full list of changes is below.

      Supported Distributions and Depreciation

      The following are the distributions that are officially supported. You still can use WeBWorK on whatever you like, of course. In particular if you install WeBWorK on a very recent version of Ubuntu or Fedora and have problems let us know.
      • Ubuntu LTS 14.04
      • Debian 7
      • Centos 6
      • Fedora 20
      We also want to support RedHat 6 but can't reliably test on it because the distribution is not freely available. If you install on RedHat 6 and run into trouble let us know as well. We are also removing or depreciating support for some things that have been replaced by newer components.
      • RedHat 5 is no longer supported because its version of Perl is not new enough.
      • The old version of the Homework Sets Editor and the Classlist Editor are no longer shown by default and are no longer maintained. If they are enabled via a configuration file they will still (mostly) work, but are not kept up to date.
      • With the increased use of JavaScript, more of our functionality has become dependent on to what were once mostly cosmetic themes. Right now the only two supported themes are math3 and math4, and math3 is only partially supported for the newest of features. Older themes can be found in the unsupported-themes directory.

      Fresh Install

      If you want to use version 2.9 with a fresh installation you should take a look at the WeBWorK installer, ww_install.  The installer has been upgraded to support Apache 2.4 and, consequently, newer distros. You can use it by following the instructions on the previous link or here. Note: ww_install also installs the components necessary to switch your installation to the develop branch or even the WeBWorK 3 branch (ww3) without installing new software. (You will need to upgrade your course tables, and run OPL update though.)


      If you want to upgrade an existing installation to version 2.9 you can do so very easily assuming you are using git.  For this to work we are assuming that you are using a git based WeBWorK installation with the standard installation locations.  If you aren't using git it might be best to move to a new installation.   To upgrade to the latest version of WeBWorK do the following:
      cd /opt/webwork/webwork2

      cp htdocs/site_info.txt htdocs/site_info.txt.bak
      git pull
      cp htdocs/site_info.txt.bak htdocs/site_info.txt
      cd /opt/webwork/pg
      git pull
      sudo apachectl restart
      cd /opt/webwork/libraries/webwork-open-problem-library
      git pull
       Some things to watch out for when running these commands
      • If you don't have standard locations, then alter the paths above.
      • If you are not using the master branch you may need to alter the git commands.  More information on using git to upgrade can be found here.
      • Depending on your distribution, restarting apache may require a different command
      • This works if you are upgrading from 2.8.  If you are upgrading from 2.7 or earlier you may also need to use check_modules and CPAN to install new modules, as well as upgrade your course databases.

      What's new?

      • System Updates
        • Added support for Apache 2.4
          • This includes internal changes to WeBWorK to support Apache 2.4 as well as a new configuration file, webwork-apache2.4-config, which should be used for Apache 2.4
          • Remember that you also need to use the new Apache 2.4 directives when setting up your server: MaxConnectionsPerChild in place of MaxRequestsPerChild, and MaxRequestWorkers in place of MaxClients.
        • Removed jsMath. This display method has been entirely superseded by MathJax
        • Added maketext calls to places where they are missing. This is the first step for localizing these strings. They now need to be translated.
        • Fixed depreciation warnings for Perl >5.16
      • General Interface Updates
        • Added an unsupported-themes folder and moved everything but math3 and math4 into that folder.
          • Newer pages assume things like the presence of jQuery or certain bootstrap elements. These are missing from old themes that haven't been kept up to date. Using these themes will cause unexpected results. You can re-enable them, at your own risk, by moving them from unsupported-themes back to themes.
        • Added LaTeX editing support for MathView. This has two uses.
          • If you enable MathView and use essay questions then MathView can be used to typeset LaTeX equations in an essay box.
          • If you set $options{PGMathView} = 1; in a config file (not available in the main course config page) it will make the MathView editor available for inserting LaTeX equations into the PG Editors
        • Cleaned up how MathView looks on small screens.
        • Cleaned up support for long set names. They still don't look great but they are better. (This includes allowing a lot of long set names to wrap. This is better than them overlapping other elements, but they can look weird if they are long names in a narrow area.)
        • Added a button which can collapse or expand the navigation bar on the left hand side of the page. The bar starts collapsed on phones and expanded otherwise.
        • Improved how the Problem Sets and Problem Set page are formatted. They have links instead of buttons for set names and problem numbers. The set names wont drop below the checkmark/radioboxs anymore.
        • Cleaned up the tables for the Homework Sets Editor and Classlist Editor. The select box in each is its own column, the select all feature has been moved to the table header. The select all, unselect all and clear buttons have been removed since they are not needed.
        • Cleaned up some other math4 display things, including removing extraneous buttons and removing extraneous button coloring.
      • Library Browser Updates
        • Fixed a bug where the Hide Problem button froze browsers when used on a problem with "More Like This". Also improved how Hide Problems and More Like This work together.
        • Added support to the OPL for cross-listing taxonomy in the library. Now things like "Vector Geometry" can appear in more than one subject.
      • Achievement Updates
        • Fixed Achievement Editor bugs, including errors on exporting and selections disappearing.
        • Changed achievement scoring so that dropped and audit students aren't scored. When you score achievements the success message contains a link to the scoring file.
      • Under the Hood Updates
        • Changed how Essay Answers works to speed things up. It no longer reports if you have problems that need grading on the Problem Grader links, but the pages load faster.
      • Homework Sets Editor Updates
        • Fixed a bug with the Datepicker and set names with a "." in them. Also improved how Datepicker works with the import date shift functionality.
          • Fun fact some of our strings get parsed so often that "." needs to be replaced with "\\\\\."
      • Classlist Editor Updates
        • Cleaned up the classlist table so that abbreviations for statuses aren't shown and the email address isn't shown. The actual status and a link to send an email are shown instead.
        • In ClassListEditor2 filtering on Permissions or Status now filters on the actual Permission or Status, not the code used to store those in the database.
      • Problem Editor Updates
        • Fixed a bug where "open in new window" sometimes didn’t open in a new window in PGProblemEditor2. Also made it so the "open in new window" checkbox is responsive to what you are selecting.
        • Upgraded PGProblemEditor2 so append works with gateway sets.
        • Added support for "Append" back to PGProblemEditor3.
        • Changed the behaviour of PGProblemEditor3 so that if you "Save As" or "Append" it will open the new file for editing instead of remaining on the old file.
      • PG Updates
        • Fixed a bug where applet based questions weren't working properly.
        • Fixed a bug with images in problems not having the right size in tables in IE.
        • Added a statistics macro and pcctables macro for pg to defaults.config.
        • Added some statistics functionality to PG, including more flexibility for Bernoulli variables and the ability to use arbitrary discrete probability distributions.
        • Added smarter fraction support to PG. Now toFraction now turns things like .33333 into 1/3 and not 33333/100000.
        • Added support to override arithmetic operators to distinguish between different forms of answers (
        • Added support for more accessible tables in PG using
        • Changed how MathJax is loaded by PG. It should be a little faster on pages loading a number of problems at the same time.
        • Upgraded PG to support the newest versions of Perl.
        • Improving how tables work in PG. Among the little improvements are how table headers work, how images work in tables, and accessibility concerns.
        • Fixed a bug with contextTypeset that was causing bugs when students used asciiMath formulas in Essay questions.

      What could possibly go wrong?

      • You can get a red warning error about using a theme which does not exist. You should go to Course Configuration and select either math3 or math4.
      • If you are upgrading from an older version you may need to update your course tables and install new modules.
      • If you don't also upgrade PG to the latest version you might not be able to have problems load correctly on the latest versions of Perl.

      by goehle ( at June 20, 2014 01:33 PM

      June 18, 2014

      Geoff Goehle

      The Life of Code

      Yet another thing to come out of the recent WeBWorK::Asheville code camp was a crystallization of our development protocols.  We now have a more formal process for how code gets reviewed and integrated into the openwebwork repository.  In this blog post I will go through the process of creating, submitting, and eventually merging a feature into WeBWorK.  This is just an overview and will likely be out of date.  You can read up on the most current iteration here.

      Suppose we want to make a feature which adds a chat client to WeBWorK so students who are logged in at the same time can chat with each other.  You might think the first thing you need to do to create a new feature for WeBWorK would be to write some code.  In truth the process starts much earlier.  Before you can write code you need to have a good coding environment.  Of course, it all begins with a WeBWorK installation.  Your development installation should not be the same machine that you use for students as it will be unstable.  I recommend a virtual machine setup as described here.  Your setup can be whatever you would like it to be, however, as long as it is based off of git.

      Git is our source code management system with a central repository hosted by at this address.  If you don't already have one, you should sign up for an github account.  Next you need your own personal copy, or fork, of the openwebwork repository.  Go here and click the fork icon in the upper right hand side, and follow the instructions.  Now do the same thing for the pg repository here.  Now you need to link your github repository to your development machine.  If you used the installation script, the openwebwork repository is a remote on your development machine called origin.  The following code will add your personal repository as a remote called personal
      cd /opt/webwork/webwork2
      git remote add personal
      cd /opt/webwork/pg
      git remote add personal
      Now we have git setup and linked to our repository.  You can pull code from a repository using git pull remote-name branch-name and you can push code to a repository using git push remote-name branch-name.  You can check out what remote repositories you have setup using git remote -v and you can see what branches you have by using git branch -v.

      Be warned, git is an extremely flexible code management system, but it has a very steep learning curve.  Its quite easy to get yourself in a situation where you have to start over because you messed up your commits or your branches.  If you haven't used git before I recommend trying out the interactive tutorial here.   You should also feel free to ask questions, either on the forum or the freenode #webwork channel. 

      You might think we could start coding, but we have another decision to make first.  All new WeBWorK features should go in their own "feature branch".  You (almost) never make commits directly to one of the openwebwork branches.  Furethermore, different features should go into different branches so they can be kept separate until they are merged into the main code.  So my chat feature will go into a branch which I will call studentchat.  However, before I can create that branch I need to choose where I want the feature to end up.  There are generally three choices.
      • master:  The master branch always represents the current release.  You should only target a feature for this branch if you are making a small fix for an important bug.
      • release/x.y: This branch contains the beta version of the next WeBWorK release.  It is primarily used for testing and bugfixing purposes. In particular feature branches should only target this branch if they are fixing bugs for an upcoming release, or if they add a feature which is either superficial or has little chance of breaking anything.
      • develop:  The develop branch is where we generally collect new features. Features are only merged into this branch when they are more or less done, which means that develop should be relatively stable.  Anything that is merged into develop represents something that will be included in the next release branch.  In development terms this means your feature has to be out of "alpha" and well into "beta". Most developers should base their feature branches off of develop.
      My chat feature is a reasonably substantial addition so I will be aiming to get it included into develop.  If you are unsure about what you should target either go with develop or ask someone.  Now that I have decided where my branch is going I can create it as follows
        git checkout -b studentchat origin/develop
        If I wanted to target release/x.y I would change origin/develop to origin/release/x.y  Because we make this choice at the time we create the feature branch, we have to have a good idea of where our feature will end up before we even start writing it.

        Note:  With rare exception, branches created or "branched" from a certain branch have to be merged back into that same branch.   So this studentchat feature branch could only be merged into develop.  Sometimes you decide that you targeted the wrong branch after you have already started on your feature.  In this case you can use the following command to move your changes from, say, develop, to release/x.y
        git rebase --onto release/x.y develop studentchat
        Whew! And all of that before we even wrote our first line of code.  We are at the good part now, though.  We can start implementing our chat client.  In addition to practicing good code techniques, like integrating and using existing code, striving for consistency, and being security conscious, we should do the following:
        1. Commit often with git commit -a, but only commit changes that are relevant to the feature you are implementing.  
          • If you end up finding a bug, don't just fix it as part of the feature branch.  Use git stash to stash your current changes, switch to a new branch (maybe one based off of release/x.y) and fix the bug there.  Commit the changes to that branch, then switch back to your feature branch and pick up where you left off with git stash pop.  
        2. Pull often with git pull origin develop, fixing conflicts if necessary.  Only pull from the branch you are targeting.  
          • If you based a branch off of develop then the safest thing to do is to only ever pull develop into your feature branch.  There are situations where its OK to pull other branches, even other feature branches, into your feature branch, but its almost always more trouble than its worth. 
        A well disciplined work flow might look something like this:
        code for chat feature
        git commit -a
        git pull origin develop
        fix conflicts
        git commit -a
        write more code, but find a bug
        git stash
        git checkout -b bugfix origin/release/x.y
        fix bug
        git commit -a
        git push personal bugfix
        git checkout studentchat
        git stash pop
        write more code for chat client
        git commit -a
        and so on ...
         Note:  If your feature requires you to change code in pg then you will need to create a separate branch in pg for your changes there.  This is because pg and webwork2 are two different git repositories.  As a result, if you are writing code for both the you will need to have two branches and, eventually, two separate pull requests.  On the other hand, if you are only changing pg code then you don't have to make a branch for webwork2.

        Eventually you will be done with your feature.  Of course, by done I mean you have finished writing your feature and you have tested it.  Remember, even code that is submitted to develop should be out of alpha and well into beta.  There are four types of testing which you should be aware of:
        • Ad Hoc Testing: This is the most common type of testing and just involves using your code for a while and seeing if anything breaks. This is a good first step but is not sufficient.
        • Black Box Testing: This is where you systematically go through and test the functionality of your feature. Click on everything that can be clicked and make sure it does what it is supposed to do. For very small features or bug fixes this can be sufficient. 
        • White Box Testing: This is where you test the code itself. First, click the green "Compare, Review, and Create a Pull Request" button on your personal github repository. Use the edit button to select your feature branch and its target branch. This will show you a list of changes between your feature branch and its target. Go through change by change and for each perform an action in WeBWorK which causes the code changes to be run. Check that the results are what you expect them to be. This level of testing is needed for any sizable feature, or any code which impacts core WeBWorK functionality. 
        • Regression Testing: This is the longest and most thorough type of testing. It involves checking to see that the code you wrote didn't introduce bugs into previously working code. Some of this is done at the same time as you are doing the Black Box or White Box testing, and for many things full scale regression testing is impractical. However if you need to do this level of testing then go to the Testing Checklist and start going through the list, one item at a time, adding new items as necessary, and fixing any bugs you come across. 
        So lets suppose we have written and tested (white box and black box) our chat feature.  Now we let the little bird out of its nest into the open world.  Its time to create a pull request.
        1. Make sure your feature branch can be merged cleanly. In other words, pull it's target branch, fix any conflicts, commit the results, and push to your git repository.
        2. Go to the webwork2 repository page in your personal github account, select your feature branch from the branch dropdown and click the green button.
        3. Click "Edit" on the pull request bar and change the base of openwebwork/webwork2 to the target branch for your feature.   In our case we would use develop.   
        4. Review your pull request. In particular take a close look at the file changes.
          • Are all of the changes relevant to your feature? Did anything unexpected sneak in?
          • Do you have any "configuration" changes or changes with hard-coded path? Any site specific code?
          • Are there a reasonable number of changes? 
          • Will it be easy for a reviewer to look over your submission?
        5. Pick a title for your pull request and write a description. You should describe the major changes included in the pull request as well as fairly detailed instructions on how to test to see if the changes are working. The better your description is and the clearer your instructions are the more likely someone will be able to test and merge your pull request in a timely manner.
        6. Double check that the pull request is for the correct branch and submit. 
        One your pull request is submitted all you can do is wait.  In general it can take days for someone to get back to you about it, and it can be even longer during finals or midterms.  Eventually someone will get back to you, though, and the review process begins.  This is a little like the review process for publishing a paper, with faster turnaround (hopefully!).  The person reviewing your pull request will start by posting a message with any questions or issues or bugs that they have come across.  Don't be disheartened if they have a lot of concerns.  Testing code is like proofreading; other people are always going to find things you missed.  Next you address the reviewers questions, fix any bugs and consider making any other changes you need to make.  When you make changes, commit the result, and then push the branch to your personal github repository the pull request will automatically update with the latest changes.  There is no need to submit a new pull request.  The reviewer can then take a look at the latest version and the cycle will repeat itself over and over.... (sometimes for months).  Eventually, after hard work and patience, the feature will be polished enough that the reviewer will merge the pull request into its target branch.  Once this happens you can delete your feature branch
        git branch -D studentchat
        Congratulations!  Your code is part of WeBWorK now.   If you end up having to make any changes to it (and you almost certainly will) you will need to do so on a new feature branch, starting the whole process over again.  (This is not strictly speaking true, but it is the safest practice.)

        by goehle ( at June 18, 2014 07:58 AM

        June 06, 2014

        Geoff Goehle

        Just-In-Time: Adaptive Homework

        Another project that I have been working on for the past couple months is a brand new type of assignment which will eventually join traditional Homework Sets and Gateway Quizzes in WeBWorK.  This new set type is called Just-In-Time and is a kind of adaptive homework; different students will experience the set differently depending on their performance.  However, the usual idea behind adaptive homework is that the homework system will analyze a student's performance and generate or select problems based off of how well they are doing.  Just-In-Time sets work a little differently. For Just-In-Time sets all students will have the same basic set of problems, but if they have trouble they will be given additional problems which will provide "just in time" review of the important concepts.  Below I'll present an example of what a Just-In-Time set might look like and will also talk about how these sets can be built in WeBWorK.  Note:  This feature is currently experimental.  Get in touch with me via email if you are interested in trying it out. 

        Lets suppose we are a student starting a Just-In-Time type homework set that is intended to bring us up to speed on some common integration techniques.  When a student starts the set they will see something like this.
        One of the options for Just-In-Time sets is to restrict problem progression so that students are forced to do their problems in the order presented.  Here Problem 1 is a u-substitution problem for which the student has 4 attempts.  If they are able to solve the problem within the given number of attempts then Problem 2 will open up and they will move on.  If they do not get the right answer after 4 attempts they will need to do some review. 

        In this case the student has not successfully solved the problem.  They are then notified about the fact that this problem has open sub-problems and that they may be able to salvage their grade by getting Problem 1.4 correct.  Of course, they will have work to do before they get to 1.4!  In particular if the student hits "Next" or goes to the main problem set page they will see that there are now four new problems, 1.1 through 1.4, which they did not previously have access to.  These problems will provide review and practice, and then allow the student to retry a problem similar to the original.

        This review is not automatically generated or anything like that, it must be created by the course instructor.  However it can be done using the standard PG techniques already in place.  For example Problem 1.1 is a video review of u-stubstitution delivered via YouTube.
        Notice we ask a simple question at the end to make sure the student watched the video since we are preventing them from proceeding until they get this problem right.  Once they watch the video, Problem 1.2 presents them with an easy substitution problem, which has unlimited attempts.  For this problem the ShowMeAnother feature has been enabled.  So if a student gets the problem wrong often enough they will have the option to see a different version of the same problem, but with the solution available for study.
        The next problem, number 1.3, presents a practice trig u-sub problem for the student to work on.  While a student does have unlimited attempts on this problem, after they get it wrong 3 times they will be given access to 1.3.1.  Problem 1.3.1 has some basic trig derivatives and is further review for those who need it.  However, even if they solve 1.3.1 and practice their trig, the student will still need to go back and do Problem 1.3.  If they are able to solve 1.3 without any trouble then Problem 1.3.1 will never appear.

        After finally solving 1.3 the student will then get 4 attempts to solve Problem 1.4 which is similar to, but not the same as, Problem 1.  If they get Problem 1.4 correct then their grade for Problem 1 will be recorded as 100% in the gradebook.  If they get it wrong, and if this feature is enabled, then the instructor will be sent an email letting them know that this student was unable to successfully complete Problem 1, even after going through the review sequence.

        Whew!  And this is all for Problem 1.  Problems 2 through 4 would contain similar review structures for other integration techniques which would be unlocked only when students are unable to finish the problem, or after a certain number of attempts.  Their final grade for the assignment will be their adjusted status on Problems 1 through 4.  That is, for each problem their score will be the larger of their score for that problem and their average score from the child problems which count for that problem.  The set score is created from that.

        Where do all of these problems come from, you might wonder.  As you might guess, setting up a Just-In-Time set is significantly more work than setting up a regular homework set.  Even an assignment with only 5 top level problems might be expected to have 20 or more child problems to serve as review.  What's more these problems have to work together well and many, like the video tutorial, are unlikely to be found in the OPL.  That being said, problems in Just-In-Time sets are written using PG, just like any problem, so anyone familiar with (or interested in) problem authoring can jump right in.   In general adding problems to Just-In-Time sets is done the same as it is for regular or gateway sets. You add problems using the library browser, or the problem editors, or the "add blank problems" field as you normally would. New problems are always added to the end of the set as a new top level problem, so you will most likely have to reorder them after you add them.  Furthermore importing, exporting and assigning homework sets can all be done using the Homework Sets Editor 2 page, just like any other assignment.

        One new tool which has been created for Just-In-Time sets is the Problem Set Detail 2 page.  This can be found by visiting a homework set and then clicking the link on the left hand navigation bar that looks like --Set Name.  You should end up at a page which looks like this.

        (Notice the link on the left.)  This portion of the page should look pretty familiar. You set parameters here as you normally would.  Just like any set remember that you have to Save your changes for them to take affect.  The main differences are how you order problems. The problem section of the page looks like this.

        Notice the nesting in the problem list. This, together with the problem numbers, should indicate the child-parent relationship between the problems. Because just-in-time type sets can have so many problems you can simplify things by clicking the - icon to hide the children of a particular problem and you can click the + icon to show the children of a particular problem. This is very similar to how viewing a directory structure in list mode works.  You can expand or collapse all of the child problems by using the buttons at the top.

        To reorder, you drag a problem by clicking on the arrow icon.  When you move a problem you will be shown a little preview box indicating where the problem will go.  You can change whether a problem is a child or not by dragging the problem slightly to the right or left.  (When you do so the preview box should become indented.)  If you delete a problem there may be gaps in your numbers. This is good if you delete something while a set is active (not recommended) but otherwise you can renumber the problems consecutively using the Renumber button.

        Another big difference between this and the original Problem Set Detail page is how problems are rendered.   In this editor you click the picture icon near the problem number to render a problem. If you click it again it will hide the problem. The method used to render the problem is determined by the drop down box near the top.  If you want to render all of the problems at the same time you can use the button at the top of the list. Be warned this will take up a lot of vertical space.  Put together, on the fly rendering and the expand collapse feature should make it so that you can focus on one review sequence at a time.  You can read more about how to set up Just-In-Time sets here.  Below I will present some of the parameters associated to Just-In-Time sets.  

        Set Options

        These are the set wide options specific to this type of homework. Many of the other set wide options, such as due date, open date and answer date, will be present. If conditional release and/or reduced credit is enabled then these configuration options will also be present.
        Restrict Problem Progression 
        This option controls whether students are allowed to proceed to the next parent problem before finishing the current problem, including the current problem’s children if necessary. Note: A student "finishes" a problem if they either get 100% or runs out of attempts.
        Email Notification 
        When this option is enabled all instructors set to receive feedback emails will be sent a notification when a student finishes a problem, e.g. runs out of attempts on a problem, and all child problems which count for that problem's score, and do not have an adjusted status of 100%. This is to let the instructor know that the review for that problem was unsuccessful.

        Problem Options

        These are options for individual problems. The usual problem variables, such as weight and max attempts, will be present as well.
        Att. to Open Children 
        This controls how many incorrect attempts are needed before the child problems for this problem become visible to a student. If this is set to 0 then child problems are always visible. If this is set to -1 or a number higher than max attempts then the child problems will become visible after a student runs out of attempts. If this is set to -1 and max attempts is set to unlimited then the child problems will never be open.
        Counts For Parent 
        This controls whether this problem's score counts towards the score of its parent. In general the adjusted status of a problem is the larger of its status and the weighted average of the status of its child problems with this flag enabled.
        While it may not be as fancy as an adaptive system which chooses new problems based on a computer algorithm, this just-in-time assignment type is extremely flexible.  Instructors should be able to make use of the tree structure, as well as the fact that they can control when and how students have access to the homework problems, to do some interesting things.  I know my students would appreciate a short tutorial on the log rules next time we do logarithmic differentiation!

        by goehle ( at June 06, 2014 08:10 PM

        May 30, 2014

        Geoff Goehle

        Automated Install

        Its been a while since my last post.  However the semester ended a couple of weeks ago and summer is here, which means that I can get more WeBWorK stuff done.  I'm going to put up a number of posts over the next several months.  A lot of what I will talk about was spurred by the recent code camp at Asheville, hosted by Western Carolina University.  You can read more about that here.

        In this post I will be discussing recent strides that were made with the automated WeBWorK install script, ww_install, recently developed by Jason Aubrey.  I'll talk about what improvements we made and walk you through the installation process.

        First off, ww_install is an automated installation script that is meant to get a vanilla WeBWorK server running on a fresh linux installation with as little fuss as possible.  It is ideal for those interested in trying WeBWorK out using a virtual machine, or doing development on multiple machines.  It is also good for initially setting up production machines.  Be aware, however, that most production machines will require manual configuration to set up things like SSL, or alternative authentication methods, or an email server, and so on.

        This script was actually developed a little while ago by Jason Aubrey can be found at this github repository.  The current master branch on that repository supports the current WeBWorK master branch.  Neither it, nor the WeBWorK master branch, support Apache version 2.4, however.  (The 2.4 version comes standard with the newest editions of Ubuntu and Fedora.)  One of the main improvements we made to the install script during the code camp was to upgrade the installer to handle WeBWorK branches release/2.9 and develop, both of which do offer Apache 2.4 support.  (Another big improvement is a surprise!  Scroll to the end to see what it is...)

        As of now the most recent branch of the installer, named ww3, supports the following distributions:
        • Debian 7 - Wheezy
        • Ubuntu 14.04 - Trusty Tahr
        • CentOS 6 - <No Cute Name>
        • Fedora 20 - Heisenburg
        This reflects the general policy of WeBWorK moving forward.  We will concentrate on supporting the stable Debian version, the current CentOS version (and the equivalent RedHat), Ubuntu Long Term Support Versions, and (recent-ish) Fedora versions.    While we will not promise that things should work on the most recent Ubuntu or Fedora versions all the time, we do want people to let us know when something goes wrong.  Problems with those distributions will become problems for the officially supported distributions sooner rather than later. 

        So lets get to the good stuff.  I'm going to walk you through the process of installing a WeBWorK server on a virtual machine.  Start by installing your favorite (supported) linux distribution...
        In this example we will be installing Debian onto a VirtualBox virtual machine.   While you can install the full version of Debian, for virtual machines I recommend working with a smaller set of features.
        Here I am only installing an SSH server and the standard system utilities.  I am not setting up a desktop environment.  You can run a desktop environment through the virtual machine but it slows things down.  I prefer to set up port forwarding so that I can use the host machine's terminal and browser, which are generally more responsive.  Next set up your initial user.  I recommend using wwadmin but you can use whatever user you like, even root.  Now I set up port forwarding.  
        Here I am forwarding port 2222 to 22 on the guest machine, and 8888 to 80 on the guest machine so that I can access ssh and apache.  (Using this setup you can even access your WeBWorK installation remotely from another machine via ssh tunnelling.)  Once the installation is finished (it should be pretty quick with this few packages) start your virtual machine (you can even do so in headless mode using VBoxHeadless) and log into it using a terminal on the host machine with
        ssh -p 2222 wwadmin@localhost
         Now you will want to get the installation script and run it.  Notice that we get the script from the ww3 branch, since that contains the latest changes.  (Eventually ww3 will be merged into master and you will just get that branch.)  
        wget --no-check-config \
        sudo bash
         Of course, there is a corresponding curl command which you can use to get the file.  Now you wait while the script installs all of the necessary prerequisites.
        • You will be asked to configure your mysqul server
        • It may look like the script has stalled while installing perl modules.  This may be because the script is asking you a question but you can't see the prompt.  When in doubt ... just press Enter.
        Eventually you will get through the installation process and the real fun begins.

        The installation script will ask you a bunch of questions.  Things like:  "Where do you want to install WeBWorK?", "What is the URL of this site?", "Do you want to create a WeBWorK administrator account, or does one already exist?"  You should always read the prompts, but its also usually safe to just press enter.  There will always be a confirmation prompt and the default settings are usually pretty good.  The only time you will absolutely have to provide input is when you are creating the database.  
        Here you will be asked for the mysql root user password, and to set the WeBWorK database password.  Note:  On some terminals the prompts can be cut off.  Again, just press Enter if you are in doubt.  After this is done, the script will ask where you want to fetch the WeBWorK code from (you should choose the defaults), and will get to work.  Some parts of this can take a long time....
        Once you are done with the installation, hopefully it finished without errors, you can find the installation log in your root WeBWorK directory.  Usually this is /opt/webwork/webwork_install.log.  This log is a pretty extensive list of everything that the installer script did.  Its useful to keep around so that you can troubleshoot a broken installation, or just modify it so that it suits your needs better.  If the install fails for any reason feel free to post your issue here or ask your question on the forums or the irc channel.  

        If the installation succeeded you should be able to access your server by visiting http://localhost:8888/webwork2 on your host machine's browser. 
        And now for the surprise (you didn't peek did you...).  By default this version of the installation script installs the new prototype interface WeBWorK3.  You can try it out by visiting the WeBWorK3 link on the left hand side of the page.  You should end up at something which looks like this: 
        While its still in the early stages, one of the main features of this new interface is that it is responsive.  No more form based "submit my changes" editor pages.  This interface records your changes as you make them, making it feel much more interactive.  Its not feature complete yet, but a lot of the big things are there.  You can read about the technical side of things here and if you run into problems you can visit the WeBWorK irc channel or email Peter Staab, the main developer.  If you want to back out of the (maybe slightly unstable) ww3 branch you can do so very easily using git.  Just use the command
        git checkout release/2.9
        in your main webwork2 directory, probably /opt/webwork/webwork2.  Note: release/2.9 will be merged into develop sometime over the summer, as will the ww3 branch of ww_install, so these instructions may get slightly out of date.  

        by goehle ( at May 30, 2014 08:57 AM

        May 26, 2014

        Michael Gage

        WeBWorK::Asheville May 19 - 22, 2014

        WeBWorK::Asheville   May 19 - 22, 2014

        WeBWorK::Asheville::2014 was held at Western Carolina University from Monday, May 19 through Thursday May 22, 2014.   Thanks very much to Geoff Goehle at WCU for organizing the code camp.  

        This was a small focused code camp attended by Mike Gage, Geoff Goehle, Peter Staab, Bruce Romano, Jason Aubrey  and John Travis and devoted principally to making progress on 
        • WeBWorK 3 -- the new AJAX based interface being developed by Peter Staab
        • the install script -- automatically installing the entire WeBWorK stack being developed by Jason Aubrey

        In addition we made a start on writing a grant proposal for dissemination of WeBWorK to community colleges and to high schools, worked on refining examples that use both MultiAnswers and Scaffolding (sequential or compound problems) in the same WeBWorK question, and created a first draft of a procedure to be followed when checking in new code to the repository.  

        This camp  and previous code camps are supported by the NSF through a national dissemination grant to the MAA. 

        Read more »

        by Michael Gage ( at May 26, 2014 09:56 PM

        May 22, 2014

        Christopher Hughes

        Show Me Another feature for WeBWorK


        Imagine that you're a student working on a WeBWorK problem, and that you're stuck. You certainly could e-mail your instructor, but part of you knows that if you could just see an example that is similar to it, you would stand a good chance of getting through the problem.

        The commercial software has such a feature to show students a new version of their problem - a lot of faculty enjoy this feature, and have often said that they would consider using WeBWorK if it had something similar.

        The idea to implement such a feature in WeBWorK was first formally discussed at the Vancouver BC coding camp in June 2013. Arnie Pizer provided some great ideas and input. Alex (Jordan) and I were keen to explore it, but hadn't dived into the WeBWorK engine before; he posted on the forums but then we got side tracked (from May-September 2013) with our developmental math library (and after that, our Program Review, November 2013-January 2014).

        In February of this year, I finally had the time to dive into the WeBWorK engine code, experiment, and develop this new feature for WeBWorK: Show Me Another.

        what does it do?

        When enabled, the button opens a new tab with a newly-seeded version of the current problem. A check is done to see if changing the seed does indeed change the problem - it would be less than ideal to show the student the solution to the current problem!

        Assuming that it is possible to find a new version, a banner with information is given to the students:

        Depending upon the settings that the instructor has chosen (see below), the student has access to a selection of tools with which to experiment on this new version of the problem. For example, if a walk-through solution exists to the current problem, and if the instructor has allowed the student to see said solution, then the banner will detail this.


        The feature has a variety of different settings including a global switch available in the Course Configuration; if disabled, the button will never be visible, and nothing will be different from the instructor or student perspective

        Assuming the global switch is enabled, then a number of choices are available for the instructor, both in the Course Configuration, and from the Homework Sets Editor

        course configuration

        Instructors can choose how many times the student may use showMeAnother per problem; if they set it to -1 then the student can use it infinitely many times for every problem for which it is enabled (see Homework Sets Editor below).

        Furthermore, instructors may choose what behaviour they would like to see on the showMeAnother screen; would they like the student to Check Answers (without affecting the number of attempts to the actual problem)? Show the solution for that problem (assuming it exists)? Show the Correct answer? Show hints (even if hints are disabled for that problem set)?

        Homework Sets Editor 

        Instructors must enable showMeAnother for each problem from the Homework Sets Editor page; they enter a number that illustrates the number of attempts that the student must make at the problem before showMeAnother becomes available; if they do not want the button to become available for a particular problem, they enter -1, which is the default for each problem.

        This can also be done on a per-student basis; think of this setting as a counterpart to Max attempts.

        Exporting/Importing homework sets

        When the button is enabled, the per-problem value of showMeAnother will be included in the export feature.
        These values will also be taken into account when importing from a .def file; if no value is found, then it will be assumed to be -1.

        If the button is not enabled, then exporting and importing is not affected.


        It was great fun diving into the code for the WeBWorK engine; I particularly enjoyed the review process, and implementing the various suggestions from the review team - they thought of features and tweaks that had not occurred to me which helped to make showMeAnother a much stronger tool. I am truly grateful to Geoff Goehle for his time and energy in testing this feature. Thank you to Alex Jordan for his contributions to the project.

        I hope that this feature appeals to students and faculty alike, and that it will encourage all users to continue to enjoy using WeBWorK.

          by Chris ( at May 22, 2014 01:24 PM

          January 18, 2014

          Peter Staab

          Slides from my Joint Mathematics Meetings WeBWorK talk

          On Saturday, January 18, I gave a talk at the Joint Mathematics Meetings on the current and future impact of WeBWorK.  Here is a link to the slides.

          by Peter Staab ( at January 18, 2014 06:01 AM

          January 08, 2014

          John Travis

          WeBWorK sessions at JMM 2014

          There are a number of sessions involving WeBWorK scheduled for the upcoming Joint Mathematics Meeting in Baltimore next week.

          The Exhibit Hall display will be available from Wednesday noon through Saturday noon. Be sure to drop by and chat and perhaps volunteer to staff the exhibit for an hour or so during the meeting. If you can make plans to staff the booth, please send a note to to have access to the planning document.

          Minicourse #9. WeBWorK: An open source alternative for generating and delivering online homework problems, will be on Wednesday and Friday and presented by John Travis, Jason Aubrey, and Paul Pearson. The first session will be on Wednesday, 2:15 p.m.–4:15 p.m. and the second on Friday, 1:00–3:00 p.m.. There are currently a relatively large number of registered participants but there may be a few slots left in this mini-course designed to introduce WeBWorK and basic problem authoring. Any experienced WeBWorK folks are welcome to drop by and chip in to help these new users.

          A NSF poster session on Wednesday includes three poster displays related to various WeBWorK projects:

          Several presentations related to WeBWorK are scheduled for various paper sessions:

          There will also be several possible informal gatherings during the meeting.  The first will be at 5:30pm on Wednesday. Meet at the exhibit hall booth or entrance. Also, stop by the exhibit booth and check on dates and times for any other gatherings as well to grab a new WeBWorK business card!

          by drjt ( at January 08, 2014 10:01 AM

          November 09, 2013

          Alasdair McAndrew

          A first look a WeBWorK

          In my last post (nearly three months ago) I commented on online assessment, and in particular on Pearson’s MyMathLab Global, which we have been using for our engineering mathematics. This is quite a good, full-featured and robust package; its only fault (if indeed a fault it be) is that it’s commercial. This works fine if you are structuring your course around a set text, but over the years I have moved further and further away from such texts. I have not found a text which could be used comfortably with my extraordinarily heterogeneous student cohort, who enter our courses with all different levels of mathematics competency, and all different levels of written and spoken English.

          A good text must be carefully, simply, and clearly written, with plenty of useful examples, and above all well scaffolded exercises which move gently from the very simple to the complicated. Most (all?) textbooks in my experience suffer greatly from this. Our current preferred textbook: “Modern Engineering Mathematics” by Glyn James, has quite appalling exercises. Also, it’s bit of a heavy monster of a book, like most mathematics textbooks, and I don’t see why students should have to lug it around with them. But anyway, if you do buy the book, either in hardcopy, or in electronic form, you get a code which gives you one years access to MyMathLab and all ts trimmings.

          So this last semester I have used my own notes and exercises for engineering mathematics, and sent students off to MyMathLab for their weekly online tests. There were problems:

          1. Students who first started over a year ago (who, for example, might have had to take a break between their mathematics subjects) found that their access codes had expired. I spent the first few weeks of the semester writing almost daily emails to the publishers begging for some more access codes. To their credit, the local Pearson reps were in every way helpful, accessible, friendly, and understanding.
          2. Right towards the end of the semester I kept finding students who had fallen through the cracks, so to speak, and hadn’t been able to access their tests at all. (And had got busy with other subjects and kept forgetting about them)
          3. Some answers were marked wrong, when they weren’t. I had one student who sent me a screen shot to show that he had entered “12″ as the numerical answer to a problem, which was marked wrong: the correct answer, apparently, was “12/1″. There was also confusion with decimal places: a question would ask for say 4 correct decimal places, and no matter how carefully the students entered their answers they were marked wrong.

          I have wanted to experiment with WeBWorK for a while now, but I couldn’t find a way to install it locally – until I realized that I could install it (and a web server) on my desktop at work, which runs Ubuntu 12.04. I did this (it took most of a day, with only a modicum of swearing, and messages to the WeBWorK forums), but now I have a working system, even if only running on a desktop, rather than on a dedicated server.

          WeBWorK has many excellent features:

          1. It’s free: open source, no less. You can pay for MAA to host your course if you don’t want to set up your own local service, but even that option is very inexpensive.
          2. It has a huge problem library: the Open Problem Library has something around 25,000 problems from most areas of undergraduate mathematics. You might not find problems on, say, cohomology or modular forms, but these advanced topics would hardly be well served by an online homework system. We are assuming here we have a large class (in the many 100′s) of students in their first or second year of university study. And for this sort of basic mathematics, WeBWorK is terrific.
          3. The authoring system for creating new problems is very straightforward: just a matter of some elementary editing. I tried authoring a problem in MyMathLab once, and although it’s quite possible, life’s too short.
          4. WeBWorK seems to play very nicely with LaTeX and Sage, so that the mathematics is properly typeset, and you can have all the power of Sage available to you if you need it.

          That’s as far as I’ve got so far. You can get a feel of WeBWorK without installing it by checking out some of MAA’s “Model Courses”, of which a list is here. There doesn’t seem to be any online hosted method for browsing the problem libraries to see what’s in them, as far as I know. I think this could be something worth making available so that teaching staff can decide whether they think WeBWorK would suit them and their courses.

          In my current state of thinking, if I can get a better hosted system at my work (that is, not my desktop), I may well use WeBWorK for my next teaching semester.

          by amca01 at November 09, 2013 10:52 AM

          November 01, 2013

          Michael Gage

          WeBWorK accessibility projects


          There are a number of centers using WeBWorK that are interested insuring that our online homework system is accessible, that it meets universal design goals and can be used as widely as possible.  Using MathJax to represent mathematical equations has been a big boost in that direction.

          Here is a post from Portland Community College one of the active centers working on the accessibility aspect of WeBWorK.  The summer MathFest conference will be held in Portland, OR in August 7-10, 2013 and there are tentative plans to hold a code camp in Portland devoted to WeBWorK accessibility on the three days proceeding MathFest. Details are still being worked out but if you are interested or have ideas or suggestions email me or Alex Jordan at Portland Community College.

          Below is the report.

          -- Mike

          Report from Portland Community College
          Kaela Parks: Director of Disability Services
          Karen Sorensen: Accessibility Advocate for online courses

          Chris Hughes, Scot Leavitt, Alex Jordan: Math faculty

          Making Math More Accessible at Portland Community College

          At Portland Community College (PCC), Disability Services (DS) is tasked with ensuring the accommodation process unfolds appropriately across and throughout a district serving approximately 90,000 students per year, 50,000 of whom are seeking credit.  In recent years the options for curricular content format and delivery have changed considerably, bringing new barriers, but also new opportunities for making math more accessible. Many courses are now designed around online engagement points that tap vast databases, generating individualized browsing sessions any time of day or night, giving users valuable and almost instantaneous feedback. While Disability Services can convert known problem sets ahead of time, and hire aides to serve as readers and scribes, it is not practical, nor does it provide equally effective communication, to try and address barriers on the fly.

          The truth is that while there will always be some need for individualized accommodation, for example creating tactile representations of graphs, there is much that can and should be done on the front end to minimize the need for manual adjustments. If online engagement points comply with established Web Content Accessibility Guidelines and use proper structural markup for math content, users who rely on text to speech, braille translation, magnification, or voice recognition, can still typically get what they need. The content is built for flexibility. However, when these best practices are not honored, there is often no way DS professionals can ensure equally effective communication. We can’t reach behind the firewall and “fix” content by adding descriptions to images, putting equations into MathML, or redesigning the interface to ensure keyboard navigation.  What DS can do, and should do, is partner with Faculty, Instructional Support, and other stakeholders to help the institution recognize our shared responsibility to ensure equal access through ethical course design and curricular material adoption processes.

          At PCC, online instructors develop their own courses within the learning management system. They choose the color and formatting of their text, the media, publisher materials and third party web sites and applications to use in their courses. And since the spring of 2012, all new and revised online courses paid for development by the Distance Education (DE) department are reviewed for accessibility. But how is an instructor supposed to know what’s accessible and what isn’t?

          The Distance Education department has seen accessibility as an area that instructors need support. To that end, they hired an Accessibility Advocate who trains instructors and reviews courses for accessibility. And last fall (2012) the DE department co-sponsored with the Math department, two math faculty in their study of accessible mathematics. This subject specific study was so successful that the DE department hopes to emulate it with other academic program areas, especially in the STEM fields.

          Math faculty members, Scot Leavitt and Chris Hughes investigated both the accessibility of content generated by the instructor and that which is delivered by homework management systems. In addition to studying commercial homework management systems such as MyMathLab, they ran a battery of accessibility tests (assisted by Math faculty Alex Jordan) on WeBWorK. The results from the WeBWorK experiments were superb- the screen reader JAWS was able to navigate easily around the web page and, most importantly, could read even the most complicated mathematical expression with the greatest of clarity.

          WeBWorK is currently the only math homework management system fully endorsed by the Disability Services Office at PCC, and they are providing strong support in the creation of a dedicated server to host it. The server should be fully functional by the end of Summer 2013, and ready for wide-spread use across the college at some point within a year. Supporting WeBWorK in this way allows PCC to provide instructors with an alternative to commercial offerings that have known accessibility issues. By establishing our own WeBWorK server we ensure our community has access to a powerful homework management system that is more usable to more people more of the time. It also provides the institution with a means to ensure access for students who are enrolled in sections built around inaccessible engagement points by providing an equally effective alternative. 

          Details of the Math accessibility study and other resources can be found at

          by Michael Gage ( at November 01, 2013 07:41 AM

          October 30, 2013

          Geoff Goehle

          Past Answers

          Its been a while since I've written something here, so I thought I would make a short post about a WeBWorK page which has always surprised me because of how much it is used.  I am talking about the Answer Log page.  After the jump I'll show off some improvements we have made to the Answer Log which will be arriving in the upcoming 2.8 release.

          Instructors have always been able to get to the answer log page by following the "Show Past Answers" link at the bottom of any problem page.  Something which is less well known is that instructors can make it so that students can also see past answers.  This is done by setting "Allowed to view past answers" to "student" on the Permissions section of the Course Configuration page.    When instructors view the past answers page it will now look something like this:
          The form at the top is to allow instructors to view answers for any student and any homework set.  The problems are colored according to correctness and now they are also formatted using the AsciiMath functionality of MathJax.  This should improve the readability of the past answers page.  It's important to note that when students look at the page they will not have the ability to view other students answers, and the answers will not be colored by correctness.  Another couple of readability features (in math4 at least) are the striped table rows (since the past answer rows can get very wide) and the extra bold bars which separate answers whose difference in submission times is greater than the WeBWorK timeout time.  The page also looks reasonably good when mixed with essay answers:
          The one caveat is that if students are "creative" in how they typeset their mathematics in essay answers, MathJax may not render things correctly.

          The final new feature, which will only be helpful to instructors, is that the answer log form now supports a variety of regular expressions:
          For the User and Set field you may use the following:

          • * - this wildcard can stand for any number of characters.
          • ? - this wildcard can stand for one character. 
          • | - this is an or statement.  Its mostly useful for comparing the answers of two students. 
          For the problem field you can use comma separated lists of numbers or number ranges.  Be warned, doing a very broad search (like User: * and Set: * and Problem: 1-1000) can return a great many past answer entries.

          by goehle ( at October 30, 2013 01:05 PM

          October 12, 2013

          Anneke Bart

          MAA PREP workshop: Problem Authoring (WeBWorK)

          Last summer I did a PREP workshop on authoring problems in WeBWorK. The presentations by John Travis, Davide Cervone, Gavin Larose, Paul Pearson, and others were wonderful. This MAA sponsored workshop was conducted online and ran for several weeks on Mondays.

          If you had told me even two years ago that I would be attempting to write problems I would have shaken my head sadly, and muttered something along the lines of “the poor dear is delusional”. But here I was learning about writing essay questions, multiple choice and multiple select questions and how to randomize the variables.

          I was really glad to hear that one of my colleagues had decided to jump into authoring problems at the same time. It was really nice to have someone to compare notes with and ask questions of.

          The initial “intimidation factor” may be there for many out there, but take it from someone who has taken the jump: It is not as hard as it looks. My advice would be:
          • If you can find a colleague to work with, then that is a big plus. Being able to trouble shoot with someone close by is really useful.
          • If you do not have someone close by, you can still get a lot of help from the online forum. WeBWork Forum Link
          • Start with existing problems and modify them. I would look for problems similar to what I wanted to do and then just adapt them to my needs. This includes finding a problem and adding random variables, modifying multiple select questions, etc. I have even passed on templates to colleagues who are now coding their own reading questions.
          • Writing you own problems from scratch is hard (well, for me it is). If you cannot find a template in the problem library, then go to the WeBWorK wiki and browse through the author pages there. There are many templates available and I have found it relatively easy to adapt them. Link to WeBWork Author Resources
          • Do take your time. It can take a while to write a good problem (even when adapting an already existing one). And document your work. Include what course this is for, what text it is from (if applicable), and write the solution part. It is sometimes tempting to skip the latter when in a hurry, but then that may never get done and the comments may be very useful to students!

          I have been writing questions myself for reading quizzes I give my students, and to create questions for placement tests. But that is something I will talk about in later posts.

          - Anneke

          by Anneke Bart ( at October 12, 2013 08:10 AM

          October 09, 2013

          Michael Gage

          WeBWorK::Rochester::2013 October 4 - 7, 2013

          WeBWorK::Rochester::2013, held at the University of Rochester from Friday October 4 through Monday October 7, was dedicated to
          • putting final touches on release/2.8 of WeBWorK 
          • integrating the new user interface created by Peter Staab into the development branch of WeBWorK
          A small group of experienced WeBWorK developers participated in this code camp: Davide Cervone, David Gage, Mike Gage, Geoff Goehle, John Jones and Peter Staab. My thanks to all of them for their contributions to a very successful weekend.  My thanks also to Louise Wingrove for organizing the lodging and meals for participants.

          This camp  and previous code camps are supported by the NSF through a national dissemination grant to the MAA. ( link to  once it is back up and running again. :-)  )

          The first outcome of the camp is an updated release/2.8 which we plan to merge with the master branch on December 1, 2013.  We combined the original release/2.8 with most of the fixes and small features which have been submitted to the develop branch over the last three months.  Both release/2.8 and the develop branch have been running smoothly under moderate course loads on the MAA testcourse site and on the hosted2 site at the University of Rochester.  The activity devoted to release/2.8 over the next few months will be responding to bug fix requests, minor adjustments of features and general polishing of the instructor experience.  Very little has changed in the student interface and there have been very few requests for changes in this aspect of WeBWorK.  While not specifically adapted to mobile devices the student view of WeBWorK works acceptably well on iPhones, iPads and Android mobile devices.

          Features of release/2.8 are listed on the wiki at:  
          (You can type release/ 2.8 into the search box of the wiki to find it.)

          You can also view all of the work involved in creating release/2.8, step by step,
          viewing the commits page on github.
          The most recent commits are at the bottom.

          The will be more exposition about new features in release/2.8 (and some under advertised features of release/2.7) in subsequent blog posts.

          It should be noted that LibraryBrowser1, although it has not changed its name, has received substantial improvements in release/2.8 from the work of John Jones.  In general it should be much faster because some of the ajax calls used in librarybrowsers2&3 have been used to speed up rendering of individual problems on a library page.  When enabled, the library page also allows for the easy tagging of library problems.  (see for more details)

          The second outcome of WeBWorK::Rochester::2013 is the integration of a new AJAX/javaScript
          instructor interface, largely created by Peter Staab at Fitchburg State University, which has been merged into the develop branch of WeBWorK.  This interface provides instructors with behavior that feels more like a "google app" instead of the form based interface that we have been  used to since the mid 2000's.  Peter began work on this project during WeBWorK::Rochester::2012 held a year ago June.

          One of the early outcomes was "ClasslistEditor3" which has been available as an option in both release/2.7 and release/2.8.  The current version includes the ClasslistManager (renamed and improved from ClasslistEditor3) and HomeworkManager which combines the duties of the Library Browsers, the HomeworkSetsEditors(1&2) and the Instructor tools page. The HomeworkManager's library browsing functions are built on the experience gained from the prototype LibraryBrowser2 and LibraryBrowser3 which were written by David Gage.  All of these tools have been available for testing in their embryo form on previous releases, but they have now progressed to the point where they can usefully speed up many standard instructor tasks.

          WeBWorK::Rochester::2013 allowed Peter to explain in person his work and his vision to several of the other core WeBWorK developers. (Peter has not been able to attend any of the code camps since last June.)  We now have a clearer idea of what has to be done to finish the transition.  We were able to make significant strides in improving reliability during the code camp itself but much more remains to be done.

          The net effect of using ClasslistManager and HomeworkManager is that instructors can manipulate classlists -- add students, change passwords, or homework assignments -- create, assign, etc. immediately.  The updates of these changes to the back end server are done asynchronously and are invisible to the user.

          At the moment the develop branch is fairly wild.  Some actions don't behave as you expect or as they should;  there are many features of the older editors and browsers that have not yet been implemented in the new interface. In some cases things that work fine on small sets or classes slow down drastically when the scale is increased. We expect that it will take many months before this develop branch is ready for use on a regular basis.

          On the upside -- the student interface is not affected, and so far at least there is no affect on stored data.  Since the old editors are still available one can simply switch to them for features that are not yet implemented and then switch back to the new "managers" for their added convenience on tasks where they work well.

          One other thing we need to watch out for.  The new javaScript interface will initially be a step backward as far as localization (translation in other languages) is concerned.  We will be enlisting all the help we can for the translating effort but there is a lot of new and quite different code that has to be internationalized.  Similarly we are concerned about whether accessibility is helped or hurt by the switch to interactive apps.  We are using standard libraries, all of which are concerned with accessibility issues, but it will take time and we look for a lot of constructive feedback to preserve the gains in accessibility that we achieved between release/2.5 and release/2.8 and insure that these gains are not lost in this next release (most likely to be called WeBWorK3 ).

          For those helping with development:

          1. submit bug fixes and small feature tweaks to the release/2.8 branch
          2. submit new features to the develop branch

          In all cases make sure that your are in sync with the branch you will submit to before you
          send a pull request.  If the commit does not merge cleanly it will be returned for more work before it is even reviewed.

          -- Mike

          by Michael Gage ( at October 09, 2013 06:44 PM

          Peter Staab

          Sending Errors From Dancer

          This is my first blog post on the work I've done on WeBWorK.  This one's a little more techie, but you gotta start somewhere.

          Over the past few weeks I rewrote the backend of WeBWorK using the Dancer framework. I never completely understood the old webservice for WebWorK, but Dancer replicates much of it in a more flexible manner.  Instead of going through all the details, here's how I recently decided to have Dancer send errors in a more flexible manner.

          As an example, suppose you need to delete a particular problem set. The code for this is:

          del '/courses/:course_id/sets/:set_id' => sub {

          if (!vars->{db}->existsGlobalSet(param('set_id'))){
          send_error("The set " . param('set_id'). " doesn't exist for course " . param("course_id"),404);

          my $setToDelete = vars->{db}->getGlobalSet(param('set_id'));

          return convertObjectToHash($setToDelete);
          } else {
          send_error("There was an error while trying to delete set " . param('set_id'),424);


          The first line is the route which says to make a delete request and the parameters are the course_id and the set_id. Before, I would send back a javascript object with an error field and then try to detect it.  Now, I'm sending a http error with certain code.  The first (a 404) is a resource not found and the second (424) is a custom one.

          Now a client that connects to the webservice can know that there is an error and the type error.  It's the job of the client to interpret the error and react.

          The other nice thing about this is authentication errors.  I want (nearly) every route to be checked for authentication and permission. I've discovered that in Dancer, I can build the route:

          any ['get','put','post','del'] => '/**' => sub {



          which will be called for all routes (although I admit, I'm sure why this is called first) which I will check authentication and routePermissions. If the user isn't authenticated, then a 401 error will be sent.  The client can then listen for any error and if it detects a 401, then can ask for reauthentication.

          I tried this earlier with the before hook which is called before anything in Dancer, but evidently the send_error function doesn't work the before hook.

          The last line in the route above then passes to the next matching route.  This is away that you can chain routes together.

          You can test this new code out by cloning the repository at and make sure you then checkout the feature/new-ui-tools branch.

          by Peter Staab ( at October 09, 2013 07:20 AM

          August 18, 2013

          Alasdair McAndrew

          Online assessment in mathematics

          This semester I've taken over a large first year subject, for which the previous subject convenor had organized to use MyMathLab Global for weekly testing.  The subject is based around Glyn James' "Modern Engineering Mathematics", a book which is OK for content, and pretty awful for exercises.  This means that as users of the text, we have access to the publisher's (Pearson) online testing service.  For educators the idea is terrific: every week I simply pull from the exercise bank a set of 10 exercises corresponding to our current topic, and make them available for a week during which time students can have unlimited goes at them.  So in theory it's an easy way for students to get some easy marks, and reduce the burden of marking weekly tests by the lecturer (me) and the subject tutors.

          The subject I am teaching "Engineering Mathematics 2" is a follow on subject from – wait, you;ve guessed it! – "Engineering Mathematics 1".

          What could be better?  Well, aside from the extraordinary ease of testing, I have begun to have doubts about the efficacy of the system.

          1. It's a commercial system, which means that students have to buy a "personal access code" to use it.  A code comes with the book if they purchase it.  However, a code lasts for only 12 months, and if students have bought the book (and its code) for Eng Maths 1 (as they all should), and if there's been a break in their studies for any reason, then their code may have expired by the time they come to Eng Maths 2.  Then there are all the students who have got subject exemptions from Eng Maths 1 and never had a code in the first place.  The local publishers reps have been terrific and have provided me with lots of extra codes to hand out, but the onus is on me to get these dratted codes in the first place, and ensure the students get them.
          2. The system only tests the final answer.  A typical question is:

          Find the partial derivatives

          displaystyle{frac{partial f}{partial x}}, displaystyle{frac{partial f}{partial y}} and displaystyle{frac{partial f}{partial y}} of f(x,y,z)=sin^{-1}(xyz).

          and the students get three entry boxes for their results.  MyMathLab Global is very picky about the form of a solution, and if the form of a student solution differs from the "correct one" they are marked wrong.  For example,  we find that

          displaystyle{frac{partial f}{partial x}=frac{4yz}{sqrt{1-16x^2y^2z^2}}}.

          if a student leaves out the "1" of the "16" in error, the answers is just marked wrong.  If the student decides to write


          it is marked wrong.  MyMathLab Global doesn't seem to include a CAS to check if two answers are equivalent (aside from some very simple arithmetic and operator commutativity).  This has been a source of annoyance to my students, who may enter an answer which is mathematically correct, and yet marked wrong.  A very spirited attack on MyMathLab Global can be seen here.

        • The system checks only the final answer. We might well ask: what mathematical learning is actually being tested here? Surely we want our students to show that they have mastered a mathematical technique, process, or algorithm, and we would mark them on their working, not just on the final answer. At least, that is how I mark by hand. A question for which the student's working is fundamentally correct but the final result is wrong will still get most marks from me. I regard the final result as just one part of a question. So suppose the student sets out to answer a question, and makes a small slip somewhere along the way which effects every succeeding step, and the final result. I would give almost compete marks – there's only been one very small slip, and aside from that the logic and the working are exemplary. But MyMathLab Global can't do this. An answer is either correct, or wrong.
        • I have been hoping to check out MAA WeBWorK as a comparison, but the only server I have access to runs a linux distribution (RHEL 5.9) which can't run it. I think WeBWorK, from my understanding, is rather more nuanced in terms of mathematical equivalence of expressions, so probably overcomes item 2 above, and being open source completely overcomes item 1. I don't think it overcomes item 3, though.

          I will continue to use MyMathLab Global – its convenience is simply too great – in spite of my misgivings. But I think that online assessment in mathematics is still a very long way from testing the students' ability to do mathematics, as opposed to their ability to obtain an answer. This latter is not mathematics: it is applied arithmetic; and noble and useful as it may be, it is only a tiny part of what we, as mathematics educators, should be testing.

          by amca01 at August 18, 2013 11:14 AM

          August 05, 2013

          Geoff Goehle


          Its been a while since my last post, so I wanted to fill the space by writing something about a side project I've been working on.  Its an upgrade to the existing mathview equation editor.  The main features are bootstrap integration, real time equation rendering, and autocompletion of common functions.  Hit the jump to see more.

          The main change from the existing mathview program is the integration with bootstrap and with mathjax.  The mathview feature will only work with bootstrap enabled themes (math3 and math4) and mathjax has to be installed on the server.  Under these conditions you will be presented with a Use Mathview? option in the Display Options sidebar.  When enabled, inputs for problems will have an button attached to them as shown below.
          If you click on the icon the Equation Editor (i.e. mathview) will pop up.  At this point you have several options.

          • You can type your answer directly into the answer box.  The result will be rendered in real time in the Equation Editor preview area. (See image below)..  
          • You can click one of the functions and have it inserted into the answer box.  You can also browse the available function categories using the Operations drop down.  
          • You can drag the equation editor out of the way to a more suitable location.

          Put together, all of this basically replicates the older mathview functionality.  There are a couple of new features, however.  The most obvious is the autocompletion of functions.  Now when you are typing an answer and you write, say, "co", you will be presented with a list of standard functions which start with "co", as shown below.
          You can then select which function you want with the arrow keys and hit enter to complete the function in the input bar.  This is nice because it helps remind people what functions are available and also keeps the browser from populating the list with previous answers like it usually does.

          The second new feature is behind the scenes.  Since there are a lot of different standards for mathematical notation ( e.g. arcsin vs sin^(-1) ), the new mathview is locale ready.  By setting the option ${pg}{options}{mathViewLocale} you can use a different locale js file.  I won't describe the contents of this file here, but it allows the administrator to change which functions are displayed, what the categories are, what the names are, what functions can be autocompleted, what the notation is, and so on.  This can be set on a per class basis using the course config files, although the system administrator will have to make sure the appropriate locale js files actually exist.

          by goehle ( at August 05, 2013 02:26 PM

          July 17, 2013

          Michael Gage

          WeBWorK::Vancouver - June 27 - 30, 2013

          Hi again,

          Close on the heels of our participation Sage Edu Days comes the code camp at Vancouver.

          We had a significant number of first time participants at this code camp:  Alex Jordan and Chris Huges, Portland Community College; Liz Brauer, ECE department at Northern Arizona University, Nandor Sieben, Math Dept at Northern Arizona University; Nora Franzova, Langara College, Vancouver; Aori Nevo, Stevens Institute of Technology, NJ.  One of the purposes of these code camp is to widen the pool of developers who feel comfortable making contributions to the WeBWorK code base.

          Additional guests dropped in for part of the code camp: Davor Cubranic, Statistics dept at UBC; John Hsu, Center for Teaching, Learning and Technology at UBC; and Steven Wittens, developer of MathBox.

          The remaining participants were Anneke Bart, St. Louis University; Robert Beezer, U. of Puget Sound; Mike Gage, U. of Rochester; David Gage, U. of Waterloo; Djun Kim, Myplanet Digital; Arnie Pizer, U. of Rochester and John Travis, Mississippi College.

          Here are a few highlights from the code camp:

          • Yoav Freund and Matt Elkerj from UCSD gave a presentation via Skype on using webwork and machine learning to discover patterns in the past_answer log that indicate that a student is floundering and to step in before this occurs with additional scaffolding. They have a video on you tube at which presents this very well. There is clearly a lot more that can be done but this is a very intriguing start.
          • Greg Kraus from NCSU gave a presentation on the accessibility of WeBWorK including both strengths (of which there are many -- especially when using MathJax) and weakness that can be improved. He has an initial document at 
            • listing specific items that can be fixed in the presentation layer.  
              • If you would like access to this document (and in particular if you might have time to work on some of the fixes) please write Greg (greg_kraus) at
            • Aori Nevo has already started preliminary work on this and contributed a few patches to github.
            • Alex Jordan and Chris Hughes from Portland Community College report that their IT/accessibility team is delighted with WeBWorK particularly compared to MyMathLab. I've asked them to write a blog post on their findings when they get a chance.
          • Steven Wittens gave a marvelous demonstration of MathBox, a mathematics presentation tool built on top of three.js which allows you to "show" not explain. Examples of a similar presentations are at:
          • David Gage created a package for Vagrant which allows you to set up a development site for WeBWorK on your own computer in about 5 mouse clicks and in 5 minutes. Watch David's blogpost for more details. 
          • David Gage and Aori Nevo worked on a prototype white board that allows many students to draw and the drawings are echoed on the instructors board. It's built with web components which simplifies the code. There is a lot of potential here. 
          • John Travis and Jason Aubrey presented two different ways in which Sage and WeBWorK can interact.
            • JT has a number of examples of using sage as an applet (sagelets?). The technology is available in WeBWorK-2.7 (the current master branch) 
            • Jason has added a macro that will take a program written for Sage (essentially in python) and ship it off to the sage engine to be processed and to return an answer. You might use this to symbolically integrate a function which WeBWorK would not be able to do. (This is available in release/2.8 and develop branches). 
            • Both of these approaches integrate WeBWorK and the sage cell server being developed by Jason Grout and others. The technology works very well once all the connections are set up. It could still use some work with graceful failure and error reporting when one or the other of the webservices is not connecting properly.
          • Alex and Chris also worked on the surprisingly difficult problem of correctly checking factored polynomials in the context of algebra and pre-calculus. It's difficult because one does not want to accept mathematically equivalent answers (e.g. unfactored polynomials) but the exact definition of simplified form is not well defined. (is 2x+1 correct? or 2(x+1/2). How about 2x+2 and 2(x+1)?). They are using Sage as one approach to the answer. It also gave rise to some serious but amusing questions about factoring polynomials over finite fields as possible approaches to the solution. I'm sure the algebra/pre-calculus students will be thrilled.
          • Mike and Nandor added code that makes much more portable. (Chromatic requires a compiled C program in order to check coloring on graphs.) This should make Nandor's graph theory questions (in the NAU sublibrary) much more widely available.
          • Nora was able to convert the entire placement test for Langara College into WeBWorK format.
          • Liz was adding to her collection of Electrical engineering problems (in the NAU subsection of the OPL). Thanks to Davide Cervone we are dealing successfully with the use of j as the representative of the square root of minus one. :-)
          • There were additional tweaks to the Library Browser, the OPL and also to MathView -- the very recent HTML5 based equation editor which is

          Thanks to all who participated in another very successful code camp.  And our particular thanks
          to Nora Franzova at Langara College and to Djun Kim from UBC/Myplanet Digital  for making all of the local arrangements.

          Those who will be at MathFest in Hartford, Connecticut this August should drop by the WeBWorK
          exhibit (part of the MAA exhibit) and say hi to John Travis and any other WeBWorKers who show up.

          -- Mike

          by Michael Gage ( at July 17, 2013 10:51 AM

          July 10, 2013

          John Travis

          The two summer PREP workshops associated with WeBWorK have gone well.  The first one dealing with problem authoring is now over but was bursting at the seams and capped out with 42 participants.   The organizers are considering offering the workshop again next summer for folks who missed out this time.

          A second workshop geared toward the creation of model courses has held two online meetings and participants are actively working offline in subject area groups.  This workshop will conclude this week with a gathering starting on July 10 at the MAA Carriage House in DC.

          Those interested in writing better and more interesting questions may take interest in the course page utilized by the participants during the problem authoring workshop and located at on the test server at MAA.  You may log into this course using the guest login and view much of the material left behind.  In particular:

          • the MathObjects presentation slides given by Davide Cervone are located in the set Workshop2.  
          • many Problem Techniques described by Paul Pearson and Gavin LaRose are located in the sets Workshop3_AM and Workshop4_AM.
          • new ways to integrate with Sage are presented by Jason Aubrey and myself in Sage_Applets.
          With respect to model courses, groups are actively working on creating and/or refining model courses in High School Math, Beginning Algebra, College Algebra, Calculus and Differential Equations.  The plan is to have functionally complete collections of homework sets for interested users to test out this coming fall and spring. 

          Further, development is also underway to make implementing these models relatively painless.  Results will be announced in this forum as soon as we know them.  We will be looking for test sites so let any of the organizers (Jason Aubrey, Anneke Bart, Djun Kim or myself) know if you are interested. 

          Stay tuned...

          by drjt ( at July 10, 2013 06:24 AM

          July 05, 2013

          Michael Gage

          Sage Edu Days 5, June 19 - 21, 2013

          Thanks very much to the folks at Sage and at UTMOST for inviting us to participate in the Sage Edu Days 5 in Seattle, WA  June 19 - 21.  Jason Aubrey, Mike Gage and John Travis from the WeBWorK team participated.  This is the third year that we have collaborated with Sage to augment the interaction between the two software applications.

          As a result we have a new macro file "" which simplifies the creation of sagelets -- plugin interacts that augment the power of WeBWorK questions.  In addition Jason Aubrey created AskSage(), a new subroutine which allows WeBWorK to query an instance of sage to have an expression evaluated.  These provide two different mechanisms for  providing high level CAS support to WeBWorK questions by remotely accessing a sage cell server.  The sage cell server has been under development by Jason Grout and the rest of the Sage team over the last three to four years.

          The new sage interaction features are available in WeBWorK  release/2.8 which is now "on deck" in the webwork github site: Release/2.8 is undergoing final testing and bug squashing before being merged into the "master" branch.   To try it out you will want to use  the branch release/2.8 of both webwork2 and pg.  These are available by simply executing "git fetch" on an up-to-date installation of WeBWorK.  (See Github   and  release/2.7 for details.) While you are at it you should download the new changes to webwork-open-problem-library and run OPL-update to update the library database so as to incorporate the improvements made at the WeBWorK OPL workshop in Charlottesville, VA earlier this month. Preliminary release notes are available at

          More refinements and many more examples demonstrating the new sage interaction features are expected from the upcoming work at WeBWorK::Vancouver June 27-30  and the PREP13 model course workshop in Washington, DC, July 10-13.

          Release/2.7 was incorporated into the "master" branch on June 15, 2013.  The separate 2.7 branch on the openwebwork site will soon be removed.

          -- Mike

          by Michael Gage ( at July 05, 2013 09:43 AM

          July 01, 2013

          John Jones

          WeBWorK OPL Workshop - Charlottesville, VA - June 2013

          This was the first in a series of workshops to improve the organization of the Open Problem Library for WeBWorK.  The standard way to search the OPL is by a taxonomy of mathematics with three levels: Subject, Chapter, and Section.  In the past, problem writers were freely allowed to expand this, and as a result, there are many overlapping parts to the taxonomy.

          For the workshop, we selected 3 "Chapters" which included 5000 OPL problems (from single variable, differential calculus).  The participants worked in groups to revise the taxonomy for those chapters, then reclassified the relevant problems into the new taxonomy.

          We have also introduced the notion of "level" to WeBWorK problems, and described 6 levels.  Participants classified problems by level as well.  In the development version of WeBWorK, one can now use level as a search criterion (on the Advanced Search part of the Library Browser), and there is an information link located there to describe the different levels.

          Next, we went through problems looking for ones which were essentially duplicates, and ones which were very similar.  The criteria is

          • given two similar problems, if one cannot make a reasonable pedagogical argument as to why people would want each of the problems (given the existence of the other), then one problem would be flagged for removal.  
          • if we have a group of two or more problems where different people might want to assign different problems from the group, but they are all very similar, then they were marked as a "more like this group".

          Mathematicians are often have very strong views on even minor differences in problems.  The OPL strives to be useful to everyone, but it cannot contain everyone's variation on a specific problem.  There will always be a place for people to make minor modifications to OPL problems to suit their courses.  So, in deciding if two problems are duplicates, we had to use some judgement: if only one in a hundred math teachers would need a particular variation of a problem, we marked one as a duplicate.  If we thought one in ten might want the variation, we kept it (and marked the problems as a more like this group).

          When problem A is marked as a duplicate of problem B, problem A is replaced with a special "pointer file".  If someone uses problem A in a homework set, the student gets problem B.  Also, problem A will no longer show up in the library browser.

          More Like This

          In picking "more like this" groups of problems, we would also pick out a good example from the bunch and marked it as the leader of that group.  In a sufficiently recent version of the library browser, if a "more like this" group is encountered, only the leader is shown.  It is has an "M" in the list of action icons for the problem.  Clicking on the M reveals the other problems from the group (and the M becomes an L).  Clicking on the L hides all but the leader again.  The time to reveal/hide the more-like-this problems is very short, so teachers can easily peek at the problems to see if there are things of interest, but reduce the clutter of problems in general.

          Locking the Taxonomy

          As mentioned above, in the past the OPL Subject/Chapter/Section taxonomy could be expanded dynamically.  We plan to essentially fix it.  New problems contributed to the OPL will either need to be tagged using the existing taxonomy, or have approval from an OPL editor to expand the taxonomy.  The current version is stored in a file called Taxonomy, which comes with the OPL (it is next to the Textbooks file at the top of the OPL).  It is easy to read/write by both computers and humans.  It also specifies the order the entries will appear in the OPL menus, so they can be in a more logical order (in the past it has been alphabetical).


          We will have several workshops to improve more of the OPL.  If you think you might be interested or have questions, contact Jeff Holt ( or John Jones (

          by John Jones ( at July 01, 2013 10:59 AM

          June 29, 2013

          Anneke Bart

          WeBWork, Raleigh, NC, 2013

          We just completed a 3-day workshop on WeBWorK. It's interesting to see the directions this is taking.
          • We can now assign free response questions. This was more or less introduced last summer if I remember correctly and the feature has been tested by several people. The submitted answers are listed, and each answer can be graded by choosing a grade 0-25-50-75-100 form a drop down menu. One of my colleagues who tried it said it was fast to grade and easy to use.
          • Instructors can use "achievements" if they choose to. This will assign "badges" to students if they do certain things. For instance if they work a certain number of problems, do problems before a certain time in the morning, etc. This is part of the so-called game-ification of WeBWorK. I have not tried it yet, but when I asked my students last Fall more than half indicated they might find this kinda fun.
          • There are now more problem sets for statistics. Problem sets have been classroom tested at several colleges and universities and are ready to be rolled out. This will be very nice to have as we may expect to be teaching more stats courses due to the increased requirements at the MCAT. 
          • Related to the previous item is work being done to incorporate the statistics package R into WeBWorK. This is still some time off (months???), but that will help developers create more problems more easily.
          Those are some of the big ticket items, but there is a lot going on behind the scenes!

          by Anneke Bart ( at June 29, 2013 03:29 PM

          June 07, 2013

          Geoff Goehle


          Hello everyone.  Another code camp has come and gone and I wanted to write a bit about what I helped out with at Ann Arbor.  My time was, to a large extent, split between three things:  continuing to polish and upgrade the math4 theme, designing the basic objects that will form the core of the next WeBWorK (or WeBWorK::Brewery as it is currently called), and finally prototyping a brand new addition to the achievement system. 

          With math4, the biggest issue was getting the most frequently used pages of WeBWorK to validate as html5 when in the math4 theme.  This includes all of the pages that students can see, including the login/logout page, grades, achievements, Problem Sets, Problems, the Problem page and the Gateway Quiz page.   These pages should render more consistently across a wider array of devices and if you plug their html into the w3c validator you get a green bar which is not dissimilar to our own green bar.

          In addition to working on validation, I also put more twitter bootstrap scaffolding into the system.template file.  This will also help the pages render more consistently and allows us to use the bootstrap "responsive" functionality to make WeBWorK look better on phones.  For example, you can see below that when you use a phone to view WeBWorK the page layout is much more vertical and the side navigation bar is hidden.

          One additional tweak worth mentioning is the new "Description" field for sets.  This can be edited on the Set Detail page (the page where you can define the set headers).  If you add a description to the text box and save, then it will appear as a tooltip whenever you hover over the set name on either the main Set List page or the HomeworkSetsEditor2 page.  This means that you can now use shorter set names, which look much better, while also having the ability to present a lengthier description of the set in a way that doesn't take up screen space. 

          Moving on to the second project, I won't say much here about the results of our database discussion.  That is covered much more thoroughly in David Gage's blog post here and here.  I will mention that the notions of problem pools (a collection of problems from which each student will be assigned a random problem with a random seed) and teams/groups of students are core structures in the new database.  Also the system is intended to be modular enough that (eventually) administrators will be able to choose between several different types of database back-ends when they install WeBWorK.  The general vision is that the database back-end, the WeBWorK code, the PG code, and the web front-end will all be able to function relatively independently of each other and could at some point all be swapped out for alternatives. 
          The last thing I did was throw together a rough prototype of a new MathPets feature (too rough to show here).  The basic idea is that each student will have a little animated "MathPet" that lives at the bottom of the Problem page (and may eventually travel to other pages).  When they get a problem right or earn an achievement it "feeds" their MathPet and it will grow a little.  On the other hand, if they miss a homework set their pet becomes sad and may get sick.  Every time they earn a level (or at certain levels) their math pet will evolve into a different type of pet (and maybe even into one of several different options).  I think this idea is a lot of fun, but unfortunately it is one of those things which is really sold on the graphics.  While the code for this feature isn't too terrible, my artistic ability is minimal at best.  I will need help if I'm going to realize the full potential of this idea.  So if this feature sounds interesting to you, and you are or know of a good graphics designer that's willing to work on the cheap (i.e. free) be sure to let me know!

          by goehle ( at June 07, 2013 09:06 PM