Release notes for WeBWorK 2.13

                                     WeBWorK                                    
                         Online Homework Delivery System                        
                                  Version 2.13                          

                    Copyright 2000-2017, The WeBWorK Project                    
                              All rights reserved.                              

Details
-------

Prerelease date: July 8, 2017 -- the advanced beta version of 2.13 is currently on github

Release date: July 25, 2017

Github repository:  https://github.com/openwebwork
Modules:    webwork2   pg   ww_install

Cautions

WARNING: It is likely that new modules will be required since the release 2.12. There are also changes to the configuration files which will need to be integrated into your existing configurations.

In particular we have switched the CPAN module for sending mail from Mail::Sender to Email::Sender since the former has been deprecated for some time and the latter is the recommended perl module to use for this purpose.

This will be an invisible change for the end user, and in many cases, except for an easy configuration change in site.conf, it will be invisible to the installer as well. In some cases some fiddling with configuration parameters may be required to get the local SMTP site and the WeBWorK site to talk to each other.

The process for upgrading to 2.13 is described below. This assumes that you have been using release 2.12 or some other recent release that you installed using git from the github.com repository.

If you are setting up WeBWorK for the very first time you will probably want help with directory structure, permissions (always a hassle) and installing infrastructure tools. If you are installing webwork on a fresh OS install such as Ubuntu or Debian, try using the ww_install script. [1] This will save you lots of time. As of this writing the install script does not include the new CPAN modules Email/* and you will need to install them by hand as described below.

If you prefer to have more control and to install all of the modules by hand there is a collection of installation manuals of which I recommend Installation_Manual_for_2.12_on_Ubuntu_16.04 as the most up-to-date. There are only slight differences between the instructions for the recent releases of WeBWorK and the directions are quite similar for the different flavors of unix. The directory names may sometimes be different but a google search can help clarify how to translate instructions from one OS to another.

Once you get to the section in the manual where the WeBWorK and PG software is being installed you can refer to the instructions below. Let's get started:

Upgrade Instructions

1) Go to /opt/webwork/webwork2

>git branch
* develop
  master

>sudo git branch -t rel2.13 origin/master
Branch rel2.13 set up to track remote branch master from origin.

>sudo git checkout rel2.13
error: You have local changes to 'lib/WeBWorK/ContentGenerator/Feedback.pm'; cannot switch branches.
 #### this means that you have made local changes you might want to save.  
    Use `git stash` or `git commit` or just move the file to a safe place.

>sudo git checkout -f rel2.13   # the `f` forces the change and overwrites the file if you didn't move it.
Switched to branch 'rel2.13'

>sudo git pull
 .... long list of updated files

2) Now go to /opt/webwork/pg and run

>git branch
* develop
  master

>sudo git branch -t rel2.13 origin/master
Branch rel2.13 set up to track remote branch master from origin.

>sudo git checkout rel2.13
Switched to branch 'rel2.13'



>sudo git pull
 .... long list of updated files

The -t flag has your branch "track" the version of master at the origin site. If there are updates on github.com/webwork2 to the master branch a simple sudo git pull will update your local directory.

3) Depending on your system there may be new perl modules which have been added since 2.12. Run check_modules.pl and look for missing modules. They may include:

  cd /opt/webwork/webwork2/bin
 ./check_modules.pl
   ** Digest::SHA not found in @INC
   ** Email::Sender not found in @INC
   ** Data::Dump not found in @INC
      cpan HTML::Scrubber found and loaded
   ** Statistics::R::IO not found in @INC   

In this case the first three modules are missing. You will need to install them with

   sudo cpan Digest::SHA
   sudo cpan Email::Sender
   sudo cpan Data::Dump
   sudo cpan Statistics:R::IO

and so forth.

You should also run

  cd /opt/webwork/webwork2/bin
 pdflatex check_latex.tex

and look for missing latex packages. They may include:

• fullpage.sty
• preview.sty

If you are missing fullpage it is in the texlive-latex-extra package in apt based distributions and texlive-preprint in yum/dnf distributions.

Configuration file update

4) (new in 2.13 ) IMPORTANT

Check site.conf.dist and transfer new configuration options over to site.conf. In particular the section which configures the mail features of WeBWorK have changed. $tls_allow=1; should be commented out and replaced by $mail{tls_allow}=0;. There are more detailed instructions in the new site.conf.dist file

 cd /opt/webwork/webwork2/conf
 diff site.conf site.conf.dist | less

Most of the differences occur when you have filled in local site urls and path ways. These do not need to be changed. You want to look in the mail section where the configuration variable $tls_allowed=1 has been removed and replaced by $mail{tls_allowed}=0. The zero setting means that ssl (or tls) is not used and this is usually the best configuration to start with. If the local smtp server requires ssl then you will need to set this variable to 1. Consult with your local IT people or whoever runs the mail server for help on this.

< $tls_allowed = 0;
---
> $mail{tls_allowed} = 0;
> #$tls_alowed=0;  #old method -- this variable no longer works.
>
> # errors of the form
> #  unable to establish SMTP connection to smtp-gw.rochester.edu port 465
> # indicate that there is a mismatch between the port number and the use of ssl
> # use port 25 when ssl is off and use port 465 when ssl is on (tls_allowed=1)
>
>
> # Set the SMTP port manually.  Typically this does not need to be done. 
> # port 25 is used if no SSL is on and 465 if ssl is on
>
> #$mail{smtpPort} = 25;

5) Check localOverrides.conf.dist and look for changes to add to the existing localOverrides.conf.

• (new in 2.12) Library Statistics - This is a new feature (see below). There are variables which control whether or not global and local problem statistics are shown in the library browser.

# These flags control if statistics on opl problems are shown in the library
# browser.  If you want to include local statistics you will need to
# run webwork2/bin/update-OPL-statistics on a regular basis.  
$problemLibrary{showLibraryLocalStats} = 1;
# This flag controls whether global statistics will be displayed
$problemLibrary{showLibraryGlobalStats} = 1;

• (new in 2.12) Problem Re-Randomization - The problem rerandomization feature (see below) has default options which need to be set.

# The default prPeriod value (re-randomization period) to use for the newly created problem.
# It is suggested to use the value of -1, which means that the course-wide setting would be used
# Setting this to -1 defaults to the use of course-wide settings (suggested)
# Setting this to 0 disables periodic randomization regardless of the course-wide setting
# Setting this to a positive value will override the course-wide setting
$problemDefaults{prPeriod} = -1;

################################################################################
# Periodic re-randomization
################################################################################
# switch to enable periodic re-randomization
$pg{options}{enablePeriodicRandomization} = 0;
# course-wide default period for re-randomization, should be an integer
# the value of 0 disables re-randomization
$pg{options}{periodicRandomizationPeriod} = 5;

6) (new in 2.12 and 2.13) Check your apache configuration file (e.g. webwork.apache24-config)and look for changes to add to your existing configuration file. If you are updating from 2.12 you probably will not need to make any changes. If you are updating from an older version you may need to add the following:

• Code to set the location of the pg directory as an environment variable

$WeBWorK::Constants::PG_DIRECTORY = $pg_dir;
$ENV{PG_ROOT} = $pg_dir;

• A line to set the server admin email

$ENV{WEBWORK_SERVER_ADMIN} = $webwork_server_admin_email;

• There are additional print statements and a "warn" statement in that will report information about your settings.

It will warn you if you have a mismatch between the correct port and the chosen protocol (whether to use

ssl (a.k.a tls) to contact the smtp server). To begin assume that ssl will not be needed and will 

not be used in contacting the smtp server.

Update course files from courses.dist

7) Check your courses.dist/modelCourse/course.conf and look for changes against your course.conf files. They may include:

• Removing a number of outdated settings.
• Commented out settings for controlling which instructors can see which students.

# The following hashes control which users are allowed to see students from which
# sections.  This is typically  used for large multi-section classes with many students, ta's and
# professors.  When set users will only be allowed to see students from the appropriate section in the following:
# -  Instructor Tools
# -  Student Progress
# -  Email
# -  Problem Grader
# -  Show Answers
# They will be able to see students from other sections on other pages.  
# These variables generally should be set here and not in defaults.conf.

# $viewable_sections = { user_id1 => [1 ,2 ,3] , # list of viewable sections for user_id1
#		       user_id2 => [1],
#		     };

# $viewable_recitations = { user_id1 => [1 ,2 ,3] , # list of viewable recitations for user_id1
#			  user_id2 => [1],
#		     };

Restart the apache server

8) Restart the webserver.

sudo apachectl restart

Upgrade the admin course, MathJax and the OPL

9) Run the upgrade admin database script. You will not be able to access your admin course until this is done, and you will need to do that to upgrade the course databases.

upgrade_admin_db.pl         #(if this doesn't work use)
perl upgrade_admin_db.pl 

Next, log into the admin course and upgrade the databases for all of your courses. Your server will not be functional until you perform this step.

10) Download the latest version of the OPL. Note: If you have library statistics enabled this will also update your statistics tables.

cd /opt/webwork/libraries/webwork-open-problem-library
git pull origin
/opt/webwork/webwork2/bin/OPL-update

11) Download the latest version of MathJax.

cd /opt/webwork/MathJax
git pull

Upgrade course databases

If you are upgrading from release 2.12 you will not have to upgrade any course databases -- there have been no changes. If you are upgrading from an older version, or if you have courses which were created under older versions and not updated then you will need to upgrade the courses from the web.

1. Login in to the admin course https://yourSite/webwork2/admin. 2. Choose the "Upgrade Courses" tab. 3. Follow instructions, choosing which courses you want to upgrade. 4. There will be an intermediate report telling what changes will be made -- click "continue" 5. There will be a final report on the changes successfully completed -- click "done"

Basic tests

Because the underlying mechanisms for sending email have changed it is important to run two tests to check that WeBWorK is still communicating with the SMTP (the post office) which you have designated (in site.conf) to handle mail.

1. Create a new course test_course</course> from the admin page 2. You should be listed in this new course as an admin 3. Add a student to the course 4. Login as the student and from the front page use the "email the instructor" button to send a feedback notice 5. Check that you receive this feedback (at your admin email address in the course). Remember the subject starts with [WWfeedback] and you may be shunting such messages off to another mailbox. 6. Login as a professor or admin and from the email page send email to your self (and perhaps to the student you have added). 7. Check that this mail comes through.

Features

New Features in release 2.13

• R server
  • Version 2.13 of WeBWorK has the built-in capability of running R code. One can write problems in PG that calls R to be run on a rserve (R server).
  • To learn how to set this up see R_in_WeBWorK. This will also get you started writing statistics problems that call R.
  • (XXXXX links to OPL problems that use R) -- need to construct a page lilnking to problems that use R
• Localization features
  • With release 2.13 it is now possible to use maketext(english phrase) in PG (as well as in WeBWorK2) to enable the creation of a dictionary that will translate

phrases into other languages. (https://github.com/openwebwork/webwork2/pull/704) and numerous maketext() pull requests following that which add additional maketext() entries.

** We hope to have full multilingual compatibility, including the ability to use alphabets outside the standard English alphabet, for release 2.14

• CodeMirror -- color highlights for editing in the PGProblemEditor (https://github.com/openwebwork/webwork2/pull/724)
  • Color highlighting is only implemented for editing within the PGProblemEditor.
  • $options{PGCodeMirror}=1; enables the use of CodeMirror instead of a plain text editor.
  • $options{PGMathView} = 0; disables the use of PGMathView as a minimal LaTeX editor for entering equations in PG code.
  • See http://webworkgoehle.blogspot.com/2016/08/codemirror-and-pg.html for more details.
  • Some people use the FileManager editor for editing PG problems, but it can also be used for editing other files. CodeMirror

has not yet been adapted for this editor.

• LTI -- The LTI connection between Canvas, Blackboard, Moodle and WeBWorK was introduced in release 2.12 but best practices for configuring the connection are emerging.
  • Here are the quick instructions that enable LTI "homework" connections for a single course.
    • Make a copy authen_LTI.conf from authen_LTI.conf.dist
    • In defaults.config make sure that authen_LTI.conf is enabled and set global settings for the grading mode and debugging:

   include("conf/authen_LTI.conf");
  $debug_lti_parameters = 0; 
  $LTIGrademode=“”;

• • • At the end of course.conf for the course using LTI place these statements:

   # Settings to connect this course to 	Canvas via  LTI.
   $LTIGradeOnSubmit = 1;
   $debug_lti_parameters = 0;
   $LTIBasicConsumerSecret = "XXXsecretXXX";
   $LTIGradeMode = "homework";
   $external_auth = 0;  # this allows both LTI and direct authorization to work together (use carefully, LTI registered students who try to
                     # login directly will be told they have an incorrect password since LTI creates a random practice that is never used. 

• • • For more details read Goeff Goehle's blog posts (links are below) and the comments in authen_LTI.conf.dist, and LTIAdvanced.pm

Features from release 2.12

(These features were introduced in release 2.12 but are only beginning to be widely used. There have been minor fixes introduced in release 2.13)

• LTI Grade Passback: This is a complete rewrite of the LTI authentication module including upgrades to allow grades to be passed back from WeBWorK to the LMS (e.g. Blackboard, Moodle, etc...). The authentication feature includes the ability to automatically create students logging into WeBWorK for the first time. The grading feature has two modes. You can either have WeBWorK pass back a single "homework" grade for the entire course, or you can set it up on a set by set basis. You can read more about this module at the following links. Note: The new LTIAdvanced module is not a strict upgrade from the older LTI authentication module LTIBasic. The older module still works, but should be considered legacy code. The older module can serve as a useful model for customizing the new module for local institutional needs.
  • http://webworkgoehle.blogspot.com/2016/03/webwork-lti-authentication.html - This contains information about how to set up the new module and use it for authentication. See also LTI-Advanced Authentication
  • http://webworkgoehle.blogspot.com/2016/03/webwork-lti-grading.html - This contains information about how to use the grade passback features of the module. See also LTI-Advanced Grading
  • https://github.com/openwebwork/webwork2/pull/687 - This is the github pull request.
• Hardcopy Themes: This adds support for different hardcopy themes. Instructors will set the theme that students have access to on the Course Configuration page, and can also select the different themes on the instructor download handouts page. Right now there are two themes: the traditional two column WeBWorK theme and a new "one column" theme which the full page width. The system is also set up so that universities can add their own themes for tests or with branding. Note: You will need to make sure the fullpage latex package is available on your server to use this feature.
  • http://webworkgoehle.blogspot.com/2016/02/hardcopy-themes.html - This contains information about how to use the hardcopy themes feature, and also how to set up your own themes.
  • https://github.com/openwebwork/webwork2/pull/682 - This is the github pull request.
• Past Answer Refactor: This is a rewrite of the existing Past Answer's page. First, it replaces the old "wildcard/regex" feature in Past Answers with multiselect lists. Now instructors will explicitly select which sets, users and problems they would like to search for past answers. It also adds the ability to download answers from the Past Answer page as a CSV.
  • https://github.com/openwebwork/webwork2/pull/677 - This is the github pull request.
• OPL Statistics: This adds problem usage statistics to the Library Browser. When enabled and configured you should see both local and 'global' statistics on how many times problems have been assigned, students average scores and, average attempt numbers. Here local statistics are generated from the data on your WeBWorK server and reflects how your courses use problems. Global statistics are generated from over a dozen different universities and includes information on 23000 problems.
  • You can donate data from your school by running update-OPL-statistics and then upload-OPL-statistics.
  • http://webworkgoehle.blogspot.com/2016/01/opl-statistics.html - This page contains more information about the statistics feature.
  • https://github.com/openwebwork/webwork2/pull/665 - This is the github pull request.
• Translator Refactor: This is an extensive under the hood revamping of how PG tracks problem resources, aliases, answer blanks and evaluators. In general the new approach uses less "dead reckoning" and is smarter about treating answer blanks and evaluators as different components of the same "object". You should not see any changes in functionality.
  • https://github.com/openwebwork/webwork2/pull/653 - This is the github pull request.
  • https://github.com/openwebwork/webwork2/pull/656 - This is the github pull request.
  • https://github.com/openwebwork/webwork2/pull/657 - This is the github pull request.
  • https://github.com/openwebwork/webwork2/pull/672 - This is the github pull request.
  • (new in rel2.13) The bug which caused past answer recording to fail for multiAnswer problems has been fixed. (https://github.com/openwebwork/pg/pull/302, https://github.com/openwebwork/webwork2/pull/767)
• Advanced Problem Debugging: Users with admin privileges will have the ability to see detailed debugging information about problems. There are four options available. AnswerGroupInfo shows information about answer "groups". This is most helpful for multianswer style problems. Show Auxiliary Resources shows information about auxiliary files used by the problem. AnswerHashInfo shows information about the answer hash, including answers, filters and evaluators. PGInfo prints the PG environment, including all available variables.
  • https://github.com/openwebwork/webwork2/pull/654 - This is the github pull request.
• Periodic Randomization: This feature allows you to force problems to generate a new seed after a fixed number of attempts. It is sort of "max attempts lite" in that students can keep trying, but they have to start over. The feature is enabled via the Course Configuration page (in the Optional Modules section). Instructors can set the default number of attempts before re-randomization. The number of attempts before re-randomization can also be set on a per problem basis.
  • Periodic Randomization: This is a wiki resource for this feature.
  • https://github.com/openwebwork/webwork2/pull/626 - This is the github pull request.
• parserWordCompletion: This new macro allows authors to ask free response fill in the blank questions with interactive help. As a student types their answer into the answer blank, jQuery's autocomplete feature generates a drop-down list of allowable answers that match what has already been typed.
  • https://github.com/openwebwork/pg/blob/master/macros/parserWordCompletion.pl - As with many macro files, the best documentation is actually contained at the beginning of the macro itself.
  • https://github.com/openwebwork/pg/pull/258 - This is the github pull request.
• Grades: The Grades page was overhauled with a number of small modifications. The total percentage score for homework is now shown and the "C" has been replaced by a 100. There is also a course homework grade at the bottom of the Grades table. This score is generated from the total number of points earned divided by the total number of points available and should match the score generated by selecting all sets on the Scoring Tools page. (This can be turned off on the Course Configuration page if WeBWorK's grading scheme doesn't match with what you use.)
  • http://webworkgoehle.blogspot.com/2016/03/grades-external-grades-and-email.html - You can read more about the changes here.
  • https://github.com/openwebwork/webwork2/pull/678

(rel 2.13) Smaller Changes and bug fixes

see also updates in rel 2.12 -- Release_notes_for_WeBWorK_2.12#Smaller_Changes

• (hotfix added after initial release) fix error in draggableProofs macro <https://github.com/openwebwork/webwork-open-problem-library/pull/343>
• Numerous maketext() calls added in preparation for allowing full utf8/unicode (multi-language) compatibility in rel 2.14.
• URLS for seeking help on PGProblemEditor page have been updated and are now configured in defaults.conf for easy maintenance.
• fix closing markers for inequalities in MathObjects <https://github.com/openwebwork/pg/pull/322>
• fix closing markers for inequalities in MathObjects <https://github.com/openwebwork/pg/pull/322>
• fix parentheses about vectors in IJK mode <https://github.com/openwebwork/pg/pull/321>
• fix problem using identity matrix as a constant <https://github.com/openwebwork/pg/pull/320>
• Make Unions report more uncombined sets. Fix error for uncombined sets.<https://github.com/openwebwork/pg/pull/315> <https://github.com/openwebwork/pg/pull/306>
• * Convert vectors to column matrices for some matrix operations <https://github.com/openwebwork/pg/pull/307>
• Add differentiation to fractions and add cmp_defaults

<https://github.com/openwebwork/pg/pull/308>

• Allow omission of height from image subroutine and preserve aspect ratio
• Fix crash caused by loading too many problems and their library tags into a library page. https://github.com/openwebwork/webwork2/pull/707
• Fix bug in "skip archiving this course" https://github.com/openwebwork/webwork2/pull/711
• Fix bug with "force renumber"
• MYSQL update to quote passwords
• OPL statistics bug fixed. https://github.com/openwebwork/webwork2/pull/723
• D2L LTI grade passback fixed https://github.com/openwebwork/webwork2/pull/726
• Defenses added to prevent XSS (cross-site scripting) https://github.com/openwebwork/webwork2/pull/739
• Prevent SQL exploit from scoring page: https://github.com/openwebwork/webwork2/pull/774
• configure PGProblemEditor help buttons in default.config https://github.com/openwebwork/webwork2/pull/775
• Prevent SQL injection exploit by instituting queries using parameters for all database calls: https://github.com/openwebwork/webwork2/pull/779
• updated directories searched for macros to include OpenProblemLibrary/macros/MC and OpenProblemLibrary/macros/UniSiegen

Installer

The ww_install installer has been updated to support version 2.12. It has been tested on all of the distributions listed below. The installer has also been refactored to be more modular when it comes to different distributions. If you are interested in modifying the installer to work with your distribution post an issue on the ww_install github. You can read more about it at WeBWorK Installer Guide

The ww_install installer has not yet been updated to release 2.13. This means that you will need to manually install some cpan modules -- notably Email::Sender

    sudo cpan Email::Sender

Supported Distributions

The following are the distributions and features that are officially still supported. They have all been tested with the webwork installer. You still can use WeBWorK on whatever you like, of course. In particular if you install WeBWorK on a very recent version of Ubuntu or Fedora and have problems let us know.

• Ubuntu 15.10
• Ubuntu 16.04 (LTS)
• Debian 8
• CentOS 7
• Fedora 22
• Fedora 23

What could possibly go wrong?

• I cannot access the admin course. This could be because you have not run upgrade_admin_db.pl yet.
• I cannot generate a "one column" hardcopy. You may be missing the fullpage latex package. In apt based systems its part of the package texlive-latex-extra and in yum/dnf based systems its part of texlive-preprint
• I see little vertical lines next to MathJax rendered formulas. This is caused by an incompatibility between older versions of MathJax and Chrome. Update MathJax with cd /opt/webwork/MathJax and git pull to fix the problem.
• I do not see links to some of the editors in the Main Menu, including the links to Problem Set Detail 2, or I get an error about "SimplePGEditor". To fix this remove the definition of the %showeditors from localOverrides.conf. This hash should be defined in defaults.config and if you want to change things in localOverrides.conf you should set hash elements individually.
• I can't get LTI to work. First make sure that you are using the new version of LTI. (Make a fresh copy of authen_LTI.conf from the distribution file.) Also enable the $debug_lti_parameters flag. For security reasons the default error messages from the LTI module are very vague.
• I dont see the Library Statistics. Make sure that you have enable the appropriate flags in localOverrides.conf (see the first section of this page) and run OPL-update

Installation Notes

(Please add additional suggestions on how to solve installation problems you have encountered. If you are registered on the wiki you can edit this document. )

references