upgrading WeBWorK from svn to git

These are notes about upgrading a web-server

from Ubuntu 11.10 running WeBWorK 2.5.0+ installed using Subversion
to Ubuntu 12.04 running WeBWorK 2.6 installed using Git

The old Subversion repository (somewhere on WeBWorK.maa.org) is frozen. Although it is possible to do a Subversion CheckOut from the active repository on Github, that Github repository has different numbering from the MAA repository — the result is that using svn switch will not work. Some type of full (and updatable) download from Github will be required to stay up-to-date. This is a record of a complete switch to Git.

This procedure defers changes to a running system to the very end. The steps are adapted from the Quick instructions for updating WeBWorK to the GitHub version section of http://webwork.maa.org/wiki/Github This work was done using an SSH connection through a VPN tunnel; all commands were given using a terminal interface (i.e., no GUI). These steps are recorded from a session which followed several dry runs and two test runs, both with some minor mistakes.

0: upgrade operating system

ensure installed operating system (Ubuntu 11.10 server) is up-to-date
```
$  sudo  aptitude  update
$  sudo  aptitude  safe-upgrade
```
upgrade to next release (12.04 server, LTS=Long Term Support): This is a step to be done only when users will not be disrupted. If primary access is to WeBWorK via a browser, then executing sudo apache2ctl halt may suffice; but advance warning to all would be polite.
```
$  su
#  do-release-upgrade
#  exit
```
Note: procedure began with cautionary message about doing this major act via an SSH connection, took about 2 hours with several occasions for me to make a decision. I actually did sudo do-release-upgrade and got a few warnings — I revised this record to use a full su with the hope it would be cleaner.

confirm that symbolic link /etc/apache2/conf.d/webwork.conf still points to $WEBWORK_ROOT/conf/webwork.apache2-config

1: upgrade cpan and Perl modules used by WeBWorK

Note: Ubuntu 11.10 server had Perl 5.12 while Ubuntu 12.04 has Perl 5.14.2

upgrade cpan

$  su
#  perl  -MCPAN  -e  shell
cpan[1]>  upgrade  Bundle::CPAN

install 2 modules used by WeBWorK 2.6

cpan[2]>  install  JSON
cpan[3]>  install  Text::CSV

report status of modules used by WeBWorK: upgrade where needed

cpan[4]>  r  Apache2::Request Apache2::Cookie Apache2::ServerRec
cpan[5]>  r  Apache2::ServerUtil Benchmark Carp CGI Data::Dumper
cpan[6]>  r  Data::UUID Date::Format Date::Parse DateTime DBD::mysql DBI
cpan[7]>  r  Digest::MD5 Email::Address Errno Exception::Class File::Copy
cpan[8]>  r  File::Find File::Path File::Spec File::stat File::Temp GD
cpan[9]>  r  Getopt::Long Getopt::Std HTML::Entities HTML::Tagset
cpan[10]>  r  HTML::Template IO::File Iterator Iterator::Util
cpan[11]>  r  Locale::Maketext::Lexicon Locale::Maketext::Simple
cpan[12]>  r  Mail::Sender MIME::Base64 Net::IP Net::LDAPS Net::SMTP
cpan[13]>  r  Opcode PadWalker PHP::Serialization Pod::Usage Pod::WSDL
cpan[14]>  r  Safe Scalar::Util SOAP::Lite Socket SQL::Abstract
cpan[15]>  r  String::ShellQuote Text::Wrap Tie::IxHash Time::HiRes
cpan[16]>  r  Time::Zone URI::Escape UUID::Tiny XML::Parser
cpan[17]>  r  XML::Parser::EasyTree XML::Writer XMLRPC::Lite RQP::Render
cpan[18]>  Q
#  exit

Oops: during my first test run, I gave just a plain upgrade command. CPAN then started to upgrade every installed module — after an hour and several errors, it quit. The report/upgrade procedure (see above) uses only modules in webwork2/bin/check_modules.pl (except the two installed in previous step).

2: confirm key environment variables point to WeBWorK stuff

$  echo  $WEBWORK_ROOT	##  /opt/webwork/webwork2
$  echo  $PATH		##  ends with  $WEBWORK_ROOT/bin
$  echo  $PG_ROOT	##  /opt/webwork/pg

3: download WeBWorK files from github

new directory names will not clash with old.
protection and ownership are set immediately after download.

webwork2 & mathjax (do not overlook "." at end of first chmod)

$  cd  /opt/webwork
$  git  clone  https://github.com/openwebwork/webwork2  webwork2_github
$  cd  webwork2_github
$  chmod  -R  u+rwX,go+rX  .
$  chmod  -R  g+w  DATA  logs  tmp  htdocs/tmp
$  find  DATA/ logs/ tmp/ htdocs/tmp  -type d  -exec chmod g+s {} \;
$  sudo  chown  -R  www-data:wwdata  DATA  logs  tmp  htdocs/tmp
$  cd  htdocs
$  rmdir  mathjax	##  it is empty (and replaced by next command)
$  git  clone  https://github.com/mathjax/MathJax.git  mathjax

pg (do not overlook "." at end of chmod)

$  cd  /opt/webwork
$  git  clone  https://github.com/openwebwork/pg  pg_github
$  cd  pg_github
$  chmod  -R  u+rwX,go+rX  .

OPL (note that this git clone commmand has only one argument)

$  cd  /opt/webwork/libraries  ## current location of NationalProblemLibrary
$  git  clone  https://github.com/openwebwork/webwork-open-problem-library.git

4: record configuration choices for old (current) installation

go to directory with old configuration files
```
$  cd  $WEBWORK_ROOT/conf
```

database.conf

$  diff  database.conf.dist  database.conf  >  ~/ww25-db.txt

global.conf

$  diff  global.conf.dist  global.conf  >  ~/ww25-gc.txt

webwork.apache2-config

$  diff  webwork.apache2-config.dist  webwork.apache2-config  >  ~/ww25-wac.txt

5: configure the new system (before making it active)

go to directory for creation of new configuration files

$  cd  /opt/webwork/webwork2_github/conf
$  cp  site.conf.dist  site.conf
$  cp  localOverrides.conf.dist  localOverrides.conf
$  less  README

defaults.config — do not edit this file

Following steps will require coordination with previous configuration. Consult all three files created by diff in step 4.
site.conf — my changes involved
- $server_root_url
- $database_password
- $mail{smtpServer}
- $mail{smtpSender}
- $problemLibrary{root} = "/opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary";
- $problemLibrary{version} = "2.5";
- $siteDefaults{timezone}
localOverrides.conf — my changes involved
- $problemLibrary{root} = "/opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary";
- $problemLibrary{version} = "2.5";

check that actual changes match intention

$  diff  site.conf.dist  site.conf
$  diff  localOverrides.conf.dist  localOverrides.conf

webwork.apache2-config — needed because /etc/apache2/conf.d/webwork.conf is symbolic link to it. (I had no changes to make to this file.)
```
$  cp  webwork.apache2-config.dist  webwork.apache2-config
```
courses: modelCourse (basis of new courses) will be symbolic link to latest version
- go to appropriate spot
```
$  cd  /opt/webwork/courses
```
- move old version out-of-the-way
```
$  mv  modelCourse  ../modelCourse-old
```
  Note: if kept in courses directory, then the Upgrade Courses command will present it for upgrade (which will fail because it is not a real course).
- create symbolic link to latest version, then check link
```
$  ln  -s  $WEBWORK_ROOT/courses.dist/modelCourse  modelCourse
$  ls  -ld  modelCourse
```

cautious changes to the running system

Although the following sequences of commands can be executed fairly quickly, a cautious approach might involve selective uses of sudo apache2ctl halt and sudo apache2ctl restart to avoid disruption during periods when the system is in an inconsistent state (part old & part new).

6: keep old stuff in case we need to revert

webwork2 & pg: rename, create symbolic links to original names

$  cd  /opt/webwork
$  mv  webwork2  webwork2-old
$  ln  -s  webwork2-old  webwork2
$  mv  pg  pg-old
$  ln  -s  pg-old  pg

NPL: rename original, create symbolic link of new OPL to old name
```
$  cd  /opt/webwork/libraries
$  mv  NationalProblemLibrary  NPL
$  ln  -s  webwork-open-problem-library/OpenProblemLibrary  NationalProblemLibrary
```
Note: current or un-archived courses with sym-link of Library to /opt/webwork/libraries/NationalProblemLibrary will continue to work (because old NPL is subset of new OPL).

desirable: check that old system still works
Note: A not-yet-existing test-suite would be useful here.

7: switch to new system

revise symbolic links to point to new stuff
```
$  cd  /opt/webwork
$  ls  -l
$  ln  -snf  webwork2_github  webwork2
$  ln  -snf  pg_github  pg
$  ls  -l
```
Note: if ls -l does not show the revised links, then use explicit "rm webwork2 ; ln -s webwork2_github webwork2", etc.

restart apache, then check that all modules for WeBWorK are installed
```
$  sudo  apache2ctl  graceful
$  check_modules.pl  apache2  |  less
```
Note: if check_modules shows anything other than tth to be missing, then use cpan (see step 1) to install it and restart apache (and repeat check_modules).

BUG: despite explicit instructions to NOT create database.conf (in webwork2/conf), the apache2ctl graceful command failed with error message Syntax error on line 45 of /etc/apache2/conf.d/webwork.conf: Failed to read include file /opt/webwork/webwork2/conf/database.conf (has it been created from the corresponding .dist file?): No such file or directory at (eval 929) line 12.\n Action 'graceful' failed. The Apache error log may have more information. Note: tail -20 /var/log/apache2/error.log showed no related item
Work Around: cp database.conf.dist database.conf

upgrade old courses
- login to admin course
- invoke Upgrade Courses command
Note: when Upgrade Courses gave warnings about ownership, I used "sudo chown -R www-data *" in the courses directory to fix that problem.

8: update database for problems

$  OPL-update

first message is Use of uninitialized value $WeBWorK::Constants::WEBWORK_DIRECTORY in concatenation (.) or string at /opt/webwork/webwork2/lib/WeBWorK/Constants.pm line 38. Several threads in our forums discuss this message. What is the authoritative fix?
when localOverrides.conf had a typo for $problemLibrary{version} and this script could not find a Textbooks file, it nonetheless reinitialized the mysql database. I suggest the database be reinitialized only after the Textbooks file has been found.
when localOverrides.conf did not have the line $problemLibrary{version} = "2.5"; (because I had already added it to site.conf), this script reported "Library version is 2; using NPLtables". Since localOverrides.conf.dist has no $problemLibrary{version} line, this implies the value (2.5) set in site.conf was not used.
times to build database after it was reinitialized
- textbook data: 2.5 minutes
- 2000 problems: 9 minutes (from start)
- 4000 problems: 21 minutes
- 6000 problems: 29 minutes
- 8000 problems: 37 minutes
- 10000 problems: 46 minutes
- 15000 problems: 58 minutes
- 20000 problems: 72 minutes
- 25000 problems: 93 minutes
- 29900 problems: 112 minutes
Library Browser reports 23844 problems — hooray for removal of duplicates.

9: (gasp) reverting to old system

revise links, check that change, restart apache

$  cd  /opt/webwork
$  ln  -snf  webwork2-old  webwork2
$  ln  -snf  pg-old  pg
$  ls  -l
$  sudo  apache2ctl  graceful

switch from OPL back to NPL (? John, Mike, Arnie: I suspect this would not be needed. ?)

$  cd  /opt/webwork/libraries
$  ls  -l
$  rm  NationalProblemLibrary
$  mv  NPL    NationalProblemLibrary
$  ls  -l
$  NPL-update