Github

Quick instructions for updating WeBWorK to the GitHub version

• This description is primarily for those who are switching from versions of WeBWorK hosted on SVN (versions prior to 2.5) to those hosted on GitHub.
• Those who are already using version 2.6 and have downloaded the software from GitHub should consult the section on updating branches since the update instructions are shorter. If you already have files /opt/webwork/webwork2/conf/site.conf and /opt/webwork/webwork2/conf/localOverrides.conf then it is likely that are already using software from the WeBWorK GitHub distributions.
• You will need root access to the apache webserver via "sudo" in order follow the instructions below.
• You will need to prepend "sudo" to many of the commands below in order to obtain the permissions needed to make these changes.

Preserve current WW version and download software from GitHub

• We'll assume that our current webwork installation is located in the directory

/opt/webwork/

with subdirectories

 /opt/webwork/webwork2
 /opt/webwork/pg
 /opt/webwork/courses

• For safety we will move the webwork2 and pg directories to "old" versions and create links that redirect pg and webwork2 to these renamed directories

 cd /opt/webwork
 mv webwork2 webwork2_old
 mv pg pg_old
 ln -s webwork2_old webwork2
 ln -s pg_old pg

• At this point our current webwork installation should work as before. Calls to webwork2 and pg are directed to the "old" (actually our current) versions of webwork2 and pg.

• Next we will create new versions of webwork and pg from the code base at github.com

 git clone https://github.com/openwebwork/webwork2 webwork2_github
 git clone https://github.com/openwebwork/pg pg_github

• Particularly if you are using "sudo" you should check that the new webwork2_github and pg_github have the same permissions and owners as the original webwork2_old and pg_old
• The commands "chmod -R .....", "chown -R ", and "chgrp -R" will change the permissions, owners, and groups respectively on a directory, all of it's subdirectories and all of its files. Most often only the owner and group need to be changed.

• Set permissions on subdirectories
  • We assume that your webserver is operating with user name www-data (other choices might be wwhttpd, _www etc.)
  • The webserver needs to own some of the subdirectories of webwork2

  cd /opt/webwork/webwork2_github
  chown -R www-data DATA logs tmp 
  chown -R www-data htdocs/tmp
  

• Update mathjax

Configure WeBWorK

• We set up the configuration files for the new code base. There are now three configuration files which replace global.conf: site.conf, defaults.config, and localOverrides.conf. The idea is that local configurations are set by modifying site.conf and localOverrides.conf while defaults.conf contains configurations that may be updated by new releases of WeBWorK but don't vary from one site installation to another.
• The configuration lines in site.conf and localOverrides.conf all have analogs in global.conf so you should consult your old global.conf file for the information about your site needed to fill out lines in site.conf and localOverrides.conf.
• If site.conf and localOverrides.conf files already exist in your old installation you can copy them over from the old code base instead of creating new versions of the files as below.

  cd /opt/webwork/webwork2_github/conf
  cp site.conf.dist site.conf
  cp localOverrides.conf.dist localOverrides.conf
  cp webwork.apache2-config.dist webwork.apache2-config

• Set up site.conf
• Set up localOverrides.conf
• Set up database.conf.dist. Because this configuration file is almost never customized to a local site we do not make a database.conf version. Instead database.conf.dist If one does have a database.conf version than this is read instead of of database.conf.dist and with every update one has to verify that no database changes have been made.

 cd /opt/webwork/webwork2_github/conf
 mv database.conf database.old  # move an existing database.conf out of the way

• Set up webwork.apache2-config

• Update /opt/webwork/courses/modelCourse from /opt/webwork/webwork2/courses.dist/modelCourse

 cd /opt/webwork
 mv /opt/webwork/courses/modelCourse /opt/webwork/courses/modelCourse.old
 cp -RPpi /opt/webwork/courses/modelCourse /opt/webwork/courses/modelCourse

• This means that new courses created from modelCourse will have new resources, such as those used by the MathAchievements feature.

Switch over to new version of WeBWorK

• Next we switch our links so that they point to the new code base.

 cd /opt/webwork
 rm webwork2
 rm pg
 ln -s webwork2_github webwork2
 ln -s pg_github pg

• Run check_modules.pl and update CPAN modules if needed.
• Restart webserver.
• Go to admin course, click on the "upgrade courses" tab and update the course databases.

Enjoy

• If something goes wrong and we need to back out to our previous version of WeBWorK simply change the links as described below and restart the apache server.

 cd /opt/webwork
 ln -s webwork2_old webwork2
 ln -s pg_old pg

What could possibly go wrong?

• Everyone's environment is different, and everyone sometimes misses a step in the instructions so things might not always work the first time.
• Don't panic!
• On restarting the server you might get a (very long) error message saying something like: "HTML::Scrubber not found after looking in ......". This happens if a new CPAN module is used which is not immediately available on your system. This can be installed with

  sudo cpan HTML::Scrubber 

• If you have not updated for some time there might be several CPAN modules that need to be added.
• Perhaps things seem to work at first but then fail when trying to enter a problem set with some information about "missing fields in table". This usually indicates that you course has an out-of-date database. (This error will not happen with a freshly built course, only one that was built with a previous version of WW.)
  • Sign in to WeBWorK from the web and go to the admin page. The url will be something like:

 http://your.site.edu/webwork2/admin

• • Select the "upgrade courses" tab
  • Follow the instructions and upgrade the course that triggered the error (and any other courses that you expect to continue to use. You will be presented with a page where you can select which courses should be upgraded, followed by another page that informs you what database changes will be made, and then a page reporting the changes that have been made.

• If the database error is not resolved
  • Check that the file database.conf has been moved out of the way (see above) so that the new database.conf.dist file is used for accessing the database.

Quick instructions for using other branches of webwork

You may want to use the current stable candidate release branch of WeBWorK instead of using the "master" branch. You might want to do this because the upcoming release branch has a new feature that you want to use right away, or you might like to help out with the final testing of the new branch. These instructions tell you how to download the new branch, how to switch to it, and how to easily revert back to the original "master" branch if there are difficulties or when you are done testing.

• These instructions tell you how to obtain recent branches of the webwork2 repository. The analogous procedures on the pg directory will obtain the various branches of the pg repository.

• To download the information about the branches of WeBWorK in addition to the master branch.

 cd /opt/webwork/webwork2
 git remote -v
      origin	https://github.com/openwebwork/webwork2 (fetch)
      origin	https://github.com/openwebwork/webwork2 (push)

This is the default setup. The reply to the remote command says that "origin" is connected to the standard repo at github.com.

 git fetch origin
 git branch -a
   * master
    remotes/origin/HEAD -> origin/master
    remotes/origin/develop
    remotes/origin/master
    remotes/origin/release/2.7
 git branch -t release/2.7 origin/release/2.7
    Branch release/2.7 set up to track remote branch        release/2.7 from origin.
 git branch
    * master
    release/2.7

• This sequence of commands does the following
  • Obtain information about all of the branches at the "origin" repository.
  • List all of the branches available. The * indicates that we are currently using the "master" branch.
  • The "git branch -t..." sets up a local branch which tracks (syncs with) the current release candidate (which is "release/2.7" in this case)
  • The final check "git branch" shows that you now have access to two versions on the software, downloaded to your machine. The * indicates that you are currently using the master version.
• Update pg (for some version changes it is necessary to update both pg and webwork2 but often they can be updated independently. The pg and webwork2 version numbers do not always match since pg is updated more frequently.)

 cd /opt/webwork/pg
 git fetch origin
 git branch -a
   * master
    remotes/origin/HEAD -> origin/master
    remotes/origin/develop
    remotes/origin/master
    remotes/origin/release/2.6
 git branch -t release/2.6 origin/release/2.6
   Branch release/2.6 set up to track remote branch        release/2.6 from origin.
  

• Switch over the webwork2 release candidate release/2.7 and to pg version release/2.6

  cd /opt/webwork/webwork2
  git branch
    * master
    release/2.7
  git checkout release/2.7
  cd /opt/webwork/pg
  git checkout release/2.6

• Run /opt/webwork/webwork2/bin check_modules.pl to make sure that you have the required CPAN modules installed.
• Now restart your apache server (the command for this might be "sudo apachectl graceful")
• Go to the webwork admin page http://your.server.edu/webwork2/admin and upgrade the relevant course databases.

Notice: With the new configuration files updates to webwork2 become relatively painless. Usually you will not have to update any configuration files. Default configurations for new features will be provided in the defaults.config.dist file. Your local directory configuration structures remain unchanged in sites.conf as do your local overrides in localOverrides.conf. At some point you may want to customize some of the default.config.dist configurations using localOverrides.conf. You should seldom, if ever, modify default.config.dist. If you do you become responsible for managing and merging changes to that file.

It is also easy to back out to a previous release.

• To switch back

  git branch
    master
    * release/2.7
  git checkout master

and restart your apache server.

What could possibly go wrong

• You get what look like database errors of some kind -- missing fields in tables, etc. when looking at a course.
  • Most likely you didn't upgrade the course databases from the web using the WeBWorK admin page
• You get a long message about "not finding HTML::Scrubber" looking in ..... various directories
  • The message looks scary but simply indicates that some required CPAN module has not been installed on your site. Perhaps you forgot to run check_modules.pl or perhaps the maintainers neglected to include a required module in the new version of check_modules.pl. (If this is the case please report the bug.) They are easily installed using the cpan command or by importing them from your distributions packages.

Quick instructions for updating one of your branches

• To update one of the standard webwork branches

 cd /opt/webwork/webwork2
 git branch 
    * master
    release/2.7
 git pull

This updates the current branch "master" to agree with any changes that have been made on the openwebwork repo. If you wish to update the release/2.7 which is tracking release/2.7 on the openwebwork repo then follow the same procedure but first switch to the release/2.7 branch using the command

  git checkout release/2.7

• Those wishing to update from other sources than github.com/openwebwork should read the WeBWorK developer instructions for using the advanced features of git.

Understanding Github and git

Git and Github Overview

Git is the name of a distributed version control system. It plays the role of the CVS or SVN control systems that WeBWorK has used in the past but the details of how it works are somewhat different. For obtaining up-to-date copies of software there is not much difference between git and previous VCS. Developers however should read closely since git has a different conceptualization of how version control should be handled. The difference allows considerably more flexibility for collaborative development. Consolidation no longer depends on a single site or single administrator. Updates are also more modular and therefore easier to check and repair or remove if they cause trouble.

Git has many capabilities and can be used in many different ways. This page outlines a subset of git commands and workflows that seems to work well for WeBWorK.

Github.com is a site which facilitates the collaborative development of open source projects. These sites are open (read-only) to the world. WeBWorK has a master site at https://www.github.com/openwebwork where accepted modifications are being consolidated. The following repositories are available at that site.

• webwork2 -- the course management face of WeBWorK
• pg -- the macros that help render the PG questions

• webwork-open-problem-library -- (formerly NationalProblemLibrary) -- a collection of homework questions contributed from institutions around the world.
• webwork-model-course-library -- a collection of mathematics courses that can be used "out of the box" -- under construction.
• wwmoodle -- plugins that connect moodle and webwork
• ww_question_server -- another plugin project connecting moodle question types to the WeBWorK webservice
• admintools -- a collection of command line scripts used by larger sites serving WeBWorK

Following WeBWorK Development

You can get a view of development using git by clicking on the "Network" button on a github site. For example at https://github.com/mgage/webwork2 click on the "Network" button to see a graph of changes incorporated into Mike Gage's (bleeding edge) version of WeBWorK. Clicking on the commit dots along the lines will show the commit comments describing what was done with this commit. With git you concentrate on the "dots" the individual commits, more than the branch lines. You can often grab individual commits and apply them to your own branch; cherry picking some features for your branch while leaving other features behind. Your branch and the branch you are borrowing from must have a shared history somewhere in the past. Having a fairly recent shared history minimizes the chances that there will be conflicts when you try to apply the commit or "patch".

You can scroll the network graph to explore the past history of WeBWorK and when (and to some extent by who) each feature was added. Please follow or improve on the style of the later commit comments. Earlier commits represent a learning phase. :-)

The top lines indicate worked done by Mike Gage locally on his laptop and eventually merged into his master branch and uploaded to the github repository. The other lines indicate work uploaded by other developers to their sites and where their development repos stand compared to Gage's.

Workflow

On your local git repository the following workflow seems to minimize clashes with work being done elsewhere. Commit early and often to your local branch.

clone the remote repo  
git checkout -b my_new_feature (start a new branch to work on a single feature)
 ..work and commit some stuff
git pull origin master    (pull changes from the upstream github repository into your master copy)
git rebase master         (apply updates to your my_new_feature branch so that it looks like it branched from the updated master copy)
 ..work and commit some stuff
git pull origin master 
git rebase master
 ..finish the feature
git checkout master
git merge my_new_feature  (merge your changes back into your local master copy)
git push origin master    (publish the changes in your master copy to your github account)
git branch -d my_new_feature  (delete this branch. Since you don't want to use it with rebase after you have submitted to the github repository? )

Issue pull request to openwebwork

The use of "rebase" means that your chain of commits will all appear together, one after another, without any intervening commits from other developers which makes it easier to understand the flow of development. In general, even without rebase issues, it is better to delete your feature branch once you have merged it with your master branch and tested it in order to avoid clutter. git will not allow you to delete a branch (easily) if it has not been merged with the master branch.

Individual developers for webwork also have their own sites (analogous to the branches in SVN, but more easily accessible)

• Michael Gage: https://www.github.com/mgage
• Jason Aubrey: https://www.github.com/aubreyja
• David Gage: https://github.com/whytheplatypus
• Arnie Pizer: https://www.github.com/apizer

Writing a good commit line

Writing good commit lines is important. Imagine that you will be reading a series of commits in order to understand how the software development is progressing. Write accordingly.

When making commits I found this advice to be useful.

• Write a good first line that describes the feature you are committing . (It's like the subject line of an email but more important) This is the line that summarizes the commit so that when searching the commit tree one can find where a feature was added.
• Good style: Start with a verb in the present tense. These read well in the commit tree. For example:

Add slider bar to library browser.
or
Fix bug in level_curve_checker  subroutine

• Follow the subject line with a blank line.
• Then add as much as you want to explain what changes were made and any other information you want preserved. (A diff of the changed files is preserved automatically.)

• You can use git log to see how previous commits were worded. (Don't copy any of my first examples, but the later ones are getting better.)

• when possible make each commit an atomic unit. One step in the development process

Download WeBWorK via svn to an svn repository

• Quick-instructions. Type the following in the directory where you wish to have the repository. For example:

          cd webwork
          svn checkout https://github.com/openwebwork/webwork2/trunk webwork2
          svn checkout https://github.com/openwebwork/pg/trunk pg  
          svn checkout https://github.com/openwebwork/webwork-open-problem-library/trunk   webwork-open-problem-library
          
 will produce the directories webwork2 and pg inside the directory webwork containing all the files from 
 the master versions of webwork2 and pg at the github.com/openwebwork repository respectively.

• For more details see the documentation by the github folks: https://github.com/blog/966-improved-subversion-client-support

References for GitHub and Git

• Pro Git book http://progit.org/

(unreviewed at the moment -- YMVV)

• http://gitref.org/ quick reference
• https://help.github.com/articles/using-pull-requests
• https://github.com/blog/966-improved-subversion-client-support
• git cheat sheet: http://byte.kde.org/~zrusin/git/git-cheat-sheet-medium.png
• Version Control by Example by Eric Sink which is available at http://www.ericsink.com/vcbe/
• http://nvie.com/posts/a-successful-git-branching-model/ a potential workflow model for WeBWorK development -- we're still working on refining a workflow
• see also Git for more references

Version_Control