Installing from WW2.15 Ubuntu20.04 Server Virtual Machine Image

From WeBWorK_wiki
Jump to navigation Jump to search


These instructions cover the installation of the Ubuntu Server 20.04 LTS 64 bit operating system and WeBWorK 2.15 using the WeBWorK Virtual Machine Image.

The WeBWorK Virtual Machine Image is an .ova file which is an "open, secure, portable, efficient and extensible format for the packaging and distribution of software to be run in virtual machines" (see http://en.wikipedia.org/wiki/Open_Virtualization_Format) and is supported by VMware, VirtualBox, QEMU/KVM, AWS, Google Cloud, etc.

This image file has been tested on

  • VMware Workstation 15 Player
  • VirtualBox 6
  • QEMU/KVM

This "server" version contains everything you need to run a WeBWorK server (e.g. WeBWorK, Apache2, MySQL, R server, lighttpd, log rotation, etc.) installed and configured.

Contents

Installing from WW2.15 Ubuntu20.04 Server Virtual Machine Image

Overview

After installing from the WeBWorK Virtual Machine Image, you will have a full fledged Ubuntu Server 20.04 LTS system with WeBWorK 2.15, Apache2, MySQL, R server, lighttpd, log rotation, etc. installed and configured. If your network uses DHCP, networking will be automatically configured for your system. If it uses static IP addresses, you will have to configure networking. Also it is imperative that you CHANGE THE PASSWORDS for the OS user wwadmin (who has sudo privileges), for the MySQL user webworkWrite who has access to MySQL, for the WeBWorK user admin who has admin privileges and for the WeBWorK user profa who has professor privileges (see below). Finally you should set up SSL (https) access to WeBWorK if students will be using your server.

There are more detailed instructions for Ubuntu Server 20.04 and WeBWorK 2.15 at Installation Manual for 2.15 on Ubuntu 20.04 Server.

Download the ova image

There is only a 64 bit distribution. The 64 bit distribution should work with most modern cpu's. Download the sha1 checksum and the .ova file from the site below. The ova is a 4.5 GB file.

  • MAA
  1. http://webwork.maa.org/ww-downloads/WW2.15_Ubuntu20.04_Server.ova.sha1
  2. http://webwork.maa.org/ww-downloads/WW2.15_Ubuntu20.04_Server.ova

Verify the SHA1 checksum of your downloaded file WW2.15_Ubuntu20.04_Server.ova agrees with the one in WW2.15_Ubuntu20.04_Server.ova.sha1.

OVA and OVF files

The .ova file is simply a tar bundle containing an .ovf file, one or more .vmdk files, a .mf file and possibly other files.

  • The .ovf file is an XML file which describes the packaged virtual machine and is human-readable.
  • The .vmdk file(s) contain the disk images(s).
  • Possibly other files
  • The .mf file contains SHA1 checksums for the above files.

You can import a virtual machine either from an .ova file or from the .ovf, .vmdk, .mf collection. Sometimes importing from the .ova file may fail whereas importing from the .ovf or .vmdk file will succeed.

You can extract the files in WW2.15_Ubuntu20.04_Server.ova with the command 

$  tar -xvf WW2.15_Ubuntu20.04_Server.ova

You then can look at (and possibly edit) the human readable WW2.15_Ubuntu20.04_Server.ofv file. If you do edit the WW2.15_Ubuntu20.04_Server.ofv file, you will also have to compute the new SHA1 checksum and replace the old one in the WW2.15_Ubuntu20.04_Server.mf file for the files to be usable.

Installing the WeBWorK Virtual Machine Image

Import the file WW2.15_Ubuntu20.04_Server.ova into your virtualization software package (e.g. VMware, VirtualBox, QEMU/KVM). The ova file was created on VMware Workstation 15 Player running on a Windows 10 Pro host. It has been tested on

  • VMware Workstation 15 Player running on a Windows 10 host (Home and Pro editions)
  • VMware Workstation 15 Player running on a Ubuntu 20.04 Desktop host
  • VirtualBox 6 running on a Windows 10 host (Home and Pro editions)
  • VirtualBox 6 running on a Ubuntu 20.04 Desktop host
  • QEMU/KVM running on a Ubuntu 20.04 Desktop host

See Specific Virtual Environments below for installation information on specific virtual environments.

Processors, Memory, Hard Disk, Networking

The WeBWorK Virtual Machine Image was created with the following parameters:

  • 20 GB dynamically allocated disk drive in VMDK format (single file) of which 10 GB is used
  • 1 GB memory
  • 1 cpu

The above resources are very minimal and (except for maybe disk space) correspond to a AWS t2-micro level machine which is free.

Assuming you have not changed things when importing the image, some of these configurations may remain in effect (they all will for VMware Workstation 15 Player running on a Windows 10 Pro host). You should adjust these resources either when you import the .ova file or later after you have tested things. Adjusting the number of processors and memory should be straightforward. Expanding the hard disk is more complicated. See Specific Virtual Environments below and also consult the documentation for your virtual machine environment. After you have expanded the disk drive, you will still have to make the extra space available to Ubuntu (see Increase Disk Space below).

Setting up networking may be the most tricky part. If you have problems, look at the "Networking" section under your specific environment below see (Specific Virtual Environments) and/or look at Debugging Networking Issues.

Import the .ova file

There may be specific information for your situation below. See

Your server

After importing, your virtual WeBWorK server will be identical to a system created by following the instructions Installation Manual for 2.15 on Ubuntu 20.04 Server with all Optional Configurations A-E implemented except that

  1. Optional C (SSL) is not implemented
  2. The PGbasicmacros.pl file has been slightly changed (see PGbasicmacros.pl below)

The implementation of Optional C (SSL) should be done locally after you have checked that everything is working (see below).

Note that on some virtual environments, you may need to take additional actions. See the section Specific Virtual Environments below.

You should read through the instructions Installation Manual for 2.15 on Ubuntu 20.04 Server in order to understand how your server has been set up. Especially look at Terminal Window Notation and Use in the Installation Manual for 2.15 on Ubuntu 20.04 Server to understand the notation we use in these instructions.

PGbasicmacros.pl

The PGbasicmacros.pl file (in the directory /opt/webwork/pg/macros) has been slightly changed to fix a small bug. The original file is PGbasicmacros.pl.bak1 in the same directory. This may or may not be addressed in a WeBWorK 15 hotfix.

Boot Your Server

Log into your server

After booting you should see a login prompt (you may have to press <Enter>).

  • Log in as "wwadmin" with the password "wwadmin" (more on accounts and passwords below). "wwadmin" has sudo privileges.

If your network uses DHCP, networking may be automatically configured for your system (but you may have to edit some files, see below). If not you will have to set it up manually.

Test your ip address

Run the command 

$ ip address show

and look at the output, something like 

 ...
 link/ether 00:0c:29:4f:2c:1d brd ff:ff:ff:ff:ff:ff
 inet 192.168.76.128/24 brd 192.168.76.255 scope global dynamic ens33 
 ...

(not the LOOPBACK inet 127.0.0.1/8 address). Here the ip address is 192.168.76.128 .

If you do not see the ip address, you have a networking problem which is not unusual at this point. More specifically, you should not have a problem using VMWare, but will have a problem using VirtualBox or QEMU/KVM. Look at the "Networking" section under your specific environment below see (Specific Virtual Environments). If that doesn't help look at Debugging Networking Issues. You have to fix this before you can go on to the next step.

Accessing Your Server from a Terminal Emulator on your Host

At this point you can login to your server from your host machine using SSH (non secure telnet and FTP are not allowed but secure SSH and SFTP are) using your favourite terminal emulator program.

You can do all of the remaining installation from a terminal emulator on your host. The advantage of doing this is that you can copy commands from these instructions (with copy from the Edit menu or ^C) and paste them into a terminal window (with paste from the Edit menu list or <Shift> <Ctrl> <V> or <Shift> <Insert> depending on your application).

My advice is to first test accessing your server from your host machine and check that everything is working properly. We will do this with using the NAT network adapter and you new server's ip address (not domain name). This is fine for testing but is not a good permanent solution. After testing that WeBWorK is working properly, if you want to allow access from the web (e.g. if you will have students using the system) you can reconfigure your system using a suitable network adapter and you new server's registered domain name. In any event, after testing, read the section Make the WeBWorK Configuration Permanent below. For the most part, instructions on allowing access from the web are beyond the scope of this document. Here we give instructions on accessing your server from your host machine.

I am assuming your network has been set up automatically.

The Guest IP is the IP address your guest WeBWorK server is using. You can find it (after you login) by entering the command 

$ ip address show

and looking at the output, something like 

 ...
 link/ether 00:0c:29:4f:2c:1d brd ff:ff:ff:ff:ff:ff
 inet 192.168.76.128/24 brd 192.168.76.255 scope global dynamic ens33 
 ...

(not the LOOPBACK inet 127.0.0.1/8 address). Here the ip address is 192.168.76.128 .

If your system is set up with NAT using these rules it means that at this point you can only access your server from a web browser running on your host machine, from a terminal emulator running on your host using ssh, or from the terminal on the guest once you login.

Actually establishing the connection depends on both your virtual machine environment and your host environment. Look at the documentation below for your situation.

Now login to your server as "wwadmin" with the password "wwadmin" from your terminal emulator running on your host.

Set the Timezone for your server

To find out what timezone your server is set to run the command 

$ timedatectl

and you will probably see 

...
Time zone: America/New_York (EDT, -0400)
...

which may not be where you live. The timezone naming convention uses a “Region/City” format and to find the correct one for your location run the command 

$  timedatectl list-timezones

Look through the list and find your timezone, e.g. "America/Los_Angeles". Then set the timezone (you have to be root), e.g. 

$ sudo timedatectl set-timezone America/Los_Angeles
[sudo] password for wwadmin: <wwadmin password>

and then 

$ timedatectl

to check it was set correctly.

Checking for and installing hotfixes

Follow the instructions at Checking for and Installing Hotfixes in the Installation Manual for 2.15 on Ubuntu 20.04 Server.

WeBWorK configuration

Most WeBWorK configuration is done in the files /opt/webwork/webwork2/conf/site.conf and /opt/webwork/webwork2/conf/localOverrides.conf. These files provide system-wide configuration settings, and defaults for course settings. Any setting in these files can be overridden in the course.conf file for a particular course. To override a setting for a course, just put the new setting (using the same syntax as is in localOverrides.conf) in the course.conf file. An instructor can only edit the course.conf file herself (for her own course) if she has "admin" privilege which by default professors do not have. But most things instructors may want to customize and many others (language, timezone, permissions, display modes, email, ...) can be set using the Course Configuration page from within the course and such setting override those in the configuration files.

Usually the "admin" user is added as an admin in all new courses and she can grant "admin" privileges to anyone (but she should be very careful in doing this).

Actually there are three main configuration files, site.conf, defaults.config and localOverrides.conf. The reason there are three configuration files is to make upgrading WeBWorK easier.

  • site.conf: This file contains global variables which are required for basic configuration. It will not be overridden when you update WeBWorK but its distribution version, site.conf.dist will be.
  • defaults.config: This file contains initial settings for many customizable options in WeBWorK. Do not edit defaults.config. It will be overridden next time you upgrade.
  • localOverrides.conf This is where you should add all local customizations. It will not be overridden when you update WeBWorK but its distribution version, localOverrides.conf.dist will be.

There are several options that must be set for WeBWorK to work with your system. The rest of the file consists of customization options.

Edit the site.conf file

Now backup and edit site.conf 

$ cd /opt/webwork/webwork2/conf
$ cp site.conf site.conf.bak1
$ nano site.conf

First we have to edit information about the Apache2 server setup. Search for 192.168.204.128 and replace that by your the new ip address you found above (192.168.76.128 in our example above).

But wait, this can be tricky. If you set up port forwarding (as we had to for VirtualBox), then instead use the code 

 $server_root_url   = 'http://localhost';

The edited line should look like the above line when we use port forwarding (i.e. running under VirtualBox) and like the line below when there is no port forwarding (i.e. running under VMware or QEMU/KVM) 

$server_root_url   = 'http://192.168.76.128';

where of course your ip address may be different. The "http://" is important.

Remark. First of all, let me admit I don't understand the above - it just works. If this is not set correctly (and I don't really understand what "correctly" means), then when you test the Library Browser, e.g. by clicking on Library Browser on the Main Menu, then on Open Problem Library (actually it should already be selected so it will be greyed out) and select a Subject, you will not be able to select a Chapter and you will see an error message similar to 

183 setmaker.js: /webwork2/instructorXMLHandler: Forbidden.

If you google this message, you will find a lot of people have seen this error and the most common reason is that $server_root_url has not been set correctly, whatever "correctly" means. A common error is to use forget http:// or to use http:// when you should use https:// or to just have the wrong domain name or ip address.

We now continue editing site.conf

WeBWorK uses the DateTime module. DateTime is supposed to be able to determine the local timezone itself without you having to enter it but this often fails so it is best to just set it here. For is a list of timezones recognized by DateTime run the command 

timedatectl list-timezones

These timezones are more refined than standard time zone usage in that they include switches to daylight savings time (e.g. some parts of a time zone may make the switch and others may not). For example if your server is in the eastern US, on the list you will see America/New_York and you should enter $siteDefaults{timezone} = "America/New_York"; which is the default. Read the documentation in this section of the the site.conf file for more information on selecting time zones and formatting dates.

Search for $siteDefaults{timezone} and enter your local timezone if it is not correct.

Here is some information on email although this probably won't really work until you connect WeBWorK to the outside world. You might want to hold off on this until then. WeBWorK sends mail in three instances. The PG system sends mail to report answers to questionnaires and free-response problems. The mail merge module is used to send mail to course participants, i.e. to report scores. The feedback module allows participants to send mail to course instructors.

To send mail, WeBWorK needs the address of an SMTP server. Normally you will use the address of your school's SMTP server. If the local machine is running an SMTP server, use localhost. IMPORTANT: Our instructions above did not install an SMTP server so you will have to install and configure one if you do not use your school's SMTP server. When connecting to the SMTP server, WeBWorK must also send an email address representing the sender of the email (this has nothing to do with the From address on the mail message). Edit the lines 

$mail{smtpServer} = ;  # e.g. 'mail.yourschool.edu' or 'localhost'
$mail{smtpSender} = ;  # e.g.  'webwork@yourserver.yourschool.edu'

entering the appropriate information. Be sure to use single quotes and NOT double quotes around email addresses otherwise Perl will treat @ as an array variable.

Then save the file and Quit.

The defaults.config file

If you want WeBWorK questionnaires or similar things from different courses to be mailed to a central person or persons (e.g. the WeBWorK administrator), in defaults.config, you will see the lines 

$mail{allowedRecipients}     = [
   #'prof1@yourserver.yourdomain.edu',
   #'prof2@yourserver.yourdomain.edu',
];

But we are not supposed to edit the defaults.config file, so if we want to do this, we will copy this to localOverrides.conf and edit it appropriately. Note that we should move this setting to the site.conf file.

Edit the localOverrides.conf file

Now backup and edit localOverrides.conf 

$ cd /opt/webwork/webwork2/conf
$ cp localOverrides.conf localOverrides.conf.bak1
$ nano localOverrides.conf

First we have to replace all occurrences of 192.168.204.128 by your the new ip address you found above (192.168.76.128 in our example above). But again, this can be tricky. If you set up port forwarding (as we had to for VirtualBox), then 127.0.0.1 gets forwarded to the ip address you found above so below you want to enter 127.0.0.1, not 192.168.76.128. You can do this one by one or by a global search and replace. To use global search and replace enter the command: 

^ \
Search (to replace) [...]:192.168.204.128
Replace with:192.168.76.128
A

where of course replace 192.168.76.128 by your actual ip address (or 127.0.0.1 in the case of port forwarding). Note that in the port forwarding case since we are already using Lighttpd, in the localOverrides.conf file we have 192.168.204.128:8080 so you want to replace 192.168.204.128 by 127.0.0.1, NOT 127.0.0.1:8080.

As we said above, if you want WeBWorK questionnaires or similar things from different courses to be mailed to a central person or persons (e.g. the WeBWorK administrator), in localoverrides.config, add and then edit the lines 

$mail{allowedRecipients}     = [
   #'prof1@yourserver.yourdomain.edu',
   #'prof2@yourserver.yourdomain.edu',
];

Of course you have to remove the comment character #.

Then save the file and Quit.

Now restart apache so that our changes to the conf files takes effect. 

$ sudo apache2ctl restart
[sudo] password for wwadmin: <wwadmin password>

Set up WeBWorK to use SSL

This step configures apache so that access to WeBWorK will be through an encrypted Secure Sockets Layer (SSL) with an https: URL. It is optional but you should certainly do this if students will be using your server. However, I would hold off on this until you have tested that everything is working properly.

Follow the instructions at Implement Optional C (SSL) in the Installation Manual for 2.15 on Ubuntu 20.04 Server.

Finish up

Once access to your WeBWorK server is set up (as it should be now), if all you want to do is test out WeBWorK yourself without any other users, you are all set at this point but please read the rest of these instructions so that you understand how insecure your WeBWorK server is right now.

Test that Things are Working Properly

Connect to http://192.168.76.128/webwork2 where of course you should use your actual ip address. We will test out a few important parts of WeBWorK. If you run into problems, you should look at the Apache error log which is located at /var/log/apache2/error.log. And you should look at Test that Things are Working Properly in the Installation Manual for 2.15 on Ubuntu 20.04 Server for help.

You should see the WeBWorK Welcome page with "Course Administration" and "myTestCourse" listed. At some point you may want to "hide" them so that they are not listed on the Welcome page (more on this later).

Click on "myTestCourse" and login with login name "profa" and password "profa". At this point you are a regular professor. There is also an administrator "admin", a regular student "jsmith" and several guest or practice users (who don't require a password but can view problems and "check" answers without them being "submitted" for credit). More on these below.

Now click on Homework Sets on the Main Menu and click on Demo. Then look at the problems. Mathematical equations should be typeset. Continue looking at problems to see if everything is working properly. Look through the problems in the other sets.

Next click on Problem List to bring back the Problem List Page and click on Download PDF .... The page is a little complicated because you are a professor so you see the professor view (students see a very simple page) but you can just scroll to the bottom and click on Generate hardcopy for selected users and selected sets.

Look through the problems in the other sets.

Now test the Library Browser. Click on Library Browser on the Main Menu. Click Open Problem Library (actually it should already be selected so it will be greyed out) and select a Subject, Chapter and Section and then hit View Problems. The first 20 of your selected problems will be displayed.

Next test that WeBWorK problems using R run properly. You are already in the Library Browser. Select "Statistics" as Subject, "Bayesian inference" as Chapter and "Posterior distribution" as Section and then hit View Problems. If the problems display with no error messages, all should be well. To be totally sure, click on the "eye" (Try it) in the upper right corner and test the problem. If there are no error messages, congratulate yourself. Everything works.

If you are new to WeBWorK or even if you are a pro, you should probably log in as that student to see what the student interface looks like. It's much simpler than the professor or admin interfaces. Login with login name "jsmith" and password "jsmith". The "admin" view is very similar to the "prof" view but you have the ability to change things about the course that mere professors do not have. Finally you can click on "Guest Login" and see what that looks like.

Make the WeBWorK Configuration Permanent

Now that everything is working properly, it is time to make the WeBWorK configuration permanent. We configured WeBWorK using your WeBWorK guest server's current ip address (found with ip address show) and used that when editing the site.conf and localOverrides.conf files. Since the network is setup with DHCP enabled, it means that the current ip address is not permanent. If it changes, WeBWorK will break (the most obvious symptom will be mathematical expressions in problems will not be displayed). We have two options to fix this.

  1. The preferred method is to use the registered domain name for your WeBWorK system in place of the ip address everywhere in the site.conf and localOverrides.conf files. It only occurs once in site.conf but it occurs many times in localOverrides.conf. For an easy global search and replace method see Edit the localOverrides.conf file above.
  2. The other method is to setup networking to use a fixed specific ip address for your server and use that everywhere in the site.conf and localOverrides.conf files. For this your have to edit the 00-installer-config.yaml and cloud.cfg.d files. See the Networking section under VirtualBox 6 running on a Windows 10 Pro host below for information on editing these files. You can search the web for information on the correct syntax to use.

Also if you are still using port forwarding (which you shouldn't in a permanent installation), things get more complicated as seen above in the sections Edit the site.conf file and Edit the localOverrides.conf file.

Remember that any time you edit WeBWorK's configuration files, you have the restart Apache2 for the changes to take effect. For networking changes to take effect, you should reboot the server.

Passwords

It is IMPERATIVE that you CHANGE THE PASSWORD for the OS user wwadmin (who has sudo privileges). Otherwise anyone can connect to your server and pretty easily gain root access. Also it is IMPERATIVE that you CHANGE THE PASSWORD for the MySQL user webworkWrite who has login privileges to MySQL. Otherwise anyone can connect to MySQL server and pretty easily gain access to the WeBWorK database. Finally it is IMPERATIVE that you CHANGE THE PASSWORD for the WeBWorK user admin who has admin privileges both in the admin course and in the myTestCourse and for the WeBWorK user profa who has professor privileges in the myTestCourse. Otherwise anyone can connect to WeBWorK server and pretty easily gain access and do anything they want.

Change the password for wwadmin

$ passwd
Changing passwd for wwadmin:
(current) UNIX password: wwadmin
Enter new UNIX password: <new wwadmin password> 
Retype new UNIX password: <new wwadmin password>
passwd: password update successfully
$

Do not forget the new <wwadmin password> that you just entered. Below when we refer to <wwadmin password>, we mean the new <wwadmin password>.

Change the password for webworkWrite

Now we change the passwords for the MySQL user webworkWrite. First we edit site.conf. 

$ cd /opt/webwork/webwork2/conf
$ nano site.conf

Search for $database_password = "wwadmin"; and replace this by
$database_password = "database_password"; where of course you should replace 'database_password' with your own password. We refer to this password as <database_password>. Remember it as we will need it shortly. Then save the file and Quit.

Then restart Apache so the changes take effect. 

$ sudo apache2ctl graceful
[sudo] password for wwadmin: <wwadmin password>
$

and start MySQL 

$ sudo mysql
[sudo] password for wwadmin: <wwadmin password>

You should see 

Welcome to the MySQL monitor ...
mysql>

Now lets check the MySQL users. 

mysql> SELECT user,authentication_string,plugin,host FROM mysql.user;

You will see a table with six users (debian-sys-maint, mysql.infoschema, mysql.session, mysql.sys, root and webworkWrite). You should see that five users have a valid password (which will be displayed in encrypted form) and root is authenticated by a socket.

Now we will change the password for the webworkWrite User 

mysql> ALTER USER 'webworkWrite'@'localhost' IDENTIFIED BY '<database_password>';

where of course you should replace <database_password> by whatever you used above (use the single quotes but no angle braces). Then 

mysql> FLUSH PRIVILEGES;
use your up arrow key to run the command
mysql> SELECT Host, User, authentication_string FROM mysql.user;

and you should see that webworkWrite has a new passwords (which will be displayed in encrypted form).

Then exit MySQL 

mysql> exit
Bye
$

If you want to check that you set the password correctly, do the following: 

mysql -u webworkWrite -p -h127.0.0.1 --protocol=tcp
Enter password: <database_password>

and you should see 

Welcome to the MySQL monitor ...
mysql>

Now exit 

mysql> exit
Bye
$

Finally a note on the MySQL root password. In Ubuntu systems running MySQL 5.7 (and later versions), the MySQL root user is set to authenticate using the auth_socket plugin by default rather than with a password. However in securing MySQL (see Installation Manual for 2.15 on Ubuntu 20.04 Server#MySQL Security and Performance Issues) we had to set a password for the MySQL root user and that password was set to "wwadmin" even though it is not used. If you ever change how the MySQL root user is authenticated (you shouldn't!!), remember this.

Change the password for admin

Change the passwords for the WeBWorK user admin in two courses. Login to both the WeBWorK admin course (http://.../webwork2/admin) and myTestCourse (http://.../webwork2/myTestCourse) with Username "admin" and Password "admin". Then change the passwords from "admin" to something more secure. Note that in WeBWorK to change the password of the user you are logged in as, select "User Settings" in the "MAIN MENU". Or you can login as "profa" and use the "Classlist Editor" to change the password.

Change the password for profa

Change the passwords for the WeBWorK user profa. Login to myTestCourse (http://.../webwork2/myTestCourse) with Username "profa" and Password "profa". Then change the passwords from "profa" to something more secure. Or you can login as "admin" and use the "Classlist Editor" to change the password.

Change the password for jsmith

Change the passwords for the WeBWorK user jsmith if you want. jsmith is just a regular student so she can't do any real damage but you may still want to change the password. Login to myTestCourse (http://.../webwork2/myTestCourse) with Username "jsmith" and Password "jsmith". Then change the passwords from "jsmith" to something more secure. Or you can login as "admin" or "profa" and use the "Classlist Editor" to change the password.

More House Keeping

Hide the admin and myTestCourse courses

Log out of myTestCourse if you are logged in and go to the WeBWorK Welcome page. Click on Course Administration and login as admin with the new admin password you set for the admin course. Select "Hide Inactive Courses" and select the courses you want to hide and hit "Hide Courses". If you go back to the WeBWorK Welcome page, you will see no courses listed. You can still access these courses directly by 

http://192.168.76.128/webwork2/admin 
http://192.168.76.128/webwork2/mytestcourse

where of course you should use your actual ip address.

Adjust Apache's configuration according to your server's memory

Look at Edit mpm_prefork.conf in the Installation Manual for 2.15 on Ubuntu 20.04 Server and adjust "MaxRequestWorkers" according to the rule of thumb given there.

Set up access to Apache's server-info and servo-status

This isn't really necessary but you should look at the info.conf and status.conf section in Installation Manual for 2.15 on Ubuntu 20.04 Server. Actually this is something that you should do after connecting your WeBWorK server to the outside world.

Increase disk space

This is a two part process. The first step is to use your virtualization software to expand the disk capacity. How to do this obviously depends on your specific virtualization software. You will find information on this in Specific Virtual Environments below. The second step is to repartition the disk and expand the file system. Let us assume you have completed the first step and expanded the disk capacity from it's initial 12 GB to 20GB although in practice you will probably want to make it larger. First enter 

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sda2        12G  9.3G  1.9G  84% /
...

to see how much disk space we have initially. Now run parted as root: 

$ sudo parted
[sudo] password for wwadmin: <wwadmin password>

and you will see something like 

GNU Parted 3.3
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted)

Now enter the "print" command 

(parted) print
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sda: 21.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  2097kB  1049kB                     bios_grub
 2      2097kB  12.9GB  12.9GB  ext4

(parted)

We need to know the number of the partition we want to resize. We can see it is 2 from the above. Now enter the "resizepart" command 

(parted) resizepart
Partition number? 2
Warning: Partition /dev/sda2 is being used. Are you sure you want to continue?
Yes/No? Yes
End?  [12.9GB]? 20GB
(parted)

Now enter the "print" command again 

(parted) print
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sda: 21.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  2097kB  1049kB                     bios_grub
 2      2097kB  20.0GB  20.0GB  ext4

(parted)

Notice we now have a 20 GB disk. Now quit parted. 

(parted) quit
Information: You may need to update /etc/fstab.

Now we resize the file system. The above information tells us that we are working on partition 2 on /dev/sda, so we use /dev/sda2 in the command below 

$  sudo resize2fs /dev/sda2
[sudo] password for wwadmin: <wwadmin password>
resize2fs 1.45.5 (07-Jan-2020)
Filesystem at /dev/sda2 is mounted on /; on-line resizing required
old_desc_blocks = 2, new_desc_blocks = 3
The filesystem on /dev/sda2 is now 4882300 (4k) blocks long.

and run df -h and we should see something like 

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sda2        19G  9.3G  8.2G  54% /
...

indicating that we now a lot more space on our disk. Yea!

File and Directory Locations and System Information

This installation of WeBWorK and Ubuntu follows the instructions given in Installation Manual for 2.15 on Ubuntu 20.04 Server. You can look there to find the locations of the WeBWorK files. All optional configurations (A-E) are implemented except that

  1. Optional A is implemented with the wwtmp directory set up as a separate directory but not as a separate partition and
  2. Optional C is not implemented. Optional C configures Apache (and optionally lighttpd) so that access to WeBWorK will be through an encrypted connection (TLS/SSL) with an https: URL. This has to be done locally and you may have already implemented this.

All system files are in their standard Ubuntu locations. The Ubuntu installation is identical to that obtained from the Ubuntu 20.04 Server LiveDVD distribution (specifically ubuntu-20.04-live-server-amd64.iso).

Consult Category:Administrators for other WeBWorK documentation for system administrators.

Specific Virtual Environments

Below you will find additional information about installing the ova, networking, accessing your server and expanding the virtual disk in specific virtual environments. Here is a list of the specific environments we have information on:

VMware Workstation 15 Player running on a Windows 10 Pro host

Importing the ova File

For VMware Player select Player, File, Open and then browse to the location of the WW2.15_Ubuntu20.04_Server.ova file and import the file.

You may get a warning message but on retry it will work (strange since the ova image was created on VMware Player).

Now continue reading the instructions in the section Boot Your Server above.

Networking

Networking should work right out of the box.

Accessing your server from your host

The Guest IP is the IP address your guest WeBWorK server is using. You can find it (after you login) by entering the command 

$ ip address show

and looking at the output, something like 

 ...
 link/ether 00:0c:29:4f:2c:1d brd ff:ff:ff:ff:ff:ff
 inet 192.168.76.128/24 brd 192.168.76.255 scope global dynamic ens33 
 ...

(not the LOOPBACK inet 127.0.0.1/8 address). Here the ip address is 192.168.76.128 .

From a browser

Assuming your WeBWorK server is set up as above with ip address 192.168.76.128, from a web browser running on your host machine connect to http://192.168.76.128 and you should see the Apache2 Ubuntu Default Page. Lighttpd is also running (it is only used for internal WeBWorK calls) and you can test it by connecting to http://192.168.76.128:8080 and again you should see the Apache2 Ubuntu Default Page (but now you will not see a graphic for the Ubuntu Logo). Actually on my Windows 10 Pro host, I can connect from Chrome, Firefox, and Brave but not from Edge. I haven't tried to figure out what is blocking Edge. To test WeBWorK, connect to http://192.168.76.128/webwork2/ and after a few seconds you should see the WeBWorK Welcome page. http://192.168.76.128:8080/webwork2_files/ should bring you to the WeBWorK Place Holder page.

Note that even through you can view some WeBWorK pages at this point, WeBWorK will not work properly until you go through the WeBWorK configuration above. At this point you should just continue reading here about connecting from a terminal emulator.

From a terminal emulator

I'm using MobaXterm but any terminal emulator will be similar. Click on Session and select SSH. For "Remote Host" enter 192.168.76.128 and specify the user name as wwadmin (or you can leave it blank and enter it at login) and leave the port set to 22. Since my host is secure, I find it convenient to let MobaXterm store my password but this is probably not a good idea in general.

Now login to your server as "wwadmin" with the password "wwadmin" from your terminal emulator running on your host.

Expand the disk drive

Make sure your server is powered off and open VMware Workstation 15 Player. Select your server and select "Edit virtual machine settings". Select "Hard Disk (SCSI)" and then "Expand..." and then enter the new Maximum size you want.

You still have to repartition the disk and expand the file system on you guest. For that see Increase disk space above.

Continue the Installation

Return to the Accessing Your Server from a Terminal Emulator on your Host instructions above.

VirtualBox 6 running on a Windows 10 Pro host

Importing the ova File

For Oracle VM VirtualBox Manager select File, Import Appliance..., and then browse to the location of the WW2.15_Ubuntu20.04_Server.ova file and open the file. Click Next, and then Import. Note that your new server will probably be called "vm". If you select "vm" and click on "Settings" you can change the name. Also you can easily up the memory and the number of processors.

Now continue reading the instructions in the section Boot Your Server above.

Networking

Initially networking will be broken. Do the following from your guest WeBWorK system 

$ sudo lshw -C network
[sudo] password for wwadmin: <wwadmin password>

and you will see something like 

      ...
      logical name: enp0s17
      version: 02
      serial: 08:00:27:30:28:b6
      ...

Remember the logical name and the serial (really the MAC address) from your system as we will need them below. We have to backup and then edit two files. For the first one: 

$ cd /etc/netplan/
$ sudo cp  00-installer-config.yaml  00-installer-config.yaml.bak1
[sudo] password for wwadmin: <wwadmin password>

Now edit the 00-installer-config.yaml file 

$ sudo nano  00-installer-config.yaml

It should look like (I had to stretch the window to see the whole file): 

# This is the network config written by 'subiquity'
network:
  ethernets:
    ens33:
      dhcp4: true
  version: 2

Now replace "ens33" by whatever you found as the logical name above ("enp0s17" in our example above). It is important to keep the indenting exactly the same. Then save the file and quit.

For the second file: 

$ cd /etc/cloud/cloud.cfg.d/
$ sudo cp  50-curtin-networking.cfg 50-curtin-networking.cfg.bak1
[sudo] password for wwadmin: <wwadmin password>

Now edit the 50-curtin-networking.cfg file 

$ sudo nano 50-curtin-networking.cfg

It should look like: 

network:
  config:
  - mac_address: 00:0c:29:4f:2c:1d
    name: ens33
    subnets:
    - type: dhcp4
    type: physical
  version: 1

Now replace "ens33" by whatever you found as the logical name above ("enp0s17" in our example above) and "00:0c:29:4f:2c:1d" by whatever you found as the serial above ("08:00:27:30:28:b6" in our example above). Then save the file and quit.

Now reboot your server,e.g. 

$ sudo reboot
[sudo] password for wwadmin: <wwadmin password>

login and run the command 

$ ip address show

and look at the output, something like 

... 
2: enp0s17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
   link/ether 08:00:27:30:28:b6 brd ff:ff:ff:ff:ff:ff
   inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s17
...

We need the Guest IP which is the IP address your guest WeBWorK server is using. Here we can see it is 10.0.2.15. Make a note of what it is on your system. We will need it below.

In VirtualBox using a NAT network we have to setup port forwarding to allow access from the host. Power off your guest, select it and click on Network. Make sure NAT is the network adapter type. Click on "Advanced" and then on "Port Forwarding". Add (by clicking on the plus symbol) the following 4 rules:

Port Forwarding
Name Protocol Host IP Host Port Guest IP Guest Port
lighttpd TCP 127.0.0.1 8080 10.0.2.15 8080
ssh TCP 127.0.0.1 2222 10.0.2.15 22
https TCP 127.0.0.1 4443 10.0.2.15 443
http TCP 127.0.0.1 8081 10.0.2.15 80

Double check that you have entered everything correctly and then hit "OK". And hit "OK" again to exit "Settings".

Now boot your server.

Accessing your server from your host

We need to employ the port forwarding rules above.

From a browser

Assuming your WeBWorK server is set up with the port forwarding rules above, from a web browser running on your host machine connect to http://127.0.0.1:8081 and you should see the Apache2 Ubuntu Default Page. Lighttpd is also running (it is only used for internal WeBWorK calls) and you can test it by connecting to http://127.0.0.1:8080 and again you should see the Apache2 Ubuntu Default Page (but now you will not see a graphic for the Ubuntu Logo). To test WeBWorK, connect to http://127.0.0.1:8081/webwork2/ and you should the WeBWorK Welcome page. http://127.0.0.1:8080/webwork2_files/ should bring you to the WeBWorK Place Holder page. On my Windows 10 Pro host, I can connect from Chrome, Firefox, Brave and Edge. Note that Edge didn't work for VMware above.


Note that even through you can view some WeBWorK pages at this point, WeBWorK will not work properly until you go through the WeBWorK configuration above. At this point you should just continue reading here about connecting from a terminal emulator.

From a terminal emulator

I'm using MobaXterm but any terminal emulator will be similar. Click on Session and select SSH. For "Remote Host" enter 127.0.0.1 and specify the user name as wwadmin (or you can leave it blank and enter it at login) and change the port to 2222. Since my host is secure, I find it convenient to let MobaXterm store my password but this is probably not a good idea in general.

Now login to your WeBWorK guest server as "wwadmin" with the password "wwadmin" from your terminal emulator running on your host.

Expand the disk drive

Make sure your guest id shutdown. In the main VirtualBox window, click File, Virtual Media Manager. Then select the virtual hard disk in the list (probably "WW2.15_Ubuntu20.04_Server-disk1.vdi" assuming you imported in vdi format) and use the “Size” slider at the bottom of the window to change its size. Click “Apply” when you're done.

You still have to repartition the disk and expand the file system on you guest. For that see Increase disk space above.You still have to repartition the disk and expand the file system on you guest. For that see Increase disk space above.

Continue the Installation

Return to the Accessing Your Server from a Terminal Emulator on your Host instructions above.

VMware Workstation 15 Player running on a Ubuntu 20.04 Desktop host

Importing the ova File

For VMware Player select File, Open a Virtual Machine and then browse to the location of the WW2.15_Ubuntu20.04_Server.ova file and open the file.

You will get a warning message but on retry it will work (strange since the ova image was created on VMware Player).

Now continue reading the instructions in the section Boot Your Server above.

Networking

Networking should work right out of the box.

Accessing your server is exactly the same as in VMware Workstation 15 Player running on a Windows 10 Pro host above except that instead of using a terminal emulator, just open a terminal window on your host and ssh into your new system, e.g. 

$ ssh wwadmin@192.168.76.128

where of course you need to use the ip address of your WeBWorK guest server.

Now login to your server using the password "wwadmin".

Expand the disk drive

Make sure your server is powered off and open VMware Workstation 15 Player. Select your server and select "Edit virtual machine settings". Select "Hard Disk (SCSI)" and then "Expand..." and then enter the new Maximum size you want.

You still have to repartition the disk and expand the file system on you guest. For that see Increase disk space above.

Continue the Installation

Return to the Accessing Your Server from a Terminal Emulator on your Host instructions above.

VirtualBox 6 running on a Ubuntu 20.04 Desktop host

Importing the ova File

Importing the ova File is exactly the same as in VirtualBox 6 running on a Windows 10 Pro host above.

Now continue reading the instructions in the section Boot Your Server above.

Networking

Accessing your server involves the same procedure as in VirtualBox 6 running on a Windows 10 Pro host above. So you have to

  1. Find the name and MAC address of the virtual nic (network interface card)
  2. Edit 00-installer-config.yaml
  3. Edit 50-curtin-networking.cfg
  4. Reboot your WeBWorK guest
  5. Find your WeBWorK guest's ip address
  6. Set up port forwarding

See VirtualBox 6 running on a Windows 10 Pro host above for details.

Except that instead of using a terminal emulator, just open a terminal window on your host and ssh into your new system, e.g. 

$ ssh -p 2222 wwadmin@127.0.0.1

Now login to your server using the password "wwadmin".

Expand the disk drive

The procedure is the same as in Expand the disk drive the Windows 10 Pro case above.

You still have to repartition the disk and expand the file system on you guest. For that see Increase disk space above.

Continue the Installation

Return to the Accessing Your Server from a Terminal Emulator on your Host instructions above.

QEMU/KVM running on a Ubuntu 20.04 Desktop host

First install Qemu-KVM . See e.g. https://help.ubuntu.com/community/KVM/Installation. And also install the optional: Install virt-manager (graphical user interface).

Importing the ova File

First cd to whatever directory you downloaded the WW2.15_Ubuntu20.04_Server.ova file to, (maybe "Downloads"). Then untar it 

tar -xvf WW2.15_Ubuntu20.04_Server.ova

and run the command 

sudo virt-convert WW2.15_Ubuntu20.04_Server.ovf -D qcow2 --destination /var/lib/libvirt/images/

This will take awhile but eventually your guest WeBWorK server will start up automatically in a Virt Viewer window. Login as wwadmin with password wwadmin and then go on to the next section.

Networking

Accessing your server involves the same procedure as in VirtualBox 6 running on a Windows 10 Pro host above. So you have to

  1. Find the name and MAC address of the virtual nic (network interface card)
  2. Edit 00-installer-config.yaml
  3. Edit 50-curtin-networking.cfg
  4. Reboot your WeBWorK guest
  5. Find your WeBWorK guest's ip address

See VirtualBox 6 running on a Windows 10 Pro host above for details.

Except that

  1. You do not need port forwarding
  2. Instead of using a terminal emulator, just open a terminal window on your host and ssh into your new system, e.g.
$ ssh  wwadmin@192.168.122.233

(where of course you need to use the actual ip address of your guest WeBWorK server).

Now login to your server using the password "wwadmin".

Expand the disk drive

You can do most of this from the graphical Virtual Machine Manager (select the machine, then Edit, Virtual Machine Details). Below are commands to this from the command line. I think you have to actually expand the disk from the command line but you can get information and add cpu's and/or memory from the graphical Virtual Machine Manager.

I'm assume the name of your WeBWorK guest is wwserver and it is Shutoff. First find the location of the disk. Run the command 

$ sudo virsh domblklist wwserver
[sudo] password for wwadmin: <wwadmin password>

and you will see something like 

 Target   Source
-------------------------------------------------------------------------
 sda      /var/lib/libvirt/images/WW2.15_Ubuntu20.04_Server-disk1.qcow2

Now get some info about the disk 

$ sudo qemu-img info /var/lib/libvirt/images/WW2.15_Ubuntu20.04_Server-disk1.qcow2
[sudo] password for wwadmin: <wwadmin password>

and you will see something like 

image: /var/lib/libvirt/images/WW2.15_Ubuntu20.04_Server-disk1.qcow2
file format: qcow2
virtual size: 12 GiB (12884901888 bytes)
disk size: 6.76 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

and now lets resize it to 20 GB by adding 8 GB 

sudo qemu-img resize /var/lib/libvirt/images/WW2.15_Ubuntu20.04_Server-disk1.qcow2 +8G
[sudo] password for wwadmin: <wwadmin password>
Image resized.

You still have to repartition the disk and expand the file system on you guest. For that see Increase disk space above.

Continue the Installation

Return to the Accessing Your Server from a Terminal Emulator on your Host instructions above.

Debugging Networking Issues

If after reading the information above on networking you are still having problems, maybe the information below will be of help.

One method

There are probably easier and better ways to debug, but the following worked for me. I imported the WeBWorK ova into VirtualBox 6 running on a Windows 10 Pro host. I will refer to my WeBWorK guest server as the WW guest. Networking (using a NAT Network) did not work. The symptoms: 

$ ip address show

does not return an ip address and the WW guest can not access the outside world.

In VirtualBox I built another guest from the ubuntu-20.04-live-server-amd64.iso using a NAT Network. Here networking works. I will refer to this system as the UB guest and I compared the two along with a lot of googling about the problem. I found that in the UB guest the information given by 

sudo lshw -C network
[sudo] password for wwadmin: <wwadmin password>

specifically the "logical name" and "serial" (which is the MAC address) agreed with the information in the files 

/etc/netplan/00-installer-config.yaml

and 

/etc/cloud/cloud.cfg.d/50-curtin-networking.cfg

BUT in the WW guest the information did not agree. This led to the information given in the Networking section of VirtualBox 6 running on a Windows 10 Pro host above.

Ports on your WeBWorK guest

Run the command 

$ sudo lsof -i -P -n | grep LISTEN
[sudo] password for wwadmin: <wwadmin password>

and you see something like 

systemd-r  697 systemd-resolve   13u  IPv4  19596      0t0  TCP 127.0.0.53:53 (LISTEN)
sshd       772            root    3u  IPv4  21834      0t0  TCP *:22 (LISTEN)
sshd       772            root    4u  IPv6  21845      0t0  TCP *:22 (LISTEN)
lighttpd   810        www-data    4u  IPv4  22509      0t0  TCP *:8080 (LISTEN)
mysqld     856           mysql   31u  IPv6  23312      0t0  TCP *:33060 (LISTEN)
mysqld     856           mysql   33u  IPv4  23654      0t0  TCP 127.0.0.1:3306 (LISTEN)
Rserve     865      rserveuser    3u  IPv4  22878      0t0  TCP 127.0.0.1:6311 (LISTEN)
/usr/sbin  946            root    4u  IPv6  21273      0t0  TCP *:80 (LISTEN)
/usr/sbin  956        www-data    4u  IPv6  21273      0t0  TCP *:80 (LISTEN)
/usr/sbin  957        www-data    4u  IPv6  21273      0t0  TCP *:80 (LISTEN)
/usr/sbin  958        www-data    4u  IPv6  21273      0t0  TCP *:80 (LISTEN)
/usr/sbin  959        www-data    4u  IPv6  21273      0t0  TCP *:80 (LISTEN)
/usr/sbin  960        www-data    4u  IPv6  21273      0t0  TCP *:80 (LISTEN)

Ports on your Windows 10 Pro host

Open a Power Shell command window as an administrator and run the command (it can take awhile) 

PS C:\> netstat -ab

Active Connections

 Proto  Local Address          Foreign Address        State
 TCP    0.0.0.0:135            desktop:0              LISTENING
 RpcSs
[svchost.exe]
 TCP    0.0.0.0:443            desktop:0              LISTENING
[vmware-hostd.exe]
 TCP    0.0.0.0:445            desktop:0              LISTENING
Can not obtain ownership information
 TCP    0.0.0.0:903            desktop:0              LISTENING
[vmware-authd.exe]
 TCP    0.0.0.0:913            desktop:0              LISTENING
[vmware-authd.exe]
 TCP    0.0.0.0:2869           desktop:0              LISTENING
Can not obtain ownership information
 TCP    0.0.0.0:3306           desktop:0              LISTENING
[mysqld.exe]
  ...
 TCP    0.0.0.0:6000           desktop:0              LISTENING
[XWin_MobaX.exe]
 TCP    0.0.0.0:49664          desktop:0              LISTENING
...
 TCP    127.0.0.1:2222         desktop:0              LISTENING
[VirtualBoxVM.exe]
 TCP    127.0.0.1:2222         desktop:52916          ESTABLISHED
[VirtualBoxVM.exe]
 TCP    127.0.0.1:2222         desktop:52917          ESTABLISHED
[VirtualBoxVM.exe]
 TCP    127.0.0.1:4443         desktop:0              LISTENING
[VirtualBoxVM.exe]
...

Ports on your Linux host

Run the command 

$ sudo lsof -i -P -n | grep LISTEN
[sudo] password for ****:

and you should see something like the following 

systemd-r   436 systemd-resolve   13u  IPv4  18620      0t0  TCP 127.0.0.53:53 (LISTEN)
vmware-au  1284            root   10u  IPv6  33012      0t0  TCP *:902 (LISTEN)
vmware-au  1284            root   11u  IPv4  33013      0t0  TCP *:902 (LISTEN)
sshd       8786            root    3u  IPv4 114131      0t0  TCP *:22 (LISTEN)
sshd       8786            root    4u  IPv6 114133      0t0  TCP *:22 (LISTEN)
cupsd     12124            root    6u  IPv6 138503      0t0  TCP [::1]:631 (LISTEN)
cupsd     12124            root    7u  IPv4 138504      0t0  TCP 127.0.0.1:631 (LISTEN)
VirtualBo 17065         wwadmin   48u  IPv4 185297      0t0  TCP 127.0.0.1:8081 (LISTEN)
VirtualBo 17065         wwadmin   49u  IPv4 185298      0t0  TCP 127.0.0.1:4443 (LISTEN)
VirtualBo 17065         wwadmin   50u  IPv4 185299      0t0  TCP 127.0.0.1:8080 (LISTEN)
VirtualBo 17065         wwadmin   51u  IPv4 185300      0t0  TCP 127.0.0.1:2222 (LISTEN)

Notice that port forwarding for VirtualBox has been set up correctly.


-- Main.ArnoldPizer - 10 May 2020

Retrieved from "https://webwork.maa.org/mediawiki_new/index.php?title=Installing_from_WW2.15_Ubuntu20.04_Server_Virtual_Machine_Image&oldid=19550"