Installation

Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
Number of replies: 35

Hello everyone.

Due to a Perl vulnerability for 5.20.2, this is a good opportunity for us to look at upgrading our WW 2.16 to the latest WW 2.18 and also Perl to version 5.38.2. We are running MySQL 5.7.

Our current version of WW is 2.16 running on RHEL 8.2 with Perl 5.20.2 and MySQL 5.7 database.


I tried to do an in-place upgrade and to my surprise, i did get it going to point but it took several weeks and encountered a lot of problems along he way and the LTI 1.3 just does not work with Moodle... but it was a good try and learning opportunity!

Anyway, I think a better approach is to build a new server with the following key requirements:

1. Redhat 8.2 (we are RedHat place)

2. Perl 5.38.2 installed via perlbrew

3. MySQL 8.0

4. WeBWork 2.18 (as per install doc for 2.18 but for Ubuntu)

I want to ask the community of expert people like you on what is best and least amount of problems that I should consider when I do this approach to upgrading WW 2.16 to WW 2.18.

This is my idea/approach.

1. Build that new server and install the perl, packages and WW 2.18 as per the docs

2. Install the MySQL 8.0

3. Backup the existing 5.7 MySQL and upgrade it to MySQL 8 on the new server

4. Modify the site.conf file to connect to the upgraded database

Questions:

1. Would my simplistic plan work out?

2. Should I keep MySQL at 5.7 or upgrade it to MySQL 8.x? Pros and Cons?

3. If I upgrade the DB, would there be schema changes in both the database and in WW 2.18 that may cause errors that it won't work?

4. Any other tips you can provide is helpful

Thank you

Lawrence





In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Danny Glin -

I was able to get WeBWorK running in RHEL8 with the packed-in version of perl (5.26).  I'm trying to avoid perlbrew since it adds another level of complexity to the installation.  All the necessary perl packages were available either from the dnf repositories or CPAN.  IIRC I had to install Mojolicious from CPAN rather than dnf because the version offered by RHEL wasn't new enough to work with some of the plugins.

RHEL doesn't offer some of the necessary LaTeX packages, so I installed texlive directly according to the instructions at https://webwork.maa.org/wiki/Installation_Manual_for_2.17_on_Oracle_(and_related)_Linux#LaTeX.  I got rid of the "--no-interaction" option so I could choose a smaller installation since the full installation takes a lot of disk space and only a couple of packages are needed.

It's worth noting that MariaDB is more widely used with WeBWorK than MySQL.  Either one should work, but you're likely to get better help with MariaDB.

In terms of the DB upgrade, my experience is that WeBWorK is rather indifferent to the version of MariaDB/MySQL.  Most likely you should be able to do a mysqldump of the DB on your old 5.7 server and import it into your new 8.0 server.  You will also have to copy the WeBWorK courses folder from your old server to your new server if you want to migrate your existing content.  Once you get everything up and running you will need to upgrade all the courses from the admin course.

In reply to Danny Glin

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
This is really good information Danny.

I was thinking if the Perl that came with it would work and it look like it does.

How did you install cpan or cpanminus? I will use the latest one from their site by wgrt instead of using the one that came with red hat maybe

Not sure if Perl 5.26 is happy with our security team but I’ll ask

Also what is iirc?

Upgrading courses is just through the interface or via command line if I remember .. thanks for the reminder
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Danny Glin -

I installed cpanminus from the repository (dnf install perl-App-cpanminus).

IIRC = If I recall correctly

You will probably have to use the script to upgrade the admin course (upgrade_admin_db.pl).  Once you're able to log into the admin course you can upgrade the other courses from there.

In reply to Danny Glin

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -

I have  test server with 2.18 and I want to start fresh again.

How or what is the best way to remove or uninstall WW, Perl, etc... everything on it and start from the beginning and use the RHEL packages instead?

or should I still try to add an EPEL repo and use their Perl instead?

L


In reply to Danny Glin

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -

Hey @Danny

How did you install both these 2: 

1. libapreq2 

2. perl-libapreq2

have a hard time trying to install this

I believe I was able to use/install as default perl 5.32.1 from the Rhel repo and not use 5.26

I want to use the Oracle docs for 2.17 release as a base to install WW 2.18 for RedHat

thanks

In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -

You don't need either of those for WW 2.18.

In reply to Glenn Rice

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -

did not know that... what about mod_perl?

In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -

WW 2.18 does not use mod_perl.  It uses Mojolicious instead.  You can install it with cpanm.

In reply to Glenn Rice

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -

great... I just installed mod_perl and will remove it

In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
Which option 1 or 2 is the one that most people implement when the time has come to choose "Serving the Webwork2 Mojolicious App"

We don's use NginX so option 3 is not considered.

Either

Option 1: Serving Directly via Hypnotoad

or

Option 2: Serving via Proxy by Apache2 (in my case just Apache)

Which one is "easier" to troubleshoot without knowing much about the application.

Pros and cons of each option?

@Glen Rice, which was your setup? Option 1 or Option 2?

thanks
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Alex Jordan -

FWIW, I ran into trouble trying to serve using hypnotoad on a server where apache needed to be running for a different application. There was some kind of port clash that maybe could have been resolved, but I just moved to option 2 in that case, so apache was serving both (as a proxy for WW).

On other servers, I use option 1. I just find it simpler to know that apache is not running and is not involved if there is some issue to investigate.

In reply to Alex Jordan

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -

so if this is a dedicated WW server, then best to use Option 1 I suspect?

In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -

Yeah, if your server is only serving WeBWorK, then Option 1 is best.  However, option 2 is perhaps easier if you have installed WeBWorK previously as it is most like what you would be familiar with.

I do not recommend using Apache instead of Apache2.  I would be surprised if Apache does not give security concerns.  It is very old and not developed anymore.

In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
I am almost there but am unable to install these 5 in my redhat 8.x

1. cm-super

2. libossp-uuid-perl

3. libuniversal-can-perl

4. libuniversal-isa-perl

5. pdf2svg

How do I install these 5 in redhat?

By the way, I had to install mod_perl because if I did not and installed SOAP:Lite, it is looking for my source Apache to make/build! ... it was looking for the apache-x.x/src folder
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
any ideas? I can't install these and don't want to continue until I get these resolve... I think the cm-super is a font type and not a package correct?

but the rest, it cannot be found even in cpan as say OSSP:uuid, etc .. no such package they said
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -
cm-super is not strictly necessary. It is a TeX font package. If it is not present then problems that use fonts from that package won't work in hardcopy. I don't remember what uses that though. I actually don't have that installed on my server, and haven't seen issues.

For libossp-uuid-perl, install Data::UUID from cpan.

For libuniversal-can-perl, install UNIVERSAL::can from cpan.

For libuniversal-isa-perl, install UNIVERSAL::isa from cpan.

Instead of installing pdf2svg, install dvisvgm. Make sure to set $pg{specialPGEnvironmentVars}{latexImageSVGMethod} = "dvisvgm"; in localOverrides.conf.  Note that you do NOT need both pdf2svg and dvisvgm.  You just need one.  The default in defaults.conf is set to "pdf2svg", and to avoid giving all of the details we just say to install both in the installation manual.  We really need to change the default to be dvisvgm though.  It is better.
In reply to Glenn Rice

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
hey Glen.. I tried those 3 and it said it can't find them .. I tried different variations of case sensitivity combos,// nothing worked :-(

thanks for the dvisvgm tip
In reply to Glenn Rice

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -

interesting... now it worked foir the 3 of them!


thanks!

In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
Almost there...

I want to check the status by going to our site at http://ourserver.site.com/webwork2 but got this error.. how can I troubleshoot this on redhat 8.9?

Service Unavailable
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

I have a textlive error as well and will put that as a separate thread
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
when I tried to do check_latex, it gave me this error

[me@myserver bin]$ check_latex

 Error in tempdir() using check_latex_XXXXXXXX: Could not create directory check_latex_XhCGeGdA: Permission denied at /opt/data/web.webwork/webwork2/bin/check_latex line 45.

i change the bin directory to be writeable and ran it again and this time is says

[me@myserver bin]$ check_latex
sh: /usr/bin/pdflatex: No such file or directory

[me@myserver bin]$ which pdflatex

/usr/local/texlive/2023/bin/x86_64-linux/pdflatex

[me@myserver bin]$  echo $PATH

/usr/local/texlive/2023/bin/x86_64-linux:/home/adm_lng/perl5/bin:/home/adm_lng/.local/bin:/home/adm_lng/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/webwork/webwork2/bin
It is already in my path so why can't it see it? and still using the /usr/bin/pdflatex that does not exist?

should I just do a symbolink to it  /usr/local/texlive/2023/bin/x86_64-linux/pdflatex 

and like that it now works


In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -

You need to change $externalPrograms{pdflatex} in site.conf to be correct for your system.

In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
trying to start it and got this error

queue
Loaded: loaded (/opt/webwork/webwork2/conf/webwork2-job-queue.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2024-01-24 11:45:55 PST; 4s ago
Process: 330545 ExecStart=/opt/webwork/webwork2/bin/webwork2 minion worker -m production (code=exited, status=217/USER)
Main PID: 330545 (code=exited, status=217/USER)

Jan 24 11:45:55 cclsw33 systemd[1]: Started webwork2 job queue.
Jan 24 11:45:55 cclsw33 systemd[1]: webwork2-job-queue.service: Main process exited, code=exited, status=217/USER
Jan 24 11:45:55 cclsw33 systemd[1]: webwork2-job-queue.service: Failed with result 'exit-code'.


sudo systemctl enable /opt/webwork/webwork2/conf/webwork2.service
Failed to enable unit: Unit file /opt/webwork/webwork2/conf/webwork2.service does not exist.


I figured it out,,, I forgot to copy the dist file and changed the group and owner




In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
In Library Browser, "Open Problem Library" button is inactive and an error message is shown:

"You are missing the directory templates/Library, which is needed for the Problem Library to function. It should be a link pointing to /opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary, which you set in conf/site.conf. An attempt was made to create the link, but that failed. Check the permissions in your templates directory."

I think the settings are correct.. how can I start to troubleshoot this?


In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
I also ran "/OPL-update"

and if I remember correctly in 2.16, I added much more problem libraries.. should it not show them onscreen when I run that command?

I remember a script that I ran on the cli and it took time because it was updating the OPL'S
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -
The OPL-update script no longer takes a long time. It doesn't scan the library directory anymore. It just downloads the database directly from GitHub (built in a workflow). That never added other libraries though. If you added other libraries, you did so in localOverrides.conf.

You need to do as the error message showed you, and fix the permissions. There are directions on how to do this in the installation instructions. See https://webwork.maa.org/wiki/Installation_Manual_for_2.18_on_Ubuntu#Set_File_and_Directory_Permissions.
In reply to Glenn Rice

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -

Yes I read that and made no difference and sane error comes up


You are missing the directory templates/Library, which is needed for the Problem Library to function. It should be a link pointing to /opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary, which you set in conf/site.conf. An attempt was made to create the link, but that failed. Check the permissions in your templates directory."


The first 2 sentences maybe the key not permissions?

Just to confirm, which directories should be owned by whom? Mine is red hat so it should be Apache.apache I believe


And what should the permission be? Want to confirm

I even tried to do a chmod 777 on that directory and still has that error


In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -
Yes, the OPL-update script will not fix the error about the templates/Library link being missing.

No, the permissions are the issue. The first 2 sentences about the link being missing is what is causing you to receive the message. However, webwork2 tries to create that link and fails because the permissions on the templates directory are not correct. The permissions not being correct on the templates directory are going to cause you a lot more problems later if not fixed now.

Run

sudo chgrp -R www-data /opt/webwork/courses

and

sudo chmod -R g+w /opt/webwork/courses
Make sure to change www-data in the first command to be the user that the webwork2 app is being run by on your server.  That may be apache on a Redhat system.  This will ensure that all permissions for all files and directories in the courses directory are correct.
In reply to Glenn Rice

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
I believe I did do those 2 commands and replaced www-data with apache.

"...You are missing the directory templates/Library..."

where does this "templates/library" reside under?

under /courses I am assuming?

[me@myserver webwork2]$ cd /opt/webwork/webwork2/
[me@myserver webwork2]$ pwd
/opt/webwork/webwork2
[me@myserver webwork2]$ sudo chgrp -R apache DATA /opt/webwork/courses
[me@myserver webwork2]$ sudo chmod -R g+w DATA /opt/webwork/courses
[me@myserver webwork2]$ ls -al DATA
total 4
drwxrwsrwx.  3 apache apache   21 Jan 23 12:46 .
drwxr-xr-x. 17 apache apache 4096 Jan 23 20:47 ..
drwxrwsrwx.  2 apache apache   20 Jan 23 12:46 uploads

[me@myserver webwork]$ pwd
/opt/webwork
[me@myserver webwork]$ ls -al
total 12
drwxr-xr-x.  6 apache apache   64 Jan 23 12:46 .
drwxr-xr-x.  7 root   root     78 Jan 19 23:37 ..
drwxrwsr-x. 60 apache apache 4096 Jan 24 21:05 courses
drwxr-xr-x.  3 apache apache   42 Jan 23 12:47 libraries
drwxr-xr-x. 14 apache apache 4096 Jan 23 12:46 pg
drwxr-xr-x. 17 apache apache 4096 Jan 23 20:47 webwork2


Just tried it again after this and same problem exist.. should it be owned by root?
the folder "/opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary" exists..
I think it is the creation of the "templates/Library" may be missing and how to implement it




In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -
Execute
ls -l /opt/webwork/course/courseId
where cousreId is the course that you are working with.
What does that show?

Note that the /opt/webwork/webwork2/DATA directory is not related to this at all.

The courses directory definitely should not be owned by root. It should be owned by the same user that the webwork2 app is being run as which definitely should not be root.
In reply to Glenn Rice

OPL-update question; also course names

by Peter Lert -

Hi Glenn,

I am supporting the WW system at my institution - and trying to learn how to do so. I can acquire root privileges on the WW server, running in Ubuntu. Based on your comments, I *think* that I can update the OPL on my WW with the following:

# runuser -g wwadmin www-data
$ cd /opt/webwork/libraries/webwork-open-problem-library
$ OPL-update

(In other words, and contrary to the wiki info at https://webwork.maa.org/wiki/Open_Problem_Library, there is no need for a "git pull" before the last command). Is this correct? Alternatively, should I just run OPL-update as root and then chown -R wwadmin:www-data  /opt/webwork/libraries/webwork-open-problem-library?

On a completely different note, I'd like to be able to assign the Course Title and Institution for our courses via shell scripts rather than using the Course Admin tool for each course. So far I have been unable to find the settings necessary to do so. I was hoping that a couple of statements inserted into each course.conf file would suffice. Alternatively, maybe there's a wwdb incantation to accomplish the same. Can you give me any clues?

Thanks for all your support to the community!



In reply to Peter Lert

Re: OPL-update question; also course names

by Glenn Rice -
If you are using Ubuntu, then you should look at the Ubuntu installation manual for WeBWorK and the release notes.

Note that you should run the OPL-update script as your usual user, not as the root user.  You should not need the runuser command that you gave, and generally you should not become root.  Commands that need root permissions should be run via sudo.  See the note at the end of https://webwork.maa.org/wiki/Installation_Manual_for_2.18_on_Ubuntu#Notation.

In fact, if you have added /opt/bwebwork/webwork2/bin to your PATH as the installation instructions tell you to do, then you can just run OPL-update without your cd command also.

In any case, there is no need for "git pull".  The OPL-update script takes care of fetching the latest OPL release.

The course title and institution are assigned to a course when it is created.  There is a script for creating a course.  It is /opt/webwork/webwork2/bin/addcourse.  However, it does not have options for setting the title and institution.  You could modify it to do so.  One thing to note is that the course institution is actually never used by webwork.  It is never shown in the user interface, and never used for anything.  You can set it when you create a course, or you can change it in the admin course via the add course page, but that is as far as it goes.   So unless you are using it for something via scripts or in some other way, I wouldn't concern yourself with that.

You can't set the course institution or title from the course.conf file since those are database settings.  You could use the wwsh script to change those settings.  The wwdb script is rather limited on what it can do, and is also rather out of date.  I am not even sure it works anymore.
In reply to Glenn Rice

Re: OPL-update question; also course names

by Peter Lert -
Thanks so much, Glenn! I'll run the OPL-update at the end of the term, just in case it might create a problem now that we are underway, and look into how to set the course title with wwsh. I also appreciate your kind and well deserved scolding re root access - I been lazy. Thanks again.
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Lorenzo Ng -
Another question I have is upgrading the course and the best way of doing it.

Ivclicked on Upgrade Courses and got a bunch of courses and selected upgrade

Upgradev: WeBWork-Experience-TNG-Test sql_single Database tables need updating.



I followed them all and when I returned to Upgrade Courses and they are still there.. should they now be upgraded? why do they still show up?




Attachment Capture2.PNG
In reply to Lorenzo Ng

Re: Best Approach upgrade 2.16 to 2.18 on RHEL 8.2 & MySQL 5.7

by Glenn Rice -

That is the correct way to upgrade courses.  The upgrade has successfully completed.  The course will still show up because you have some columns that are no longer in the schema.  You can delete those by checking the checkbox by "Select/unselect all fields missing in schema for deletion."  That is perfectly safe to do.  Those columns are not needed anymore, and won't have any data in them that you need to keep.