Installation

Error encountered when upgrading courses

Error encountered when upgrading courses

by Dave Meng -
Number of replies: 17
Hi,

Our school system administrator helped to upgrade our webwork from 2.5 to 2.10.The operating system now is ubuntu 14.04.3 running on a virtual machine.  After I ran upgrade_admin_db.pl, I was able to get into the admin course. When I ran Upgrade courses, I had the error messages like the following with lots of the courses:

 000-engr120-Watson sql_single Directory structure or permissions need to be repaired. Database tables need updating.

Your modelCourse in the 'courses' directory is out of date or missing. Please update it from webwork/webwork2/courses.dist directory before upgrading the other courses. Cannot find MathAchievements directory /opt/webwork/courses/modelCourse/templates/achievements nor MathAchievements picture directory /opt/webwork/courses/modelCourse/html/achievements

how do I update database tales?

Also, from the command line, when I run git pull origin, I got the following error message

fatal: Not a git repository (or any of the parent directories): .git

What did I miss/

Thank you,

Dave Meng


In reply to Dave Meng

Re: Error encountered when upgrading courses

by Michael Gage -
This page will help some.  Since you have upgraded from 2.5 you might want to look at more than one of the release notes.

you need to run git commands from either the webwork2 directory or 
the pg directory (or a subdirectory of those).

Try running   git branch  
in one of those directories -- it will show you which branches (probably master) that you have.  If this doesn't work then check with your systems 
administrator to make sure that webwork has been installed from the github.com/openwebwork site.

To upgrade modelCourse  first move your courses/modelCourse directory  to some other directory then copy the directory webwork/webwork2/courses.dist/modelCourse into your webwork/courses directory. That will probably get rid of the error message you report. 

To upgrade courses you go to the    ...../webwork2/admin course using your webbrowser and choose the tab marked "upgrade courses".  

Hope that helps.

In reply to Michael Gage

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Micheal,

Thank you for your help.
I did what you said. I renamed  my courses/modelCourse directory  to some other names then copy the directory webwork/webwork2/courses.dist/modelCourse into my webwork/courses directory. Then I ran "upgrade courses". The original error message is gone. But I still got the the following message: "Database tables need updating". When I tried to log into a specific course, I still saw error messages.
I am attaching the error messages in a file. 

I was reading the release notes. Since we jumped directly from 2.5 to 2.10, it seems that we missed quite a few things. Do we need to go through 2.6, then 2.7, .. to 2.10? Is there a guideline how to upgrade from 2.5 to 2.10?

I figured out the git error. I was running the command outside the repository directory.

Thanks.

Dave


In reply to Dave Meng

Re: Error encountered when upgrading courses

by Alex Jordan -
The specific error message indicates that the modelCourse course is missing some folders. If you add those folders to all existing courses, then perhaps the update tool in the admin course will work for updating the data tables. (And perhaps it already has been able to do that part, and it's just the missing folders that it is objecting to.)

For instance, you can:
cp -r /opt/webwork/webwork2/courses.dist/modelCourse/templates/achievements /opt/webwork/courses/modelCourse/templates/achievements

to copy the contents of the achievements folder that comes with the distribution over to the modelCourse in the courses folder. Then you may want to check that the file permissions are in order for what you just copied over. Then do the same with .../modelCourse/html/achievements.

Do this for each course that has the specific error message mentioned above. That is, copy the two achievements folders (one in templates, one in html) from
/opt/webwork/webwork2/courses.dist/modelCourse
over to
/opt/webwork/courses/<each course that needs it>
and then if needed, set the permissions.

Again, once all this is in order, see if the admin course's update tool tells you that all of the database tables are in order.


If you are in
/opt/webwork/
or in
/opt/webwork/courses/
or any subfolders of
/opt/webwork/courses/

then you are not in a git repository, and that would explain the git message you got. There are four separate git repos (at present) for a WeBWorK installation. They are at

/opt/webwork/webwork2/
(the main reconrd-keeping and web interface engine)

/opt/webwork/pg/
(the problem processing engine)

/opt/webwork/libraries/webwork-open-problem-library/
(the collection of problems in the OPL and their taxonomy)

/opt/webwork/mathjax
(javascript rendering of math content)

When you ran git pull origin, which of these were you trying to update? You may want to visit each one, then:

git status
(check that you are on master branch)

git fetch origin
(retrieve all updates from GitHub without yet applying them)

git status
(see how you compare to master from GitHub, which currently means 2.10)

git pull
(apply any differences that your branch has with GitHub)
In reply to Alex Jordan

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Alex,

Thank you for your help.

I copied the files like you said into the respective directories. But it did not work.  As a matter of fact, I checked and found that each course already contained the directory "achievements". (maybe because I have ready run "upgrade courses"?). When I log into each course, I still saw error messages. I posted the error message in my reply to Michael above.

For the git pull error, I figured out my mistake. I was running the command outside the repository directory.

Thanks,

Dave


In reply to Dave Meng

Re: Error encountered when upgrading courses

by Michael Gage -
Hi Dave,

Have you rerun the "upgrade courses" process from the admin course since you made the change to modelCourses?

One other thing -- since you upgraded from 2.5 you may still have a database.conf file in your webwork2/conf directory -- and it is possibly out 
of date with the current database.conf.dist. 

Because very few sites customize their database.conf files we have made it so that webwork will read the database.conf.dist file directly unless there is a database.conf file present.  

I think that you can move your database.conf file out of the webwork2/conf directory (for safe keeping) and leave only the database.conf.dist file.  

then rerun the "upgrade courses" process from the web admin course and see what happens.

-- Mike
In reply to Michael Gage

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Michael,

I have run "upgrade courses" many times now, no luck.
We did not have database.conf in that directory. There is only one database.conf.dist file. However, when I took a peek of the file, the comment file is like the following. Does this indicate that this is an old file?

Thanks,

Dave

----------------------------------------------------------------------------------------------
#!perl
################################################################################
# WeBWorK Online Homework Delivery System
# Copyright ▒ 2000-2007 The WeBWorK Project, http://openwebwork.sf.net/
# $CVSHeader: webwork2/conf/database.conf.dist,v 1.38 2007/08/13 22:59:51 sh002i Exp $
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of either: (a) the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version, or (b) the "Artistic License" which comes with this package.
#
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE.  See either the GNU General Public License or the
# Artistic License for more details.
################################################################################

In reply to Dave Meng

Re: Error encountered when upgrading courses

by Michael Gage -
I don't think that means its out of date.  We're just a bit lazy about updating copyright on all the files.

The error message you sent me earlier says that it is missing a column "displayMode"  -- probably in the user or user_setting table. 

When you upgrade the course in which this error message appears do you end up with a message on the admin page in which everything is shown as green?  Sometimes people don't push the button on the the admin display which actually updates the course. (The page displays what will be updated, but it doesn't actually update the database until you push the button and go to the next page -- and then it will recheck to verify that it has upgraded everything. )  If you have properly upgraded everything should be green and the button should say "done" or something like that.


In reply to Michael Gage

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Michael,

The error messages appear when I tried to log into each individual course.
On the admin page, I saw either one of following two messages in red by each course except the admin course, which has a green "Database table ok".
"Database tables need updating." or "Directory structure or permissions need to be repaired. Database tables need updating".
I looked at the courses with those two different error messages. They seem to have the same kind of permission. Alex also mentioned above that I might need to change permissions. I am not sure what I need to change.
At the end, I saw this warming message: "Harmless: 000-engr120-Watson_set has extra column &#39" for all the courses with red error message.
and the following error message:
------------------------------------------------------------------------

Error messages

DBD::mysql::db do failed: Incorrect table name 'old-modelCourse-math242-pre201112-02winter_global_user_achievement' at /opt/webwork/webwork2/lib/WeBWorK/DB/Schema/NewSQL/Std.pm line 826.

Call stack

The information below can help locate the source of the problem.

  • in WeBWorK::DB::Schema::NewSQL::Std::handle_error called at line 297 of /opt/webwork/webwork2/lib/WeBWorK/DB/Schema/NewSQL/Std.pm
  • in WeBWorK::DB::Schema::NewSQL::Std::tableFieldExists called at line 222 of /opt/webwork/webwork2/lib/WeBWorK/Utils/CourseIntegrityCheck.pm
  • in WeBWorK::Utils::CourseIntegrityCheck::checkTableFields called at line 110 of /opt/webwork/webwork2/lib/WeBWorK/Utils/CourseIntegrityCheck.pm
  • in WeBWorK::Utils::CourseIntegrityCheck::checkCourseTables called at line 2187 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator/CourseAdmin.pm
  • in WeBWorK::ContentGenerator::CourseAdmin::upgrade_course_form called at line 450 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator/CourseAdmin.pm
  • in WeBWorK::ContentGenerator::CourseAdmin::body called at line 155 of /opt/webwork/webwork2/lib/WeBWorK/Template.pm
  • in WeBWorK::Template::template called at line 530 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm
  • in WeBWorK::ContentGenerator::content called at line 376 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator/CourseAdmin.pm
  • in WeBWorK::ContentGenerator::CourseAdmin::content called at line 202 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm
  • in WeBWorK::ContentGenerator::go called at line 380 of /opt/webwork/webwork2/lib/WeBWorK.pm
  • -----------------------------------------------------------------
In reply to Dave Meng

Re: Error encountered when upgrading courses

by Michael Gage -
The upgrading process on the admin page is not succeeding. The common reason for that is that people forget to move on to the last page, but there could be others.  The "harmless" error messages are just that -- they alert you that you have old courses in your database that don't have the standard table structure.
The extra "published" column doesn't get in the way -- it's just ignored.


The error message about the incorrect table name may be what is causing the process to fail without completing the upgrade process.

You might be able to work around it somewhat by trying to upgrade just one or two courses -- that might avoid running into the bad course.
I think however that you'll need systems admin help to access the mysql database to clear out the bad table and perhaps also direct access to the webwork/courses directory to remove some tables there.  It looks like someone may have saved an old copy of the modelCourse in the courses directory and the upgrade process is hanging when it tries to upgrade it.  It should be moved out of the courses directory.
In reply to Michael Gage

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Michael,

After our system administrator moved a few old modelCourse files out of the courses directory, the "Upgrade courses" functions correctly. It successfully updated all the tables. Thank you so much. A big relief.

Now I have more problems. With 2.5, we have NPL, now we use OPL. I downloaded OPL and installed it on our system. I edited localPverrides.conf to reflect the change(see the attached file). But when I log into admin course, I see that button name changed to "OPL Directory". No buttons for ASU, Rochester, etc. When I click on the "Open Problem Library " button, I see "subject","chapter","section" buttons. If I click those buttons, nothing shows. Do I still miss something here?  When I log into an individual course, and click the library browser, I do see those individual buttons, but I get following error message:

"You are missing the directory templates/Library, which is needed for the Problem Library to function. It should be a link pointing to/opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary, which you set in conf/site.conf. I tried to make the link for you, but that failed. Check the permissions in your templates directory"

I google searched  the error message. I saw an old post from you for someone with the same problem in old version. You mentioned to delete the Library link.  It worked. Is there a script that we can run to change all the courses?

Thanks again.

Dave


In reply to Dave Meng

Re: Error encountered when upgrading courses

by Michael Gage -
Glad you have the first problem solved.  You've pretty much identified the second problem.  The alias "Library" in each course/template directory is currently pointing to the location of the old NationalProblemLibrary and you need to make it point to the new one:  ..../webwork-open-problem-library/OpenProblemLibrary.

Deleting the link AND then visiting the library browser  will refresh the Library link properly. (Sometimes people still see error messages until they make that first click on the library browser which resets the Library link.)

I couldn't find a script but this is what I use. Put it in bin/changeLibraryLink.pl.
#!/usr/bin/env perl

# utility script
# usage:  ls  /opt/webwork/courses | changeLibraryLink.pl


while (<>) {
        next if $_ =~/tar\.gz/;
        chomp;
        ` sudo ln -sf /opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary  /opt/webwork/courses/$_/templates/Library`;
        ` sudo ln -sf /opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary/Rochester /opt/webwork/courses/$_/templates/rochesterLibrary`;
        ` sudo ln -sf /opt/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary/Union  /opt/webwork/courses/$_/templates/unionLibrary`;
         print " /opt/webwork/courses/$_/templates/Library relinked \n";

}

You will want to read and understand the script first.  It accepts each courseName and creates a new link for Library, rochesterLibrary, and unionLibrary.  You probably only want the Library link so comment out the

lines with unionLibrary and rochesterLibrary. 

ls /opt/webwork/courses |changeLibraryLink.pl  pipes every course name that is found in /op/webwork/courses into changeLibraryLink.pl which then creates the appropriate Library link in that course.


Hope this helps.



In reply to Michael Gage

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Michael,

Thanks for sharing the script. The Library link works now. I still have one more problem. Right now, each individual school library button works. But on the top row of buttons. The first one "Open Problem Library" does not work. When I press that button, I see a drop down menu: Subject, Chapter, Section. When I choose "Subject", I only get "All Subjects", nothing else. What did I miss? I installed webwork new myself, I see there should be a lot of subjects showing there when I choose subject.

Thanks,

Dave
In reply to Dave Meng

Re: Error encountered when upgrading courses

by Danny Glin -
This is a common (but probably unrelated) problem.  The most common causes for this are:
  1. $server_root_url not being properly set in site.conf
  2. Issues with SSL.
The former is easy to troubleshoot.  The latter can be a little more complicated.  If you have verified that $server_root_url is correct, then take a look at the apache error logs, and also in the javascript console in your browser.  One of these will hopefully give you a descriptive error.

This is a common enough problem that we should probably have a troubleshooting page on the wiki about it.

Danny
In reply to Danny Glin

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Danny,

I checked the $server_root_url in site.conf file. It is set correctly. I will ask our system administrator to help to check the second one.

Thank you for your help.

Dave
In reply to Dave Meng

Re: Error encountered when upgrading courses

by Danny Glin -
It sounds like some of these issues may be a result of the web server not having write permissions in the course templates directory.  This would cause the Library link creation to fail, and also "Upgrade courses" to be unable to fix the directory structure.

Try re-running the commands at the end of this section, and then visit the Library Browser again.  It should hopefully fix things for you.

Danny
In reply to Danny Glin

Re: Error encountered when upgrading courses

by Dave Meng -
Hi Danny,

Thank you for your help. When said the to re-run the commands at the end of the section, do you mean the following scripts? I checked the permission and ownership. I seems that everything is correct. I still have one more problem.
Please see my last reply to Michael above. 
 
Thanks,

Dave


$ sudo su
[sudo] password for wwadmin: <wwadmin password>
# cd /opt/webwork/webwork2/
# chgrp -R wwdata DATA ../courses htdocs/tmp htdocs/applets logs tmp /opt/webwork/pg/lib/chromatic
# chmod -R g+w DATA ../courses htdocs/tmp htdocs/applets logs tmp /opt/webwork/pg/lib/chromatic
# find DATA/ ../courses/ htdocs/tmp logs/ tmp/ -type d -a -exec chmod g+s {} \;
# exit
$
In reply to Dave Meng

update -- Re: Error encountered when upgrading courses

by Dave Meng -
Hi,

I just want to give an update. After our system administrator ran ./OPLupdate again, the "Open Library Problem" and "Advanced Search" buttons work now. Everything seems to work now. But I still don't know why it worked.
Looking back, maybe after I changed the the library links as Michael suggested, I should run ./OPLupdate?

Thank you guys for all your help. I really appreciate it.

Dave