WeBWorK Main Forum

Library browser stopped working - error

Library browser stopped working - error

by Lars Jensen -
Number of replies: 20

Dear Colleagues,

Suddenly my library browser stopped working and produces the error below when I click "View problems." I don't get it, because I have done no major updates since I installed 2.15. I did check my modules, and it looks like Statistics::R::IO is not installed, but that is not installed on our test server either, and the library browser on the test server works fine....

Any ideas?

Sincerely,
Lars.

Warning messages

  • Couldn't find the OPL global statistics table. Did you download the latest OPL and run load-OPL-global-statistics.pl? at /opt/webwork/webwork2/lib/WeBWorK/Utils/LibraryStats.pm line 95.

Error messages

Table 'webwork.OPL_global_statistics' doesn't exist at /opt/webwork/webwork2/lib/WeBWorK/Utils/LibraryStats.pm line 97.

Call stack

The information below can help locate the source of the problem.

  • in WeBWorK::Utils::LibraryStats::getGlobalStats called at line 1089 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator/Instructor/SetMaker.pm
  • in WeBWorK::ContentGenerator::Instructor::SetMaker::make_data_row called at line 1729 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator/Instructor/SetMaker.pm
  • in WeBWorK::ContentGenerator::Instructor::SetMaker::body called at line 155 of /opt/webwork/webwork2/lib/WeBWorK/Template.pm
  • in WeBWorK::Template::template called at line 610 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm
  • in WeBWorK::ContentGenerator::content called at line 233 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm
  • in WeBWorK::ContentGenerator::go called at line 386 of /opt/webwork/webwork2/lib/WeBWorK.pm

In reply to Lars Jensen

Re: Library browser stopped working - error

by Lars Jensen -

....and I tried to run load-OPL-global-statistics.pl as suggested elsewhere, but when I do, it complains that:

Access denied for user 'webworkWrite'@'localhost' (using password: YES)

Lars.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Glenn Rice -

You might check that you have the correct database password for the webworkWrite user set in site.conf.

That is the message you get if the password is incorrect.

In reply to Glenn Rice

Re: Library browser stopped working - error

by Lars Jensen -

Hi Glenn,

My password for webworkWrite as listed in site.conf allows me to login to mariadb from he command line, so that is not the problem, apparently. 

Sincerely,
Lars.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Danny Glin -

I'll start with a couple of things I always check when errors appear despite having made no changes on the server:

  1. Restart apache.
  2. Check if the disk is full.

It's amazing how often one of these things is the problem.

Have you verified whether the global statistics table actually exists in the DB?  If it does, then it indicates that there is something wrong with the database interaction.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Glenn Rice -

Yeah, if it works from the command line, that shouldn't be the problem.

I noticed that you said you tried to run the "load-OPL-global-statistics.pl" script.  There is no script by that name that I know of.  Did you try "update-OPL-statistics.pl"?  Or did you just type the wrong script name in the earlier comment.

In reply to Glenn Rice

Re: Library browser stopped working - error

by Lars Jensen -

Hi Again,

Thanks for the pointers. Unfortunately they didn't help. So I ran the update-OPL-statistics.pl script followed by an OPL-update, and nothing changed. Scripts ran fine. No errors, but still the same error message as in my original post when I enter a course and click "View problems" in the Library browser....

I also tried running restore-OPL-tables.pl, but to no avail.

What is the load-OPL-global-statistics.pl for? It won't run. The webworkWrite password is correct, but it still complains about the password.

Disk space on my system is not the issue either.

Would it help to do a database restore?

And what could have caused this? My test server is essentially identical in its setup, and works fine.

Sincerely,
Lars.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Glenn Rice -

Okay, so I gave you the wrong script to run.  There is a mess of scripts that needs to be cleaned up which apparently caused confusion to me also.  The script that you need to run is "OPL-update".

In reply to Glenn Rice

Re: Library browser stopped working - error

by Glenn Rice -
If running that script gives you the "Access denied for user ..." message, then you may have a permissions issue with the webworkWrite user, or else you have database corruption.

Try running "show GRANTS for 'webworkWrite'@'localhost';" and see that the permissions match with those shown at https://webwork.maa.org/wiki/Installation_Manual_for_2.15_on_Ubuntu_20.04_Server#Set_up_the_webwork_database.

You might also try just deleting the webworkWrite mysql user, and recreating it according to the directions at the above site.
In reply to Glenn Rice

Re: Library browser stopped working - error

by Arnold Pizer -
Hi Lars,

https://webwork.maa.org/wiki/Installation_Manual_for_2.15_on_Ubuntu_20.04_Server#Initialize_the_OPL_local_statistics_database_table
and
https://webwork.maa.org/wiki/OPL_Problem_Statistics
might provide some useful information.

Also you might want to update the OPL (https://webwork.maa.org/wiki/Installation_Manual_for_2.15_on_Ubuntu_20.04_Server#Updating_the_OPL). Maybe somehow the file OPL_global_statistics.sql which is distributed with the OPL got corrupted.

It seems like you managed overcome your problem with the password for the webworkWrite.
In reply to Lars Jensen

Re: Library browser stopped working - error

by Danny Glin -

In site.conf, check that $database_dsn is in the following format:

$database_dsn = "dbi:mysql:webwork:[hostname]";

where [hostname] is the name or IP address of your database server.

Most other aspects of WeBWorK can handle the alternate format ("dbi:mysql:database=webwork;host=[hostname]"), but the load-OPL-global-statistics script depends on the specific format given above.

In reply to Danny Glin

Re: Library browser stopped working - error

by Lars Jensen -

Hi Everyone,

Thanks for all the suggestions. No luck so far. This issue seems to have to do with new 2.15 changes because I've never had this issue before...

We run 2.15. In the link Arnie provided, it states that,

"Changes planned for WeBWorK-2.15 will require running load-OPL-global-statistics to load the updated global statistics data instead of update-OPL-statistics which was used in older versions and did additional unrelated work."

Well, my problem is that I get a password error for webworkWrite when I try to run load-OPL-global-statistics. (I know the webworkWrite password is OK, as I can login from command line, and also student scores are properly recorded in the database.) Also, the update-OPL-statistics  runs fine.

Arnie, you also ask:
It seems like you managed overcome your problem with the password for the webworkWrite. 
Well, I don't think I had a problem. Perhaps the problem is with the load-OPL-global-statistics script, which won't take my webworkWrite password....

I have also updated the problem database as Arnie suggested.

I have run OPL-update multiple times, I have rebooted the system, and I checked Danny's suggestion. We don't have a dedicated database server, and the line Danny refers to reads: $database_dsn ="dbi:mysql:webwork";

As for Glenn's suggestion, show GRANTS for 'webworkWrite'@'localhost'; gives this reply, which, I think, is correct:

MariaDB [(none)]> show GRANTS for 'webworkWrite'@'localhost';

+---------------------------------------------------------------------------------------------------------------------+

| Grants for webworkWrite@localhost                                                                                   |

+---------------------------------------------------------------------------------------------------------------------+

| GRANT USAGE ON *.* TO 'webworkWrite'@'localhost' IDENTIFIED BY PASSWORD '*52FE49CDA4D3042E0443621A6443517526A21F6D' |

| GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER, LOCK TABLES ON `webwork`.* TO 'webworkWrite'@'localhost' |

+---------------------------------------------------------------------------------------------------------------------+

So i'm not sure what to do. I do have a webwork database backup....but does that contain the OPL global statistics table that I need? Should I do that - it seems that there should be a better way???

Sincerely,
Lars.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Lars Jensen -

Hi Everyone,

The Library Browser is now working again. Based on Arnie's suggestion, I played around with $problemLibrary{showLibraryGlobalStats} and $problemLibrary{showLibraryLocalStats}. Setting the former to 0 made the Library Browser work again. Thanks, Arnie.

So the conclusion seems to be that there is a bug in the scripts somewhere....Perhaps in load-OPL-global-statistics ??

Sincerely,
Lars.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Glenn Rice -
I am not sure that is an entirely valid conclusion. The script works fine for me and others. It isn't a particularly involved script. It is only 43 lines of code if you ignore empty lines and comments (82 lines total). The code is rather straight forward.

It is more likely that you have something in your configuration that is conflicting. The fact that you are getting the access denied message is a pretty clear indicator of this.
In reply to Glenn Rice

Re: Library browser stopped working - error

by Lars Jensen -

Hi Glenn,

What setting could that be?

Does the load-OPL-global-statistics.pl work for you, or are you talking about upload-OPL-statistics.pl? 

The reason I ask is that you said earlier in this thread that you don't recognize a script by the  name load-OPL-global-statistics, and you suggested using upload-OPL-statistics.pl. They both produce a webworkWrite password error for me that we know to be fake.... 

To contribute to the confusion, there is also another script called update-OPL-statistics.pl, which runs fine, but did nothing to fix my library browser problem.

Also, why would the load-OPL-global-statistics.pl give a password error when we know the password is correct? 

Is there a direct way to check the integrity of the OPL-global-statistics file?

Sincerely,

Lars.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Glenn Rice -

I am not really certain what is causing the problem for you.  All of the scripts that have been mentioned here work for me.

Note that if you run OPL-update it actually runs the other two as well if you have either of $problemLibrary{showLibraryLocalStats} or $problemLibrary{showLibraryGlobalStats} set to 1.  Well, actually it is supposed to run both of them.  There is a typo in OPL-update though, so it does not run update-OPL-statistics.pl (it is missing the .pl in the filename), but it does run load-OPL-global-statistics.pl.

Just out of curiosity, do you happen to have a file name .my.cnf in your user's home directory?  If so does it have the webworkWrite user in it, and does the password there match the password in your site.conf?

In reply to Glenn Rice

Re: Library browser stopped working - error

by Lars Jensen -

Hi Glenn,

Thanks for your reply. No, I don't have a .my.cnf file in my home directory.

What else does the setting of $problemLibrary{showLibraryGlobalStats} impact? It must have an impact on "View problems" in the library browser, because if it is set it to 0 I can view problems. If it is set to 1, I cannot...

Could a non-standard character in the webworkWrite password be an issue? I have a multiplication sign "*" in it.

Sincerely,
Lars.

In reply to Lars Jensen

Re: Library browser stopped working - error

by Lars Jensen -
Hi Glenn, et.al,

The culprit, it turned out, was special characters in the webworkWrite password. As soon as I changed the password to alphanumeric characters, load-OPL-global-statistics.pl ran without errors, and the Library Browser worked normally again. I think this should be considered a bug.

Sincerely,
Lars
In reply to Lars Jensen

Re: Library browser stopped working - error

by Alex Jordan -

I was loosely following this thread. I just checked, and as it happens, the database password on my server also has an asterisk. Also one other non-alphanumeric character. I tried as below, with the same result as Lars:

:/opt/webwork/webwork2/bin$ ./load-OPL-global-statistics.pl
ERROR 1045 (28000): Access denied for user 'webworkWrite'@'localhost' (using password: YES)

So then I looked to see if the Library Browser was having the same issues when browsing the OPL, and yes, it was. I get a regular WeBWorK error screen with this error, as reported:

Couldn't find the OPL global statistics table. Did you download the latest OPL and run load-OPL-global-statistics.pl? at /opt/webwork/webwork2/lib/WeBWorK/Utils/LibraryStats.pm line 95.

There are over 70 active courses right now, but I guess no one has used the Library Browser since this was introduced. (It was probably only introduced last week when I moved to the develop branch plus Glenn's problem grader).

I manually undid the change from commit 00d4f3017df3d65da87ab909419f7eac1956a5f6, and then the script runs. (Although with a warning "mysql: [Warning] Using a password on the command line interface can be insecure.")

So something about that change does not work as intended when the password is like mine and Lars'. But I don't understand what arcane perl aspect would cause the issue here.
In reply to Alex Jordan

Re: Library browser stopped working - error

by Glenn Rice -

So you were correct Lars.  This was a bug after all.  I have put in a pull request (https://github.com/openwebwork/webwork2/pull/1189) that fixes this issue.  The problem is that the database password was being shell quoted.  Previously this was needed as the password was used in a shell command.  After the change in the commit that Alex referenced it no longer is used in the shell command, and doing the shell quoting causes problems if the database password contains characters that are shell escaped (like *).

In reply to Glenn Rice

Re: Library browser stopped working - error

by Arnold Pizer -

Just as an aside, Glen writes "There is a typo in OPL-update though, so it does not run update-OPL-statistics.pl (it is missing the .pl in the filename), but it does run load-OPL-global-statistics.pl."  This explains why in the installation instructions I had to add a section  https://webwork.maa.org/wiki/Installation_Manual_for_2.15_on_Ubuntu_20.04_Server#Initialize_the_OPL_local_statistics_database_table to run the command update-OPL-statistics.pl separately.  I never tried to trace down what had changed to cause that to be necessary.  I'll have to try to remember that I should remove that for 2.16.

Also OPL statistics are no longer being collected as far as I know.   It would be a good idea to setup a server to hold the data and start collecting that data again.  The current data is pretty old.


Arnie