WeBWorK Main Forum

Intermittent "Failed to initialize course environment" error

Intermittent "Failed to initialize course environment" error

by Brett Hendricks -
Number of replies: 7
The following message has started showing up in our error_log (a student brought the problem to my attention when he had trouble logging on, and this was the error message he forwarded):

Failed to initialize course environment: Could not evaluate global environment file /opt/webwork/webwork2/conf/global.conf: Can't locate object method "get" via package "APR::Table" at /opt/webwork/webwork2/lib/Apache/WeBWorK.pm line 74.

By the error_log entries (only 16 in the last 4 days -- not a lot, I know, but it would be nice to solve the problem rather than just live with it and perhaps find out later that it was a harbinger of worse to come. Also, it's the only error we're getting on the system.) it seems to have happened on login, logout, while an instructor was accessing setmaker, and just viewing problems. Redoing a request or refreshing clears the error, so the problem is still at the level of "user annoyance."

We are running RHEL 5 with RedHat's build of Apache 2.2.3. 1 GB RAM. Only serving 6 courses. perl-status shows ARP::Table is loaded.
APR::Table 0.009000 Fri Sep 19 04:30:28 2008 /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/APR/Table.pm

httpd.conf tweaks are
Timeout 300
KeepAlive On
MaxKeepAliveRequests 25
KeepAliveTimeout 5
StartServers 5
MaxSpareServers 10
MaxClients 8
MaxRequestsPerChild 100

Any ideas?
In reply to Brett Hendricks

Re: Intermittent "Failed to initialize course environment" error

by Michael Gage -
My first step would be to reboot the server -- in fact I would actually stop the server, make sure that all of the apache children have died and then restart the server. (Using the "graceful" command won't always stop a hung child.)

Intermittent behavior such as you describe occurs most often when there is one child process which is bad while the others are working properly. Every so often your request is served by the bad process.

I think that APR::Table is one of the modules installed with mod_perl2 -- you might make sure that it can be found by the server somewhere in the perl library -- if the problem persists after a restart.

-- Mike

In reply to Michael Gage

Re: Intermittent "Failed to initialize course environment" error

by Aleksey Nogin -
Mike,

I did a full restart and made sure there were no hanged children processes after that. The issue temporarily disappeared after restart, but than came back (a day or so later?). I restarted apache again last night and have not seen the problem since than - yet.

Aleksey
In reply to Michael Gage

Re: Intermittent "Failed to initialize course environment" error

by Aleksey Nogin -
Sorry, I got two different "intermittent error" threads confused. Please ignore my previous post.

Aleksey
In reply to Michael Gage

Re: Intermittent "Failed to initialize course environment" error

by Fuzzy Rogers -
I'm getting these every 100th request or so. Wondering if anyone ever found a fix? My other server uses apache 1.3 and does not exhibit this.

FWIW- CentOS 5.4, Apache/2.2.3, mod_perl2 2.000004, WW 2.4.7

Error:
"Error messages

Failed to initialize course environment: Could not evaluate global environment file /opt/webwork/webwork2/conf/global.conf: Can't locate object method "get" via package "APR::Table" at /opt/webwork/webwork2/lib/Apache/WeBWorK.pm line 74."

Thanks-
Fuzzy
In reply to Fuzzy Rogers

Re: Intermittent "Failed to initialize course environment" error

by Danny Glin -
Based on the evidence it seems to be a Redhat problem of some sort. I'm running Scientific Linux 5.5 (another RHEL clone) and I get these every once in a while. The upgrade to 5.5 from 5.3 seemed to reduce the problem greatly, as I used to get them much more frequently.
Sometimes restarting apache helps.
Sorry I don't have a real solution.
Danny
In reply to Danny Glin

Re: Intermittent "Failed to initialize course environment" error

by Gavin LaRose -
Hi all,

We've seen the APR::Table error sporadically, and have found it to be chronic and page load dependent. We've found in every case that if we reload the page the error will go away. Our hypothesis is that it's probably a thread initialization problem, and that it shows up when the apache process hands the WeBWorK job off to a new thread that doesn't have all of the required Perl loaded before the it tries to execute the WeBWorK request. That said, we don't know much about thread management in apache, so we don't know if that's even credible.

We're RHEL5 as well.

Gavin
In reply to Gavin LaRose

Re: Intermittent "Failed to initialize course environment" error

by Fuzzy Rogers -

Until we move or upgrade servers our best solution so far is to shutdown httpd for a few seconds and restart it daily around 4am. (service httpd stop , sleep 20, service httpd start)
That has significantly reduced the number of errors received on a per day basis. Currently this particular server is handling 2300 students and I'm seeing less than 20 of these errors per day (homework due M,W,F).
Take care,
Fuzzy