UsingWW: Gavin LaRose - img mode errors/"not enough space"

$inactiveTopic$ img mode errors/"not enough space"

topic started 12/9/2004; 10:42:56 AM
last post 12/10/2004; 7:31:59 AM

$user$ Gavin LaRose - img mode errors/"not enough space" $blueArrow$
12/9/2004; 10:42:56 AM (reads: 1091, responses: 4)

Hi all,

I'm using WeBWorK-2.0.1 on apache 1.3.29 with mod_perl 1.29, running on a Solaris 8 box with two processors and 8GB of memory. The WeBWorK installation is on a partition that's about 30GB big. I've set the system up to use image mode for previewing and equation display. And I have two classes with a total of about 1500 students.

I've now had two occasions where image display and previewing (in image mode) start failing. The error message that shows up when viewing a problem is the following:

Warning messages
    * cd /opt/www/webwork2/webwork-2.0.1/tmp/ImageGenerator.feYu2Ve5
      && [latex command] equation > latex.out 2> latex.err
      returned non-zero status -1: Not enough space at
      /opt/www/webwork2/pg-2.0/lib/WeBWorK/PG/ImageGenerator.pm 
        line 340.
[some messages snipped]
    * Warning: something's wrong at
      /opt/www/webwork2/pg-2.0/lib/WeBWorK/PG/ImageGenerator.pm 
        line 345.
    * Unable to read logfile
      /.../tmp/ImageGenerator.feYu2Ve5/equation.log at 
      /opt/www/webwork2/pg-2.0/lib/WeBWorK/PG/ImageGenerator.pm 
        line 351.
[other similar messages snipped]

At first sight I thought "drat, I filled up the filesystem." But that doesn't seem to be a problem. df reported that I had 25GB free, and on the order of 3 million inodes available for use. When I got these errors before I restarted the webserver, that didn't help, and so I cleaned out some files and things started working again. I was also working pretty fast, so I didn't stop to figure out what happened exactly. I was a little more careful this time.

What I did: I cleaned out the webwork WeBWorK_home/tmp directory, and deleted all files in WeBWorK_home/htdocs/tmp/equations that were over two weeks old. This increased the amount of space available in the filesystem to almost 28GB and the number of free inodes by about 33,000. I was still getting the errors, however, but after restarting the webserver they went away.

Does anyone have insight on what's going on here? Has anyone seen similar errors? Has anyone used image mode for this number of (or more) students?

Thanks,
Gavin

<| Post or View Comments |>

$user$ John Jones - Re: img mode errors/ $blueArrow$
12/9/2004; 1:19:48 PM (reads: 1273, responses: 0)

Hi Gavin,

We have more than that number of students on at least one of our systems and we haven't had problems of this sort. It is curious that improvement doesn't come until you restart the web server. My first guess would have been that filehandles aren't being closed by the continuously running mod_perl processes, but df isn't being told about it (this is a guess).

In apache/conf/httpd.conf, you might check MaxRequestsPerChild. If it is large, then the httpd processes will be running longer. So, if long running httpd processes are part of the problem, you might want to adjust this variable. We have it set to 50.

The other thing I would do is wait for things to go bad, have webwork leave image related files behind, and then go in by hand and try to run the relevant latex/dvipng commands to see if more clues are given in a shell window.

John

<| Post or View Comments |>

$user$ Sam Hathaway - Re: img mode errors/ $blueArrow$
12/9/2004; 3:06:12 PM (reads: 1253, responses: 0)

Gavin,

You might want to try running lsof if you run into this again. That would tell you about any dangling file descriptors.

Sam

<| Post or View Comments |>

$user$ Davide P. Cervone - Re: img mode errors/ $blueArrow$
12/9/2004; 3:45:28 PM (reads: 1262, responses: 0)

Gavin:

You might want to check to see if disk quotas are enabled, and if the web server or MySQL server has a limit set.

Davide

<| Post or View Comments |>

$user$ Gavin LaRose - Re: img mode errors/ $blueArrow$
12/10/2004; 7:31:59 AM (reads: 1287, responses: 0)

Hi all,

Thanks for the very useful comments. I'll check with lsof if it happens again. /etc/httpd.conf has MaxRequestsPerChild set to 0 (infinite requests), which they note might be a problem for Solaris, but at the same time I trust the systems guys who set up the apache install that we're using. So I'm not sure if I think that's a problem or not. I may set it to something high and see if that makes a difference in the long term (by which I mean next semester; at this point it's unlikely we'll see the problem again this term).

Disk quotas aren't an issue, but that's something I would have likely not thought about. I've set all limits on MySQL very high.

Best,
Gavin

<| Post or View Comments |>

Using WeBWorK

Forum archive 2000-2006

Gavin LaRose - img mode errors/"not enough space"

Gavin LaRose - img mode errors/"not enough space"