UsingWW: File Manager, upload error

I recently installed WebWork using these instructions:

http://webwork.maa.org/wiki/Installation_Manual_for_2.4_on_Ubuntu_10.04

on a fully patched Ubuntu 10.04.3 machine (64-bit). Everything went smoothly and everything seems to be working except uploading files using the filemanager. A pdf of the error page is attached.

The permissions on the directories all seem to be consistent with the installation instructions. Also, if I put a file into a course directory (using terminal access to the WW server), I am able to delete it using the filemanager. This suggests that the web server has read/write permissions to the relevant directories.

My conjecture is that wherever WebWork is trying to create the tmpfile mentioned in the attached error page, that directory does not have the correct permissions. Can someone tell me where WW is trying to create this file? Thanks,

Albert Schueller

upload_error_page.pdf

Re: File Manager, upload error

by Jason Aubrey - Monday, 15 August 2011, 4:22 PM

Hi Albert,

The location used for temporary storage of uploaded files is set in global.conf, and unless you changed it for some reason it will be:

$webworkDirs{uploadCache} = "$webworkDirs{DATA}/uploads";

So, you should find that directory at /opt/webwork/webwork2/DATA/uploads

The permissions on DATA are set with

cd /opt/webwork/webwork2/
chgrp -R wwdata DATA ../courses htdocs/tmp logs tmp
chmod -R g+w DATA ../courses htdocs/tmp logs tmp
find DATA/ ../courses/ htdocs/tmp logs/ tmp/ -type d -a ! \( -name .svn -prune \) -exec chmod g+s {} \;

And, I think the crucial part for the DATA/uploads directory is the "-exec chmod g+s {}" which ensures that subdirectories of DATA/ inherit the same group ownership and permissions as DATA.

Hope this helps,
Jason

Re: File Manager, upload error

by Albert Schueller - Monday, 15 August 2011, 5:09 PM

So I'm still getting the same error. Rather than search through all of the directories in the commands above, I just executed them as root from the command line, one at a time. Each ran cleanly--no errors. In addition,

root@math:/opt/webwork/webwork2# id www-data
uid=33(www-data) gid=33(www-data) groups=33(www-data),1002(wwdata)

indicates that the web server is a member of the wwdata group. The following process listing:

root@math:/opt/webwork/webwork2# ps aux | grep www-data
www-data 1446 0.0 0.2 98180 11240 ? S Aug11 0:00 /usr/bin/perl /usr/sbin/apt-cacher -R 3 -d -p /var/run/apt-cacher.pid
www-data 6444 0.0 1.3 233352 53584 ? S 09:25 0:00 /usr/sbin/apache2 -k start
www-data 10929 0.0 2.1 294668 86552 ? S Aug14 0:00 /usr/sbin/apache2 -k start
www-data 10930 0.0 2.1 292536 84536 ? S Aug14 0:00 /usr/sbin/apache2 -k start
www-data 10933 0.0 2.1 293180 85196 ? S Aug14 0:00 /usr/sbin/apache2 -k start
www-data 11335 0.0 1.3 233100 53060 ? S 14:03 0:00 /usr/sbin/apache2 -k start
root 11424 0.0 0.0 7628 1024 pts/2 S+ 14:07 0:00 grep --color=auto www-data
www-data 13367 0.0 2.1 292992 84904 ? S Aug14 0:00 /usr/sbin/apache2 -k start
www-data 13369 0.0 2.0 291920 83968 ? S Aug14 0:00 /usr/sbin/apache2 -k start
www-data 17033 0.0 2.1 293260 84972 ? S Aug14 0:00 /usr/sbin/apache2 -k start
www-data 17035 0.0 2.0 291448 83424 ? S Aug14 0:00 /usr/sbin/apache2 -k start
www-data 17265 0.0 1.3 233352 53612 ? S Aug14 0:00 /usr/sbin/apache2 -k start

shows that the web server is running as www-data. Is it possible the WW scripts are suid'ing to some other user? (Also, I've confirmed that the default "uploads" dir in global.conf is as you suggest, i.e. $webworkDirs{uploadCache} = "$webworkDirs{DATA}/uploads";)

Albert

Re: File Manager, upload error

by Jason Aubrey - Tuesday, 16 August 2011, 9:22 AM

Hmm, this is a mystery (to me). Here are some things I find strange:

1) the call stack never mentions FileManager.pm or Upload.pm,
2) the call stack mentions WeBWorK::ContentGenerator::loginstatus
3) your error message "CGI open of tmpfile: Permission denied" comes directly from CGI.pm.

So, here are some questions:
1) Just to confirm, what are the owner, group and permissions of the DATA and DATA/uploads directories (e.g. ls -l output)
2) do you see any related error messages in the apache error log?
3) what version of CGI.pm do you have, and in the file /usr/share/perl/5.10/CGI.pm is the line '$CGITempFile::TMPDIRECTORY = '/usr/tmp';' commented out? (It should be.)
4) What kind of file are you trying to upload?

You might also try this: In webwork2/lib/WeBWorK/Constants.pm, set
$WeBWorK::Debug::Enabled = 1 and $WeBWorK::Debug::LogFile to a log file you can tail and try uploading the file again.

If there are nearby debug(""); statements in the code, that might give some better clue of where things are breaking, if not, then maybe you can put them in near where things are going wrong to help track down the problem.

Re: File Manager, upload error

by Albert Schueller - Tuesday, 16 August 2011, 2:00 PM

Attached are a pair of screen shots (looks like I can only attach one file to a post, second file will be in a followup) first showing the landing page for the filemanager link and then after browsing for and selecting a file for upload. At this point, if I click the "Upload" button, I get the aforementioned error page.

Answers to Questions:

1) Perms:

root@math:/opt/webwork/webwork2# ls -l
total 48
drwxr-xr-x 4 wwadmin wwadmin 4096 2011-07-13 13:28 bin
drwxr-xr-x 3 wwadmin wwadmin 4096 2011-07-13 13:28 clients
drwxr-xr-x 5 wwadmin wwadmin 4096 2011-08-15 14:15 conf
drwxr-xr-x 4 wwadmin wwadmin 4096 2011-07-13 13:28 courses.dist
drwxrwsr-x 4 wwadmin wwdata 4096 2011-07-13 13:28 DATA
drwxr-xr-x 5 wwadmin wwadmin 4096 2011-07-13 13:28 doc
drwxr-xr-x 13 wwadmin wwadmin 4096 2011-07-13 13:28 htdocs
drwxr-xr-x 8 wwadmin wwadmin 4096 2011-07-13 13:28 lib
-rw-r--r-- 1 wwadmin wwadmin 1531 2011-07-13 13:28 LICENSE
drwxrwsr-x 3 wwadmin wwdata 4096 2011-08-14 12:48 logs
-rw-r--r-- 1 wwadmin wwadmin 544 2011-07-13 13:28 README
drwxrwsr-x 3 wwadmin wwdata 4096 2011-08-15 14:24 tmp

root@math:/opt/webwork/webwork2/DATA# ls -l
total 4
drwxrwsr-x 3 wwadmin wwdata 4096 2011-08-15 14:03 uploads

2) Apache2 error log:

Use of uninitialized value $string in substitution (s///) at /opt/webwork/webwork2/lib/Apache/WeBWorK.pm line 321.
[Tue Aug 16 10:32:25 2011] [error] [client 10.33.1.144] [/webwork2/FALL_2011_126_C/instructor/file_manager/] CGI open of tmpfile: Permission denied\n * in CGI::read_multipart called at line 553 of /usr/share/perl/5.10/CGI.pm\n * in CGI::init called at line 370 of /usr/share/perl/5.10/CGI.pm\n * in CGI::new called at line 32 of /opt/webwork/webwork2/lib/WeBWorK/CGI.pm\n * in WeBWorK::CGI::new called at line 466 of /usr/share/perl/5.10/CGI.pm\n * in CGI::self_or_default called at line 3 of (eval 1203)\n * in CGI::br called at line 842 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm\n * in WeBWorK::ContentGenerator::loginstatus called at line 152 of /opt/webwork/webwork2/lib/WeBWorK/Template.pm\n * in WeBWorK::Template::template called at line 491 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm\n * in WeBWorK::ContentGenerator::content called at line 195 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm\n * in WeBWorK::ContentGenerator::go called at line 353 of /opt/webwork/webwork2/lib/WeBWorK.pm, referer: http://math.whitman.edu/webwork2/FALL_2011_126_C/instructor/file_manager/?effectiveUser=admin&user=admin&key=E6OppoM1EWih97PBqE4kmFiLVN9nyRfc

3) The Perl version is 5.10.1, the version string from the top of CGI.pm is:

$CGI::revision = '$Id: CGI.pm,v 1.263 2009/02/11 16:56:37 lstein Exp $';
$CGI::VERSION='3.43';

and the line:

# $CGITempFile::TMPDIRECTORY = '/usr/tmp';

is commented out.

4) The file is a WW generated exported homework .def file.

Tried turning on debugging, with logfile /tmp/upload.log, but no file was created. I can try harder to figure out why that didn't work if it'll help. In general, I really appreciate your helping with this. Even without the upload capability, the system is pretty functional. It's just that I think I'd have an easier time selling it to my dept colleagues if this worked (since it appears to be the de facto way of moving homework assignments from one class section to another).

Re: File Manager, upload error

by Albert Schueller - Tuesday, 16 August 2011, 2:01 PM

The second screen shot.

Re: File Manager, upload error

by Bill Farr - Tuesday, 16 August 2011, 3:03 PM

Albert,

A simple question. What is the group that your httpd process is

running as? It looks to me like the owner of the process is

www-data, which is a member of the wwdata group, but this

may not provide the proper permissions if the group is not

set appropriately in httpd.conf. What value are you using for

Group in httpd.conf?

Regards,

Bill

Re: File Manager, upload error

by Albert Schueller - Tuesday, 16 August 2011, 5:32 PM

Here's the relevant line from /etc/apache2/apache2.conf (which I'm pretty sure is playing the role of httpd.conf on my system):

# These need to be set in /etc/apache2/envvars
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

And from /etc/apache2/envvars:

export APACHE_RUN_USER=www-data
export APACHE_RUN_GROUP=www-data

So it looks like it's running as a member of the www-data group.

Albert

Re: File Manager, upload error

by Bill Farr - Tuesday, 16 August 2011, 6:13 PM

Albert,

The DATA subdirectory is owned by wwadmin:wwdata. I think the group

for your server should be wwdata, not www-data.

Bill

Re: File Manager, upload error

by Albert Schueller - Tuesday, 16 August 2011, 8:49 PM

Given that the web server is running as www-data and that userid is listed as a member of wwdata, I think it should be ok. I'm really hesitant to change the web server's group to wwdata since it does more than just serve up the WebWork pages. It also serves up dept pages for example. Chances are your suggestion wouldn't mess up any of the other tasks, but it doesn't seem to be the "correct" solution. Thanks.

Re: File Manager, upload error

by Jason Aubrey - Wednesday, 17 August 2011, 4:24 PM

Well, it might be worth a try when you think nobody will notice.

Also, is there anything in that directory DATA/uploads? If so, it might help to debug this if we knew the permissions on those files.

If that directory is non-empty, maybe deleting the current contents of that directory might fix the problem (but first note the permissions in case it doesn't help).

Re: File Manager, upload error

by Albert Schueller - Wednesday, 17 August 2011, 5:32 PM

I tried my upload test again and a tmp file appears in DATA/uploads that is a copy of the file I'm trying to upload:

-rw-r--r-- 1 www-data wwdata 698 2011-08-17 14:15 ee2474c1-7b71-423a-955d-8f320de5a960.file

So the error comes when the WW script tries to copy the tmp file to the final destination. It's puzzling that the tmp file is 644, but the apache2 error log claims that the script doesn't have permission to open it.

I tried deleting all the files in the DATA/uploads, but the error persists.

Re: File Manager, upload error

by Jason Aubrey - Friday, 19 August 2011, 9:39 AM

Well, I'm stumped. Those are the same permissions as on temp files in my correctly working installation.

The final destination will be

/opt/courses/<course name>/templates

What are the permissions on that directory?

Also, I notice you have two courses on that server now - do you get the same error in the other course?

Jason

Re: File Manager, upload error

by Albert Schueller - Friday, 19 August 2011, 5:23 PM

Permissions on the second section (same as the first):

root@math:/opt/webwork/courses# ls -ld FALL_2011_126_D/
drwxrwsr-x 7 www-data wwdata 4096 2011-08-13 14:18 FALL_2011_126_D/
root@math:/opt/webwork/courses# ls -ld FALL_2011_126_D/templates/
drwxrwsr-x 9 www-data wwdata 4096 2011-08-13 14:54 FALL_2011_126_D/templates/

Tried uploading in the second section (D) and get the same error. Thanks for your efforts on this, I understand if you want to walk away from this. I'll be sure and post something if I ever figure it out.

Re: File Manager, upload error

by Arnold Pizer - Saturday, 20 August 2011, 10:36 AM

Hi Albert,

From everything I can see it looks like your permissions are set correctly. Apache2 is also set up correctly:

export APACHE_RUN_USER=www-data

export APACHE_RUN_GROUP=www-data

So I don't know what might be the trouble. If you have a spare laptop or old machine (or a empty partition on one), you could download and install from the LiveDVD --- it only takes 7 quick steps. That will give you Ubuntu 10.04.2 and you can check that uploads work. Then upgrade to 10.04.3 and check again. If things still work, I guess you can then try to see what is different on your server.

http://webwork.maa.org/wiki/Installing_from_WW2.4.9_Ubuntu10.04.2LTS_Vanilla_LiveDVD

The LiveDVD is at the address above.

Arnie