WeBWorK Main Forum

archiving sets as def files

archiving sets as def files

by Nandor Sieben -
Number of replies: 7
A perl script to archive every problem set of every course as a def file.
This might be useful for transferring courses between servers.
The script creates a def_files subdirectory containing a directory
for each course with def files. You need to edit "mysql.school.edu"
and "mypassword".


# archiveSetsAsDefFiles 
# Archives all problem sets as def files  
# Author: Nandor Sieben  

sub mysql {
my ($sql) = @_;
qx[mysql -s -u webworkWrite --host=mysql.school.edu -pmypassword webwork << EOF 

`mkdir def_files`;
@courses= mysql(q[show tables like '%_set';]);

for $course (@courses) {
  $course =~ s/_set.*\n//;
  next if $course =~ /admin/;
  next if $course =~ /problem_library/;
  print "$course\n";
  `mkdir def_files/$course`;
  @sets=mysql(qq[select set_id from $course\_set;]);
  for $set (@sets) {
#    print "$set"; 
    chop $set;
    open F, ">def_files/$course/set$set.def";
    $problems=mysql(qq[select source_file, value, max_attempts from $course\_problem WHERE set_id = '$set' ;]);
    $problems =~ s/\t/,\t/g;
    print F " 
openDate          = 01/01/2011 at 01:00am MST 
dueDate           = 01/01/2011 at 01:00am MST 
answerDate        = 01/01/2011 at 01:00am MST 
paperHeaderFile   = 
screenHeaderFile  = 
problemList       =  
    print F $problems;
    close F;

In reply to Nandor Sieben

Re: archiving sets as def files

by Gavin LaRose -
Hi Nandor,

This looks like something that is worth adding this to the contributed scripts for managing large installations in the Admin section of the wiki ( http://webwork.maa.org/wiki/Contributed_Admin_Scripts_(Large_Installations) ). I've taken the liberty of doing that; if you have concerns let me know (or just delete it).

In reply to Nandor Sieben

Re: archiving sets as def files

by Dick Lane -
Thanks, Nandor, for this useful tool.
Thanks, Gavin, for adding it to the wiki.

Perhaps one or two variants can provide some options.

1)  Header file(s):  current default is to use Arnie's ASimple... files; some instructors may have written header files specific to some of their assignments.  That info is in the database.
a)  Extracting it and including it in the exported .def would be useful.
b)  OTOH: handling any actual course-specific header files may be best handled by the current archive-course procedure.

2)  special customizations: The email and macros subdirectories might have some course-specific stuff; the course.conf file (one level up from the templates directory) might also have some customizations.  OTOH: perhaps an instructor knowledgeable enough to use that flexibility should inherit the responsibility to archive it (hence this type of script might ignore that ;-).

PS: I suggest Nandor's showActive.sh script also be added to the wiki.  (If I'm about to update the OS in a way that will require rebooting the machine, I like to do that when no students are logged-in.)
In reply to Dick Lane

Re: archiving sets as def files

by Nandor Sieben -
The issue with my script and also with the showActive script is that they require the database password. I am pretty sure this could be done automatically using the webwork config files. Could someone help me with this.
In reply to Nandor Sieben

Re: archiving sets as def files

by Nandor Sieben -
Please tell me if this is a horrible thing to do. What is the better way?


@config= `cat $ENV{WEBWORK_ROOT}/conf/global.conf`;

for $line (@config) {
  if ($line =~ /\$database_username.*=.*"(.+)"/) {
    $database_username = $1;
  if ($line =~ /\$database_password.*=.*"(.+)"/) {
    $database_password = $1;

print "$database_username,$database_password\n";

In reply to Nandor Sieben

Re: archiving sets as def files

by Jason Aubrey -
Hi Nandor,

You can have a look at webwork2/bin/wwsh and webwork2/bin/NPL-update for how those scripts get access to the database. What you have isn't horrible, but it's probably more fragile. For example, in the most recent code, the database password is in a config file called prelocal.conf.

In reply to Nandor Sieben

Re: archiving sets as def files

by Kurt O'Hearn -
This is a tangential note to the discussion, but I found a bug in the showActive script. The SQL command

qx[mysql -s -u $database_username -p$database_password webwork << EOF

should be

qx[mysql -s -u "$database_username" -p"$database_password" webwork << EOF

to prevent shell variable interpolation of metacharacters (e.g., '&').