Forum archive 2000-2006

Arnold K. Pizer - Hardware for WeBWorK

Arnold K. Pizer - Hardware for WeBWorK

by Arnold Pizer -
Number of replies: 0
inactiveTopicHardware for WeBWorK topic started 11/5/2003; 5:13:26 PM
last post 11/6/2003; 8:58:38 PM
userArnold K. Pizer - Hardware for WeBWorK  blueArrow
11/5/2003; 5:13:26 PM (reads: 1587, responses: 8)

We recently received the  following query.

Dear Prof. Gage,

I am an assistant professor in the Johns Hopkins University. We plan to
use WeBWork for some calculus courses, maybe eventually up to a scale 
of 500 to 600 students. Here we need your valuable help.

Now our IT guy wants to know how much space WeBWork may require for 
such a class, including the temporary files and data files, so that he can
properly configure the server. Will you please give us some estimates?

Thanks for your kind consideration!

Best regards,
Qiao

<| Post or View Comments |>


userArnold K. Pizer - Re: Hardware for WeBWorK  blueArrow
11/5/2003; 5:17:46 PM (reads: 1763, responses: 0)

Here's our answer:

Dear Qiao,

Mike Gage has forwarded your email to me. 

Thank you for your interest in WeBWorK. I'm sure you are aware that Johns Hopkins used WeBWorK for many years under the direction of James Martino.  Jim is still at Johns Hopkins but has moved from the math department to the Digital knowledge Center, an R&D unit in the Hopkins university library structure.  His email is jrm@jhu.edu .  You certainly should get in touch with him if you haven't already.  He can tell you what he ran the courses on at Hopkins. If you talk with Jim, say hi for Mike and I.

First you should have your IT guy look at http://webhost.math.rochester.edu/webworkdocs/discuss/msgReader$1572 which gives some important info.  Also have him look at http://webhost.math.rochester.edu/webworkdocs/discuss/msgReader$1808 which is interesting and gives more info. about people having load problems.  Note that in Arizona State's case their problem was due to bad hardware they hadn't noticed and in Dartmouth's case the problem was that someone wrote a (very bad) problem that did a tremendous amount of needless calculations.  But the comments on server configuration are still valid.

At Johns Hopkins you are not talking about many users but I would still try to use a dual processor server if possible.  Concerning disk size I'll give you some figures below but disk space is so cheap that I would certainly err on the large size. 

The basic WeBWorK system files will fit on 10 MB but if you want to turn the access log on (it's off by default) that can become huge maybe 500 MB or more.  The tmp files at Rochester need about 4-5 GB (we have twice as many users as you anticipate) --- note that we have them on a separate partition that is not backed up.  A course (with logs, DATA, many problem template) should easily fit in 50 MB but this assumes people compress the database from time to time.  Most professors never do this (it requires one click) so maybe 500 MB is a safer size.   Also professors should remove unneeded tmp files (again it requires one click) but most don't.  You can get by with a pretty small (by current standards) amount of space if you want to but it is definitely not worth the trouble.  I see Dell is charging $224.10 for a 36GB 10K RPM Ultra 320 SCSI Hard Drive and I'm sure there are cheaper venders.  That size would be more than sufficient for you. [Actually look on http://www.pricewatch.com/ where you can find good 73 GB drives for about $200].  I would give 2 GB for system files, 10 GB for tmp files (in a separate partition), and 10 GB for courses.   The system and courses don't have to be on separate partitions so you can combine them into one partition.  You can get by with a lot less but it's not worth your time to worry about it.

I hope this helps. Please fell free to contact us again.  Others may find this info useful --- would it be OK with you if I post this on http://webhost.math.rochester.edu/webworkdocs/discuss  ?

Sincerely,

Arnie

<| Post or View Comments |>


userArnold K. Pizer - Re: Hardware for WeBWorK  blueArrow
11/6/2003; 2:36:35 PM (reads: 1779, responses: 0)

Here is another sequence of queries and answers.

My name is Zed Pobre, and I am the system administrator for the
Department of Mathematics at Louisiana State University.  I have been
asked to implement a Webwork system to accommodate 2500 students for
the Fall 2004 semester, later to be expanded to handle 6000 students.
I am very much hoping that you can tell me what kind of hardware I am
going to need to provide reasonable response time for 500 to 1500
simultaneous users, and whether it would be a better idea to try to
jump directly to Webwork 2 or build the entire environment around
Webwork 1.9.

Thanks in advance for any help you can provide.  I unfortunately don't
have the budget to run my own scalability tests (though the university
has offered to pay for the final server once I give them firm specs),
so if I have to do it on my own I'm going to have to just guess.

--
System Administration <merlin@math.lsu.edu>;
Louisiana State University
Department of Mathematics

<| Post or View Comments |>


userArnold K. Pizer - Re: Hardware for WeBWorK  blueArrow
11/6/2003; 2:37:02 PM (reads: 1768, responses: 0)
Dear Zed,

I'll attempt to give you a few guide lines.  First look at http://webhost.math.rochester.edu/webworkdocs/discuss/msgReader$1928 and the especially the two links given there.  We have a list of hardware people are using at http://webhost.math.rochester.edu/webworkdocs/sites/systems/ but it is very much out of date.  We should  try to update this with current info.

There are not many places with the number of users you anticipate.  Arizona State has a lot of users and John Jones <jj@asu.edu>; can give you some info.  Irvine also has I believe a lot of users but they are or at least were using multiple servers --- you might want to contact someone there.  Indiana also has a lot of users on heavy Sun hardware. Rutgers is another place with a lot of users.  Utah also I think.

Actually the load you will experience is mostly dependent on the size of the largest class (assuming that if you have a number of large classes, they don't have assignment due at the same time).  At Rochester we have about 1300 users with a largest class of 300.  With our relatively old Dell 4600 (one of the earliest dual Xeon machines with 2 Ghz processors with 1 GB of memory I think), we no longer try to schedule assignments on different days.  I actually don't know when assignment are due since with this hardware, we have no load issues at all.  So we may be close to the lower range of your "500 to 1500 simultaneous users."

You can get the fastest Dell 4600 with dual 3 Ghz Xeons and 2 GB of memory for about $6,000. This is one fast machine and might handle the load.  For $10,000 you can get an essentially equivalent (dual 2.5 Ghz Xeons but with larger cache) Dell 6600 but this gives you the option of adding two more processors.  That ( a quad processor) would certainly handle the load.  You probably should add more disk space than the above quotes contain and maybe more memory but you can get disks for a lot cheaper than Dell's prices.  Another option might be to get a Dell 2600 with dual 3 Ghz Xeons and 2 GB of memory for $4,500 and if that didn't handle the load, then get a second one and split the courses.   I can certainly say that we have been extremely happy with the Dell hardware but you can build or buy cheaper servers.  In any case if you use Intel chips, I wouldn't get anything less than a dual Xeon machine.

I would suggest setting up WeBWorK 1.9 and then 2.0.  After setting up 1.9, 2.0 is not much more work.  2.0 is still on the cutting edge.  You can run a course simultaneously on both.

I hope this info helps.  Please fell free to contact us again.  Others may find this info useful --- would it be OK with you if I post this on http://webhost.math.rochester.edu/webworkdocs/discuss  ?

Sincerely,

Arnie

<| Post or View Comments |>


userArnold K. Pizer - Re: Hardware for WeBWorK  blueArrow
11/6/2003; 2:37:48 PM (reads: 1761, responses: 0)
On Thu, Nov 06, 2003 at 11:38:01AM -0500, Arnold Pizer wrote:
>
> You can get the fastest Dell 4600 with dual 3 Ghz Xeons and 2 GB of memory
> for about $6,000. This is one fast machine and might handle the load.  For
> $10,000 you can get an essentially equivalent (dual 2.5 Ghz Xeons but with
> larger cache) Dell 6600 but this gives you the option of adding two more
> processors.  That (a quad processor) would certainly handle the
> load.

    After checking the prices on this, it's not much more expensive to
add a second 2 processor 4600 system as it is to upgrade a 6600 to 4
processors.  I submitted specs to the chairman recommending two $7400
2650s with 3.2GHz Xeons, 4GB RAM, and 2 hard drives (RAID1), with the
idea of just adding or replacing them as distributed systems as time
goes on.

    I'll try load-testing by pushing all classes onto one server to
start with next semester, and as soon as I hear a complaint about
speed, I'll split them up between the two and mirror changed data
between the two nightly.  I don't have the infrastructure for this
currently, but I'm looking into setting up Kerberos and AFS, moving
actual storage to other machines so I can do instantaneous failover if
something happens.  Do you know anyone who has done this before?


> I hope this info helps.  Please fell free to contact us again.  Others may
> find this info useful --- would it be OK with you if I post this on

    Thanks, it gave me a starting point and a list of other people to
bug.  Feel free to post this, and if I forget to send you the results
of this by next July, feel free to send me e-mail asking.

--
System Administration <merlin@math.lsu.edu>;
Louisiana State University
Department of Mathematics

<| Post or View Comments |>


userArnold K. Pizer - Re: Hardware for WeBWorK  blueArrow
11/6/2003; 2:40:22 PM (reads: 1767, responses: 0)
<!--StartFragment -->
On Thu, Nov 06, 2003 at 11:38:01AM -0500, Arnold Pizer wrote:
>
> You can get the fastest Dell 4600 with dual 3 Ghz Xeons and 2 GB of memory
> for about $6,000. This is one fast machine and might handle the load.  For
> $10,000 you can get an essentially equivalent (dual 2.5 Ghz Xeons but with
> larger cache) Dell 6600 but this gives you the option of adding two more
> processors.  That (a quad processor) would certainly handle the
> load.

    After checking the prices on this, it's not much more expensive to
add a second 2 processor 4600 system as it is to upgrade a 6600 to 4
processors.  I submitted specs to the chairman recommending two $7400
2650s with 3.2GHz Xeons, 4GB RAM, and 2 hard drives (RAID1), with the
idea of just adding or replacing them as distributed systems as time
goes on.

I think this is a good plan. It's what I would do in your situation.  Since it's so cheap, don't skimp on hard disk space.


    I'll try load-testing by pushing all classes onto one server to
start with next semester, and as soon as I hear a complaint about
speed, I'll split them up between the two and mirror changed data
between the two nightly.

I'm not sure I totally understand this.  Since the real problem (if you have one)  will occur at the times homework is due and this is determined on a course by course basis, it would be simple to just move some courses to the second server.  There is no need to share data between courses. 

 I don't have the infrastructure for this
currently, but I'm looking into setting up Kerberos and AFS, moving
actual storage to other machines so I can do instantaneous failover if
something happens.  Do you know anyone who has done this before?

No, but I'm not really up on this. I would ask the folks at Irvine.  A couple of years ago, they set up a load balancing system with several servers serving maybe 2500 students.  I'm not sure what they are doing now but it looks like they have expanded into Physics and Chemistry (http://homework.ps.uci.edu/webwork/). The sys admin there is "Domingos R. Begalli" <drb@uci.edu>; and you could contact him.



> I hope this info helps.  Please fell free to contact us again.  Others may
> find this info useful --- would it be OK with you if I post this on
> http://webhost.math.rochester.edu/webworkdocs/discuss ?

    Thanks, it gave me a starting point and a list of other people to
bug.  Feel free to post this, and if I forget to send you the results
of this by next July, feel free to send me e-mail asking.

OK, Thanks.  Good luck with the project.

Arnie

<| Post or View Comments |>


userArnold K. Pizer - Re: Hardware for WeBWorK  blueArrow
11/6/2003; 2:53:06 PM (reads: 1757, responses: 0)
On Thu, Nov 06, 2003 at 02:30:40PM -0500, Arnold Pizer wrote:
> >    I'll try load-testing by pushing all classes onto one server to
> >start with next semester, and as soon as I hear a complaint about
> >speed, I'll split them up between the two and mirror changed data
> >between the two nightly.
>
> I'm not sure I totally understand this.  Since the real problem (if you
> have one)  will occur at the times homework is due and this is determined
> on a course by course basis, it would be simple to just move some courses
> to the second server.  There is no need to share data between courses.

    For failover, if necessary.  If one of the machines explodes, I'd
like to be able to have the second one pick up the slack with as
little downtime as possible.  Especially noting that the department
wants to use the system for testing as well as homework, there will be
a collective heart attack if half of our courses drop out the morning
before a test.
 
[...]
    Thanks.

--
System Administration <merlin@math.lsu.edu>;
Louisiana State University
Department of Mathematics

<| Post or View Comments |>


userJohn Jones - Re: Hardware for WeBWorK  blueArrow
11/6/2003; 4:20:43 PM (reads: 1823, responses: 0)
Hi,

As mentioned earlier in the thread, we are servicing a fairly large number of students. I think we are over 4000 this semester. We have two machines - both dual processor pentiums, a 2.66GHz and a 2.40GHz. Earlier this semester, one of the machines was having hardware problems, so we had to shut it down and run off of just one.

Currently, we are running soley on the 2.66GHz machine, and that seems to hold up pretty well with the student load. Other variables which may be relevent to speed:

  • We run dvipng only mode for displaying problems
  • We use vlatex for quicker generation of pdf (faster than dvips and ghostscript - probably comparable to pdflatex)
  • We run a hybrid somewhere between WW 1.8 and 1.9, but not 2.0 (yet)
  • We cache images of graphs. Some of our problem sets are full of images (as many as 8 graphs in a single problem). They are much faster with graph caching.
  • We keep dvipng/graph images for a week, but then delete them. Without that, we ran out of disk space and, well, it wasn't pretty. Of course, if an image is needed again, it is regenerated.
Our load sometimes creeps up for short times, but is mostly fine.

When we were having problems, I was shifting courses back and forth between machines. The course would be unavailable for < 1 minute in the process (disconnect course, tar it up, sftp it, untar, run script to fix permissions).

The biggest problem we ran into with this was that students/teachers would bookmark/link to the login script. When the course moved, those links were dead and students got the course not found error. Also, our main page of links to courses would change when we changed machines. In general that should not be a problem, but some browsers won't refresh a page even if the student shift-clicks on the refresh button (some versions of AOL). They got very frustrated. Even though it was clearly their browser's fault, they blame WeBWorK.

I don't synch course files between the machines. We have RAID mirroring on each machine, so even if it half-explodes, we should be able to recover.

John

<| Post or View Comments |>


userZbigniew Fiedorowicz - Re: Hardware for WeBWorK  blueArrow
11/6/2003; 8:58:38 PM (reads: 1770, responses: 0)

At Ohio State we currently have around 1100 students this quarter using WeBWorK, with two of our largest courses consisting of 300 students each.  We are using the course management system Moodle as the front end to all WeBWorK functions except for initial course setup.  We haven't had any performance problems and seem to have plenty of reserve capacity.  The hardware specs for our new server are given below.

Zig

sysinfo

Host Name is              webwork2
Host Aliases is           webwork2
Host Address(es) is       140.254.92.37
Host ID is                fe8c255c
System Model is           GenuineIntel 3057 Mhz Intel(R) Xeon(TM) CPU 3.06GHz
Main Memory is            4 GB
Virtual Memory is         4 GB
Number of CPUs is         2
CPU Type is               Intel(R)
App Architecture is       i686
Kernel Architecture is    i686
OS Name is                Linux
OS Version is             2.4.20-20.9bigmem
OS Distribution is        Red Hat Linux release 9 (Shrike)
Kernel Version is         Linux version 2.4.20-20.9bigmem (bhcompile@stripples.devel.redhat.com) (gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)) #1 SMP Mon Aug 18 11:18:32 EDT 2003

sudo /sbin/fdisk -l
Password:

Disk /dev/sda: 36.7 GB, 36703934464 bytes
255 heads, 63 sectors/track, 4462 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sda1   *         1      3939  31639986   fd  Linux raid autodetect
/dev/sda2          3940      4461   4192965   fd  Linux raid autodetect

Disk /dev/sdb: 36.7 GB, 36703934464 bytes
255 heads, 63 sectors/track, 4462 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdb1   *         1      3939  31639986   fd  Linux raid autodetect
/dev/sdb2          3940      4461   4192965   fd  Linux raid autodetect

Disk /dev/sdc: 36.7 GB, 36703932928 bytes
255 heads, 63 sectors/track, 4462 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdc1             1      4462  35840983+  83  Linux

Disk /dev/sdd: 36.7 GB, 36703934464 bytes
255 heads, 63 sectors/track, 4462 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdd1             1      4462  35840983+  fd  Linux raid autodetect

<| Post or View Comments |>