Hello,
I currently administer a WeBWorK server of about 500 students (HP stock machine, Fedora 15 OS, 4 GB RAM, 400 GB hard drive, quad core). Because our pilot of WeBWorK has been very successful this semester, the department here wishes to scale the system to support about 4000 students. I'm interested in hearing what other setups and/or advice other institutions have for (essentially) building from the ground up.
Also, I am particularly interested in the cluster vs. server rack debate. Our School of Computing and Information Systems here boasts professional-grade server rooms with air conditioning, server racks, backup power systems, etc. However, the mathematics department currently has not such luxuries (and I doubt we'll be able to purchase anything similar, or ask the School of Computing to support us).
Also, if I went the cluster route, is there any good virtual management software to maintain a set of homogeneous linux boxes?
I understand this is a VERY broad question, but any feedback or suggestions would be very much appreciated! The department here wishes to scale the success of our pilot rather quickly, and I want to make sure that this goes well.
-Kurt
Specifications and system design for a large WeBWorK system (4000+ students)
by Kurt O'Hearn - Number of replies: 7
In reply to Kurt O'Hearn
Re: Specifications and system design for a large WeBWorK system (4000+ students)
by Jason Aubrey -
Hi Kurt,
If you haven't already done this, my first recommendation would be for your department to start talking about this with central IT on your campus.
They may have policies (either required or recommended) regarding applications which handle student data, and procedures for bringing a campus wide application live.
Also, they may be able to provide the department with a spot in the campus professional-grade server room, and possibly support integrating the system in the existing IT environment (e.g. for authentication, IT help desk, etc.) and administering the system.
Maybe they can't offer any of this, but I'd at least check with them before going it alone.
Jason
If you haven't already done this, my first recommendation would be for your department to start talking about this with central IT on your campus.
They may have policies (either required or recommended) regarding applications which handle student data, and procedures for bringing a campus wide application live.
Also, they may be able to provide the department with a spot in the campus professional-grade server room, and possibly support integrating the system in the existing IT environment (e.g. for authentication, IT help desk, etc.) and administering the system.
Maybe they can't offer any of this, but I'd at least check with them before going it alone.
Jason
In reply to Jason Aubrey
Re: Specifications and system design for a large WeBWorK system (4000+ students)
by Kurt O'Hearn -
Hi Jason,
Our IT department will not offer any administrative support since IT here mainly deals with Windows and Mac machines (I personally believe they do not have the expertise to assist us with WeBWorK). As for infrastructure, they may provide space in their server room, but anything beyond this is unlikely. IT may also be able to help us in purchasing hardware if we limit ourselves to the 'stock' options they provide (but I may prefer to shop elsewhere depending on pricing).
As for other services IT provides, we do use BlackBoard and LDAP here. However, I am under the impression that WeBWorK integration into BlackBoard is currently not mature enough to bother pursuing. And, as for LDAP, I will deal with that hurdle when the time comes.
And regarding protocols - I do not know the exact regulations regarding student grades (I know we operate in accordance with FERPA), so I will inquire more into what guidelines we must follow, and what guidelines IT suggests we follow.
I appreciate your information, and welcome any additional thoughts, especially system design discussion (high-end custom server rack type machines, or a cluster of PCs and virtualization) and hardware specifications. I personally have no experience with virtualization software, so if we decide here to try and save costs with a cluster, I have some learning ahead of me.
-Kurt
Our IT department will not offer any administrative support since IT here mainly deals with Windows and Mac machines (I personally believe they do not have the expertise to assist us with WeBWorK). As for infrastructure, they may provide space in their server room, but anything beyond this is unlikely. IT may also be able to help us in purchasing hardware if we limit ourselves to the 'stock' options they provide (but I may prefer to shop elsewhere depending on pricing).
As for other services IT provides, we do use BlackBoard and LDAP here. However, I am under the impression that WeBWorK integration into BlackBoard is currently not mature enough to bother pursuing. And, as for LDAP, I will deal with that hurdle when the time comes.
And regarding protocols - I do not know the exact regulations regarding student grades (I know we operate in accordance with FERPA), so I will inquire more into what guidelines we must follow, and what guidelines IT suggests we follow.
I appreciate your information, and welcome any additional thoughts, especially system design discussion (high-end custom server rack type machines, or a cluster of PCs and virtualization) and hardware specifications. I personally have no experience with virtualization software, so if we decide here to try and save costs with a cluster, I have some learning ahead of me.
-Kurt
In reply to Kurt O'Hearn
Re: Specifications and system design for a large WeBWorK system (4000+ students)
by Nandor Sieben -
We have two webwork servers working with a shared file system on a third file server and a shared database on a fourth database server. This setup is mainly to avoid down time. The servers are powerful alone to handle the load we have which can go as high as 130 students.
I found that the NPL is too slow on the file server, so both servers have their own NPL directory. The shared file system has the classes directory.
In reply to Nandor Sieben
Re: Specifications and system design for a large WeBWorK system (4000+ students)
by Nandor Sieben -The webwork servers are:
Intel Xeon 2.50GHz 8 CPU with 4 cores.
You can see the CPU load compared to the number of students working at:
https://webwork.math.nau.edu/webwork2_files/reports.html
In reply to Nandor Sieben
Re: Specifications and system design for a large WeBWorK system (4000+ students)
by Mark Hamrick -
My question here is how do you find out the number of students that are globally logged in at one time? I would also like to know the number of students that are logged into a course at one time.
I believe I can tell roughly how many students are globally logged in at one time by using the number of database connections.
At my location we currently have over 5000 students, split between to virtualized Ubuntu 10.04 servers running on top of two RHEL 5.0 servers. I chose this method so that I could put both on one server if something started to fail. We would loose one day however, as the backups occur nightly.
I believe I can tell roughly how many students are globally logged in at one time by using the number of database connections.
At my location we currently have over 5000 students, split between to virtualized Ubuntu 10.04 servers running on top of two RHEL 5.0 servers. I chose this method so that I could put both on one server if something started to fail. We would loose one day however, as the backups occur nightly.
In reply to Kurt O'Hearn
Re: Specifications and system design for a large WeBWorK system (4000+ students)
by Gavin LaRose -
Hi Kurt,
Here at Michigan we run a WeBWorK server that is supporting on the order of 6000 students this semester. We have a single server (RHEL6, 24GB memory, four hex-core 3GHz processors) for this. This appears to have plenty of horsepower for us: over the last week (which includes one testing deadline) our load average has been about 0.12, with a max of about 1.
We have this on a rack in a server room, because we have that luxury.
Gavin
Here at Michigan we run a WeBWorK server that is supporting on the order of 6000 students this semester. We have a single server (RHEL6, 24GB memory, four hex-core 3GHz processors) for this. This appears to have plenty of horsepower for us: over the last week (which includes one testing deadline) our load average has been about 0.12, with a max of about 1.
We have this on a rack in a server room, because we have that luxury.
Gavin
In reply to Kurt O'Hearn
Re: Specifications and system design for a large WeBWorK system (4000+ students)
by Jason Aubrey -
Hi All,
To follow up on this, we've started collecting performance data on webwork servers at
http://webwork.maa.org/wiki/WeBWorK_performance
It would be really helpful if you could add the timing log data from your server here. To do so go to
webwork2/logs/
and run
webwork2/bin/timing_log_check.pl timing.log
Also, if you never noticed this timing log before you might find that it is very large and you might want to set up a logrotate/chron job to rotate this log every month or so and generate a timing log report.
(E.g. the timing log on my server that I just posted the report on was 6.6 GB and contained data on 4,664,624 requests over a year and a half!)
If you do generate such reports regularly, please add them to the wiki!
Thanks,
Jason
To follow up on this, we've started collecting performance data on webwork servers at
http://webwork.maa.org/wiki/WeBWorK_performance
It would be really helpful if you could add the timing log data from your server here. To do so go to
webwork2/logs/
and run
webwork2/bin/timing_log_check.pl timing.log
Also, if you never noticed this timing log before you might find that it is very large and you might want to set up a logrotate/chron job to rotate this log every month or so and generate a timing log report.
(E.g. the timing log on my server that I just posted the report on was 6.6 GB and contained data on 4,664,624 requests over a year and a half!)
If you do generate such reports regularly, please add them to the wiki!
Thanks,
Jason