WeBWorK Main Forum

dealing with a high server load

dealing with a high server load

by Timothy Prescott -
Number of replies: 4

My department hosts a local math contest every year for middle and high school students.  Last year, we couldn't have the event in person, and decided to try having people test through our locally installed WeBWorK (2.15 at the time, now 2.16).  Things did not go well.

It turns out that having ~200 people simultaneously log in and generate a gateway proctored test was more than our local server could handle.  Testers were reporting that logging in and/or starting their test was taking several minutes, which was a substantial portion of the allotted time.

Is this a typical load that our (assumed to be typical) server should have been able to handle?  Would the MAA server have been able to handle this traffic?  Are there settings we could tweak to help our server?  Are there ways we could adjust the administered tests to make things easier on the server?  (Would reducing the amount of MathJax used make things easier?  Is it possible to tell WeBWorK that a test isn't actually random and can be identical for all testers?)  And finally, web searches turn up a few ways to test a server's capability to respond to high demand; is there anything specific to WeBWorK that would allow us to test our server's capacities?

In reply to Timothy Prescott

Re: dealing with a high server load

by Glenn Rice -

One thing that you can do is try to tweak the settings in /etc/apache2/mods-available/mpm_prefork.conf.  You might want to look at https://www.techrepublic.com/article/how-to-enable-apache-mpm-prefork-to-improve-performance/.  There is some advice on this, as well as a perl script that you can run to test your settings.

You can also do some things to make alleviate the load.  The first thing you could do is stagger the starting times for the students so that they are not all asking the server to do the initial test generation at one time.  Another thing you can do is make sure that your test has each problem on a separate page. This won't help with the initial test generation, but will help during the exam.

MathJax does not put load on the server at all, with the exception of the initial download of the javascript.  Once the browsers have cached that, the server load due to MathJax will be pretty much none.

In reply to Glenn Rice

Re: dealing with a high server load

by Ama Chefe -

thanks for this suggestion.

I had similar issue and it ended up with MariaDB going down and App Armor trigger.

Does the by any means help with DB connections?

In reply to Ama Chefe

Re: dealing with a high server load

by Danny Glin -
I have never come across WeBWorK load causing MariaDB to crash. WeBWorK is much more taxing on apache than the database, so usually if a WW server crashes it is because apache has run it out of memory.
Can you give more details on what happened when MariaDB went down?
In terms of DB connections, generally WW opens one DB connection per apache process, so your MariaDB server needs to be able to handle at least as many connections as you have apache processes (as determined by MaxRequestWorkers in your apache configuration).
In reply to Timothy Prescott

Re: dealing with a high server load

by Danny Glin -
The gateway quiz mode wasn't designed for large-scale simultaneous exams, so experiences such as yours are common.  The (former) MAA server was probably similar in specs to what you're running, so there is a good chance that it would have similar issues.

There are a lot of things going on when a student starts a gateway quiz, so it's hard to nail down which aspects are having the most impact on performance.  At the start of a quiz, the version has to be generated for that student, which involves a lot of calls to the database, and then each of the problems has to be rendered.  This makes it hard to create standardized performance tests, since a genuine test would have to have a large number of different accounts starting a test at the same time.

One thing that may help would be to present the entire contest as a single multi-part question.  This would reduce the number of calls to the database, and also the number of files that need to be accessed.  It's hard to say how much this would change things as it would very much depend on the specific hardware.