Features & Development

Cache for Gateway Quizz

Cache for Gateway Quizz

by Robert Mařík -
Number of replies: 3

Hello, I started do use WeBWorK this semester. It is wonderful tool.

However, as I understand, there are problems with server load during quizzes. Is it possible to implement some caching? And does it help? I once developed webpage which needed many computations (Mathematical Assistant on Web, was quite popular in my country, but does not exist aymore). I used the database where I stored md5 sum of a request and the answer. Thus it was possible to start the computation only if there was not a corresponding item in the database. This reduced the server load a lot.

Using the cache it should be possible to render the problems for each student a day before the quizzes start and this could help to reduce the server load during exam. However, I am not sure, if the bottleneck is the time required to render the problem from pg to html or something else.

Robert Marik

In reply to Robert Mařík

Re: Cache for Gateway Quizz

by Alex Jordan -

> Using the cache it should be possible to render the problems for each student a day before the quizzes start and this could help to reduce the server load during exam

I think I understand how this could help, but it would only help with the first time the problem is presented to the student. When the student types in some answers and submits them, all of the quiz problems need to be re-processed. Depending on the quiz settings, a student may be able to submit over and over again. When they do this with each question, on a quiz with many questions, it is very inefficient. So I think there is opportunity to increase efficiency with quiz operations by only processing those problems for which a student has submitted a new answer.

You could combine that with pre-rendering the questions as you suggest. I pause because traditionally you can fix a bug with problem code (or macros on which it relies, or course config settings, or ....) and expect that all users will see the effect of the fix, and pre-rendering would subvert that. But maybe there could be some way to tell the system that it should redo the pre-rendering.

In reply to Alex Jordan

Re: Cache for Gateway Quizz

by Gavin LaRose -
This is a great question. I think there are two issues with the load that gateway/quiz assignments impose on the system. The first is that when a student starts an assignment, WeBWorK has to create a new version of the assignment for them. I'm not sure why this seems to be a heavy lift, but it appears to be. This could be resolved by pre-creating that the first version of the assignment fo the student when the global set is assigned to the student. The only slightly tricky part of this would be to then fork the creation of the next version when the first version is started, so that it can proceed in the background when the server has processor time to deal with it.

The second issue, of processing all of the problems on the quiz when it's submitted, is one that depends a little bit on the set up of the quiz (e.g., I think that if the quiz has multiple pages only those on the current page are processed with a preview---but all will be processed on submit). I don't have a sense of the degree to which this load is a significant issue.

In reply to Gavin LaRose

Re: Cache for Gateway Quizz

by Nathan Wallach -
Given that problems are typically customized per student with random parameters - and that the "seed" range is pretty large, I suspect pre-computing and storing cached "rendered" problems would be both CPU and storage intensive, and it would not help once answers are being submitted. I suspect that load peaks occur both at the start of a quiz (many students opening the quiz in a short time period) and at the end (many submissions very close to to the deadline).

I think that the issue is the current "single build in" PG renderer per Apache process, as rendering each PG file takes time, and causes the memory ballooning issue which the current generation of the WW server software experiences.

I think that the goal should be to improve WW so that it can easily provide more or less "power" without requiring provisioning a fixed large amount of resources based on expected peak loads. Anyone running in the cloud or with access to a Kubernetes cluster or the like would hopefully be able to then provision for "typical" demand as a standard, but either schedule or "auto-scale" for higher demand when necessary.

I think that the best long-term direction will be to separate the "PG renderer" out of the Apache process, using the new "standalone render" or something similar. Hopefully that will allow somewhat parallel processing of multi-problem rendering requests - or at least provide a means to better control the memory issues which probably is the bigger problem than the CPU resource issue. (Less memory ballooning would allow more Apache processes to run, so many more students could be served in parallel.)

This is something which is not likely to be ready for WW 2.16, but could be a major goal for development in the near future.


Posted also to the GitHub issue requesting feature requests for Gateway quizzes:   https://github.com/openwebwork/webwork2/issues/1156   with a reference to this forum discussion.