The gateway quiz mode wasn't designed for large-scale simultaneous exams, so experiences such as yours are common. The (former) MAA server was probably similar in specs to what you're running, so there is a good chance that it would have similar issues.
There are a lot of things going on when a student starts a gateway quiz, so it's hard to nail down which aspects are having the most impact on performance. At the start of a quiz, the version has to be generated for that student, which involves a lot of calls to the database, and then each of the problems has to be rendered. This makes it hard to create standardized performance tests, since a genuine test would have to have a large number of different accounts starting a test at the same time.
One thing that may help would be to present the entire contest as a single multi-part question. This would reduce the number of calls to the database, and also the number of files that need to be accessed. It's hard to say how much this would change things as it would very much depend on the specific hardware.