Hi Danny,
There is a bug somewhere that causes this to happen. See
http://webwork.maa.org/moodle/mod/forum/discuss.php?d=6243 for other reports. It also can happen under other unknown circumstances although this seems very rare (but it did happen to me once). At this point we do not have any idea where the bug is and haven't yet had the time to track it down. My first thought (which I haven't checked out) is that maybe if a student is active when the change is made, the current setting are saved for them.
Any additional information people have on this will help us trace it down.
Arnie
WeBWorK Main Forum
Changing a source file for a question after a set is open
This forum has a limit to the number of forum postings you can make in a given time period - this is currently set at 10 posting(s) in 1 day