WeBWorK Main Forum

"The selected problem set is not a valid set"

"The selected problem set is not a valid set"

by Siman Wong -
Number of replies: 9
After I created a gateway quiz, I try to "take it" myself (as the instructor) to see if it works, only to be greeted with the error message

   The selected problem set (name of test) is not a valid set for (me).  One or more of the problems in this set have not been assigned to you.

Right now the quiz is set to be invisible and it's been assigned to myself only.   I thought that's the issue, until I export this quiz, and then reimport it under a new name --- so the duplicate has the identical settings (invisible to everyone etc).  BUT:  I can take this new version!

So what's going on, what settings should I change to make this error message goes away?  And how can I test things out so that I can be sure it will work when I give the test? --- I created a separate test account and everything worked there before I moved the files to the actual account, so I'm baffled (and panicking...)  Thanks for your help!


In reply to Siman Wong

Re: "The selected problem set is not a valid set"

by Alex Jordan -

Are the problems in the quiz numbered consecutively? If not, use the Hmwk Sets Editor to renumber them consecutively from 1, and see if the issue persists. This bit "One or more of the problems in this set have not been assigned to you." could be about a mismatch between a problem's labeled number and its sequential number in the set. This is the most common explanation for this message.


In reply to Alex Jordan

Re: "The selected problem set is not a valid set"

by Siman Wong -
Thanks for your reply. The problems are consecutively numbers -- as I was composing the quiz, every time I add/remove a problem I always selecte "Force problems to be numbered consecutively from one" in the Hmwk Sets Editor. And just in case I clicked that again (and Saved Changes, of course) but the issue persists.

And like I said, when I exported this quiz and then re-import it under a new name, the re-imported version seems to work fine-- as me -- and (presumably?!) has the same settings as the initial set. But I don't know how it might be like for students...
In reply to Siman Wong

Re: "The selected problem set is not a valid set"

by Nathan Wallach -
I took a look at lib/WeBWorK/ContentGenerator/GatewayQuiz.pm and the message "One or more of the problems in this set have not been assigned to you." is triggered by the test if ( ! defined( $mergedProblems[$pIndex] ) )  which seems to be related to the need for consecutive problem numbering, as Alex wrote.

My guess is that the database records for the "assigned set" are not properly updated when changes are made to the main set definitions. Most likely this has to do with the Gateway specific notion of "versioned" sets, but I cannot be certain of that.

If you are comfortable with SQL, looking at the data inside the relevant SQL tables might give useful information.

I recommend checking whether unassigning the assignment to your user and then reassigning it helps. Typically I would expect that to remove any database records for the user account for the specific assignment. I'm not an expert on GateWay quizzes, and it could be that this does not work that way for Gateway quizzes due to manner in which GateWay quizzes use "versioned" sets. (Sorry, I'm not dredging deeper in the code to try to check). 

I recommend that the issue be reported on GitHub, with steps to trigger the issue, so hopefully someone will look for the bug in the code.
In reply to Nathan Wallach

Re: "The selected problem set is not a valid set"

by Siman Wong -
Unassigning to all users, and then reassigning them, removes this error message --- THANKS SO MUCH! When I first started composing the quiz, at one point I did delete some problems, added more, and rearranged them without renumbering the whole set. I started using the renumbering button consistently later on. I don't know what if any one these steps might have triggered the bug. And like I said, the weird thing is that if I export and the re-import this quiz under a different name then I have no problem. Let me give the quiz first (!) and then try to reproduce the bug and submit a bug report. Q: What's the github address? Thanks again for all the help!
In reply to Siman Wong

Re: "The selected problem set is not a valid set"

by Glenn Rice -

What you are experiencing is rather well known behavior for gateway quizzes and occurs for homework assignments also.  Modifying sets after they are assigned and opened causes problems.  As such, I am not sure that it is something that you should file a bug report for.

In reply to Glenn Rice

Re: "The selected problem set is not a valid set"

by Nathan Wallach -
Happy the workaround helped, and it makes it pretty clear that something in the database records get broken during some sort of changes to an assigned quiz.

I'm pretty sure that removing a single question from a regular assignment changes the assignment suitably for all users.

Renumbering problems in an open assignment breaks the expected connection between problems and their answers, but as far as I know, it does not totally make it impossible to open the assignment.

Here - changes to a gateway quiz made it impossible to view the quiz. I think the issue is worth tracking in GitHub, even if we decide it is too difficult to fix it.
However, it seems there is already an issue as well as a proposed fix to prevent the error message:
https://github.com/openwebwork/webwork2/issues/899
https://github.com/openwebwork/webwork2/pull/1128

Now with the report here - it seems that the issue could be "deeper" than just the error message preventing access, as it seems that the user to which the gateway was assigned has different "problem data" than the "main" quiz settings. If that sort of situation remained when a quiz went live - some students might get a a different quiz than what was intended. It it probably something to look into. At the least - it would be helpful if WW suggested an "unassign+reassign" if changes which can cause this sort of inconsistency were made.
In reply to Nathan Wallach

Re: "The selected problem set is not a valid set"

by Glenn Rice -
The issue is also not "deeper" in terms of a mismatch between the user "problem data" and the "main" quiz settings in regards to the gateway quiz consecutive numbering issue. The student data and the main quiz data do match. A quiz with non-consecutive numbering simply fails to work for all, and that is fixed in the mentioned pull request. Merging that pull request will not cause the situation that you are claiming, that a student will get a different quiz than was intended. If a quiz is created from scratch with non-consecutive numbering, then that is what is intended. The reason that a gateway quiz fails to work with non-consecutive numbering is simply due to incorrect indexing in the GatewayQuiz.pm file.

There are problems with homework assignments when the main settings are modified. You mentioned the connection between problems and answers, but that goes deeper. The connection is between problems the student has and problems in the main settings. If you change the source file for a problem after a student has opened the problem, the student will still have the original source file used, and not the one that is changed to. If you change the value of a problem, the student's value is not changed. Most settings are not changed for the student once the student has opened the problem. You have to go to the individual student settings to make these take effect, or do the unassign+reassign cycle. However, usually the unassign+reassign cycle is not the correct thing to do in this situation. If a student has already opened a homework assignment and started to work on it, then that will delete all of the student's work.

Generally speaking, modifying a homework assignment or gateway quiz after it has been assigned and students have begun to work on it is a bad idea. The thing to remember when you are creating an assignment before it has been assigned is to make sure changes take effect for your user. The easy way to do this is the unassign+reassign cycle. That is really the only time that should be used.
In reply to Glenn Rice

Re: "The selected problem set is not a valid set"

by Glenn Rice -
I want to add that the way that many proprietary online homework systems (e.g. Pearson MyLab) deal with this is to simply not allow the instructor to change the problems in a homework set once students have begun to work on it. That is probably what we need to implement in WeBWorK. There really is no sane way to modify a set after students have started working on it.
In reply to Glenn Rice

Re: "The selected problem set is not a valid set"

by Alex Jordan -

> There really is no sane way to modify a set after students have started working on it.

If the id of the problem in the database were separated from the label, that seems like an approach that could work. As in:

Create problem set:
id 1, label 1, active
id 2, label 2, active
id 3, label 3, active

Open the problem set and students do work:
id 1, label 1, active, studentDataFoo
id 2, label 2, active, studentDataBar
id 3, label 3, active, student DataBaz

Remove #2 from the problem set:
id 1, label 1, active, studentDataFoo
id 2, label empty, inactive, studentDataBar
id 3, label 2, active, student DataBaz
# "label empty" could imply "inactive", relieving the need for that column

Change order of problems:
id 1, label 2, active, studentDataFoo
id 2, label empty, inactive, studentDataBar
id 3, label 1, active, student DataBaz

Add a new problem:
id 1, label 2, active, studentDataFoo
id 2, label empty, inactive, studentDataBar
id 3, label 1, active, student DataBaz
id 4, label 3, active


I might go further and separate "label" into a an actual label, and an index. As in the problem with id "3" could have index "1" and label "Warm Up Exercise 1".