WeBWorK Main Forum

The selected problem set is not a valid set

The selected problem set is not a valid set

by Doug Torrance -
Number of replies: 14
Hello!

One of my students is getting the following error when trying to access our most recent assignment in our WeBWorK course:

The selected problem set (Homework_9) is not a valid set for studentname.

When viewing from the main course page, the homework set appears, but there is no hyperlink.

I checked, and the student has indeed been assigned this homework set. No other students are reporting this issue.

Our WeBWorK instance is hosted locally on our own server. I'm the sysadmin and have full privileges to fiddle around with any settings.

I'm stumped -- any suggestions?

Thank you!
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Danny Glin -
I've seen similar behaviour when something dies in the middle of assigning the set to the user.

The following has often worked for me to fix similar issues:
Go to the Instructor Tools page (not the Homework Sets Editor. Many people don't even know that this page exists). Highlight the student and the relevant set, then click "Assign selected users to selected sets". In my experience using this page rather than the Homework Sets Editor will fix any partially completed assigning, but will leave alone anything that was already completed.

In your case since it sounds like the student hasn't been able to access the assignment at all you could unassign it from that one student then reassign it. Just beware that unassigning permanently deletes all student progress on the assignment.
In reply to Danny Glin

Re: The selected problem set is not a valid set

by Doug Torrance -
Thank you for the reply!

I tried unassigning/reassignment the assignment both under the Homework Sets Editor and Instructor Tools, but no change yet on the student's end.

Any other suggestions? Perhaps there's something I can do manually in the database? Or a log somewhere printing what might have gone wrong when I assigned it?

Thank you!
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Nathan Wallach -
Doug - I'm still pretty new to WW, but have started reading/editing code.

The error message you reported is triggered in only one place in the WW code, in webwork2/lib/WeBWorK/ContentGenerator/ProblemSet.pm when the "invalidSet" flag was set.

That flag is set (as opposed to tested) in only 2 places. (I'm ignoring webwork2/lib/WeBWorK/ContentGenerator/GatewayQuiz.pm for now, which seems to be both special and quite active regarding this flag). The line numbers listed below are from the 2.14 RC, so may not match your code. I think the second hit is probably the first place to look...
  • webwork2/lib/WeBWorK/ContentGenerator/Problem.pm:484: $self->{invalidProblem} = $self->{invalidSet} = 1 unless defined $sourceFilePath;
    • This seems to be for when there is a bad path to a specific problem file, so does not seem particularly relevant to the case at hand, unless there is some problem which uses "random" versions for different students, in which case it could be happening to this and some other students.
    • The code there seems to be trying to log an error message via a Perl "die" call: "Problem $problemNumber in set $setName does not exist" which would probably end up in some log file (I'm not sure where, sorry).
    • As suggested below, you can force some logging for when this issue is triggered to help debug it.
  • webwork2/lib/WeBWorK/ContentGenerator.pm:201: $self->{invalidSet} = $authz->checkSet();
    • It is calling the checkSet() function from webwork2/lib/WeBWorK/Authz.pm which can set all sort of error values to the flag when something is wrong, but does not seem to log anything.
    • You might want to log the value given to the flag, if it is not 0...
    • Getting the flag value into the logs would be easiest to do in ContentGenerator.pm.
    • Putting something like the code below just after the line of ContentGenerator.pm where checkSet() is called should get what you want into the logs.
    • But you will need to set (temporarily!) a value for the config variable $courseFiles{logs}{activity_log} in webwork2/conf/defaults.config so that the activity log file will be created/written. (See the warning about it getting huge in the config file - so don't leave it live all the time.)

Proposed code:

if ($self->{invalidSet}) {
WeBWorK::Utils::writeCourseLog($ce, 'activity_log', "Error: invalidSet was set to $self->{invalidSet} in ContentGenerator.pm");
}


Nathan
In reply to Nathan Wallach

Re: The selected problem set is not a valid set

by Doug Torrance -
Thank you so much for the detailed response! Now that I'm back on campus and able to SSH into our server, I'm poking around to see if I can learn more.
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Danny Glin -
Hmm... It looks like this type of issue doesn't get logged anywhere. If there is an error in processing a page a log entry gets generated in the apache error log, but since the page is properly loaded here that doesn't happen.

When I try to access a set that is not assigned to me I get more information than just the message you quoted. It is followed by further detail. In my case the full message is
The selected problem set (Assignment_4) is not a valid set for [username]:

Requested set 'Assignment_4' is not assigned to user [username].

When you act as the student do you see the same error? Has the student sent you a screenshot of the error?

In reply to Danny Glin

Re: The selected problem set is not a valid set

by Doug Torrance -
I can access the assignment with no issues when I act as the student.

The student did send me a screenshot of the course homepage with no hyperlink on the assignment, but not of the error message. They later came to my office and reproduced the issue on both their laptop and my desktop, confirming it wasn't just some browser issue. I don't recall any further detail on the error message, but I could be misremembering.
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Danny Glin -
This behaviour is exactly what I would expect if it were before the open date for the assignment. In this case the student would not be able to access the assignment, but the instructor would.

I'm assuming that the rest of the class is not having problems with this assignment?

If it were a setting that was changed for the one student, then unassigning and re-assigning should restore all of the settings to the defaults, making it behave like it does for every other student.

In terms of the database, when a homework set is assigned to a user, one record is created in [coursename]_set_user for the whole set and one record is created in [coursename]_problem_user for each problem in the set. You can see how these look by running the following mysql commands:

SELECT * FROM [coursename]_set_user WHERE set_id='Homework_9' and user_id='studentname';

For a freshly assigned set the first three columns should be populated and everything else should be NULL, as the values are inherited from the global values.

SELECT * FROM [coursename]_problem_user WHERE set_id='Homework_9' and user_id='studentname';

This should return a row for each question in the set. You can compare the output by running the commands for a different student and seeing if anything looks different. You can safely run these commands on a production database since SELECT statements do not make any changes.
In reply to Danny Glin

Re: The selected problem set is not a valid set

by Doug Torrance -
Thanks again for the detailed response!

The one difference I noticed with the student in question was the 'lis_source_did' column of the [coursename]_set_user table was NULL. For the other students, it was populated with a long hexadecimal-looking string.

From glancing at the source code, it looks like this might be related to the LTI authentication. We're using Canvas.

Any ideas?

Thanks!
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Danny Glin -
You are correct that the lis_source_did column has to do with LTI authentication. It is set by the LMS to allow WeBWorK to pass grades back.

The fact that it is null would suggest that something odd is happening with this particular student. Is the student's status somehow different in Canvas? Sometimes the LMS will not send a source_did if the user is not a student (e.g. TA, guest, etc.).

The missing source_did may or may not be related to why the student can't access the assignment. Since you're using LTI there are two ways sets get assigned to user. Based on this being "Homework_9" I'm assuming that she didn't have any issues with the previous assignments?

Is it possible to get screenshots of what she sees on the main page, and also the message she gets when she tries to access that assignment?
In reply to Danny Glin

Re: The selected problem set is not a valid set

by Doug Torrance -
I got a screenshot from the student and also turned on LTI debugging. Indeed, there seems to be an issue on that end.

WTAKIi5.png
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Doug Torrance -
Never did figure out exactly what was going on. I ended up setting $LTIGradeMode to 0, which fixed the problem.

Now I have to manually enter the grades in Canvas, but it's a small class, so it's not a huge deal.
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Danny Glin -
If that fixed it then it sounds like something to do with the student's status in Canvas.

If you have a chance, can you compare the debug info of the problem student with one that works?
In reply to Doug Torrance

Re: The selected problem set is not a valid set

by Nathan Wallach -
Doug - Can you take another look at the [coursename]_set_user table in the database. You wrote:

The one difference I noticed with the student in question was the 'lis_source_did' column of the [coursename]_set_user table was NULL. For the other students, it was populated with a long hexadecimal-looking string.

  • Does this student have non-NULL values for 'lis_source_did' for other assignments?
  • Are there any other NULL values for 'lis_source_did' in the table and if so is there any noticeable pattern?
Could you also confirm that this student was able to do prior assignments when the LTI grade passback was enabled.

Thanks,
Nathan
In reply to Danny Glin

Re: The selected problem set is not a valid set

by Nathan Wallach -
Dear Doug and Danny,

This sounds like the type of bug which deserves attention to understanding what is getting corrupted in the database tables, so that either a fix can be set up to automatically try to overcome such cases, or so an information mail can be automatically sent the course staff that such a problem was detected.

Doug - if you did not yet follow Danny's advice, could you consider making a (file level) copy of the SQL database table files in the course before doing it. Then together with someone who better understands the tables, we can try to determine what records are corrupted and how to detect such issues automatically.

Looking at the documentation in webwork2/docs/new-DB-sql and webwork2/docs/new-DB-structure and some sample course DB tables, something which affects individual users is likely to be in course_set_user and/or course_problem_user, but the main course_set and course_problem tables (and maybe some other tables) may be important to being able to figure out what is corrupted.

One difficulty to sharing such tables to allow debugging the issue will be figuring out how to scrub restricted information (user names, passwords, etc.) if tables are to be shared. It might be simple to work out a methodology of having local privileged users do the necessary investigations in SQL and just share the minimal amount of information on broken records.

Nathan