WeBWorK Problems

Canvas to Webwork LTI re-setting student scores to zero.

Canvas to Webwork LTI re-setting student scores to zero.

by tim Payer -
Number of replies: 2

Greetings:

I have another Canvas-Webwork LTI glitch that I hope you might be able to help me with. I realize that not many in the Webwork community have Canvas as an LMS but I will describe the error in hopes of some feedback:

The LTI link between Canvas and Webwork is sending zero scores to the Canvas gradebook for three assignments for over one third of the class: 16 out of 48 students.

Here are the specifics:

1.) The three affected assignments (Quiz 3 Prep, Quiz 4 Prep and Quiz 5 Prep) are all “extra credit” which in Canvas means that after the assignment is closed the point value in the problem is set to zero to push the completed points scored to the homework total and thus these points become “extra credit” within that category.

2.) More than a third of the students who completed the assignment had their non-zero scores set to zero by the LTI. Manually over-writing the score results in the LTI resetting the scores to zero in 20-80minutes. Multiple attempts to re-open the assignment for re-scoring have failed to restore these non-zero scores. Breaking the LTI link also Failed to prevent the resetting of the assignment to zero for these students. Changing the assignment to a non-submission type that it is “independent” of the webwork link also did not stop the LTI from resetting these 16 students to zero scores.

3.) I wrote the Canvas techs and they feel fairly certain that the issue is on Webwork’s LTI connection. I have attached their response below.

4.) On Sun, Dec 1, 2019 at 3:39 PM Instructure Support <support@instructure.com> wrote:

Hi Tim!

Thanks for reaching back out to us to let us know. We took a deeper look and found that it is still the "Webwork for Stat 109" LTI that is making those grade changes. We found this by checking who graded those submissions via the API. Even though the LTI has been removed from those assignments, the LTI was still granted permission to edit the gradebook and has record of the assignment ID's, so it is probably just syncing and passing the grades that it has on record back to Canvas.

I talked to a coworker about what we can do about it. The easiest option will probably just be to make new assignments. Since those new assignments won't be on Webwork's record, they won't be able to be edited. Beyond that, you will need to either remove the LTI configuration from Canvas or get in contact with Webwork and ask them what you can do about removing those assignments from the list of assignments that sync.

If you have any questions, please feel free to reach out to us. We are available 24/7 and happy to help.

Thank you!

Gareth Sanders
L1 Canvas Support
Support@instructure.com


Please let me know if this helps determine what the source of this error is.

In reply to tim Payer

Re: Canvas to Webwork LTI re-setting student scores to zero.

by Nathan Wallach -
I'm not using Canvas, but we are using LTI (with Moodle) and based on the description, if there is a problem on the WW side, then I doubt that the issue is specific to Canvas.

It is interesting why those users grades are now considered to be 0 by WW. Could some change have been made to their WW data to delete the scores that WW had for them at an earlier time (based on which the non-zero scores were sent by LTI farther in the past)?

It would help to know what data the WeBWorK databases contain (score records, source_did set/not set, etc.) for the students whose scores are being changed to zero, and how that differs from the data for the other students.

I am assuming that you have $LTIGradeMode="homework" for per assignment grades. In that case, each "UserSet" has a value set which provide the data needed to pass the grades back to the LMS. Moreover, it seems that $LTIMassUpdateInterval is set, which causes the periodic updates, which is why the scores get changed back to zero again and again.

As far as I can tell, there is no existing WeBWorK feature to disable the mass-update for specific assignments, but one can be added in the future. Such a feature would provide some manner of avoiding such issues via course.conf.

It could be that you can turn off on the Canvas side the "grade receive" feature; but it is not certain that such a change will truly prevent Canvas from accepting additional LTI score updates based on the value of "lis_result_sourcedid" which was already stored on the WeBWorK side. (The fact that changing the type of task did not help seems to indicate that this may not work.)

They only current workaround I can think of off hand would be to manually delete (via direct access to the databases) the value set for the "lis_source_did" in the database records in the table COURSENAME_set_user for each relevant user_set of these assignments, which would prevent further grade updates being sent by WeBWorK, as WW would no longer have the data needed to send the updates for the specific sets.

Something like:
  • select set_id,count(*) from _set_user group by set_id;
  • # Find the correct values of set_id to change
  • update COURSENAME_set_user set lis_source_did=NULL where
  • set_id='"First_SetId_to_changed' or set_id='"Second_SetId_to_changed';
should do the trick.

Nathan
In reply to Nathan Wallach

Re: Canvas to Webwork LTI re-setting student scores to zero.

by D. Brian Walton -
This is quite a late response, but what follows is my understanding of what is happening. WeBWorK does not keep track of which scores have been pushed to the LMS. Consequently, it pushes every grade for every student to Canvas, sending the percentage scored on the assignment. Canvas calculates a new score based on the product of the reported percentage and Canvas's points possible for the assignment. If the score is different than what is recorded, it updates the Canvas database and posts a notification. If the calculated score is the same, then there is no update required.

So if Canvas has changed the score to 0 based on the Extra Credit setting, then Canvas is being instructed on this grade push to set the scores to zero.