WeBWorK Main Forum

Webwork LTI settings prompting multiple Canvas notifications

Webwork LTI settings prompting multiple Canvas notifications

by tim Payer -
Number of replies: 8
Greetings,

Has any one had the problem of multiple Canvas Homework update notifications for students when using the LTI webwork connection with grade pass-back for homework assignments?

I am experiencing this and it is a bit exasperating for the numerous emails.
If I extend the due date for one student the entire class is notified and each time the student makes a submission, each student in the class is notified that the homework score has been updated.

Some students are receiving 4-6 emails in a given hour and this is occurring in assignments that are past due. I did make some entries with a test student from the canvas page in a webwork assignment, but the frequent updates to the entire class are too much.

The Canvas folks suggested that I "mute" the assignment which will stop the sending of messages related to the homework, but then students can not see their grade for the muted assignment.

I do have admin privileges, on the webwork side, and the Canvas folks are certain that the prompts are originating from the webwork connection.

Do you have any suggestions for trouble shooting the frequent messages?

Thanks, tim
In reply to tim Payer

Re: Webwork LTI settings prompting multiple Canvas notifications

by D. Brian Walton -
Tim,

I have been dealing with the very same issue this week and believe that I have figured it out. Unfortunately, I'm not sure how to resolve the issue without a modification of the WeBWorK database itself.

Here is my setup. LTI login and grades per homework set. Score is updated on mass_update. I was submitting grades every hour to keep Canvas and WeBWorK close to the same. When students started complaining, I switched to every 12 hours to reduce the number of notifications.

Grades are recorded when WeBWork passes a percentage complete to Canvas. Canvas multiplies that percentage times the score possible and records the grade.

If the new grade computed differs from the grade that is recorded, a notification is sent. (No notification seems to be sent if the grade is unchanged.) Unfortunately, Canvas does not record the fully computed grade, but rounds it to maybe the nearest hundredth. (The gradebook only shows tenths, but I thought I saw in a student notification that there were more.)

Unfortunately, Canvas doesn't look to see if the recorded grade is unchanged, just if the submitted grade is different than the recorded grade. When a student's score has a round-off error, Canvas thinks it needs to send a notification every time WeBWorK talks to Canvas.

If your server is set to pass grades back on submit, instead of in batches, then they might actually be getting a notification every time they answer a question.

The only fix I can imagine is if the SubmitGrade routine added a new WeBWorK database entry in the set_user table and tracked the last grade posted by LTI passback. Only if this grade was changed would a new submission be posted.

I have just submitted this as a bug/feature request.
http://bugs.webwork.maa.org/show_bug.cgi?id=3963

By the way, students can change their Canvas notification settings to get grade updates daily instead of immediately.

Sorry I don't have better news.

- Brian
In reply to D. Brian Walton

Re: Webwork LTI settings prompting multiple Canvas notifications

by tim Payer -
e want to find the source of the repeated notification of grade updates in Canvas that use an LTI with Webwork.

We are trying to debug the repeated notification of grade updates in Canvas that use an LTI with Webwork. And we are also looking for a stop gap measure.

Currently we have the default MAA subscription plus these flags in our local course.conf file.

# Settings to connect this course to Canvas via LTI. $LTIGradeOnSubmit = 1; $debug_lti_parameters = 0; $LTIBasicConsumerSecret = (DELETED) $LTIGradeMode = "homework";

Past due HW scores are getting repeated notifications of updated scores for SOME but not ALL students. We have questions concerning debugging about the timing and what information gets sent to the LMS, Canvas.


1.) Our understanding for the LTI connection is that one update per student is sent upon each correct submission for a given problem. They key point is that only the updated homework set score is sent via LTI. OR is it possible that ALL scores are sent from Webwork to Canvas every time there is an updated score for a single Homework?


2.) Regarding the timing of these updates: Does webwork accumulate updated scores and then send them to Canvas en mass? Even for multiple students at once?

3.) Webwork displays student scores to the second decimal. Is it the second decimal value being sent to Canvas or is it something else? In the canvas grade book all student scores are rounded to integer scores (0, 100). We suspect that the difference in these second decimal scores and integer displays could be a factor in triggering the repeated updated messages in Canvas. Note that all our Homework sets have problem weights that sum to 100 points.

4.) What if We were to reset the LTI file notification rate? (As a stop gap measure) Currently we are using "$LTIGradeOnSubmit = 1;". We are assuming that if this was set to zero, that the notifications would be sent from the MAA server en mass every 24 hours?



Any help would be most appreciated,

Tim
In reply to tim Payer

Re: Webwork LTI settings prompting multiple Canvas notifications

by Danny Glin -
WeBWorK offers two methods for sending grades back to the LMS: on submit and scheduled in bulk. These are independent in the sense that you can enable one or the other, or both.

You have the on-submit version explicitly enabled in your configuration ("$LTIGradeOnSubmit = 1;"). It's possible that the scheduled passback has been enabled globally for the MAA server.

The variable that controls the mass update is $LTIMassUpdateInterval. If you use the default LTI configuration, it is set to 86400 (measured in seconds, so every 24 hours). To disable the scheduled update, you can try putting "$LTIMassUpdateInterval=-1;" in your course.conf file.

In reply to Danny Glin

Re: Webwork LTI settings prompting multiple Canvas notifications

by tim Payer -
Thank You Danny!

I think we are going to let the students manage the response rates from their Canvas page individually as some students actually want the instant updates.

I appreciate the update, though.

Tim
In reply to tim Payer

Re: Webwork LTI settings prompting multiple Canvas notifications

by tim Payer -
Hi Danny,

The frequent, unwarranted, email messages of updated grades continue in our LTI connection between Canvas and Webwork.

We have inserted the line of code:

$LTIMassUpdateInterval=-1;
within our course.conf file

But it seems to have no effect.

1.) Question
Would it matter where we place this code line in our course.conf file?
Currently we have placed this code just after the line of:

$LTIGradeOnSubmit = 1;
Would the placement before or after this line have an effect?

The canvas folks have a complaint that our particular Calculus courses (using a wewbork with LTI) are "pushing" the memory capacity and they would like it to stop.

I saw the post by Brian that he is testing a solution, but I have not yet seen a recent post.

I would hate to have to pull out of the LTI connection and return to posting grades via csv files, but I am afraid that this is looking more likely as the students are beginning more homework assignments as the semester proceeds these numerous "updating grade" messages will only increase.

Any further suggestions would be most appreciated.

Sincerely, Tim
In reply to tim Payer

Re: Webwork LTI settings prompting multiple Canvas notifications

by Andras Balogh -
Why do you have $LTIGradeOnSubmit = 1; if you don't want frequent updates?
Don't you want something like

$LTIGradeOnSubmit = 0; (don't immediately update)
and maybe
$LTIMassUpdateInterval = 86400; #update once a day



In reply to Andras Balogh

Re: Webwork LTI settings prompting multiple Canvas notifications

by D. Brian Walton -
I agree with Andras. You definitely do not want $LTIGradeOnSubmit=1 because that is going to send a grade update to Canvas after every submission, and many students are going to submit each problem multiple times.

Also, I was going to post tomorrow, but in light of the question here, I'll make it today. My test run is going very well. Grades are being updated but only grades that have changed are being sent. Students who had been forwarding me the rogue notifications have been silent for the last 5 days.

If others want to implement the same change, it involves adding some lines to one file. (/opt/webwork/webwork2/lib/WeBWorK/Authen/LTIAdvanced/SubmitGrade.pm) The proposed fix is provided as an attachment to the relevant bug report.

http://bugs.webwork.maa.org/show_bug.cgi?id=3963

(There is one extra modification to that file in that I have commented out a debug line that was giving me an error for students that had dropped the course and has not yet been resolved.)

I hope this helps.

Sincerely,
- Brian Walton
In reply to D. Brian Walton

Re: Webwork LTI settings prompting multiple Canvas notifications

by D. Brian Walton -
Tim,

I just wanted to let you know that I am currently testing a possible fix. I never found out why only some grades were resulting in notifications. In particular, I have verified that Canvas only looks at whole percentages. The code definitely rounds the reported percentage even though the gradebook could hold more precise information. So my hypothesis about how Canvas actually decides whether to send students notifications doesn't seem to work.

The solution I am testing is that Canvas allows the LTI tool to query the recorded grade as well as update the grade. So I have added some code that queries the grade and compares it with the grade that is about to be reported. The grade update only occurs when a change is observed. If no change is observed, the grade passback step is skipped.

I have posted my attempt at a fix on the bug report page and am currently testing whether it works properly.

- Brian