We have a problem set with an especially long name. It comes from a chapter of our algebra textbook that automatically generates the problem set name from the section name. The problem set name is:
I've changed the problem set to have a shorter name, and reimported it for all affected faculty. So we have no emergency here. I'm just curious to understand what happened. And why, if there is a cap at some point on the set's name length, does the system still allow us to import/create that set in the first place.
With the switch to utf8mb4 and 4bytes devoted to each character the cap on the length (in characters) of various items is effectively decreased.
That’s almost certainly the answer to the question “what is different”?
I’m not sure why the set is allowed to be imported — it may be that we need to add some additional checks. At a guess the name is
imported and truncated silently.
The name is only 82 characters long I’m surprised its running up against a cap. We'll have to investigate more fully. Would you post this on the issues in github.com/openwebwork/webwork2 issues?
Take care,
Mike
As Mike said, this is a consequence of the reduced field length sizes.
UTF-8 tables under one of the common mySQL/MariaDB table engines require the primary key to be under about 1000 bytes, including a few bytes needed for overhead, which leaves about 240 characters total for the primary index fields.
Several VARCHAR(255) fields were reduced to VARCHAR(240), but the change which is most restrictive was to lib/WeBWorK/DB/Record/PastAnswer.pm as 3 fields jointly build the primary key.
From the output of
git diff b40bbff11f16bccd9471459c989dcd4602773768 lib/WeBWorK/DB/Record/PastAnswer.pm
- course_id => { type=>"VARCHAR(100) NOT NULL", key=>1},
- user_id => { type=>"VARCHAR(100) NOT NULL", key=>1},
- set_id => { type=>"VARCHAR(100) NOT NULL", key=>1},
+ course_id => { type=>"VARCHAR(80) NOT NULL", key=>1},
+ user_id => { type=>"VARCHAR(80) NOT NULL", key=>1},
+ set_id => { type=>"VARCHAR(80) NOT NULL", key=>1},
We could somewhat mitigate this issue if we set a short size limit on course_id, and maybe also user_id. user_id is somewhat more dangerous to limit to strictly, as LTI and LMS systems may be setting it using an email address or hask-key ID they use. However, we could certainly set an smaller upper limit on the course_id for the future.
Is there a fix to this? I am getting similar error but not for set_id, it is for "frozen_hash':
DBD::mysql::st execute failed: Data too long for column 'frozen_hash' at row 1 at /app/webwork/webwork2/lib/WeBWorK/DB/Schema/NewSQL/Std.pm line 837.
The correct answer also gets marked incorrect.
The problem in my case seems to be that students have to enter a 5x5 or larger matrix. I guess that also gets added to the set_id and other stuff. ADo I have to decrease the matrix size?
That is a different issue. The answers for the problem are not a part of the key for the database row, and so that data is not limited in the same way. There is still a limitation on the data size based on the chosen data type of the last answer column, and there are an increasing number of cases that are indicating that that column needs to have its data type changed to allow for more data.
Do you by chance have MathQuill enabled? That increases the amount of data that is stored in the last answer column.
I don't have MathQuill enabled. But I do have achievements enabled. In a sandbox course disabling and deleting achievements removes the error message. Does this sound reasonable? Is there something else that would decrease the data?
Achievements are stored in another table, but the same problem can occur there. The frozen_hash column of that table stores local data for an achievement. In particular the still_not_right.at achievement stores all of the last answer data in that column (with some other data). That achievement can easily exceed the VARCHAR(1024) size limits. This is a known issue.