If you restore a course from an archive, then the database tables will use the encoding that was used when the course was archived, so even if your new server is set to use utf8mb4, any courses imported from previous versions might not be. Moreover the "Upgrade Courses" feature in Course Administration doesn't handle upgrades of character sets and collations, so it wouldn't fix this issue.
In your case the error appears to be coming from the comment field, so it may be sufficient to log into the course and use the Accounts Manager to delete any text from the comment field for the affected user(s).
If you want to make sure that everything is set up for utf8mb4, you can try running the upgrade-database-to-utf8mb4.pl script, but I recommend backing up your database before doing this.