Achievement Evaluator

The achievement evaluator is the system by which WeBWorK checks to see if achievements have been earned or not. Every time a student submits a problem WeBWorK goes through the list of assigned achievements for that student. For each achievement it loads up a perl script, called the achievement evaluator, and runs that script (in a Safe container). If the script returns 1 then the achievement is awarded and if it returns 0 then the achievement is not awarded. This means that all of the heavy lifting for assigning achievements is done by these perl script achievement evaluators.

Editing Achievement Evaluators

Each achievement as an evaluator file which can be set using the edit tab of the Achievement Editor page. The evaluator itself can be edited using the "Edit Evaluator" link. This will bring you to a text editor page where you can make changes, save, or save as a new editor which can replace the existing one or be used in a new achievement. The evaluators are assigned to a particular achievement using the edit tab and the evaluator files are all stored in the achievements directory in the templates folder. At heart an achievement evaluator is nothing more than a perl script that returns a true or false value. For example, here is the evaluator script for an achievement which is awarded when a student finishes a homework set (100% on every problem) between midnight and 2am.

my @timeData = localtime(time);

#test to see if it is between midnight and 2am return 0 if it isn't
if ($timeData[2] > 1) {
     return 0;
}

#if it is check to see if we have finished the set return 0 if we haven't
foreach my $problemRecord (@setProblems) {
     if ($problemRecord->status != 1) {
          return 0;
     }
}
    
#if we got this far then the student should be awarded the achievement
return 1;

The evaluator system is very flexible. You can implement any achievement which can be tested using a perl script.

Evaluator Environment Variables

When the achievement evaluator is run there are a number of environment variables available for use. Most of these variables are read only, but some of them will persist across evaluator calls. The available variables are listed below.

• $problem : The problem data. Changes to this variable will not be saved! This variable contains the problem data. It is a hash pointer to a standard WeBWorK problem data structure. The following variables are a sample of those stored in $problem.
  • $problem->status : the score of the current problem
  • $problem->problem_id : the id of the current problem
  • $problem->set_id : the id of the set containing the problem
  • $problem->num_correct : the number of correct attempts
  • $problem->num_incorrect : the number of incorrect attempts
  • $problem->max_attempts : the maximum number of allowed attempts

• $set : The set data. Changes to this variable will not be saved! This variable contains the set data. It is a hash pointer to a standard WeBWorK set data structure. It contains the following values (not all values shown).
  • $set->open_date : when the set was open
  • $set->due_date : when the set is due

• @setProblems : The problem data for all the problems from this set. Changes to this variable will not be saved! This is an array of problem hashes. Each element of the array has the same hash keys as the $problem variable above

• $counter : The users counter associated to this achievement. Changes to this variable will be saved! If this achievement has a counter associated to it (i.e. solve 20 problems) then this is where you store the students counter for this achievement. This variable will initially start as . If you do not store your counter data here students will not be able to see their progress on the Achievements page.

• $maxCounter : The goal for the $counter variable for this achievement. Changes to this variable will not be saved! If this achievement has a counter associated to it then this variable contains the goal for the counter. Your achievement should return 1 when $counter >= $maxCounter. This variable is set using the edit tab of the Achievement Editor page.

• $tags : This contains the metadata for the problem. This is a hash pointer to a standard tags structure. Note: These values are not super stable and will change as the library tagging is updated. It contains the following values (not all values shown).
  • $tags->DBsubject : The subject tags for the problem
  • $tags->DBchapter : The chapter tags for the problem
  • $tags->DBsection : The section tags for the problem

• $localData : This is a hash which stores data for this user and achievement. Changes to this variable will be saved! This hash will persist from evaluation to evaluation. You can store whatever you like in here and it can be accessed next time the evaluator is run. Two things to keep in mind. First, The data in this hash will not be accessible by other achievements. Second, the first time a variable is accessed it will have the value .

• $globalData : This is a hash which stores data for this user and all achievements. Changes to this variable will be saved! This hash will persist from evaluation to evaluation and, like $localData, you can store whatever you like in here. This data will be accessible from every achievement and is unique to the user.
  • There are two variables stored in this hash that are maintained by the system.
    • $globalData->completeSets : This is the number of sets which the student has earned 100% on.
    • $globalData->completeProblems : This is the number of problems which the student has earned 100% on.
  • Warning: The achievements are always evaluated in the order they are listed the Achievement Editor page. To make matters more complicated, achievements which have already been earned are not evaluated at all. The up-shot of this is that when modifying variables in $globalData you need to either write your code so it doesn't matter which order the evaluators are run, or you need to pay very close attention to which evaluators are run and when.
  • The Achievement Items are kept track of using this hash.

In addition to the above variables there are two special variables available to "Level" achievements. Level achievements behave differently than other achievements. When a level achievement is earned the system uses the icon and title of the level achievement as the icon and title of that students level. There are two additional variables available to level achievements.

• $achievementPoints : The number of achievement points the current user has. Changes to this variable will not be saved.
• $nextLevelPoints : The number of points needed to earn the a level. Changes to this variable will be saved if a new level is earned. These variables are used to construct the users level progress bar.