** Using WeBWorK **

There are a couple of issues, here.

First, AnswerHints() usually only does its testing when the student answer is incorrect, but there is an exception to this that is not well documented: it will also perform its check when the test answer (the on in the AnswerHints()) is the correct answer. This is not explicitly stated in the documentation (though it should be), but it is implicit in the example given in the file:

        ANS(Vector(1,2,3)->cmp(showCoordinateHints=>0)->withPostFilter(AnswerHints(
                Vector(0,0,0) => "The zero vector is not a valid solution",
                "-<1,2,3>" => "Try the opposite direction",
                "<1,2,3>" => "Well done!",
                ["<1,1,2>","<2,2,2>","<3,3,3>"] => "Don't just guess!",
                sub {
                        my ($correct,$student,$ans) = @_;
                        return $correct . $student == 0;
                } => "Your answer is perpendicular to the correct one",
                Vector(1,2,3) => [
                        "You have the right direction, but not length",
                        cmp_options => [parallel=>1],
                ],
                0 => ["Careful, your answer should be a vector!", checkTypes => 0, replaceMessage => 1],
                sub {
                        my ($correct,$student,$ans) = @_;
                        return norm($correct-$student)  ["Close!  Keep trying.", score => .25],
        )));

The <1,2,3> answer is the correct answer, and to get the message for that, we need to do the check when the student answer is correct.

Since your $periodic answer actually is the correct answer, the message is given when the student answer is correct. The reason you don't get this for the subroutine case is that there is no "test" answer in that case, so the special case of the test answer being the correct answer never comes into play. So you are right that using a subroutine is important in this case.

There is another problem, however, which is that the periodic property applies to MathObject Real objects, but you are using fractions, so you are actually getting a MathObject Fraction, not a Real. Unfortunately, Fractions don't support the periodic property, and so your testing is actually just doing a plain old non-periodic test. That is why you couldn't get it to work with the original Compute() code.

When you switched to Formula(), however, that changed things. Formulas are compared by comparing their values at several random inputs; but evaluating a formula that involves a fraction produces the real value, not the fraction, and so that allowed the answer to be a periodic Real rather than a Fraction. So the comparison worked as you intended.

You should not use Formula objects when the thing you are testing is a number (and not a constant-valued formula). That is because formulas will produce error messages about formulas, and that can be confusing for students.

Fortunately, however, your $periodic and $alt are never displayed, so you don't have to keep them as Fractions, but can make them Reals:

#all solutions based on the principle solution
$periodic=Real("(1/-$a)*($inv-$b)")->with(period=>Real("2*pi/$a"));

#all other solutions
$alt= Real("(1/-$a)*((pi-$inv)-$b)")->with(period=>"2*pi/$a");

You are right that the message will appear even when just doing a preview. That should probably be considered a bug in AnswerHints() (and I can't believe no one has reported it before now!) and should probably be fixed (there should be a processPreview flag to allow it, but the default should be not). So again, a subroutine is the only current way to handle that. So you are definitely on the right track.

Here is a version of the function you asked for:

$HintNoPreview = sub {
  my $values = shift;
  $values = [$values] unless ref($values) eq 'ARRAY';
  return sub {
    my ($correct, $student, $ans) = @_;
    return 0 if $ans->{isPreview};
    foreach my $value (@$values) {
      return 1 if $value == $student;
    }
    return 0;
  }
};

This takes a value (or array of values) and returns a subroutine that tests the student answer against the given one(s), returning 1 if there is a match, and 0 if not or if this is a preview. The key here is that this subroutine returns another subroutine, which is the one actually used by AnswerHints().

Then you can use

ANS($ans->cmp()->withPostFilter(AnswerHints(
  &$HintNoPreview([$periodic,$alt]) => "Although this is A solution, it is not the PRINCIPLE solution"
)));

to get the checker for both values, as you requested. Note that you do need the & in &$HintNoPreview in order to actually make the function call.

I think that covered everything.