One way to get a sense of what is happening is to add diagnostics=>1 to the answer checker, as in
ANS($f->cmp(diagnostics=>1));
This will cause the problem to generate a diagnostic table showing the values used within the comparison for both the student and correct answers and whether they were within the tolerances or not. Often this can help determine what is going wrong.
Formulas involving very large and very small values tend to be problematic, and setting the tolerances and limits properly can be very important. In my experience, one of the most important caveats is to try to avoid the zeros of the function, as relative tolerances work poorly near values of zero.
Note that in your case, the default limits of -2 to 2 in the x value will include several zeros of the function, and depending on the problem seed, that course lead to incorrect results like what you have seen. You may find that restricting the limits to a range where the function is non-zero will help, say -pi/6 to pi/6 in your case.
Alternatively, you may need to use a custom answer checker for this. For example, it could evaluate the student function at x=0 (to determine the constant used by the student) and if it is not correct, mark the answer as false, while if it is correct, it could divide the student answer by that constant and compare it to cos(2x), which will no longer be enormously large values. I haven't tried that, but I think it would work. Anyway, it is worth a try.
Davide