There is a value called
zeroLevel that determines when WeBWorK will switch to absolute error (by default it is 10^-14) and a value called
zeroLevelTol that is the tolerance to use in that case (10^-12 by default).
So if you want to distinguish values that are smaller than this range, you will need to adjust these two values. If you are using MathObjects, these are stored in the Context, for example
Context()->flags->set( zeroLevel => 1E-15, zeroLevelTol => 1E-13 );set these to smaller values. I would change these only for the particular answer checker where you need the higher precision, as it could increase the numeric instability of function checking when there are near zeros within the domain limits (the major source of problems with WeBWorK function checking).