In pg/lib/Value/Real.pm, I've added some lines to sub compare, as indicated below. The idea is to allow a context to have `tolType=>'sigfig'` with `tolerance=>3`. For comparing two numbers, each number is converted to a string using sprint and a format like `%.3g`. Then the comparison is based on these two strings.
Since `tolerance=>0.001` is the default tolerance, intended for use with `tolType=>'relative'`, I also thought it convenient to make that situation [with a tolerance in the interval (0,1)] convert to a natural number, 3 in this case. In this way this sigfig tolType can be used out of the box without additionally having to set the tolerance to, say, 3.
My limited testing is showing this to work, but I have a few questions.
- Are there any red flags anyone sees in my code? Like, am I not accounting for certain types of numbers that might land in the `compare` subroutine somehow?
- To get this submitted as a pull request, what more would be needed? For example I see some things using `tolerance` in AnswerChecker.pm that appear to be used when running diagnostics. Should `sigfig` be accounted for there?
Here is the subroutine in question, with my additions highlighted.
sub compare {
my ($self,$l,$r) = Value::checkOpOrderWithPromote(@_);
#
# Handle periodic Reals
#
my $m = $self->getFlag("period");
if (defined $m) {
$l = $l->with(period=>undef); # make sure tests below don't use period
$r = $r->with(period=>undef);
if ($self->getFlag("logPeriodic")) {
return 1 if $l->value == 0 || $r->value == 0; # non-fuzzy checks
$l = log($l); $r = log($r);
}
$m = $self->promote($m); my $m2 = $m/2;
$m2 = 3*$m/2 if $m2 == -$l; # make sure we don't get zero tolerances accidentally
return $l + (($l-$r+$m2) % $m) <=> $l + $m2; # tolerances appropriate to $l centered in $m
}
my ($a,$b) = ($l->{data}[0],$r->{data}[0]);
if ($self->getFlag('useFuzzyReals')) {
my $tolerance = $self->getFlag('tolerance');
if ($self->getFlag('tolType') eq 'sigfig') {
# convert tolerance values meant for relative to a sigfig tolerance
$tolerance = -log($tolerance)/log(10) if ($tolerance > 0 and $tolerance < 1);
# make sure nonsensical tolerances are converted to a natural number
$tolerance = (1 > int($tolerance)) ? 1 : int($tolerance);
my $format = "\%.${tolerance}g";
return sprintf($format,$a) ne sprintf($format,$b);
}
if ($self->getFlag('tolType') eq 'relative') {
my $zeroLevel = $self->getFlag('zeroLevel');
if (CORE::abs($a) < $zeroLevel || CORE::abs($b) < $zeroLevel) {
$tolerance = $self->getFlag('zeroLevelTol');
} else {
$tolerance = $tolerance * CORE::abs($a);
}
}
return 0 if CORE::abs($a-$b) < $tolerance;
}
return $a <=> $b;
}
my ($self,$l,$r) = Value::checkOpOrderWithPromote(@_);
#
# Handle periodic Reals
#
my $m = $self->getFlag("period");
if (defined $m) {
$l = $l->with(period=>undef); # make sure tests below don't use period
$r = $r->with(period=>undef);
if ($self->getFlag("logPeriodic")) {
return 1 if $l->value == 0 || $r->value == 0; # non-fuzzy checks
$l = log($l); $r = log($r);
}
$m = $self->promote($m); my $m2 = $m/2;
$m2 = 3*$m/2 if $m2 == -$l; # make sure we don't get zero tolerances accidentally
return $l + (($l-$r+$m2) % $m) <=> $l + $m2; # tolerances appropriate to $l centered in $m
}
my ($a,$b) = ($l->{data}[0],$r->{data}[0]);
if ($self->getFlag('useFuzzyReals')) {
my $tolerance = $self->getFlag('tolerance');
if ($self->getFlag('tolType') eq 'sigfig') {
# convert tolerance values meant for relative to a sigfig tolerance
$tolerance = -log($tolerance)/log(10) if ($tolerance > 0 and $tolerance < 1);
# make sure nonsensical tolerances are converted to a natural number
$tolerance = (1 > int($tolerance)) ? 1 : int($tolerance);
my $format = "\%.${tolerance}g";
return sprintf($format,$a) ne sprintf($format,$b);
}
if ($self->getFlag('tolType') eq 'relative') {
my $zeroLevel = $self->getFlag('zeroLevel');
if (CORE::abs($a) < $zeroLevel || CORE::abs($b) < $zeroLevel) {
$tolerance = $self->getFlag('zeroLevelTol');
} else {
$tolerance = $tolerance * CORE::abs($a);
}
}
return 0 if CORE::abs($a-$b) < $tolerance;
}
return $a <=> $b;
}