Miscellaneous

Adding floor, ceil, etc. to the parser.

Adding floor, ceil, etc. to the parser.

by Ross Richardson -
Number of replies: 3
I need to add a number of functions to the parser.
Most of the functionality I want is in PGauxiliaryFunctions.
However, when trying to add them into the parser (by creating a class
and then using the Context()->function->add() approach), I run into problems.
It seems that no matter what I do, I can't reference the functions in PGauxiliaryFunctions.

I noticed that when log, exp, sin, etc. were added to the parser,
they referenced CORE::log, CORE::exp, etc. directly as opposed to
the versions created in PGcommonFunctions and accessible through main, and I'm not sure exactly why.

Anyway, I would be greatful if someone could point me in the right direction. I can always just recreate the functions I want and add them to the parser, but it seems in appropriate to have many versions of the same function floating around in different contexts.
In reply to Ross Richardson

Re: Adding floor, ceil, etc. to the parser.

by Davide Cervone -
Making new functions available within the new parser is certainly possible, non-trivial. The functions that are part of the parser are more than just perl code that produces the function's value. The function must actually be part of a perl object, and that object needs to be able to do things like check the number and type of the inputs to make sure they are appropriate, and indicate the type of the result so that the parser knows how to deal with the result within larger formulas. (This has to be done without actually calling the function itself, since the parsing of an expression is not the same as evaluating it.)

There are some examples in webwork2/doc/parser/extensions (see 1-function.pg and 2-function.pg in particular). There is also a discussion of this from the old discussion board at

   https://devel.webwork.rochester.edu:8002/webwork2_files/moodle/mod/forum/discuss.php?d=2645
(see in particular my examples about a third of the way down the page.

However, when trying to add them into the parser (by creating a class and then using the Context()->function->add() approach), I run into problems.

Can you be more specific about the problems you are having? Also, could you post a code snippet showing exactly what you did? Without these, it is hard to diagnose the problem.

I noticed that when log, exp, sin, etc. were added to the parser, they referenced CORE::log, CORE::exp, etc. directly as opposed to the versions created in PGcommonFunctions and accessible through main, and I'm not sure exactly why.

The setup is a bit complicated, I admit. PGcommonFunctions.pl is a hack to make the parser coexist nicely with pre-parser based problems. Parser-based problems need to have the functions like tan() and log() be linked to the Parser-based versions of these functions, but older problems need them to be standard perl functions, not the parser-based equivalents. (When the parser was first introduced, it was not preloaded, and so its versions of those functions would not necessarily be available.)

Since WeBWorK produces warning messages if a function is redefined once it has already been defined, the Parser.pl file could not redefine tan() and log() to use its versions, and we wanted PG files to be able to load both the PGauxilaryFunction.pl and Parser.pl so the functions that were common to both were extracted to PGcommonFunctions.pl, where and these functions were made to check if the Parser.pl function was loaded to decide whether to use the parser-based versions or the traditional ones.

Note that PGcommonFuncitons.pl is not where these functions are added to the parser. This file is mainly for when the parser is not used. So the definitions of tan() and log() that you see there (in terms of CORE:log() and so one) are not used by the parser. They are used when the parser is not available. Note that they are part of the CommonFunctions package, and so must call the CORE:: versions of the functions, otherwise they would be calling themselves recursively. They don't want to use the main:: versions, because those call the CommonFunctions->Call() function to decide which version of the function to call: the parser one or the one in CommonFunctions pacakge.

The parser-based versions of the functions are defined in pg/lib/Parser/Function/numeric.pm and other related files. But you are right, they use CORE::log() rather than call the ones in main::, since the ones in main:: call CommonFunctions->Call(), and that in turn call the parser functions, and that would lead to an infinite loop. Somewhere you have to stop and call the actual perl definitions of those functions (which is what CORE:: does). It's true that the parser has its own copy of the definitions of the functions in the CommonFunctions package, but this is because the parser predates PGcommonFunctions.pl. While it would be possible for the parser code to call the versions in CommonFunctions rather than its own copies, it is probably not worth it.

I can always just recreate the functions I want and add them to the parser

If you want to add floor to the parser, you will need to make your own subclass of Parser::Function::numeric that has a floor method, and that method should call main::floor if you want to access the code from PGauxiliaryFunctions.pl. That will certainly work, but there is one drawback: the main::floor function won't work like other parser-aware functions, so if you use floor() in your PG file and pass it a formula, it will not create a formula object for you, as tan() or log() would. This probably won't be a hardship for you, but it is an inconsistency. If you try to redefine floor as suggested in the example code referenced above, you will run into the problem that required the creation of PGcommonFunctions.pl in the first place.

Note that adding new functions to CommonFunctions in PGcommonFunctoins.pl does not add them to the parser. That only makes them available when the parser isn't in use.

Finally, if you are planning to use floor() in numeric answers, you should not have any trouble, but if you want to use it in formulas, you should be aware that the method used to compare two formulas is based on the assumption that slightly different x values will generally produce slightly different results. With floor() this will not be the case, and so, for example, with limits of 0 to 1 for x values, the formulas floor(x), floor(x/2), floor(x^2), and the constant formula 0 would all be considered equal, which you might not want to be the case. So you might need to be more careful about selecting the limits for your formulas. (I'm assuming, here, that when you say you want to add a function to the parser, that means you want to allow a student to enter it in his or her answer.)

Hope some of that helps.

Davide

In reply to Davide Cervone

Re: Adding floor, ceil, etc. to the parser.

by Ross Richardson -
I think that clarifies my understanding of how things are organized.
My original hope was that I wouldn't need to have multipe versions of each
function floating around, but I imagine if I want Formulas to work out
it makes sense to make separate versions (which is what I ended up doing anyway).

I probably have enough to go on based on the above, but if there is any more Parser
documenation floating out there it would be good to know of it.

Thanks for the copious help.
-Ross
In reply to Ross Richardson

Re: Adding floor, ceil, etc. to the parser.

by Davide Cervone -
if there is any more Parser documenation floating out there it would be good to know of it.

As usual, the documentation falls far behind the code, and that is definitely the case for the Parser (not called MathObjects). There is some documentation in webwork2/doc/parser, but it is pretty sparse, and doesn't treat all the situations of interest. Many of the source files (particularly those in pg/macros) have some comments at the top that explain how to use them. I have also tried to post examples in the old discussion board (and will be in this new one) as people ask questions, partly in an attempt to provide at least some documentation, but there really needs to be much more. Mike has been working on some of that, and we hope to have more done this summer.

Your best bet may be to ask questions here, as I try to be pretty responsive to those, when I can.

Davide