Is there a standardized way to implement unit tests for subroutines in macro files? The subroutines I want to test use MathObjects. It is possible to easily test them separately, without a running WeBWorK server?
Not yet. But it's something I'm working on as well. You can see what I have so far in my github.com/mgage/pg branch "develop_plus_tableau". https://github.com/mgage/pg/tree/develop_plus_tableau
These are not quite ready for prime time and I haven't submitted a pull request to develop.
The files to look at are pg/macros/tableau.pl -- which adds tableau/matrix manipulations and operations which I wanted for the linear programming (simplex method) class that I'm teaching. The unit tests for this file are in
pg/t/matrix_tableau_tests. I'm working on this project during this semester so these files may change.
An earlier example of unit tests is the test_units.t file submitted by Boyd Duffee which tests the Units.pm file.
The files in pg/t/pg_test_problems are not unit tests in the sense of Test::More but they can be run individually from the command line or an editor using the script webwork2/clients/sendXMLRPC.pl. (This requires some of the CPAN modules used by webwork2 but not that many and requires a host that you have access to that will render .pg. You can use hosted2.webwork.rochester.edu for now. ) I use a mac and bbedit, but it should be possible for this to be adapted to any platform and editor.
The other option for editing and testing .pg without putting the problem in a homework set is: https://github.com/mgage/standaloneProblemRenderer
This is another project that "works for me" but is not ready for prime time yet. I hope to see it polished and packaged as a CPAN module so that one could download a module containing the standaloneProblemRenderer and all of the relevant CPAN and pg/ macros and libraries and edit *.pg files directly without the need of a server. There is still some small entanglement with modules that are part of webwork2 and not pg and it has only really been tested on a mac, using bbedit.
I believe that Davide Cervone also has some unit test files for the basic MathObject functions and I hope to find them, organize them and include them in the pg/t directory before I issue a pull request to develop.
If you have pull requests to my branch -- or ready to go directly to develop feel free to submit them.
-- Mike
Thanks for your reply. I am rather looking for unit tests in the sense of Test::Simple or Test::More. My point is not so much to test specific pg problem files, but to test whether algorithms in macros are mathematically correct. More concretely I want to define a number of MathObjects, run subroutines that take them as an input and compare their output with results I expect. My problem is that the subroutines use MathObjects. So I guess I need some parts of pg to test them. It seems that Units.pm is quite independent and so it seems to suffice to use it in pg/t/test_units.t. Let me give an example:
How would I implement a test that in a situation like the following (considered as part of a .pg file)
loadMacros(
"PGstandard.pl",
"MathObjects.pl",
);
$A = Matrix([
[1,0],
[0,1],
]);
$B = Matrix([
[2,0],
[0,3],
]);
$correct = ($A*$B == $B);
would evaluate $correct?
In reality I would replace $A*$B by something like mysub($A,$B) and compare its return value to a known object $C. A difference to Units.pm seems to be that I need MathObjects. Can I implement such a test outside of .pg files? I am asking, because .pg files seem not to be the right place to tests mathematical correctness of subroutines.
How would I implement a test that in a situation like the following (considered as part of a .pg file)
loadMacros(
"PGstandard.pl",
"MathObjects.pl",
);
$A = Matrix([
[1,0],
[0,1],
]);
$B = Matrix([
[2,0],
[0,3],
]);
$correct = ($A*$B == $B);
would evaluate $correct?
In reality I would replace $A*$B by something like mysub($A,$B) and compare its return value to a known object $C. A difference to Units.pm seems to be that I need MathObjects. Can I implement such a test outside of .pg files? I am asking, because .pg files seem not to be the right place to tests mathematical correctness of subroutines.
I don't have time to write anything in detail right now, but there are some test files for MathObjects listed in the ww-devel archives in the attachment file. That is a gzipped tar file, so you can extract it using
tar vfxz attachment.objonce you download the file. This shows how to load enough of PG to run MathObjects. The files assume the MathObejcts directory is in the pg folder, so you will have to change the paths to the libraries if you move it somewhere else.
It looks like some things in PG have changed since I posted the code. You should change
sub Parser::defineLog {eval {sub log($) {CommonFunction->Call("log",@_)}}};
to
sub ParserDefineLog {eval {sub log($) {CommonFunction->Call("log",@_)}}};
in MathObjects/parserTests.pl in order to get the tests to run. Some of the tests fail now because of changes in MathObjects since they were written. A number are because of changes to error messages. There are also changes in the Matrix object that allow negative exponents to mean powers of the inverse. I will have to look into the errors in the complex Re() and Im() functions.
In any case, it should show you how to call the MathObjects from the command line. If you intend to test the cmp() method, you will need to load some more PG libraries. I can give the details if that is part of your testing.
PS, all the tests currently in place use strings for the test expression, but the parserTests.pl also allows a code reference, so you can pass a subroutine that is run instead of a string.
There are some Test::More unit tests in the file:
in one of my other branches. The t directory in that branch is very much a work in progress so don't trust it too much. The file above will show you how I was implementing unit testing for the new macro file tableau.pl which will provide matrix routines for the simplex method. Davide's collection of unit tests is probably a better reference and model.