[ww-devel] PG Tests

[Davide P. Cervone]

I actually have an automated test suite for the main MathObject classes that test much of their functionality (though I admit I haven't updated them in a while).  Currently, these only test the core functions, not the answer checkers, but tests could be added for those as well.  It doesn't use anything fancy, just some perl code I wrote to do the testing, but they do provide some quality assurance.  I just ran them to see if they still work, and they did point out a change in an error message, and a change to how matrices to negative powers are handled, and they picked up an obscure error in how Complex functions promote non-MathObject arguments. That might want to be fixed.

These could certainly be used to test MathObjects without having to rung all of PG to render problems.  The 1100 tests currently in the suite took about 10 seconds to run.

Adding in the answer checker tests would at least double the number of tests.  And then there are the extensions in the pg/macros directory (all the context* and parser* files).  So there is some work to be done to get a full MathObject test suite.

More than half of the items on the proposed list are MathObjects, and I think this sort of testing makes more sense for them than straight PG problems, though I suppose the test framework could be run within a PG problem as well, if you really wanted to have everything be a PG problem.

Davide


On Aug 13, 2014, at 7:54 PM, goehle at gmail.com wrote:

> I was thinking it would be really helpful to have a "testing" HW for PG that just contains a bunch of different HW problems which test the various PG modules.  The list of files in pg/macros is pretty imposing however.  So I wanted to first get a list of "important" modules and functions and make a set of say 20 or 30 problems which tests the core features.  I've put a draft list here: 
> 
> https://github.com/openwebwork/pg/wiki/PG-Testing-List 
> 
> I would really appreciate it if people (especially those familiar with PG and its storied history) could take a look at this list and modify/change it however they see fit.  In particular 
> -  Are those the correct way to group things?  Should some things be combined and others split up?
> -  Are there things which are missing that should be there?  Do I have things listed which are really only used in corner cases? 
> -  For each problem/module what are the important things to test?  Are there options (like singleResult) which really change how the thing works? 
> -  Which contexts are really important to test? 
> -  Which graph macros are still in heavy use and how should they be tested? 
> 
> Also, if someone has already done this then I would be happy to just make use of their work. 
> 
> Thanks a bunch. 
> 
> Geoff.
> 
> _______________________________________________
> webwork-devel mailing list
> webwork-devel at webwork.maa.org
> http://webwork.maa.org/mailman/listinfo/webwork-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://webwork.maa.org/pipermail/webwork-devel/attachments/20140819/b76dfb59/attachment.html>