WeBWorK Main Forum

Possible to write new questions in Python instead of Perl?

Possible to write new questions in Python instead of Perl?

by Christian Seberino -
Number of replies: 17

Is it possible to write questions for Webwork in Python?

I know Perl can call Python scripts but I wondered if there already

existed a framework for this somewhere.

Sincerely,

Chris

In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Kurt O'Hearn -
I am interested in the answer to this question as well, as I have much greater experience with and affinity toward Python than Perl.

-Kurt
In reply to Kurt O'Hearn

Re: Possible to write new questions in Python instead of Perl?

by Christian Seberino -
Kurt

Yay! I'm not the only one interested in this! I'd love to chat further about this until we hammer out a working solution for both of us if you want. It shouldn't be that hard.

One solution: For every question, we can write a Perl template that calls a Python script for all the work.

Then every question needs TWO files....the Python code and the standard Perl glue file to connect it to Webwork. But that is still nicer that using Perl for Python fans. :) So for example, you'd have prob_1.pg, prob_2.pg, prob_3.pg in the same directory along with prob_1.py, prob_2.py, prob_3.py. The beauty is that all the *.pg files are THE SAME. They just call a Python file with the same name as itself for the details!

Here is some preliminary thoughts about the prob_?.pg template....

Every question file needs values holding the following:

answer values
answer units
text of question

We can have something like this in the Perl template:

$answer_val = <insert Perl code here to call Python script with command line switch to ask for answer value>

$answer_units = <insert Perl code here to call Python script with command line switch to ask for string denoting answer units>

$question_text = <insert Perl code here to call Python script with command line switch to ask for string denoting question text>

The answer units and question text are ALWAYS strings. The code to return the answer value would need to also return a string denoting the TYPE so that the Perl glue could do the appropriate type cast.

cs
In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Michael Gage -
I think you could do this more simply by modifying the file pg/lib/WeBWorK/PG/Translator.pm  to simply process a problem  prob1.py differently from the way it processes the file prob1.pg.  The translator already 
has a preprocessing stage (where all the \ used in the tex commands are escaped and things like BEGIN_TEXT/END_TEXT are expanded) so you could replace the current preprocessor with something that helps tie the perl and python together.  Perhaps there is a python interpreter written in perl somewhere? I couldn't find one quickly.  In any case the translator could split
depending on the problem extension and do either perl-like processing or python-like processing (or ruby, javaScript or Forth :-) ).

I'm curious about what you are going to do with all of the current PG macros written in perl -- do you need to rewrite them in python or can you tie back into the perl code in some way and use the existing macros?  

As a completely different idea -- have you looked at the PGML language (PG markdown language) that Davide Cervone wrote?  

That helps hide some of the perlisms (but it's not python either).
 
http://webwork.maa.org/wiki/PGLabs#.T8PW6Y6ZAso

https://courses.webwork.maa.org/webwork2/cervone_course/PGML?login_practice_user=true

https://courses.webwork.maa.org/webwork2/cervone_course/PGML-examples/?login_practice_user=true

Keep in touch.  I'd like to see where this goes.

--Mike



In reply to Michael Gage

Re: Possible to write new questions in Python instead of Perl?

by Jason Aubrey -
Also, are there python/ruby/javascript/<name your favorite scripting language> modules that can be used to implement the same saftey/security functionality as Safe.pm now does? It seems to me that this would have to be addressed before we would want to make an another interpreter available to problem authors.
In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Michael Gage -
Looking at this a little bit more I think your scheme might be good way to start out. Once you know which parts of the code are in python and which in perl we can tweak translator.pm and eliminate the middle .pg file. You might look at Inline::python on CPAN
In reply to Michael Gage

Re: Possible to write new questions in Python instead of Perl?

by Christian Seberino -
I used your suggestion for Inline::Python. I'm having a little trouble integrating working code into Webwork I wanted to ask you about if you don't mind...

This test code can be run by Perl and successfully defines functions visible to Perl from Python...
(The mail forum app seems to be removing tabs from my Python code so if you try to run the code you'll need to add a tab before the returns on the Python functions.)

#============================================================================

$answer_val = get_answer_val();
print $answer_val;
print "\n";

$answer_units = get_answer_units();
print $answer_units;
print "\n";

$intro = get_intro();
print $intro;
print "\n";

$question = get_question();
print $question;
print "\n";

use Inline Python => << "END_OF_PYTHON_CODE"

def get_answer_val():
return 5

def get_answer_units():
return "ft"

def get_intro():
return "A 2 ft rope is tied to a 3 ft rope."

def get_question():
return "What is the length of the new rope?"

END_OF_PYTHON_CODE

#============================================================================


The Webwork pg file below doesn't work. From the error messagse, the conflict appears to be that Webwork expects ENDDOCUMENT(); to the last line and Inline::Python expects to be at the end of a file with nothing coming after.
If I make ENDOCUMENT last then I get a syntax error. If I put the Python code AFTER the ENDDOCUMENT then it gets ignored. Can't win.

#============================================================================

DOCUMENT();
loadMacros("PG.pl",
"PGbasicmacros.pl",
"PGauxiliaryFunctions.pl",
"PGchoicemacros.pl",
"PGanswermacros.pl",
"PGgraphmacros.pl",
"PG_CAPAmacros.pl");
## ----------------------------------------------------------------------------

$answer_val = get_answer_val();
$answer_units = get_answer_units();
TEXT(beginproblem());
CAPA_import( " ${CAPA_Tools}Problem ");

TEXT(CAPA_EV(<<'END_OF_TEXT'));
get_intro();
$BR$BR
get_question();
END_OF_TEXT

## ----------------------------------------------------------------------------

TEXT("$BR",ans_rule(30),"$BR");
ANS(CAPA_ans($answer_val,
"unit" => $answer_units,
"format" => "%0.2e",
"sig" => "3 PLUS 13",
"reltol" => 1,
"wgt" => $prob_val,
"tries" => $prob_try));
TEXT(CAPA_EV(<<'END_OF_TEXT'));
END_OF_TEXT

## ----------------------------------------------------------------------------

use Inline Python => << "END_OF_PYTHON_CODE"

def get_answer_val():
return 5

def get_answer_units():
return "ft"

def get_intro():
return "A 2 ft rope is tied to a 3 ft rope."

def get_question():
return "What is the length of the new rope?"

END_OF_PYTHON_CODE

ENDDOCUMENT();
#============================================================================



In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Christian Seberino -
I made some progress...the version below solves the problem of what must be at the end of the pg file but it gives another error I need help dissecting...

ERROR caught by Translator while processing problem file:capaLibrary/type_itt_problems/test/prob_test2.pg
****************
'require' trapped by operation mask at line 12 of (eval 1563)

****************

------Input Read
1 DOCUMENT();
2 loadMacros("PG.pl",
3 "PGbasicmacros.pl",
4 "PGauxiliaryFunctions.pl",
5 "PGchoicemacros.pl",
6 "PGanswermacros.pl",
7 "PGgraphmacros.pl",
8 "PG_CAPAmacros.pl");
9 ## ----------------------------------------------------------------------------
10
11 # ============================================================================
12 use Inline::Python qw(py_eval);
13
14 py_eval(<<'END');
15
16
17 class Foox:
18 def get_answer_val(self):
19 return 5
20
21 def get_answer_units(self):
22 return "ft"
23
24 def get_intro(self):
25 return "A 2 ft rope is tied to a 3 ft rope."
26
27 def get_question(self):
28 return "What is the length of the new rope?"
29
30 # ============================================================================
31 END
32
33 my $o = Inline::Python::Object->new('__main__', 'Foox');
34
35 $answer_val = $o->get_answer_val();
36 $answer_units = $o->get_answer_units();
37 $intro = $o->get_intro();
38 $question = $o->get_question();
39 TEXT(beginproblem());
40 CAPA_import( " ${CAPA_Tools}Problem ");
41
42 TEXT(CAPA_EV(<<'END_OF_TEXT'));
43 $o->get_intro();
44 $BR$BR
45 $o->get_question();
46 END_OF_TEXT
47
48 ## ----------------------------------------------------------------------------
49
50 TEXT("$BR",ans_rule(30),"$BR");
51 ANS(CAPA_ans($answer_val,
52 "unit" => $answer_units,
53 "format" => "%0.2e",
54 "sig" => "3 PLUS 13",
55 "reltol" => 1,
56 "wgt" => $prob_val,
57 "tries" => $prob_try));
58 TEXT(CAPA_EV(<<'END_OF_TEXT'));
59 END_OF_TEXT
60
61 ## ----------------------------------------------------------------------------
62
63 ENDDOCUMENT();

In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Jason Aubrey -
This is related to my earlier message about Safe.pm.  Problem authors are essentially given a mini-programming environment.  It's "mini" in the sense that the amount of perl problem authors are allowed to use is restricted to what is necessary for problem authoring and which can't be used to wreck things. So the message " 'require' trapped by operation mask at line 12 of (eval 1563)" is telling you that the Safe compartment in which your problem file is evaluated does not allow you to use (or require) perl modules.


In reply to Jason Aubrey

Re: Possible to write new questions in Python instead of Perl?

by Christian Seberino -
I have a dev server. Please tell me how fix.

Presumably your security layer has a whitelist somewhere that we can add a few modules that we *ARE* allowed to use? That would be a nice way to avoid having to completely shut down the security just to use Python.


Thanks.

cs
In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Michael Gage -
In global.conf.dist there is a list of modules that are automatically into the PG Safe compartment.  

You may want to add your own module  PGPython.pm (for example) where you would also include some of your code for gluing Python and Perl together.

You will probably need to add both PGPython.pm and Inline::Python to the list of modules to be loaded in global.conf.dist.

Hope this helps.

-- Mike



In reply to Michael Gage

Re: Possible to write new questions in Python instead of Perl?

by Jason Aubrey -
Yes, as Mike indicated the whitelist is the $pg{modules} hash in global.conf.

A minimal first attempt at this would be to add just Inline::Python to that list, and then create a macro file that instantiates an Inline::Python object.  Then add that to your list of macros in loadMacros at the beginning of a problem. 

I haven't tested this, and I only looked briefly at Inline::Python, so it may not work and may even be way off base, but here would be my first attempt: Add Inline::Python to the list, and creat a new macro file in pg/macros called inline_python.pl containing this code.

sub _inline_python {}; 

sub python { 
 my $python_code = shift;
 use Inline::Python => $python_code; 
}

Put it in pg/macros, then in a problem do 

loadMacros( 
 standard stuff, 
 "inline_python.pl"
);

Then to actually try it:

$python_code = <<'END_PYTHON'; 

your python code 

END_PYTHON

Maybe your python code is 

def add(x,y): 
 return x + y def 
subtract(x,y): 
 return x - y 

Then you could see if you could use the subroutines in your problem: 

$a = 1; 
$b = 2; 
$c = add($a,$b); 
BEGIN_TEXT 
c equals \{ ans_rule(4) \} 
END_TEXT 
ANS($c -> cmp());

Again, absolutely no idea if this is even close, but this is what I would try.
In reply to Jason Aubrey

Re: Possible to write new questions in Python instead of Perl?

by Jason Aubrey -
Oh, and I forgot after defining $python_code to use the function in the macro:

python($python_code);
In reply to Jason Aubrey

Re: Possible to write new questions in Python instead of Perl?

by Jason Aubrey -
Also, (I should have noticed this) the 'use Inline::Python' will also get trapped by Safe.  So, you'll have to put that part back in a PGPython.pm in pg/lib as Mike suggested and include it also in the $pg{modules} list.
In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Dick Lane -
What are the major reasons to write a WeBWorK problem using Python instead of Perl?

Note: I would choose Python instead of Perl for a new project involving numerical or graphical work, and I yearn to use true multi-dimensional arrays in some WeBWorK problems.  However, in the near term, fewer than 3% of my problems have been needlessly complicated by Perl syntax.  Considering Mike's comments about modularity, qualified by Jason's about security, I wonder whether Python enthusiasts have set their sights too low.
In reply to Dick Lane

Re: Possible to write new questions in Python instead of Perl?

by Christian Seberino -
Dick L.

What did you have in mind instead?

cs
In reply to Christian Seberino

Re: Possible to write new questions in Python instead of Perl?

by Dick Lane -
Hi Christian,

In hindsight, I suspect my query should have had just the first, short-range, question.  Here's a more focused paraphrase of it.

What will be the immediate benefits of work to extend current WeBWorK (v 2.5.0) to allow Python scripting within a problem template?  Several examples with contrasting fragments of pseudo-code would be useful, especially if discussion of them highlights key differences between Perl and Python functionality or syntax.

dick