WeBWorK Main Forum

read_whole_problem_file fails erratically. Using Ubuntu 14.04

read_whole_problem_file fails erratically. Using Ubuntu 14.04

by George Jennings -
Number of replies: 11
"upgrading" to Ubuntu 14.04 LTS with the Apache 2.4.7 has caused us many problems. So far we have fixed most of them by changing to the development branch of webwork 2.9, and upgrading everything. But there is one problem that just never goes away:

The function read_whole_problem_file fails erratically. Sometimes it works for a while, sometimes it doesn't, sometimes rebooting the web server cures the problem, sometimes it doesn't. This is a big headache because the problem database includes many links from one file to another, apparently to reduce the number of duplicates, but this seems to depend on read_whole_problem_file working reliably.

Here is an example of that type of error.

ERROR caught by Translator while processing problem file:rochesterLibrary/setInt
egrals15ByParts/bennyparts1.pg
****************
ERRORS from evaluating PG file:
Undefined subroutine &main::read_whole_problem_file called at [PG]/macros/PG.pl
line 661
Died within main::includePGproblem called at line 10 of (eval 1760)



****************

------Input Read
1 # This file is just a pointer to the file
2 #
3 # "Library/Union/setIntByParts/bennyparts1.pg"
4 #
5 # You may want to change your problem set to use that problem
6 # directly, especially if you want to make a copy of the problem
7 # for modification.
8
9 DOCUMENT();
10 includePGproblem("Library/Union/setIntByParts/bennyparts1.pg");
11 ENDDOCUMENT();
12
13 ## These tags keep this problem from being added to the NPL database
14 ##
15 ## DBsubject('ZZZ-Inserted Text')
16 ## DBchapter('ZZZ-Inserted Text')
17 ## DBsection('ZZZ-Inserted Text')

-----

Here is an example of a similar error that doesn't seem to be caused by
links to a duplicate problem

ERROR caught by Translator while processing problem file:Library/Union/setIntRie
mannSums/an6_4_34_left.pg
****************
ERRORS from evaluating PG file:
Undefined subroutine &main::read_whole_problem_file called at line 27 of [TMPL]
/Library/macros/Union/unionInclude.pl
Died within main::includePGfile called at line 45 of (eval 1023)



****************

------Input Read
1 ## DESCRIPTION
2 ## Calculus: Riemann Sums
3 ## ENDDESCRIPTION
4
5 ## Tagged by cmd6a 8/9/06
6
7 ## DBsubject(Calculus - single variable)
8 ## DBchapter(Techniques of integration)
9 ## DBsection(Approximation)
10 ## Date(8/23/07)
11 ## Institution(Union College)
12 ## Author(K. Lesh)
13 ## MLT(approx_02)
14 ## Level(2)
15 ## MO(1)
16 ## TitleText1('Calculus')
17 ## AuthorText1('Anton')
18 ## EditionText1('7')
19 ## Section1('6.4')
20 ## Problem1('34')
21 ## TitleText2('Calculus: Early Transcendentals')
22 ## AuthorText2('Rogawski')
23 ## EditionText2('1')
24 ## Section2('5.1')
25 ## Problem2('19')
26 ## KEYWORDS('calculus','integration','riemann sums')
27
28 DOCUMENT(); # This should be the first executable line in the problem.
29
30 loadMacros(
31 "PGstandard.pl",
32 "PGunion.pl", # Union College utilities
33 "MathObjects.pl",
34 "unionInclude.pl", # Utilities for included files
35 "PGcourse.pl", # Customization file for the course
36 );
37
38 TEXT(beginproblem());
39
40 ###################################
41 # Setup
42
43 # The following file defines the function to be used
44
45 includePGfile("an6_4_34-define-function-text.pg");
46
47 # $offset (if present) is a number between 0 and 1 that
48 # indicates where within each interval, the point should
49 # be taken. E.g., 0 = left-end, .5 = midpoint, 1 = right-end.
50 # The default is 0.
51
52 $offset = 0; #use left endpoints
53
54 ###################################
55 # Main text
56
57 #The following file generates the problem text for each
58 #Riemann sum estimate.
59
60 includePGfile("riemann-sum-estimate-text.pg");
61
62 ###################################
63 # Answers
64
65 $showPartialCorrectAnswers = 1;
66
67 #The $ans is included from the files above
68
69 ANS(Real($ans)->cmp);
70
71 ###################################
72
73
74 COMMENT('MathObject version');
75 ENDDOCUMENT();

-----

Is there a fix for this or should we just dump Ubuntu 14.04 and apache2.4 and rebuild the server with something else?

Thanks for any help you can give,

George
--
George Jennings, Math Dept., Calif. State University, Dominguez Hills


In reply to George Jennings

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Alan Ableson -
> Undefined subroutine &main::read_whole_problem_file called at [PG]/macros/PG.pl

We have had the same problem, but it appeared to be consistent rather than sporadic. To remedy it, we would update all the problems to the actual source problem, rather than the redirection:

- Open each homework set in HWSet Editor.
- Display all the problems.
- For each problem that had an error in when displaying,
- "Edit it" - this opens a new tab and shows the source with the includePGfile command.
- Select and copy the actual source file .pg name.
- Tab back to the homework set and paste the redirect target over the original .pg path.
- Save.
[then check the homework set again: sometimes the historic redirections were 3-5 deep!]

Removing these includePGfile references has eliminated the &main::read_whole_problem_file errors we were getting. The downside is obviously that it can be a fair amount of work.

If you are comfortable writing scripts, it might be more efficient to extract the .def files, grep for includePGfile in each of them and either do the substitutions with the script, or at least automate the indification of the files at issue. Alternatively, we could try to figure out why includePGfile isn't working as it used to. :)

Alan




In reply to George Jennings

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Sarunas Burdulis -
We are having the same problem! Ubuntu 14.04, tracking Webwork master branch. Supposedly homework problems were checked before being assigned, but now the ones that use includePGproblem() are suddenly failing:

ERROR caught by Translator while processing problem file:Library/ma123DB/set10/s11_1_9.pg **************** ERRORS from evaluating PG file:

Does this look like the same issue as for fileFromPath() in IO.pm?

Sarunas

In reply to Sarunas Burdulis

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Michael Gage -
It looks like the same problem as the fileFromPath() error to me.  It seems to be caused because the new perl loads modules asynchronously in a way that we hadn't anticipated when writing the original code. 

You should be able to fix it temporarily (or at least verify what causes the problem) by using the fully qualified function name  WeBWorK::PG::IO::read_whole_problem file.  

The long term fix for this problem is probably
the pull request https://github.com/openwebwork/pg/pull/147  which 
hasn't yet gone through a complete testing. You could pull that to master (or the corresponding pull request to develop if you are using the develop branch) and help with the testing if you want.  

Otherwise we should have the testing done within a week and it will be merged into the current develop and master branches. 

In reply to Michael Gage

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Michael Gage -
You also need to pull the corresponding pulls for webwork2. --465 in this case. Thanks to George for reminding me.
In reply to Michael Gage

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by George Jennings -
Thanks! Pull webwork2 #465 and pg develop #148 (is that the same as #147?) seem to have worked for us on our production server. No complaints from students in last 36 hours.
In reply to George Jennings

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Michael Gage -
Glad it's working George.  #148 is the pull request designed to patch the develop branch. #147 patches the master branch.  Since you were already running the develop branch you used the correct pull request. (I think pulling #148 to a master branch installation might work as well but you might get a few extra features you weren't expecting. :-)   I don't advise it.)

Geoff Goehle has written some notes on development procedures on the github wiki of which the most relevant section is at https://github.com/openwebwork/webwork2/wiki/Coding-and-Workflow. (The rest of Geoff's comments on the wiki are worth reading as well. )

In order to keep the code development clear and organized we try not to mix
the develop and master branch except at release points where the development branch is merged into the master branch. At the moment these new releases appear about every 6 months (depending somewhat on the amount of time people have available to get them ready and tested). 

-- Mike
In reply to Michael Gage

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Sarunas Burdulis -
Using fully qualified function name in pg/macros/PG.pl line 650 doesn't seem to help. The error changes from


Is there a way to run the new Perl with asynchronous loading disabled?

I'll do the testing with pull request #147 on a non-production install.

In reply to Sarunas Burdulis

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Sarunas Burdulis -
I have pulled

curl https://github.com/openwebwork/pg/pull/147.patch
curl https://github.com/openwebwork/webwork2/pull/464.patch

and did 'git apply' on a fresh install of Webwork (ww_install.pl) on Ubuntu 14.04.

This seems to resolve the undefined read_whole_problem_file() and the earlier fileFromPath() issues: restarted Apache about 15 times while checking problem rendering in between.

This is all on a test system. How much testing did these patches already have been through? Before considering them on a 'production' system I tried reversing the patches:

git apply --reverse 147.patch

but I am getting 'error: macros/PG.pl: patch does not apply' for all files involved in the patch. Reversing works fine for #464.


In reply to Sarunas Burdulis

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Michael Gage -
I'm not a git expert but my practice when adding something experimental is to proceed as follows:

# check that I'm on the branch I think I'm on:
git branch
... * develop
git checkout -b develop_tmp    # creates a new branch containing
                               # same files as develop
git pull https://.....address here of the pull request.
Now I have the new pull request applied to my version of develop.

If things don't work out I can just do

git checkout develop

and I'm back where I started from. I can even do this temporarily if I want to think about some problem with the new version but I want the server to work normally while I'm thinking.  

It's a slightly different work flow pattern than with some other version control systems but once you are used to it, it works fairly well.


git experts may have a more elegant method of using git to test out patches. I'd be happy to learn.



In reply to Michael Gage

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Bryan O'Shea -
I've pulled PG request #147 and Webwork2 Request #464.  I can confirm that, after applying, the loadPGProblem macro works correctly.  I haven't noticed any unintended behavior yet.
In reply to Bryan O'Shea

Re: read_whole_problem_file fails erratically. Using Ubuntu 14.04

by Michael Gage -
Thanks  to those who have pulled and tested pg#147 and webwork2#464 
(as well as the companion fixes to the develop branch pg#148 and webwork2#465).  

I have merged those four pull requests into their respective branches.  This will fix the read_whole_problem issue (and the fileFromPath) issue. As best we can tell there are no unwanted side effects.  If you find one you can report it here and/or to bugzilla.  

Anyone pulling the most recent version of the master branch or of the develop branch from github will get these patches automatically.