Installation

Error: /webwork2/instructorXMLHandler: Forbidden

Error: /webwork2/instructorXMLHandler: Forbidden

by F. Heiderich -
Number of replies: 15
Using WeBWorK 2.13 (more particularly the branch https://github.com/heiderich/webwork2/tree/develop_uft8_ver3) and clicking "Render all" in the homework set editor of a bigger homework set (around 100 problems), the first problems render fine. Then a popup window with the error

/webwork2/instructorXMLHandler: Forbidden

appears. If then I try to render additional problems (using the the render buttons of the individual problems), I get the error this error again. The webserver error log shows the following:

Error message for command: renderProblem
<br/>faultcode: Server
<br/>faultstring: Not an ARRAY reference at /opt/webwork/webwork2/lib/WebworkWebservice/RenderProblem.pm line 547.

<br/>End error message<br/>
Use of uninitialized value in subroutine entry at /usr/share/perl5/HTML/Scrubber.pm line 181.
[Tue Aug 08 20:42:21.486878 2017] [perl:error] [pid 31773] [client 192.168.122.1:47832] [/webwork2/instructorXMLHandler] Error message for command: renderProblem \n<br/>faultcode: Server \n<br/>faultstring: Not an ARRAY reference at /opt/webwork/webwork2/lib/WebworkWebservice/RenderProblem.pm line 547.\n \n<br/>End error message<br/>\n * in WeBWorK::ContentGenerator::instructorXMLHandler::content called at line 233 of /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator.pm\n * in WeBWorK::ContentGenerator::go called at line 384 of /opt/webwork/webwork2/lib/WeBWorK.pm, referer: https://webwork-server/webwork2/TestCourse/instructor/sets2/Matrices/?user=admin&key=LLha9DqMjHCUOFGDIhngaESSpDrmielr&effectiveUser=admin

Does anybody have an idea what might be the reason for this error? In the past I experienced timeout errors in such situations (I think due to limited resources). But the "Forbidden" error is new to me.
In reply to F. Heiderich

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Michael Gage -
I don't have time to debug this right now (I'll be back for real next week.)

You can use View->Developer->Developer tools (on Chrome, or Firebug on Firefox) to get more information about which files aren't be reached.

In WebworkWebserve/Renderer.pm you can turn on

my $debugXmlCode=0; # turns on the filter for debugging XMLRPC and SOAP code
local(*DEBUGCODE);

Which will give you more information.


It _might_ have to do with the contents of the first problem. Try reordering the problems in the homework set and see what happens.

-- Mike


In reply to Michael Gage

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Michael Gage -
I'm back in town now. Has this been resolved?
In reply to Michael Gage

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by F. Heiderich -
No, the problem remains.

Chrome shows the following errors. I noticed that in the URL the http protocol occurs, while I configured the server to use https. I might have overseen this somewhere.

jquery.js:4 [Deprecation] Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/. 
send @ jquery.js:4
ajax @ jquery.js:4
(anonymous) @ problemsetdetail2.js:8
o @ jquery.js:2
fireWith @ jquery.js:2
ready @ jquery.js:2
B @ jquery.js:2
jquery.js:2 [Violation] 'DOMContentLoaded' handler took 1020ms
[Violation] Forced reflow while executing JavaScript took 307ms
[Violation] Forced reflow while executing JavaScript took 36ms
jquery.js:4 POST http://webwork/webwork2/instructorXMLHandler 403 (Forbidden)
send @ jquery.js:4
ajax @ jquery.js:4
render @ problemsetdetail2.js:319
(anonymous) @ problemsetdetail2.js:115
each @ jquery.js:2
each @ jquery.js:2
(anonymous) @ problemsetdetail2.js:112
dispatch @ jquery.js:3
i @ jquery.js:3
jquery.js:4 [Violation] 'readystatechange' handler took 2399ms
jquery.js:4 POST http://webwork/webwork2/instructorXMLHandler 403 (Forbidden)
send @ jquery.js:4
ajax @ jquery.js:4
render @ problemsetdetail2.js:319
(anonymous) @ problemsetdetail2.js:115
each @ jquery.js:2
each @ jquery.js:2
(anonymous) @ problemsetdetail2.js:112
dispatch @ jquery.js:3
i @ jquery.js:3
jquery.js:4 [Violation] 'readystatechange' handler took 1730ms


I turned on the debug output. Unfortunately the output does not help me much. Grepping the file for "error" gives lines as the following:

 errors is : scalar -- not converted
 error_flag is : scalar -- not converted
 error_message is : scalar -- not converted

Reordering the problems (making another problem the first one) did not help neither.

In reply to F. Heiderich

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Michael Gage -
I think
jquery.js:4 POST http://webwork/webwork2/instructorXMLHandler 403 (Forbidden)
is the serious error. 
Check that the site url has been defined properly (with https)
in the file webwork2/conf/site.conf
In reply to Michael Gage

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by F. Heiderich -
It is defined using https there.
In reply to F. Heiderich

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Arnold Pizer -
Hi,

Could you have changed http to https in site.conf but then forgot to restart apache? (Look for simple things first).

Arnie
In reply to F. Heiderich

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Michael Gage -
At least one of the errors is the call to

http://webwork/webwork2/instructorXMLHandler 403 (Forbidden)

  

This is wrong on a couple of levels. It should be https: instead of http: and I believe it should be https://yourschool.edu/webwork2/instructorXMLHandler.


This is the call that is creating the error. The next question is where is the address for this post being defined?

I might be wrong about it being defined in site.conf. One debugging trick I use is to change the variable that I think is responsible (e.g. the definition of the site url ) to something nonsensical (e.g. https://foobar.edu/.....) and see if I can detect the change. I have found more than once that I was editing the wrong file. :-)



In reply to Michael Gage

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Danny Glin -
Could this be the same problem as this thread?

In site.conf, if your $server_root_url is in double quotes, try it with single quotes instead. It's possible that it's mangling the url.
In reply to Danny Glin

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by F. Heiderich -
No, it is not the same problem. The error there was a 404 error. I have a 403 error.

I did tests concerning the URL that figures in the error messages in the Chrome console. It seems to be derived from the URL in the browser of the user and not depend on the value of the URL configured in site.conf. The fact that it figured http and not https was due to the fact that I accessed the server exceptionally using Chrome for debugging purposes. It is a test server and I do not have a http -> https forwarding configured there. Therefore I was accessing the server using http without noticing it (Chrome seems to prepend incomplete URLs by http). This seems to be the cause for the http in the URL in the error message.

The problem seems to depend on specific problems. I did not notice this beforehand. Two of the problems that causes this error are:

OpenProblemLibrary/TCNJ/TCNJ_MatrixLinearTransformation/problem1.pg
OpenProblemLibrary/TCNJ/TCNJ_MatrixLinearTransformation/problem2.pg
In reply to F. Heiderich

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Michael Gage -
Aahh,

In that case I suspect that this error message is the relevant one

<br/>faultstring: Not an ARRAY reference at /opt/webwork/webwork2/lib/WebworkWebservice/RenderProblem.pm line 547.

I've been aware that the xmlrpc transport breaks on certain datatypes on rare occasions. School is just starting (Wednesday -- yikes!) so while I'll move this up on my list of things to debug I may not get to until the end of the week.
In reply to F. Heiderich

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Michael Gage -
Hi Florian,

What was the final resolution of this problem? I now have an instance of it which might be duplicated on other machines. I have downloaded the 2.14 release candidate branch rel-ww2.14 and set it up inside a docker container (see instructions on the wiki for github.com/openwebwork/webwork2). It largely works as expected by the instructorXML call back error appears when trying to access features of the Library browser. It appears to be a 403 error as in your case, not a 404 error. It does not appear to depend on which problem I look at.

Setting up rel-ww2.14 on my other computers (outside the docker container) seems to work fine. Gavin LaRose thought his problem might have to do with conflicting CPAN modules and that might be true in this case as well. Since the environment in a docker container is completely controlled it should be possible to duplicate what I'm seeing on another machine if someone has the time to test it out.

-- Mike

In reply to Michael Gage

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Nathan Wallach -
I ran into the same issue with
/webwork2/instructorXMLHandler: Forbidden
being reported when trying to use the library browser in a running Docker container available with a real hostname, but no such problem when using the same WW code on Docker on my laptop via "localhost".

The Docker container had the hostname set in Dockerfile via an environment variable like this:
WEBWORK_ROOT_URL=https://myhostname.mydomain

In this case, when using the selector boxes in the library browser the
browser would give an error:
/webwork2/instructorXMLHandler: Forbidden

There were error messages in /var/log/apache2/error.log similar to what
appears below.

The error messages show that there were "routing" problems between the "server" and the target network address. The other (non-Docker) problems of this nature may also be related to similar issues, in which case editing /etc/hosts on the server may help fix them.

Adding a "hostname" line to the "app:" block of docker-compose.yml
hostname: myhostname.mydomain
solved this problem.

Explanation: The WW server inside the docker container is trying to connect to itself on its public web address, but without the "hostname" set in docker-compose.yml it does not know how to connect to that host.

An alternative solution would be to map the FQDN used to the external IP address using "extra_hosts" in docker-compose.yml, but that would needlessly route the traffic out of the docker framework and then back in. (I did not test if it works.)

In the long run, it may make sense to instead use "alias" and set up a network configuration in the docker-compose.yml by hand, as an alias can be shared by multiple "replicas".

See:
See the sections "domainname, hostname" (start of section name) and "ALIASES"
Note: A network-wide alias can be shared by multiple containers

=====================
From: /var/log/apache2/error.log
=====================

There were a lot of errors
Errors:
500 Can't connect to myhostname.mydomain:443 at /opt/webwork/webwork2/lib/WebworkClient.pm line 292.
End Errors
xmlrpcCall to listSetProblems returned no result for
[Mon Nov 19 20:22:36.111578 2018] [perl:error] [pid 1299] [client 213.57.108.112:1952] [/webwork2/instructorXMLHandler] xmlrpcCall to listSe
tProblems returned no result for \n * in WeBWorK::ContentGenerator::instructorXMLHandler::content called at line 233 of /opt/webwork/webwork
2/lib/WeBWorK/ContentGenerator.pm\n * in WeBWorK::ContentGenerator::go called at line 384 of /opt/webwork/webwork2/lib/WeBWorK.pm, referer:
https://myhostname.mydomain/webwork2/test_001/instructor/setmaker/?user=USERNAME&key=SECRET_KEY&effectiveUser=USERNAME


=====================


There were a lot of errors
Errors:
500 Can't connect to myhostname.mydomain:443 at /opt/webwork/webwork2/lib/WebworkClient.pm line 292.
End Errors
xmlrpcCall to searchLib returned no result for
[Wed Nov 21 15:54:26.079333 2018] [perl:error] [pid 1515] [client 132.68.115.60:54758] [/webwork2/instructorXMLHandler] xmlrpcCall to search
Lib returned no result for \n * in WeBWorK::ContentGenerator::instructorXMLHandler::content called at line 233 of /opt/webwork/webwork2/lib/
WeBWorK/ContentGenerator.pm\n * in WeBWorK::ContentGenerator::go called at line 384 of /opt/webwork/webwork2/lib/WeBWorK.pm, referer: http:/
/myhostname.mydomain/webwork2/test_001/instructor/setmaker/?user=USERNAME&effectiveUser=USERNAME&key=SECRET_KEY



In reply to Nathan Wallach

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Sungwook Lee -

I am currently having the same issue. It causes a number of problems, for example rendering problems or using the library to add problems to  a target assignment with the error message "305 setmaker.js: /webwork2/instructorXMLHandler: Forbidden." Apache2 error log lists the error "500 Can't connect to math.usm.edu:443 (Name or service not known) at /opt/webwork/webwork2/lib/WebworkClient.pm line 302." I added hostname to docker-compose.yml as you suggested (and I restarted apache) but the problem still persists. It also appears to have affected webwork email. What baffles me is that I have not done anything to the server including any kind of routine updates this year. There hasn't been any issue until now. Btw my ww version is 2.15.

I would very much appreciate any insight on this issue.

Thank you.

John

In reply to Sungwook Lee

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Sungwook Lee -

I resolved the issue. For some reason, DNS entry in the network setting was removed. Once I corrected it, all seems to be working fine. I realized it might have something to do with DNS when I noticed that email messages from webwork are not going out.

John

In reply to Sungwook Lee

Re: Error: /webwork2/instructorXMLHandler: Forbidden

by Sungwook Lee -
Correction: Sorry, webwork email not going out wasn't the primary clue to a possible problem with DNS, rather it was secondary one that supported my suspicion from the primary clue. The primary clue was that I could ping or ssh only IPs but not hostnames associated with the IPs.

John