## Miscellaneous

### "Expected token not present" - what does this mean?

by Maria Nogin -
Number of replies: 10
We just set up a new server.

Now a lot of our students are having trouble getting to the WeBWorK login page (from a link on Blackboard). The error message they are getting is "Expected token not present".

Any ideas on what this could mean?

### Re: "Expected token not present" - what does this mean?

by Michael Gage -
Maria,

Can you post an example of the link that the students are following from Blackboard?

-- Mike

### Re: "Expected token not present" - what does this mean?

by Maria Nogin -
It's an "external link", with the URL like this one:
http://webwork2.math.csufresno.edu/webwork2/Math75B-Nogin/

Sometimes, same students are not able to get the login page to work at some place, but then, on the same computer but in a different location, it works.

The error message they get looks like this:

Warning messages

Error messages

Expected token not present

Call stack

The information below can help locate the source of the problem.

• in Apache2::Cookie::fetch called at line 622 of /opt/webwork2/lib/WeBWorK/Authen.pm
• in WeBWorK::Authen::fetchCookie called at line 277 of /opt/webwork2/lib/WeBWorK/Authen.pm
• in WeBWorK::Authen::get_credentials called at line 219 of /opt/webwork2/lib/WeBWorK/Authen.pm
• in WeBWorK::Authen::do_verify called at line 150 of /opt/webwork2/lib/WeBWorK/Authen.pm
• in WeBWorK::Authen::verify called at line 290 of /opt/webwork2/lib/WeBWorK.pm

Request information

Method
GET
URI
/webwork2/Math76-Cusick/
 User-Agent Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; WOW64; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506; Media Center PC 5.0) Accept */* Connection Keep-Alive UA-CPU x86 Referer Accept-Encoding gzip, deflate Cookie frspplwebn2-csufresno-edu-443-PORTAL-PSJSESSIONID=KTM2ffqky1MyVqVL0kDhrD8BGJrdymvG!-755107663; https%3a%2f%2fmy.csufresno.edu%2fpsp%2fmfs%2fcsu%2ffresno%2frefresh=list:|; ExpirePage=https://my.csufresno.edu/psp/mfs/; PS_LOGINLIST=https://my.csufresno.edu/mfs; PS_TOKENEXPIRE=24_Aug_2009_21:56:32_GMT; PS_TOKEN=AAAApAECAwQAAQAAAAACvAAAAAAAAAAsAARTaGRyAgBOcwgAOAAuADEAMBR1+/L0d7ZgKOoiOZq/I2HVo06ZxgAAAGQABVNkYXRhWHicHYs5DoAwDAQnAVFS8A+iJOLsOTqE+Arf43FsbEvjtXcNvM5XNQ6V/wo7EpGJrJlNxYadi7Pl5uDRcovbYHZkpRcXMTOYngmMegx2CZYquaT+AXKYC1M=; SignOnDefault=; csuf_auth=student; csuf_info=Hilary Heinrich~hheinrich@csufresno.edu Accept-Language en-us Host

### Re: "Expected token not present" - what does this mean?

by Aleksey Nogin -
From what I understood after Googling for this error message, the problem is that some browser are sending non-standards-compliant "Set-Cookie" HTTP headers and the Apache module is rejecting them. It seems that there are ways of catching the error, ignoring the ill-formed cookies and keeping the well-formed ones. http://wwrk.maa.org/moodle/mod/forum/discuss.php?d=1999 contains what appears to be some work-around code (perhaps customized for a particular situation).

P.S. What's the easiest way to have all HTTP headers logged somewhere, so that we could figure out what the offending cookies are?

### Re: "Expected token not present" - what does this mean?

by Maria Nogin -
Mike,

It seems now that this has nothing to do with Blackboard. The first homework deadline was in a course that used Blackboard, but now we see that many students from all of our courses are having trouble connecting to the server, no matter which link they use, or even if they just type the URL. Some people can never connect, and most - sometimes can, sometimes cannot, and they don't see a clear pattern. For some, restarting the browser helps, sometimes, but again not always.

Aleksey was going to try to figure out how to fix the problem this weekend, so if you have any ideas where to look, we'd appreciate that.

Maria

### Re: "Expected token not present" - what does this mean?

by Michael Gage -
Maria and Aleksey,

The only WeBWorK module involved is Authen.pm (as far as I can tell from the error messages). I suspect that the problem mostly lies with compatibility problems with Apache2/Cookies.pm and various browsers. It looks like the error occurs when WeBWorK attempts to read the cookie on the disk. There might be more information in the apache server error log than appears on the screen.

Our production servers here at Rochester still use Apache 1.3.x so I haven't had enough experience with Apache2/Cookies to have run into the problem you describe.

I dimly remember years ago having a problem with parsing the cookies but I can't find a reference to it in the forum messages so I don't remember the precise nature of the problem or of its resolution.

sorry I can't help more. Let us know what the problem and fix are when you find them.

--Mike

### Re: "Expected token not present" - what does this mean?

by Aleksey Nogin -
Mike,

The problem is caused by a confluence of three things:
1. There is a widely used my.csufresno.edu server that
• Sometimes creates cookies that are not strictly RFC-compliant (I am not 100% sure what characters are allowed/not allowed in cookie values, so I am not 100% sure what the exact problem is).
• Sets all cookies with "domain=.csufresno.edu" (As I understand, the my.csufresno.edu server is used to authenticate users and then the authentication token can be used at a number of other csufresno.edu servers).
The libapreq2's Apache2::Cookie module is fairly conservative in parsing the "Cookie:" HTTP headers and raises the exception when a cookie is not quite RFC-compliant. So when a non-compliant my.csufresno.edu cookie arrives to Webwork, an exception is raised. Note BTW that AFAIK this is not specific to Apache 2 - the Apache 1's Apache::Cookie would have had the exact same problem.
Which it's possible to catch the exception raised by Apacha2::Cookie and then extract all the compliant cookies (dropping the non-compliant ones), Webwork code does not try to do that. Fixing any of the three would be sufficient to get the problem to go away and each of the three can be potentially fixed.
1. Maria is planning to complain about the my.csufresno.edu's cookies, although I have strong doubts that anything would come out of it (I am guessing this is some big complicated and probably proprietary solution that would not be easy to fix). Another solution would be to created an alias hostname outside of the csufresno.edu domain - but this is obviously undesirable.
2. The change log for the version 2.12 of libapreq2 says "Make the cookie parser a little more flexible." Unfortunately, when I upgraded the libapreq2 on the Webwork machine to 2.12, the problem did not go away. I guess "a little more flexible" is not flexible enough.
3. The URL I posted earlier contains some example of the code for catching the exception. I will try to see if I can incorporate something like that into the Authen.pm code.
Aleksey

### Re: "Expected token not present" - what does this mean?

by Michael Gage -
Aleksey,

The URL I posted earlier contains some example of the code for catching the exception. I will try to see if I can incorporate something like that into the Authen.pm code.

Thanks very much for all of this work. Once you find a stable fix send the code to me and I'll incorporate it into the next release version of Authen.pm.

Again thanks -- we greatly appreciate the help.

-- Mike

### Re: "Expected token not present" - what does this mean?

by Aleksey Nogin -
Mike,

The attached patch, inspired by http://marc.info/?l=apreq-dev&m=114001388709851&w=2 seems to fix this problem for us. As you'll see, the patch is very badly written (I know almost nothing about using the Perl modules and Perl objects, so I have not even tried to incorporate in into your code the "proper" way) and only updates the Apache2-specific execution path (I would imagine that the Apache1 path can be updated more or less the same way, but I would not know).

Hopefully you can still turn my patch into a proper fix, despite it being so messy.

Aleksey

P.S. This forum messes up the patch when posted as attachment, so I also posted it at http://webwork2.math.csufresno.edu/ww-cookie-patch.txt