WeBWorK Main Forum

Prevent cheating in Gateway quizzes

Prevent cheating in Gateway quizzes

by Valerio De Angelis -
Number of replies: 25

I imagine I am not the only math instructor faced with the problem of preventing cheating during tests administered online. I would be interested in learning about what others are doing or have discovered. 

A specially notable source of cheating is the availability of homework help sites such as chegg.com, where students can easily post a screenshot of an exam and receive complete answers within minutes. 

I would like to use the WebWork Gateway quizzes, but with some extra features. The main one is that students should not be allowed to backtrack. If a problem is available only for a few minutes (or whatever suitable time it takes to solve it), it becomes much more difficult to get the answer from chegg.com. But on the WebWork Gateway quizzes, as far as I can see, it possible to jump from any page to any other page at any time during the test. So I would like to know if there is some way to prevent that, and have problems presented one at time with no backtracking allowed. 

Of course any other tip on how to use WebWork for testing with safeguards against cheating would be welcome. 

Thanks

 



In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Danny Glin -
There is not currently a feature in WeBWorK which can prevent students from revisiting a problem on a quiz. One workaround would be to set up a separate quiz for each question with an appropriate time limit.

With the prevalence of websites that can answer math questions for students, authentic assessment in a distance learning environment has become very challenging. I want to continue to ask students challenging questions, and give them enough time to think about them. The problem is that giving them enough time to think about hard questions also gives them plenty of time to get answers from places like Chegg. One can ask "rapid fire" style questions where students have to answer very quickly, but this doesn't test the same skills.
In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -

I think that the only option is to trust that the students do not cheat.

There is always a possibility how get remote help. I can provide provide remote access to my PC and nobody knows, it the answers are from my keyboard or from the keyboard at another PC in another part of the world. However in my case it is somewhat easier to trust the students, since only about 10 milion people in the world speak Czech and the number of people which are ready to solve your math questions in Czech is smaller compared to English. Especially if the text is something longer than "Simplify" or "Evaluate".

I do not know chegg.com, but here are few hints which work for me.

1. Do not ask "Differentiate", ask "Find rate of change". Many people providing help during exam like simple a straightforward problems. They recognize this as something less typical than simple differentiation and refuse to help you (These people look for easier job.)

2. Do not ask "Integrate", ask "Find the change from the rate of change". The same as above.

3. You can try some psychology. :) I turned Achievements on during the course. The students have seen some messages and I told them, that this is some side effect of the attempt to map the route how they solve problems. And that this can be used to catch some signs of cheating. For example, if someone fills all answers in last 5 minutes. They are not sure what exactly I track. They probably consider the worst scenario that some artificial intelligence is monitoring the process of problem solving and sends alert to the instructor if the behaviour has marks which arise when cheating.

4. One more serious idea. I am not sure if it will work, however: You can provide any Javascript code in the theme files. This code can hide some parts of the test and this disallows the work on some problem when some condition is met. Of course, this can be broken by manipulating Javacripts in the browser, but think about the price. IMHO it is easier for many people to learn derivatives or line integrals, than to learn  how to break your code. See the file /opt/webwork/webwork2/htdocs/themes/math4-green/math4-overrides.js . Another disadvantage is that this file is server-wide so this is usable only if you run server for few courses. And needs a lot of coding and testing before you use this for real exam.

There are many different approaches to prevent cheating and the development is fast. I think that it is reasonable to keep the original mission of WeBWorK (deliver homeworks) and use some third party tools to prevent cheating. Using webcams and Zoom is one of the ideas.



In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Danny Glin -

I have resigned myself to the fact that no matter what we put in place to prevent cheating there will be students who find ways around it.  I will also second Robert's point about the original mission of WeBWorK.  As a homework system it was built with the intention of helping students learn the material, and not necessarily to provide rigorous assessment (though it does at least a reasonable job of this).

A couple of things I always have to remind myself when designing assessments to try and prevent cheating:

  1. Am I still testing the right skills?  It is easy to ask questions that can't be easily computed by programs like Wolfram Alpha, but do they still require the students to demonstrate understanding of the things we want them to learn in the course?
  2. Am I disadvantaging the honest students?  Not all students have access to the same resources, and have different backgrounds and situations.  By making questions that are hard to cheat on, am I inadvertently making it harder for other students to succeed in the course?
In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Valerio De Angelis -
Thank you for the feedback. In the meantime I realized that there is a way to give exams with no backtracking in WebWork, and that is use the "Just in time" gateway quiz type, with only one attempt. At the start, the student sees only Q1, and cannot proceed to Q2 until he has submitted an answer. He can then go to Q2, and can also go back and look at Q1, but can no longer change the answer because there is only one attempt. With a 10 question test in 30 minutes (for example), the student has only about 3 minutes per question on average and there is no time for any chegg.com shenanigans.

Another device to prevent cheating is the Respondus Lockdown browser and monitor. This is often available from within the various learning management systems (we use BrightSpace, but there is also BlackBoard, Canvas, etc.). Chances are WebWork is not integrated with those systems. But since writing my first post, I found out there is still a way to use WebWork with the Lockdown browser: add the url for WebWork to the "whitelist" of the Lockdown browser (that is, the list of url's that are permitted, that is empty by default), then prepare a quiz that contains only a single link to the WebWork exam.

Combined with some carefully written questions and other strategies as pointed out in your reply, I think this should help cut down the cheating. But there is still a lot of room for improvement.
In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Andras Balogh -

I am nut sure I understand how would the Respondus Lockdown Browser work with WeBWorK through Blackboard.

1. If $LTIGradeMode='homework'; is used then the assessment link does not work in Blackboard.

2. If $LTIGradeMode='course'; is used then the assessment is accessible directly through WeBWork.

In reply to Andras Balogh

Re: Prevent cheating in Gateway quizzes

by Valerio De Angelis -

Andras: 

thanks for your message. I do not know $LTIGradeMode.  Is it a way to integrate WebWork with some Learning Management System, and to have the grades from WebWork automatically recorded on the LMS? I have done a little web search but did not find much so I am really just guessing. In any case I will describe below how I use the LockDown Browser with WebWork. This will definitely not record the grades on the LMS, but that is not my aim.

I use BrightSpace (more or less equivalent to BlackBoard) to create a quiz that requires the LockDown Browser. The quiz has only one (mandatory) question, whose text consists in an academic honesty pledge. The student is instructed that to answer that question, they have to write their name in the answer box, and that will mean they have read and acknowledged the academy honesty pledge.   After that, and before submitting the quiz to BrightSpace, the student needs to click a link that says "Click here to access the test". The link will open the WebWork login page in a new tab of the LockDown Browser. In order for this to work, one has to add the WebWork url to the "whitelist" of the LockDown browser (because by default all websites are blocked, except those listed in the whitelist). The student then logs in to WebWork and takes the test, that I prepare as a "just-in-time" homework assignment with only one attempt, and with "Restrict Problem Progression" enabled. This way a student cannot access Question 2 until they have answered Question 1, and then even if they later go back to Question 1 they can no longer change the answer because there is only one attempt.  This means that they cannot spend too much time on a single question, and this cuts down on the cheating, because (at least for low level courses such as Pre-Calculus) cheating takes more time than just doing the work to answer the question. Once the WebWork test is complete, the student closes the LockDown Browser tab with WebWork and goes back to the BrightSpace quiz, and then clicks on "Submit Quiz".

The monitoring has two components: one of course is the Respondus monitoring system that comes with the LockDown browser and provides a video recording of the student taking the test. I rely on this to make sure that they do not use a different computer to login to WebWork directly. This is surely not a 100% secure method but I don't think any exam administered online can be, and it is the best I can come up with. The second monitoring comes from the WebWork Classlist editor page, from which I can see if a student is active and working on WebWork at the right time and for the right length of time.

Valerio


    

In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Andras Balogh -

Thank you for the detailed answer, Valerio.

The $LTIGradeMode is not just to  forward grades to a Learning Management System, but it is also used to automate the setting up of student accounts in WeBWorK, which is a big deal for us.

I understand now, that the mandatory question forces students to go through the LMS and monitoring WeBWorK is still required. I think this works if and only if the $LTIGradeMode is set to send over the overall grade for the whole course.

Still probably most of my colleagues would not want to monitor WeBWorK in person, although the WeBWorK log files could be used for checking student activity even later, and this could be even automated with some script. Warning students that it is possible to monitor them is probably enough.

Andras

In reply to Andras Balogh

Re: Prevent cheating in Gateway quizzes

by Nathan Wallach -

I wish to clarify for the record.

  • If LTI authentication is on, automatic account creation should work by default (regardless of whether grade pass-back is enabled).
  • $LMSManageUserData can be turned on/of to control whether data is updated in case it is changed on the LMS side.
  • $LTIGradeMode is only for grade pass-back, and can be left off (undefined) when no grade pass-back is needed.
  • In cases where it is enabled site wide, it can be turned off at a course level AFAIK by using undef($LTIGradeMode); in the course.conf file.

References:

In reply to Nathan Wallach

Re: Prevent cheating in Gateway quizzes

by Valerio De Angelis -
Nathan:
thank you for the clarification and for the references.

Regarding the discussion on how to prevent cheating, I only recently found out that there was a WebWork conference last November that addressed the problem, and that I wish I had been able to attend. I found these recorded videos from the conference quite useful.

https://www.youtube.com/watch?v=dmgXSWGXXyo&list=PLdzhewHoTma_pFr2xjXq5bLg73n3n_0Uk&index=4

I am now looking for a good way to have some (faint) watermark that says "[your institution here] Exam Question" all over the background of the WebWork problems that are used for exams. I think this will be a deterrent. At a minimum, it will make it impossible to just take a screenshot and post it on chegg.com, as a lot of students in my class did last semester. I would be interested in any suggestion on the best way to place such a watermark.

Valerio
In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Nathan Wallach -

You could do this by adding the some HTML+CSS code to every problem. (Some clever students would be able to bypass it).

Google helped me find the following on using CSS to add a text based watermark:

You could probably created a small "pl" file to do it, so that you could add pl file into loadMacros to do it easily.

Note: There is a feature to have a problem add CSS code from an external file to a problem. It was added to WW/PG 2.15 by:

(Shameless plug for a feature I needed for something else.)

In reply to Nathan Wallach

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -
Another option is to add the code which manipulates css styles into math4-overrides.js file. This is site wide for all courses which use the corresponding template (there are three: math4, math4-red, math4-green). With this option you just add the corresponding Jquery code. This needs admin access to the serve, I think. You have to change the file and restart apache.
In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Monica VanDieren -

I love the idea of a watermark.  If you do get it to work, please share the template/instructions.  


In reply to Monica VanDieren

Re: Prevent cheating in Gateway quizzes

by Valerio De Angelis -

I tried the html+css code pointed out by Nathan and found here

http://jsfiddle.net/fg7m3/

and it works fine, but after some testing I am not happy with the fact that the watermark interferes with the readability of the question. It could become an issue with students who have any type of reading disability. Maybe it could be improved by making the font of the problem bold and larger in size, and I plan to try that next. In any case I think this code needs to be inserted at the administrator level on the WebWork server (I do not have that access), so I would not know the specific instructions on how to insert it.



In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -

It seems that putting the attached file into /opt/webwork/course/YOUR-COURSE/templates/macros/ makes the job. The directory is acessible from webwork using File Manager. You do not need the direct access to the server.

This works provided you have the file PGcourse.pl in the header of each problem. This is probably the case if you use OPL or if you started from some template.

You can add your css rules as well (bigger font etc) which affect the original appearance of the problem.

Watermark

In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -
BTW: it seems that another method how to resolve conflict of watermark and the text is to use position:fixed. With this setting the text moves with respect to the watermark as the user scrolls the page. The watermark can be removed by manipulating css styles, but I think that you are aware of this fact.
In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Valerio De Angelis -

Robert:

I placed the file you attached PGcourse.pl in template/macros but it is not working for me. I do have loadmacros("PGcourse.pl:); at the beginning of the problem. My version of WebWork is a bit old (2.12), could that be the problem?

Thanks,

Valerio


In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -
Not sure, I started with 2.15 version. But I am sure that someone else will know.

Meanwhile you can remove my code from the file and attach some simple javascript code (such as alert("Hi");) and check if it is executed. The names of the containers could be different. Also check the html source of the page, if the code is written inside.

By the way, there is a typo in your code, you have a colon where a doublequote is expected.
In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -
BTW: it is also necessary to add pointer-events:none; to the css rules for the watermark, since the watermark is on top and we need the acsess to the buttons under.
In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Valerio De Angelis -

I removed your code and replaced it with alert("Hi"); and it did work: I got a pop-up window that said "Hi" when I loaded the problem. 

Yes, there was a typo in the code for loadmacros, it was just in my message though. 


In reply to Valerio De Angelis

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -

That's great. Perhaps the name of the container with the problem is not #output_problem_body but something else. I check the names of the containers in Firefox -> Right mouse click -> Inspect Element and here you can browse the structure of the html document.

Also check the console for errors. 


In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Valerio De Angelis -

Yes, that was exactly the problem: the container in my WebWork version is called problem_body. I now got it to work just fine. I experimented with various settings and I think the best result is by keeping the opacity of the watermark to only  0.1, and then make all the text of the problem in bold, as well as all the LaTeX content in math mode in bold (this can be done by using \boldsymbol{} ). Then I think the watermark does not interfere much at all, but it is still visible, also when I take a photo. I repeated the same line (with smaller font) throughout the page. I feel that this is going to be quite a deterrent for cheating. Thank you for your help! I couldn't have done it by myself.

In reply to Robert Mařík

Re: Prevent cheating in Gateway quizzes

by Monica VanDieren -

I tried this out and it works well - so long as the watermark doesn't overlap pop up menus or answer blocks.  For some reason if the watermark is over a pop up or an answer block, it is not possible to select the pop up or enter in information in the textblock.  Otherwise it works well.

In reply to Monica VanDieren

Re: Prevent cheating in Gateway quizzes

by Robert Mařík -

adding the following css rule solves the problem.

pointer-events:none;

just add that line to the others css rules for the watermark div  in the file provided in previous post.