Students uploading photos of work? (similar to WebAssign "show your work")
by Alan Ableson - Number of replies: 11Has anyone heard of development of a Webwork tool which would allow/require students to upload photo(s) of their work and associate it with a Webwork problem/assignment/quiz?
The idea is that students will usually work on Webwork problems on paper, and it is feasible now for them to take a photo of the page when they are done and submit it along with/just after each question. As a model, this feature exists in (Enhanced) Web Assign:
http://www.webassign.net/manual/student_guide/t_s_show_work.htm
Context: we are offering a calculus course in a distance format starting this summer. We have both student understanding and in-term test security concerns, and increasing student accountability by requiring evidence of their work seems like a good method for addressing both issues.
Any suggestions, resources or comments would be warmly welcomed.
Alan Ableson
Queen's University, Kingston, Ontario
In reply to Alan Ableson
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Jason Aubrey -
Hi Alan,
WeBWorK does not have this specific feature (of uploading photos of work), and as far as I know no one is working on implementing it.
However, Aaron Wangberg at Winona State University has developed a whiteboard for webwork that sits next to a problem and where students can show their work by writing on it with a mouse or tablet pen.
We're working with Aaron to integrate this feature into the main webwork distribution, but it hasn't been done yet.
In the meantime, I have two suggestions: First, I taught an online course this summer and I gave unproctored online exams. I wanted exactly what you're asking for here, so I gave students a set amount of time to submit their answers to the exams on webwork and then an additional 30 minutes to upload photos or scanned images of their work to Blackboard. You could do just the same with any other course management system.
The main drawback of this is that Blackboard doesn't make anything really easy to do (IMHO), and it was a pain to download all of the images and keep track of them.
However, I recently heard of a better idea that I would probably try if I do that again: You could have the students create dropbox accounts and then you could share a folder with each of them where they could put their work. It would be automatically organized by student, and you could tell them how to organize it beyond that. Dropbox has versioning and timestamps too.
Hope this helps,
Jason
WeBWorK does not have this specific feature (of uploading photos of work), and as far as I know no one is working on implementing it.
However, Aaron Wangberg at Winona State University has developed a whiteboard for webwork that sits next to a problem and where students can show their work by writing on it with a mouse or tablet pen.
We're working with Aaron to integrate this feature into the main webwork distribution, but it hasn't been done yet.
In the meantime, I have two suggestions: First, I taught an online course this summer and I gave unproctored online exams. I wanted exactly what you're asking for here, so I gave students a set amount of time to submit their answers to the exams on webwork and then an additional 30 minutes to upload photos or scanned images of their work to Blackboard. You could do just the same with any other course management system.
The main drawback of this is that Blackboard doesn't make anything really easy to do (IMHO), and it was a pain to download all of the images and keep track of them.
However, I recently heard of a better idea that I would probably try if I do that again: You could have the students create dropbox accounts and then you could share a folder with each of them where they could put their work. It would be automatically organized by student, and you could tell them how to organize it beyond that. Dropbox has versioning and timestamps too.
Hope this helps,
Jason
In reply to Jason Aubrey
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Alan Ableson -
Thanks Jason: that was very helpful, especially hearing about your experience using uploaded images with a distance course. I hadn't considered using Dropbox as an option, but quite like that idea for automatic organization of the files.
Two brief comments:
- For the record (and if it helps influence feature prioritization) I am also very interested in the whiteboard capability Aaron has developed, and would love to see it implemented in the main Webwork branch! =]
- I am pretty motivated to write a demo of a Webwork "submit your work" button and related workflow, because I can see that approach relieving administrative complications with the other file-handling methods that you mentioned. If I get a demo working, I'll post again.
One teaching-related follow-up question:
- how did you use the images students uploaded in your course? I.e. Did you do detailed grading of the students' work based on their uploaded images, or did you use the images largely as a check on whether they had done the work themselves?
Thanks again,
Alan
Two brief comments:
- For the record (and if it helps influence feature prioritization) I am also very interested in the whiteboard capability Aaron has developed, and would love to see it implemented in the main Webwork branch! =]
- I am pretty motivated to write a demo of a Webwork "submit your work" button and related workflow, because I can see that approach relieving administrative complications with the other file-handling methods that you mentioned. If I get a demo working, I'll post again.
One teaching-related follow-up question:
- how did you use the images students uploaded in your course? I.e. Did you do detailed grading of the students' work based on their uploaded images, or did you use the images largely as a check on whether they had done the work themselves?
Thanks again,
Alan
In reply to Alan Ableson
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Michael Gage -
- I am pretty motivated to write a demo of a Webwork "submit your work" button and related workflow, because I can see that approach relieving administrative complications with the other file-handling methods that you mentioned. If I get a demo working, I'll post again.
This sounds great. I don't know if this is what you had in mind, but if you can use javaScript in an ordinary webpage to grab the student's admission and upload it to a specified dropbox then I think we can use exactly the same code, place it in a macro .pl file and allow an instructor to insert such an upload feature in any problem just by adding a few lines. such as:
\{ upload_to_dropbox("dropbox address", "title of submission") \}
we could even have it grab the student name and email address from the data available to the WW problem.
Write me if this doesn't make sense or if you have a better idea.
-- Mike gage@math.rochester.edu
In reply to Michael Gage
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Michael Gage -
Would this link be of any use in this project?
http://www.highoncoding.com/Articles/229_Upload_and_Preview_Images_Using_JavaScript_and_Client_Callbacks.aspx
In reply to Michael Gage
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Alan Ableson -
Hi Michael,
Wow: that looks like the solution-in-a-box! The Dropbox idea is inspired: having a tie-in between Dropbox running on the server and a method like the javascript upload+preview system would make for a great workflow as a grader.
First draft thoughts:
-share the directory [Dropbox]/[course] between the desktop & server
-have the upload script drop images in
[Dropbox]/[course]/[homework/test name]/[userid]/Problem##_Image##.jpg
- auto-generate some indexing (e.g. thumbnail HTML file) to make moving through the images easier for the grader when the test is over.
Challenge: allow students to view & revisit their images through the Webwork interface.
Alternative: just allow students to put the files in a Dropbox directory themselves.
Downsides to alternative/reason for server side: non-standard naming and directory conventions make it hard to grade in bulk. All students have to create a Dropbox account.
All things to think about. I should have time this weekend to do some experimenting with the tools you pointed out, and I'll post an update next week (unless someone else tries it first =])
Thanks very much for the suggestions!
Alan
Wow: that looks like the solution-in-a-box! The Dropbox idea is inspired: having a tie-in between Dropbox running on the server and a method like the javascript upload+preview system would make for a great workflow as a grader.
First draft thoughts:
-share the directory [Dropbox]/[course] between the desktop & server
-have the upload script drop images in
[Dropbox]/[course]/[homework/test name]/[userid]/Problem##_Image##.jpg
- auto-generate some indexing (e.g. thumbnail HTML file) to make moving through the images easier for the grader when the test is over.
Challenge: allow students to view & revisit their images through the Webwork interface.
Alternative: just allow students to put the files in a Dropbox directory themselves.
Downsides to alternative/reason for server side: non-standard naming and directory conventions make it hard to grade in bulk. All students have to create a Dropbox account.
All things to think about. I should have time this weekend to do some experimenting with the tools you pointed out, and I'll post an update next week (unless someone else tries it first =])
Thanks very much for the suggestions!
Alan
In reply to Alan Ableson
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Michael Gage -
Sounds great. Don't limit yourself to just that tool/method -- that was the result of a very quick google search to see if something might be feasible -- and it is feasible. You might want to spend a half hour or so to see if you can find a version that is better or simpler.
Good luck and let us know how it works out. We can include it as a new macro file once it's ready and working.
-- Mike
In reply to Michael Gage
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Alan Ableson -
Some definite success to report. You can see a test site at:
http://r140.n100.queensu.ca:8090/webwork2/MATH121_test/Set_1/
(Guest login button at the bottom)
Student/File Management Process
1) Each question has an "Upload My Work" button.
2) Button sends students to a file browser PHP page, with their student ID, problem set, and problem number defined.
3) Uploaded files are saved on the server as
[ID]-[set]-[problem].[extension]
4) Dropbox is running on the server, and the files are stored in a Dropbox folder (WebworkUploadsFolder); as soon as each file is uploaded, Dropbox updates the copy on the desktop machine, timestamped and ready for grading. This is really the icing on the cake... =]
Author requirements
From the problem author perspective, the .pg code required is:
(in loadMacros)
"QUuploadfilesmacros.pl",
...
BEGIN_TEXT
...
$BBOLD Don't forget to upload an image of your work: $EBOLD
$PAR
\{ Queens::createUploadButton("Upload My Work") \}
$PAR
...
Macro file
The macro file is attached to this post. It uses the $main::envir variable to get the problem and user information, and builds them into parameters in the file uploading URL.
Upload .php file
The file uploading is kept in /var/www on the server, outside of the Webwork directories to avoid the automatic URL mangling/parsing. It also avoids the security built into Webwork (for better or worse). Full credit for the uploading tool goes to 'Max' from PHP F1:
http://www.phpf1.com/product/file-upload-script.html
Necessary changes:
1) Match style of upload page to Webwork.
2) Have a 'back to problems' button on the upload page.
3) Think security, security, security. I've got some basic server-side validation and sanitation going on, but haven't brainstormed other ways this could go wrong yet, especially risks of students overwriting or viewing other students' work.
Strongly desired changes:
1) Have a first problem/next problem button _on the upload page_: no need for students to always jump back to the problems, skip to the next, then re-upload, if they have numbered their problems on paper.
2) Improve the security of the problem parameters being passed. They are being passed as part of the URL now (index.php?studentLogin=practice9&...), and it would be better if they were submitted via POST, using the information already in the hidden inputs in the page form. I'm not sure how to/if you can hijack the javascript executed with the 'submit' button on a Webwork problem yet though, using just code in a macro or a .pg problem.
3) Allow multiple files to be uploaded; right now it's strictly one file per problem, and if an uploaded file exists, a new upload overwrites it.
4) Allow students to view what they have uploaded.
5) Instead of having the files named by student/set/problem, have them stored in directories filed that way (easier to browse all solutions of the same problem at the same time with something like Picasa etc.) The extra step is creating an intermediate directories as necessary before the file is moved to its final position.
Any comments or suggestions are welcome. Given this proof of concept has worked out, I expect to integrate this tool into our production site for our online course (May 2012).
Cheers,
Alan
http://r140.n100.queensu.ca:8090/webwork2/MATH121_test/Set_1/
(Guest login button at the bottom)
Student/File Management Process
1) Each question has an "Upload My Work" button.
2) Button sends students to a file browser PHP page, with their student ID, problem set, and problem number defined.
3) Uploaded files are saved on the server as
[ID]-[set]-[problem].[extension]
4) Dropbox is running on the server, and the files are stored in a Dropbox folder (WebworkUploadsFolder); as soon as each file is uploaded, Dropbox updates the copy on the desktop machine, timestamped and ready for grading. This is really the icing on the cake... =]
Author requirements
From the problem author perspective, the .pg code required is:
(in loadMacros)
"QUuploadfilesmacros.pl",
...
BEGIN_TEXT
...
$BBOLD Don't forget to upload an image of your work: $EBOLD
$PAR
\{ Queens::createUploadButton("Upload My Work") \}
$PAR
...
Macro file
The macro file is attached to this post. It uses the $main::envir variable to get the problem and user information, and builds them into parameters in the file uploading URL.
Upload .php file
The file uploading is kept in /var/www on the server, outside of the Webwork directories to avoid the automatic URL mangling/parsing. It also avoids the security built into Webwork (for better or worse). Full credit for the uploading tool goes to 'Max' from PHP F1:
http://www.phpf1.com/product/file-upload-script.html
Necessary changes:
1) Match style of upload page to Webwork.
2) Have a 'back to problems' button on the upload page.
3) Think security, security, security. I've got some basic server-side validation and sanitation going on, but haven't brainstormed other ways this could go wrong yet, especially risks of students overwriting or viewing other students' work.
Strongly desired changes:
1) Have a first problem/next problem button _on the upload page_: no need for students to always jump back to the problems, skip to the next, then re-upload, if they have numbered their problems on paper.
2) Improve the security of the problem parameters being passed. They are being passed as part of the URL now (index.php?studentLogin=practice9&...), and it would be better if they were submitted via POST, using the information already in the hidden inputs in the page form. I'm not sure how to/if you can hijack the javascript executed with the 'submit' button on a Webwork problem yet though, using just code in a macro or a .pg problem.
3) Allow multiple files to be uploaded; right now it's strictly one file per problem, and if an uploaded file exists, a new upload overwrites it.
4) Allow students to view what they have uploaded.
5) Instead of having the files named by student/set/problem, have them stored in directories filed that way (easier to browse all solutions of the same problem at the same time with something like Picasa etc.) The extra step is creating an intermediate directories as necessary before the file is moved to its final position.
Any comments or suggestions are welcome. Given this proof of concept has worked out, I expect to integrate this tool into our production site for our online course (May 2012).
Cheers,
Alan
In reply to Alan Ableson
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Michael Gage -
Hi Alan,
This looks very promising. I have a couple of comments that might help with the next steps. In particular they might replace the need for the external .php script and automatically supply a bit more security -- since all access would be through the current WW security set up.
(1). Take a look at the way that the "File Manager" page of WeBWorK uploads files -- it uses javaScript to help the student search for the file to upload and then upload it.
(2). The File Manager approach requires the creation of a form. This could cause a problem because the entire problem section is itself a form and HTML does not allow you to embed one form inside another. Fortunately this can be resolved (we ran into it in a previous project) by loading the text into the section BEGIN_POST_HEADER_TEXT/END_POST_HEADER_TEXT.
The contents of this section are placed after the HEADER section of the HTML page but before the form containing the problem begins. Forms (and iframes) can be safely placed in this section.
(see
http://webwork.maa.org/viewvc/system/trunk/pg/macros/PGbasicmacros.pl?r1=6902&r2=6929
)
Let me know if I can be of further help.
-- Mike
In reply to Michael Gage
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Alan Ableson -
[Keep] all access ... through the current WW security set up.
I agree that would be the best from a management, consistency and security perspective. Unfortunately it means having to learn more Webwork internals than the stand-alone solution, so the development goes a little slower the first time around... =]
Thanks for that pointer. I don't know why I didn't think of the File Manager, except that I had it in mind as an instructor-only access tool. It would obviously have most of the functionality (upload, delete, preview by clicking on the file's link) already built in.
(2). The File Manager approach requires the creation of a form. [This can be resolved by... ] loading the text into the section BEGIN_POST_HEADER_TEXT/END_POST_HEADER_TEXT.
Ah! That's what I was looking for earlier, to break out of the problem's form trap; it's good to know that code/HTML inserted in the POST_HEADER region will be outside that master form.
Thanks very much for the tips and suggestions; I'll work on this some more and report back likely in a few weeks (after the end-of-term craziness dies down). I appreciate the help offer: if any questions come up along the way, I'll get in touch.
Cheers,
Alan
I agree that would be the best from a management, consistency and security perspective. Unfortunately it means having to learn more Webwork internals than the stand-alone solution, so the development goes a little slower the first time around... =]
(1). Take a look at the way that the "File Manager" page of WeBWorK uploads files -- it uses javaScript to help the student search for the file to upload and then upload it.
Thanks for that pointer. I don't know why I didn't think of the File Manager, except that I had it in mind as an instructor-only access tool. It would obviously have most of the functionality (upload, delete, preview by clicking on the file's link) already built in.
(2). The File Manager approach requires the creation of a form. [This can be resolved by... ] loading the text into the section BEGIN_POST_HEADER_TEXT/END_POST_HEADER_TEXT.
Ah! That's what I was looking for earlier, to break out of the problem's form trap; it's good to know that code/HTML inserted in the POST_HEADER region will be outside that master form.
Thanks very much for the tips and suggestions; I'll work on this some more and report back likely in a few weeks (after the end-of-term craziness dies down). I appreciate the help offer: if any questions come up along the way, I'll get in touch.
Cheers,
Alan
In reply to Alan Ableson
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Jason Aubrey -One teaching-related follow-up question:I was trying to address two problems: (a) making sure that the students were really doing the work, and (b) allowing for partial credit on the exam problems.
- how did you use the images students uploaded in your course? I.e. Did you do detailed grading of the students' work based on their uploaded images, or did you use the images largely as a check on whether they had done the work themselves?
The first problem is really about trying to prevent cheating. I could have given proctored exams - there is an infrastructure for that and our distance education unit would have facilitated it. I also got some pressure to do proctored exams from my department, but I wanted to try it at least once with unproctored exams. I told the students they could use their textbook and notes, but could not get help from anyone or any third party resource.
I also had a "participation" component in the course that required students to post at least 3 times per week in the discussion board for the week. They could get up to 5 points for 3 types of posts: (1) a carefully written up solutions to one of the textbook homework problems, (2) a carefully written up solution to a webwork problem from the previous week, or (3) a constructive and substantive reply to another student's question about a homework problem. (The discussion board had webEQ built in so that mathematics could be easily entered.)
As I looked at the students written work from the exams I wanted to know if the quality of that work was consistent with my impressions from the work in the forums. In nearly every case, the answer was yes. This was of course not proof that the students weren't cheating, but it seemed to me to be evidence in that direction.
For the second issue - partial credit - my policy was that work had to be submitted for every problem, right or wrong. A correct (or incorrect) answer with no work got zero points, an incorrect answer with partially correct work got partial credit.
In the end the grade distribution in my online section happened to be nearly identical to the grade distribution in the two face-to-face sections that were taught simultaneously (taught by others, but the syllabus, calendar, etc. were set by me.) I was surprised by that.
Another surprising issue - the length and difficulty of my webwork exams were (I thought) about the same as the length of my regular exams. (In fact, I used many of the exact same problems, just coded up in webwork.)
However, the students found them very difficult to complete in the allotted time. So, for the first exam I gave an hour, but by the last exam I was giving fewer problems and 2 hours to complete them. I think that at least part of the issue was that I allowed students to see whether or not they had the problem correct.
The effect of this, I think, was that they would persist on problems longer than they would during a regular exam. I think that some students just couldn't move on when immediately told that their answer was wrong during the exam. So, next time I would set the exams so that students wouldn't be told whether the answer they submitted was correct or not.
Phew! Sorry for the long post. Hope it's helpful!
Jason
In reply to Jason Aubrey
Re: Students uploading photos of work? (similar to WebAssign "show your work")
by Alan Ableson -
Thanks Jason. I appreciate the details, as they are very helpful in making the whole process more concrete.
Thanks for bringing that up: I had wondered how that psychology would play out! I agree that if they're submitting all their written work, there might be an advantage to them not knowing whether they are right or wrong during the test, more like a paper test.
Very interesting: I have to admit that I would be just as surprised if it happens in my course too. There's some interesting fodder for the "lectures vs activities - where do students really learn?" debates in that result... =]
Thanks for sharing your experiences with using Webwork this way: it is appreciated.
Alan
For the second issue - partial credit - my policy was that work had to be submitted for every problem, right or wrong. A correct (or incorrect) answer with no work got zero points, an incorrect answer with partially correct work got partial credit.
As a follow-up (hopefully could be helpful for others as well as myself): - What was your impression of using the images for that level of grading, combined with Webwork?
- Was it easier than on-paper tests because of the pre-check for correct answers in Webwork?
- Was it easier than on-paper tests because of the pre-check for correct answers in Webwork?
- Was the hassle of dealing with ditigal images in various directories a lot worse/just as cumbersome compared to dealing with paper tests?
- On a separate note, how did you manage comments/feedback to the students after a test? Was it feasible to either annotate their images with text, or post comments in Blackboard with the grade?
Another surprising issue - the length and difficulty of my webwork exams.... I think that at least part of the issue was that I allowed students to see whether or not they had the problem correct.
Thanks for bringing that up: I had wondered how that psychology would play out! I agree that if they're submitting all their written work, there might be an advantage to them not knowing whether they are right or wrong during the test, more like a paper test.
I also had a "participation" component...
Great ideas for managing that communication aspect in a distance course. I especially like giving the students the option of submitting either independent work (solutions) or communication work (good comments).In the end the grade distribution in my online section happened to be nearly identical to the grade distribution in the two face-to-face sections that were taught simultaneously (taught by others, but the syllabus, calendar, etc. were set by me.) I was surprised by that.
Very interesting: I have to admit that I would be just as surprised if it happens in my course too. There's some interesting fodder for the "lectures vs activities - where do students really learn?" debates in that result... =]
Thanks for sharing your experiences with using Webwork this way: it is appreciated.
Alan