WeBWorK Main Forum

How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Christian Seberino -
Number of replies: 18
Some parents really love getting email reports of their kid's scores so they
don't have to log into Webwork themselves and view the grades.

How extract the scores from the Webwork database from a command line script so that data can be put in an email?

cs
In reply to Christian Seberino

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Davide Cervone -
Just FYI, sending student grades to parents is probably a violation of the FERPA (Family Educational Rights and Privacy Act) regulations, so you may not want to do that. I'm not sure what your institution is, but at mine, this is not allowed. Nor is sending grades via email (to anyone, including the student). Most colleges and universities that I know anything about how they handle FERPA are very strict about this sort of thing, and essentially allow discussion of grades only in person and only with the student.

Just something to think about (or check into).
In reply to Davide Cervone

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Dick Lane -
I agree with Davide's advice about FERPA rules.  (My post http://webwork.maa.org/moodle/mod/forum/discuss.php?d=3059 involves a related concern.)  Although my school is inching toward encouragement of digital signatures for faculty correspondence, I will not consider emailing grade info to a student until such communication is securely encrypted.  (In the past, for similar reasons, a student's provision of a postcard for me to report course-grade was always met with my requirement that a stamped-self-addressed-envelope be provided instead.  FERPA does not allow the student to waive its requirements for faculty and staff.)

Furthermore, my lay-interpretation of FERPA would be that a parent logging into a student's WeBWorK account is marginally acceptable IFF the student initiates the request to the parent and the student provides the username/password.  On the other hand, I suspect that if a school has published policy which restricts use of its resources to those with specific authorization, then I think a parent would need to request such authorization from the school rather than the student.


I suggest two extensions of WeBWorK's current feature allowing a student to see her/his own WeBWorK scores.

  a)  include a summary line:  total-earned  out of  total-possible

  b)  adapt the mail-merge ability in WeBWorK's email so that an instructor can upload a CSV with non-WeBWorK score info.  In this case, conformity of the WeBWorK installation with FERPA is (probably) unchanged.
In reply to Dick Lane

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Christian Seberino -
Does FERPA only pertain at the college level?  The reason I ask is that
everyone and their brother is now using web based tools in high schools where parents can log in and view scores.  They also email grades.....at least for the public schools I have 1st hand knowledge of in my area.

Nevertheless....possible to extract grade info on command line?

cs
In reply to Christian Seberino

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Dick Lane -
"... only at the college level?"

Discussion at
    http://www.naceweb.org/public/ferpa0808.htm
seems useful.  One source of official info is at
    http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html

URLs for other online sources of authoritative info will be welcomed.
In reply to Dick Lane

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Christian Seberino -
Dick

Something is being lost in translation.  

I just did a google search for "high school how access students grades"
and there were several hits like the one below on how parents
can access grades online....

http://www.dentonisd.org/grades
https://home.tamdistrict.org/

If these rules are a concern thousands of schools seems oblivious.

cs

In reply to Davide Cervone

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Christian Seberino -
Thanks for the concern.  This is for homeschoolers so I think I'm ok.
Even for public institutions it would probably be useful to extract
scores without needing to log into Webwork perhaps.

Is it possible without doing some complex MySQL query?

cs
In reply to Davide Cervone

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Andras Balogh -
Since our university is very self-conscious making sure FERPA violation does not happen, while at the same time I have access to all my son's high school grades, I looked up the FERPA law:
http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html

The law applies to all schools that receive funds under an applicable program of the U.S. Department of Education. (read: you violate FERPA, you lose funds)

FERPA gives parents certain rights with respect to their children's education records. These rights transfer to the student when he or she reaches the age of 18 or attends a school beyond the high school level. ...


In reply to Christian Seberino

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Dick Lane -
I wish this request had come with explicit context.  Perhaps something of the form,

"I realize the vast majority of WeBWorK installations are at colleges and universities subject to severe constraints about release of private data about educational status and performance of a student --- e..g., FERPA regulations in the US.  On the other hand, a few of us serve underaged students in a middle or high school or who are being home-schooled.  A parent of a minor child (age < 18 years) is entitled (in the US) to some information about educational performance of their child (during a parent-teacher conference or in some other fashion).  I request suggestions for legally providing info to a parent about their minor child's scores within WeBWorK in a simple/automated manner."
In reply to Dick Lane

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Davide Cervone -
Dick, while I think your statement of the question is great, Christian's original question was only about getting the grades via the command line, not about the legality of sending them to parents, so I would not have expected him to write it in your form. This was not a concern for him, and since I know not everyone is aware of the FERPA implications, I thought I'd suggest that he might want to think about that. So it was I who introduced that to the topic. His was only a technical question.

I'm looking into how it can be done. Sorry to have hijacked the thread.
In reply to Christian Seberino

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Davide Cervone -
I don't see an easy way to extract the scores from the command line. Of course, it would be possible to extract the perl code from the Grades.pm file to get something like this, but I don't currently have the time to do that, and it would take some modification to write out text rather than HTML.

An alternative, however, is to use the Scoring Tools page to get a CSV file containing the data, and generate the emails from that. It is not command line, but would not require the parents to log in (just you).
In reply to Davide Cervone

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Christian Seberino -
Thanks for looking into this. Do you think playing with Grades.pm is something a mere mortal like me could do with minimal experience in the Webwork codebase?

Regarding scraping the data from the Scoring Tools page, I have used tools like Selenium and mechanize.  It may be easier to automate client browser behavior in a script to scrape the data from the Scoring Tools page with these tools.

Thanks again,

Chris
In reply to Christian Seberino

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Davide Cervone -
Do you think playing with Grades.pm is something a mere mortal like me could do with minimal experience in the Webwork codebase?

I wouldn't recommend it unless you are an experienced perl programmer.

Regarding scraping the data from the Scoring Tools page, I have used tools like Selenium and mechanize.

Sorry, I wasn't suggesting that you scrape an HTML page. The Scoring Tools page lets you create a file that you can download and then load into a spreadsheet. So I was expecting you simply to take that file (in CSV format -- comma-separated values, so easily read by a script), and write a script that uses that data to send your emails. The only difference from your original request is that this data has not come from the command line.
In reply to Davide Cervone

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Christian Seberino -
Oh heh thanks.. I know....no worries...

Yes I could do it manually but I was going to do this weekly for many years
in a crontab so hence automation is desired and I'm happy to do it with Selenium and Mechanize.

Thanks again.

cs
In reply to Christian Seberino

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Gavin LaRose -
Hi all,

Not at all relevant to the question of FERPA, but I've been using a command line script to pull scoring data for a while. I've just added it to the large site admin wiki page, http://webwork.maa.org/wiki/Contributed_Admin_Scripts_%28Large_Installations%29. I named the script wwScoreGetter. It probably needs to be wrapped in a script to set the requisite environmental variables, as suggested below.

Cheers,
Gavin

Suggestive wrapper: 

#!/usr/bin/perl -w
use strict;
$ENV{WEBWORK_ROOT}='/opt/webwork/webwork2';
$ENV{MOD_PERL_API_VERSION}=2;

exec "./wwScoreGetter", @ARGV;
In reply to Gavin LaRose

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Robert Mařík -

Thank you, great script.

However does not work 6 years later in 2020. It produces the error message related to missing maketext method.

If someone is interested, it is sufficient to add the maketext method to the FakeRequest package near the and of the wwScoreGetter script. I updated in this way the script in WeBWorK wiki and with this fix it should work again (if you customize the part related to the course names as with the original script).

I find this script useful since I can automatically in cron produce (anonymized) reports on progress of students. In this way the students can compare his or her results with classmates. cron, wwScoreGetter and Rmarkdown produce nice statistics easily and automatically. Thank you again.

In reply to Robert Mařík

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Jack Dockery -
Could I get a working copy of the script?
In reply to Jack Dockery

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Robert Mařík -

I think that you still need to modify the script according to the fact what is your course name and which hw you want to include.

So feel free to modify the parts of https://webwork.maa.org/wiki/WwScoreGetter according to your needs. Also use the wrapper earlier on this page.


In reply to Robert Mařík

Re: How extract all homework scores from Webwork (for a student) in a COMMAND LINE script? (e.g. to send scores in an email to parents)

by Robert Mařík -
Another (different) way could be the direct sql access. Something like the lines below may do the job. You may need to remove the "where ... " clause which selects just students (logins start with x at our uni). And of course: adjust the name of the course. I use adminer from https://www.adminer.org to access the database. It is only one php script and much lighter than php admin. For security reason I use the user which has only read privileges.

SELECT x.user_id, sum(x.status*ifnull(x.value, y.value)) as score
FROM `AM-2021-Marik_problem_user` as x
left join `AM-2021-Marik_problem` as y
on x.set_id = y.set_id and x.problem_id = y.problem_id
where user_id like "x%"
group by user_id
order by score desc