Workshop on Online Assessment Systems for Mathematics and Computer Science
What's Next?
Call for Submissions / Call for Participation
http://atlantis.seidenberg.pace.edu/wiki/nycoasys2008
There is no fee to attend this workshop.
Where: Pace University, 163 William Street, New York, NY 10038, 15th floor
When: Friday, May 2nd 2008 – 9:30 am to 5 pm
Invited speakers include:

Dr. Michael Gage, University of Rochester, WeBWorK

Dr. David Arnow, Brooklyn College, CodeLab
Context:
Numerous webbased assessment systems have emerged in the past few years to support the teaching and learning of mathematics (e.g., calculus, precalculus, algebra, and finite and discrete mathematics) and programming (e.g., for the Java, C and C++ programming languages) in introductory classes. These systems permit instructors to deliver, and (automatically) grade homework problems and distribute their solutions. Many of them are commercial initiatives (rather than free or open source) and often linked with textbooks; students pay to use them and problem sets are administered centrally. Some of them permit instructors to select and edit problems from a problem library. Rarely, they allow instructors for writing their own customized problems.
The main objectives of these systems are to: 1) engage students in active online learning to augment traditional practices; 2) present students with immediate and customized feedback on their progress; 3) provide students with tailored and constructive support for any problem areas they encounter; and 4) offer instructors the ability to continually monitor and assess student performance (both individually and across classes), to help guide class sessions.
These systems typically manage authenticity to prevent plagiarism and support problems ranging from true / false, multiplechoice and fillintheblank type problems. To address a wider range of problems in mathematics, they need to include a particular mechanism for the treatment of symbolic mathematical formulae; for example, x+1, (x^{21)/(x1) or x+sin(x)}2+cos(x)^2 should be equally accepted as an answer. In programming, such systems must be able to measure the correctness and quality (e.g., code typography) of code fragments. Correctness is generally evaluated based on running shell scripts and unit testing mechanisms. Very few systems deal extensively with code quality.
Existing Systems:

BOSS (http://www.dcs.warwick.ac.uk/boss, Open Source, Computer Science)

CodeLab (http://www.turingscraft.com, Commercial, Computer Science)

CourseMarker (http://www.cs.nott.ac.uk/coursemarker, Commercial, Computer Science)

DevSquare (http://www.devsquare.com, Commercial, Computer Science)

Gradiance (http://www.gradiance.com, Commercial, Computer Science)

JavaBat (http://javabat.com, Free, Computer Science)

MathXL (http://www.mathxl.com, Commercial, Mathematics)

MyMathTest (http://www.mymathtest.com, Commercial, Mathematics)

OWL (http://owl.course.com, Commercial , Computer Science)

Quiz PACK (http://www.sis.pitt.edu/~taler/QuizPACK.html, Open Source, Computer Science)

Viope (http://www.viope.com, Commercial, Computer Science)

WebCAT (http://webcat.cs.vt.edu, Open Source, Computer Science)

WeBWorK (http://webwork.rochester.edu and http://atlantis.seidenberg.pace.edu/~scharff/webwork, Open Source, Mathematics and Computer Science)
(Note: This is not an exhaustive list.)
Objectives of the Workshop:
This workshop will permit attendees to discover and/or practice with a wide range of online assessment systems for mathematics and programming. The systems will be presented by their own designers, extensive users or contributors who will focus on the philosophy behind the system, the stateoftheart of the system and its future. This workshop is also intended to create a space of discussion for instructors in mathematics and computer science interested in building a community of instructors in mathematics and computer science interested in online assessment systems. The experience of each community can transfer to the other and create synergies.
Another objective is to discuss the future and features of the next generation of such systems. For example, some of the newest systems offer graphicalbased progress and performance visualization, with error highlighting of code and hints on how to correct it. They also begin to use sophisticated data mining techniques on the repository of existing submissions to provide students' learning profiles and generate dynamic questions at the appropriate level of difficulty. After the workshop we will survey the attendees to assess their impression on the features of the next generation of online assessment systems in mathematics and programming.
Themes:

The stateoftheart of online assessment systems in mathematics and programming

Teaching introductory mathematics and programming courses with online assessment systems

Experience with teaching with different online assessment systems

Comparison of different online assessment systems

Designing online assessment systems for mathematics and programming

Survey of online assessment systems in mathematics and programming

The next generation of online assessment systems in mathematics and programming
Intended audience:
This workshop is intended for instructors involved in teaching courses in mathematics and programming, and/or interested in the design and use of online assessment systems to support students' learning and their teaching.
Schedule:

The workshop will run from 9:30 am to 5 pm. Parallel session will be organized in the afternoon to permit mathematics and programming instructors to meet separately and discuss specifics, before convening again to share ideas.

Refreshments and lunch will be served during the day.
Labs:
Attendees will have access to a lab to practice with existing online assessment systems in mathematics and programming.
Call for Submissions:
If you are interested in presenting at the workshop, please submit a 300word abstract at: webworknyc2008@gmail.com with subject submission by April 15th. The abstracts will be posted on the web page of the workshop. During their presentations, speakers will be provided with a laptop, a projector and a whiteboard with pens.
Submissions will be acknowledged and authors will be notified of acceptance/rejection by April 17th.
Call for Participation:
If you do not plan to present but are interested in attending this workshop, please rsvp by sending an email to: webworknyc2008@gmail.com by April 15th with subject participation.
Contact email:
Support:
This workshop is funded by an NSF CCLI AI grant "Adapting and Extending WeBWorK for Use in the Computer Science Curriculum" (#0511385 and #0511391, 20052008)
Organizers:
Christelle Scharff, Pace University, Seidenberg School of Computer Science and Information Systems, New York, NY 10038, cscharff@pace.edu, Tel: 212 346 1016, Fax: 212 346 1863
Olly Gotel, Pace University, Seidenberg School of Computer Science and Information Systems, New York, NY 10038, ogotel@pace.edu
Andrew Wildenberg, Cornell College, Department of Computer Science, Mt Vernon, IA 52314, awildenberg@cornellcollege.edu
Richard Kline, Pace University, Seidenberg School of Computer Science and Information Systems, New York, NY 10038, rkline@pace.edu