Roadmap Details

Roadmap for webwork development

Webservices API

At the Raleigh code camp a number of us discussed putting together a solid API for the webservices and currently have a working draft. The API is a listing of all the interactions that we want to have with the webwork server (or actually the database).

Current Webservice

Separate from the current webwork pages, there is a webservice to serve information from the database in lib/WebworkWebservice.pm and lib/ContentGenerator/instructorXMLHandle.pm and a handful of additional files in /lib/WebworkWebservice. This webservice mainly serves some of the new tools (Homework Manager, Classlist Manager and with a new version of the LibBrowser). The idea would be to deprecate this webservice (see RESTful webservice below).

RESTful Webservice

Most current webservices run what is called a RESTful service. In short, it's a common way to send and request information from a webserver. There are many reasons for using RESTful webservice, but the best is that this will be based on the API listed above. Once we have a solid API, we can go about designing the backend.

The current webservice is not RESTful. A near-term goal will be to create such a webservice in perl, possibly using Perl Dancer, Catalyst or Apache2::REST. Some quick research shows Perl Dancer may be the way to go for this. The reason for using a perl webservice is that it will be the quickest way to develop such a webservice since the rest of the codebase is in perl.

A longer term goal will be to write non-perl RESTful webservices. These may include a node.js service or any other type of back end. More of this to follow.

Switching the core database

Another current trend in webservices is that of a NoSQL database, which store flexible documents. This is in contrast to a SQL (or relational) database in which everything needs to be stored as a table. The flexibility of a NoSQL database will be that data structures can be nested. For example, A ProblemSet can store information about the problem as well as an array of users that set is assigned for. The users can also be nested if need be.

Another advantage to a NoSQL database is that the documents are injected and retrieved via JSON (which is the standard interchange format for objects on the web) and no conversion is necessary. This leads to clearer code on the server side.

Some common NoSQL databases that are being used are MongoDB and CouchDB.

Changing the Webwork UI

In addition to the backend, there has been progress on the Webwork UI. The goal of the new Webwork tools are to modernize and simplify the interface. These tools use [backbonejs.com Backbone] to help with 1) separating the data from the view (in MVC style) and 2) synching the data on the client with that on the server. Backbone was designed to interact with a server in a RESTful manner, and although we currently don't have such a webservice, there are ways to manage. Unfortunately, this means writing a lot of additional code to deal with our current API (or lack thereof).

Tasks

The following is the order of some of the backend tasks listed above.

1. Finish the API.
2. Develop a perl RESTful webservice
3. Convert the new WW pages to use the new webservice (Note: this can be done in stages as the old webservice will still work.)
4. Develop a non-perl RESTful webservice and new database. The new WW pages SHOULD work exactly the same using either RESTful webservice.
5. Develop tools to convert the mySQL database to the new database.