Forum archive 2000-2006

Kenneth Appel - How can we arrange to share problems

Kenneth Appel - How can we arrange to share problems

by Arnold Pizer -
Number of replies: 0
inactiveTopicHow can we arrange to share problems topic started 5/22/2001; 1:03:50 PM
last post 5/25/2001; 12:46:28 PM
userKenneth Appel - How can we arrange to share problems  blueArrow
5/22/2001; 1:03:50 PM (reads: 2441, responses: 6)
As a new participant in WebWork, I would much prefer not to reinvent too many wheels. I am very grateful to Arnie and Mike for putting a great number of problems at our disposal and am pleased that many others will have created problems for mass use. It would be nice to know how we will be able to access the work that has been and will soon be created.

I would hope that the eventual form of the database will include not only the courses for which problems have been created but the texts with which they will work well. Probably each of us will have a course in mind and there will be some things that are covered in some texts and not others. I would assume that there will be certain problems that would be appropriate for all beginning calculus courses (Does anyone not teach differentiation of polynomials) but others that will really depend on the text material. It would be simple for the person who writes the problem to give the text and chapter he or she had in mind, but it would be even more valuable if there were a way for users to indicate other texts (chapters?) for which the problems were appropriate. This might enable a totally new user to indicate a course, a text, and the chapters of the text in the syllabus and immediately be given a collection of problems that people feel are appropriate.

If WebWork continues to expand it will be much too much work for Arnie and Mike to do all that is necessary to make it maximally useful, and those of us who profit from the immense amount they have done should really find a way to parcel out to one another some of the additional work our existence in the system will create.

I am not certain how all of this should be done, but I am certainly willing to volunteer to do my share.

Ken Appel, UNH

<| Post or View Comments |>

userMichael Gage - Re: How can we arrange to share problems  blueArrow
5/24/2001; 10:36:02 PM (reads: 2864, responses: 1)
Thanks, Ken, for bringing up a very important topic. As I see it the creation and distribution of good problems has a number of aspects:

1. Conceive of an educationally appropriate mathematics question for the course. -- (or take one of the existing problems in books, one's earlier tests, etc.)

2. Write the PG code which displays this problem and checks the answer. -- (or find a similar one in the library) -- If it's a new problem or a significant modification of an old one, add the problem to the bank of problems.

3. Make sure that the code is debugged -- in particular that it checks the answer correctly in all situations. -- Transmit any corrections back to the bank of problems. (Double check that these are really correct corrections. :-) )

4. Revise the wording of the problem to make it more effective. -- Transmit revisions back to the bank of problems (as a new version of the problem perhaps? or as a "debugged" version of the old one?)

5. Write a new problem inspired by the existing one. -- enter this into the data bank (as a new problem? a variant of an old one, a variant of a combination of several existing problems?)

Aspect 1 is and always has been the central part of creating good problem sets and software doesn't significantly change this aspect of the process, except perhaps as it makes certain questions infeasible to ask and check or contrarily makes questions which were previously infeasible, now possible.

Aspect 2: Coding a new problem requires some new skills, but we're actively working to minimize the amount of work required to create a good PG problem which can be asked and checked over the web from a good mathematics question clearly expressed on paper. At the very least we want to be able to ask all of the types of questions we asked earlier with pencil and paper (and checked by hand if the class was small enough) and perhaps in some cases ask questions that were previously not feasible. All in all, WeBWorK and the PG language seem to pose few barriers for asking and checking questions relevant to pre-calculus, calculus and other similar courses. The learning curve for writing these problems is still steeper than I would like but it's getting better. Adding a problem to the data bank is easy. Deciding whether the problem is new or a significant modification will probably require a human editor. Good indexes will help this editor and will also in discovering whether a particular mathematical question has already been coded.

Aspect 3: The problem of archiving and retrieving very similar collections of code, each purporting to have fixed a bug in previous pieces of code seems very close to the problems faced by the open source development groups surrounding projects such as apache, perl, mozilla, etc. (see As a first attempt at a storage method for new problems it seems that using one of the Concurrent Version Servers (CVS) designed by these groups would allow us to build on their expertise. (See These systems allow many different people to co-operatively edit and improve a common database of code packages. They also have the capability of "rolling back" to earlier versions if the "fixes" turn out to be mistakes.

Aspect 4: The revision problem is also somewhat related to the problem of coding a large project with many people. The CVS implementations allow one to develop "branches" -- parallel development of code packages that have different features. It also allows one to merge these branches if that becomes desirable. Whether this is the enough or the right kind of flexibility for revising problems remains to be seen.

Aspect 5: In this case one has a new problem, but one would like to maintain some connection with the problems that inspired it. It's not clear whether the "branch" mechanism of the CVS is flexible enough in this case.


Which leads to the question of indexing and cross-indexing, which is really at the heart of Ken's comments above. One way to do "on the fly" indexing is using a search engine -- and storing the problems in an SQL (search query language) database would facilitate this. (Quite likely you would only want the "latest" representative of each problem -- not the previous "buggy" or "confusing" versions of the problems represented in this database.) This kind of searching and indexing takes some skill and can produce variable results, it is pretty flexible however, since one can always think up new combinations of keywords to search for. With the help of this search engine individual researchers/instructors could compile indices to the problem data bank that they have designed for a specific course or to accompany a specific book or to complement a certain style of teaching. Because the indices would point to a development "tree", rather than a single piece of code, improvements to the code --say code that worked faster, or gave better error messages when checking answers or had improved wording would automatically be incorporated into the indices.


So (finally :-) ) here is an initial suggestion for archiving problems:

Bottom layer: A CVS which holds the problem templates. It maintains versions of the problems including both coding corrections and mathematical/educational corrections. Many users can build branches "improving" existing problems. An editorial board decides whether the branches should be folded into the main branch, entered as a separate problem, or pruned. The CVS might be mirrored at more than one institution for reliability.

Middle layer: An SQL database which points to the "current" version of the problem in the CVS and contains additional data: who wrote the problem, various key words describing the content of the problem, the level of difficulty, the availability of the problem (some authors might not allow global availability), possibly information about the problem it was derived from, and a long list of other attributes to be determined. A library board would check in new problems, verify accuracy of attributes (add new attribute categories) etc.

Top layer: Instructors and authors would create and make available lists of problems that they thought appropriate for their course or for a specific text book. These could stored anywhere on the web, including in an SQL database for easy searching. This is where a new instructor for a course would go first to obtain a good starting point for designing a course. From there they could follow links and keywords in the SQL database to find other similar problems, possibly write some of their own and generally customize the problem sets for their course to their liking.


I'm anxious to hear further thoughts on this subject. Is this scheme too complicated? not inclusive enough? too rigid? Are there suggestions for refining and implementing the scheme (both the software support and the organizational structure) or perhaps we should consider a completely different scheme.

Thanks for your attention.

-- Mike Gage

<| Post or View Comments |>

userAndy Wildenberg - Re: How can we arrange to share problems  blueArrow
5/25/2001; 9:07:57 AM (reads: 2857, responses: 0)
I think that this is a well thought out solution to a difficult problem. CVS can do everything that you say, and there are web-based front-ends so that people can download problems without having to figure out CVS (which isn't always trivial when you're getting started). However, the scheme that you're proposing will also likely take some time to develop.

<| Post or View Comments |>

userMichael Gage - Re: How can we arrange to share problems  blueArrow
5/25/2001; 9:29:03 AM (reads: 2813, responses: 0)
Hi Andy,

Have you tried ? It's an up-to-the-hour copy of our library and you can e-mail TeX versions of the problems to yourself.

The page takes a long time to load, but you can save a copy of the page to your desktop and use that instead. There are a lot of extraneous errors, and some features of the problems don't work right yet -- I still have a lot of work to do in cleaning up the scripts which run this page, but it should give you an idea of what is feasible and I would very much like suggestions as I put the finishing touches on it.

I'll be gone this weekend, but will be back Monday night.

-- Mike

<| Post or View Comments |>

userAndy Wildenberg - Re: How can we arrange to share problems  blueArrow
5/25/2001; 12:13:41 PM (reads: 2724, responses: 0)
Mike, Thanks for the link. I actually spotted the link after I posted my original and then edited out my comments asking for the compiled set of problems. Guess I should read posts from oldest to newest instead of top to bottom :-)

<| Post or View Comments |>

userArnold K. Pizer - Re: How can we arrange to share problems  blueArrow
5/25/2001; 12:46:28 PM (reads: 2739, responses: 0)
Hi Andy,

I was in the process of responding to your post and then was puzzled when I noticed that your request for a compiled list had dissappeared. Mike beat me to the punch anyway. The comprehensive list of problems in our library is very useful. At some point we will hopefully have the ability to search for and sort on key words, etc.

We have also found the low tech solution of printing out the problems in the library and scaning them quickly is very effective. We keep a paper copy of the problems in the Math Dept mail room at Rochester. If you do this, be sure your login is listed in $PG_environment{'PRINT_FILE_NAMES_FOR'} so that the actual file names of the .pg files will be printed. You should also be a professor in the course which will enable you to select and print up to 20 sets at a time. Students can only print one set at a time. If you have a very fast server, you can raise this limit.

You can even print out the problems directly from our library at Rochester. Connect to and do a System Login with username practice3 and password practice3. The reason you should use practice3 is that practice3 is the only practice user listed in $PG_environment{'PRINT_FILE_NAMES_FOR'} for this course. Note that since you can not login as a professor to this course you will have to download the problem sets one at a time. A lot of patience would be required to download all 131 sets but in practice, most people will only be interested in a subset of the sets.

<| Post or View Comments |>

userBill Ziemer - Re: How can we arrange to share problems  blueArrow
6/11/2001; 4:19:39 PM (reads: 3165, responses: 0)
I am in the process of coding the middle and top layer of this scheme for a related project, and can quickly modify it for Webworks I am sure. I will bring the prototype to the Madison meeting as a starting point.

<| Post or View Comments |>