OK, I never know whether to followup when my problem turns out to be mostly my own stupidity, but just in case:
1) The "upgrade courses" option in the admin course is a tiny bit confusing. When you choose that option, you get a list of courses to upgrade, and link at the bottom of the list saying "upgrade courses."
If you click on that link, it _doesn't_ actually upgrade the courses yet - it shows you in more detail what is wrong, and it turns out that at the bottom of this detailed report, you have to click "upgrade courses" once more to actually have it take effect.
I didn't notice this for a very long time...
2) Thanks to Mike's help, I did find the appropriate password for mysql, but it it didn't help since that wasn't the problem.
3) I had an unrelated problem. When upgrading via svn, I did that as a non-root user who had write privileges to the webwork directories. Whether because of that, or for some other reason, group and ownership were not preserved. As a result, apache couldn't write to /opt/webwork/webwork2/tmp, which also causes problems of a different sort.
Thanks Mike and Andras for the help!
-Hal