<select>, which usually means use the OS menu system. In general, these do not let you include HTML, and that means you can't include math in menu items. It looks like recent versions of Safari and Firefox are the exceptions (I wasn't aware of that), but it is not something you can count on, as you have noted.Because you can't really put HTML in the menus in general (and certainly couldn't when the pop-up code was written), the pop-up doesn't try to replace its entries with images when you are using image mode (it's only because MathJax processes the page after it is produced that it works with MathJax).
Unicode does work more universally, but not always, as it depends on having appropriate fonts available on the user's system, and some browsers (notably IE on Windows XP) don't always locate the fonts even when they are installed, so it is touch and go for that.
The only solution I can think of is to implement an HTML-based menu system instead so that the menu items can contain images or MathJax output. There are certainly menu libraries that could be used (indeed, MathJax has one of its own), but it would take some work to hook it into WeBWorK.
Davide