It looks to me as if the error message customization discussed at this problem techniques page might get part of the way to providing a better error message for the student. However, it doesn't address the questions of what better feedback would be and whether there is a better way to think about the answer checking in this case.
Another way to frame the problem would be to have a selector for "Does this function have an inverse?" and an answer blank for "If so, fill in the inverse; if not, leave the answer blank empty." But I'm not actually sure if we can ensure that the answer evaluator gets called (so that we could mark a blank answer correct) if there is no supplied answer. So that might not work.
Which is to say, I don't actually have an answer to your question, but maybe this will spur discussion.
Cheers,
Gavin