When I have done that it has been because the first version didn't work or was way to hard so it anyone who got credit is likely to have done the second version.
You can even delete problems AS LONG AS YOU DO NOT RENUMBER THEM. The difficulty is that the problem number is an part of the primary key for determining the students grade on the problem. (A flaw in our original database design 15 years ago.). That means that if you delete and renumber you will have no idea whether problem5 grades refer to the original problem 5 or the problem 5 after the renumbering. In principle you could delete the last problem without any confusion.
Arnie's solution about marking everyone's problem correct seems much superior to me and is the one that I use most often.
I often correct problems on the fly. But if the correction involves too many changes (adding new random variables or changing the order in which they are created for example) then the random number generator will generate different values for the new version of the problem. That's usually not a big deal as Paul points out.